This was bit pain on the project as the module was not behaving as expected and as part of the analysis/investigation with sitecore we end up fing fiding couple product bugs.
I like to mention those bugs here so It will be useful for someone who face same issues.
Issue -1
It started with below exception where live events were not processing by the marketing automation engine.
The error mentions “System.Configuration.ConfigurationErrorsException: A connection string with the name ‘xdb.processing.pools’ is not configured.”
2019-02-27 05:39:17.301 +00:00 [Information] XConnect Test Host Application Start, Machine: "XXXXX", Site: "XXX-sitecore9-ma-ops", AppId: "/LM/W3SVC/1175419295/ROOT" 2019-02-27 05:39:19.660 +00:00 [Information] Certificate Validation Filter Enabled, Thumbprint: 3E960CB20CC96DAA6B5A9CA1266687A4A99DA3BB 2019-02-27 05:39:19.660 +00:00 [Information] SSL Validation Filter Enabled 2019-02-27 05:47:23.684 +00:00 [Information] AppInsightsPerformanceCounters Constructor, Instance:tspl-ase-staging-sitecore9-ma-ops, Path: D:\home\site\wwwroot\App_Data\Diagnostics, CounterFilePattern: *.json 2019-02-27 05:47:25.708 +00:00 [Error] Failed to instantiate a processor of type '"Sitecore.Xdb.MarketingAutomation.Processing.LiveEventProcessor"'. System.Configuration.ConfigurationErrorsException: A connection string with the name 'xdb.processing.pools' is not configured. at Sitecore.Xdb.Processing.Queue.SqlServer.ConfigurationConnectionStringResolver.Resolve(String name) at Sitecore.Xdb.Processing.Queue.SqlServer.SqlServerProcessingPool`2..ctor(TPoolDefinition poolDefinition, SqlServerProcessingPoolSettings settings, IConnectionStringResolver resolver, IRetryManager retryManager) at Sitecore.Xdb.Processing.Queue.SqlServer.SqlServerInteractionLiveProcessingPool..ctor(ProcessingPoolDefinition definition, SqlServerProcessingPoolSettings settings, IConnectionStringResolver resolver, IPerformanceCounters performanceCounters, IRetryManager retryManager) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ClosedIEnumerableService.CallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.DependencyInjection.ServiceProvider.TransientCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Sitecore.XConnect.Configuration.Extensions.<>c__DisplayClass13_0.<UseServiceDecorator>b__1(IServiceProvider provider) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Microsoft.Extensions.Configuration.ConfigurationServiceType.<>c__DisplayClass24_0.<op_Implicit>b__0(IServiceProvider sp) at Microsoft.Extensions.DependencyInjection.ServiceProvider.TransientCallSite.Invoke(ServiceProvider provider) at Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) at Sitecore.Xdb.MarketingAutomation.Factory.ProcessorFactory.CreateProcessor(Guid contactId, ExecutionData data) 2019-02-27 05:47:25.771 +00:00 [Error] Failed to create a processor for live event request. Contact: '212ea775-cb36-0700-0000-057e672890d0'
As per the sitecore documentation there is no need for that connection string in the “Marketing operation” service.
Sitecore has registered the exception as a bug for Sitecore version 9.0.2. And created a public reference number 166086 to track status of this bug. And they thanks for reporting it.
In fact, Marketing Operations role requires access to “xdb.processing.pools” database and appropriate connection string must present. As a workaround, could I ask you just to add “xdb.processing.pools” connection string to your problematic marketing operations server. You can just copy “xdb.processing.pools” connection string from your XConnect service.
Issue -2
I was really hoping that the first exception resolve my issues however I bump upon the below exception.
2019-04-02 00:00:00.839 +00:00 [Error] An exception occurred during processing for worker '014905aa-eff2-4ca9-a90c-89832dfd9f05' System.AggregateException: One or more errors occurred. ---> Sitecore.XConnect.Serialization.XdbSerializationException: Unknown type 'Sitecore.ContentTesting.Model.xConnect.PersonalizationEvent' at Sitecore.XConnect.Serialization.XObjectReader.ReadXObject(XdbType expectedType) at Sitecore.XConnect.Client.Serialization.XObjectJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Sitecore.Xdb.MarketingAutomation.Core.Serialization.LiveEventDataJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Parsers.WorkItemBatchParser`1.ConvertExecutionData(Byte[] data, String typeName) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Parsers.WorkItemBatchParser`1.<ParseBatchAsync>d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Commands.CheckoutCommand`1.<ExecuteAndProcessResultsAsync>d__30.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.SqlServerCommand`2.<ExecuteAsync>d__17.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.SqlServerAutomationPool.<CheckoutAsync>d__24.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.Pool.AutomationPoolCountersDecorator.<CheckoutAsync>d__4.MoveNext() --- End of inner exception stack trace --- at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at Sitecore.Xdb.MarketingAutomation.Processing.PoolWorker.Process() at Sitecore.Xdb.MarketingAutomation.Core.Processing.Worker`1.StartProcess() ---> (Inner Exception #0) Sitecore.XConnect.Serialization.XdbSerializationException: Unknown type 'Sitecore.ContentTesting.Model.xConnect.PersonalizationEvent' at Sitecore.XConnect.Serialization.XObjectReader.ReadXObject(XdbType expectedType) at Sitecore.XConnect.Client.Serialization.XObjectJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Sitecore.Xdb.MarketingAutomation.Core.Serialization.LiveEventDataJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Parsers.WorkItemBatchParser`1.ConvertExecutionData(Byte[] data, String typeName) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Parsers.WorkItemBatchParser`1.<ParseBatchAsync>d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.Commands.CheckoutCommand`1.<ExecuteAndProcessResultsAsync>d__30.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.SqlServerCommand`2.<ExecuteAsync>d__17.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.SqlServer.Pool.SqlServerAutomationPool.<CheckoutAsync>d__24.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Sitecore.Xdb.MarketingAutomation.Pool.AutomationPoolCountersDecorator.<CheckoutAsync>d__4.MoveNext()<---
The issue “Unknown type ‘Sitecore.ContentTesting.Model.xConnect.PersonalizationEvent’” was then registered as a bug for Sitecore version 9.0.2. You can use public reference number 258373 to track status of this bug.
Sitecore has provide below hotfix to address the issue. Great support!
Please install the following hotfix in order to overcome the issue:
Be aware that the hotfix was built specifically for Sitecore XP 9.0 rev. 180604 (Update-2), and you should not install it on other Sitecore versions or in combination with other hotfixes, unless explicitly instructed by Sitecore Support.
Note that you need to extract ZIP file contents to locate installation instructions and related files inside it.
Sitecore Blogs 