Unify.Product.IdentityBroker.AdapterEnginePlugInKey.extensibility.config.xml

Hi,

I need some help with 2 items : time offset and sliding date window in IdB Adapter transformations for a customer. They have chris21 as the source system.

IdB version: 4.1

Q1: Sliding Date Window

When an employee starts on a new position from a specific date (i.e 05 June 2018) the data flows in the connector and the data transformation is applied a day earlier(i.e on 04th June 2018). The implication is the job title is changed the day before the correct date. the customer wants to know whats wrong.

The transformation as configured: 

Q2: Time offset

When a user is terminated in chris21, he gets a termdate set and a simple transformation is applied to set the terminated flag as "True"/"False". But for example if a user has term date is 02 july 2018, by logic, it should add 24 hrs and set terminated flag to "true" 03 july 2018 midnight, but the terminated flag is set to "true" only around mid next day(i.e 04 july 2018 midday). Need some help fixing this issue.

Please help me to know where the issue is, thanks. really appreciate and thanks in advance

I have attached the Adapter xml, if you want for reference. Thanks.

Group ID's are being duplicated on connectors when modifying group membership, and not being removed on deletion.

Reproduction steps:

1. Create multiple connectors
2. Create connector group
3. Add a few connectors to group. Save group
   1. At this point, note the configuration file adds the group ID to the connector configs
4. Modify group again, adding another connector
   
   1. At this point, note that the original connectors added to the group now have two entries for group ID in the connector config. The new connector has one entry for group ID.
5. Modify group again, removing one of the original connectors
   1. At this point, note that only one of the ID's is removed from the configuration
6. Modify group again
   1. At this point, note that the connector removed in the previous step is still marked as being a member of the group
7. Delete group
   1. At this point, note that group ID's do not get removed from the connector configuration.

When attempting to create and save a join transformation, the following error presents when attempting to save the transform without any attributes mapped:

If I edit the configuration to remove the mappings, the transformation performs as expected, which confirms that they are not actually required by the service.
The transformation should let you configure and save without any mapped attributes.

As per the documentation ( https://voice.unifysolutions.net/knowledge-bases/7/articles/2873-adapter-overview ), a change in the DN template for an adapter, should generate a warning for clearing precalculated entities and generating changes again to ensure entities are updated appropriately. 

There appears to be a bug in 5.3.1, where changing the DN template does not generate a warning on the UI or in the logs. 

It would be good if the warning could be also generated for a change in the adapter object class or container name, if appropriate.

When a date field is configured on a connector / adapter in UNIFYBroker, it displays in the entity view table in a friendly format (such as 04/Jul/2014). However, when exposed into a PowerShell transformation or downstream, it is exposed in a different format (2014-07-04T00:00:00.000)

This is also true for the Timestamp field, which displays in the entity view table as something like 04/07/2014 12:00:00 AM

It would be good if you could see the raw date format as exposed by the gateways, as well as a formatted date for easier reading.

It would also be good to get some consistency across the formats that are displayed in the entity view tables.

<img alt="Unify.Connectors.CSV - The CSV connector allows for manipulation of a source CSV file. Useful for solution testing." class="AutoThumbnail" src="1b175bc1-da20-4064-9d74-d43e5744931f.connector" style="vertical-align: middle;" title="Unify.Connectors.CSV - The CSV connector allows for manipulation of a source CSV file. Useful for solution testing.">

Only fixes when you go into "edit connector" and "save". Saves successfully and shows the correct image after (even if the csv file still doesn't exist).

On installing UNIFYBroker with the database server being remote, the following is logged by the installer:

CreateDatabase:  Error 0x80004005: failed to create to database: 'UnifyIdentityBroker', error: unknown error
MSI (s) (2C!88) [11:12:22:654]: Product: UNIFYBroker/Service v5.3.1 RC1 -- Error 26201. Error -2147467259: failed to create SQL database: UnifyIdentityBroker, error detail: unknown error.

Error 26201. Error -2147467259: failed to create SQL database: UnifyIdentityBroker, error detail: unknown error.
CustomAction CreateDatabase returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

IDB 5.3.1.0

I'm currently using a group transformation to determine the child objects for org units, and mapping their relational DN (multi valued).

I'm finding that i'm getting the following error in the adapter when it attempts to run reflection:

Adapter
Adapter 81400d1b-2122-4571-92a8-c12470291968 page errored on page reflection. Duration: 00:00:02.5781432. Error: System.Exception: Max iteration count exceeded for base key discovery: ChildOrganisationUnits
at Unify.Product.IdentityBroker.KnownEntityBase`3.GetBaseKeysIter(TKey key)
at Unify.Product.IdentityBroker.KnownEntityBase`3.GetValueOriginInformation(TKey key)
at Unify.Product.IdentityBroker.EntityToEntityValueOriginsAdapter`2.<>c__DisplayClass3_0.<Transform>b__0(GroupedNameValueCollectionKey schemaKey)
at System.Linq.Enumerable.<SelectManyIterator>d__16`2.MoveNext()
at System.Linq.Enumerable.<SelectManyIterator>d__16`2.MoveNext()
at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
at Unify.Framework.Collections.ThreadsafeHashSet`1.UnionWith(IEnumerable`1 enumerable)
at Unify.Product.IdentityBroker.Adapter.WriteReflectionPageAdapterChanges(IAdapterEntityPartitionUpdatableContext adapterContext, IEntityValueOriginContext originContext, IDictionaryTwoPassDifferenceReport`4 report)
at Unify.Product.IdentityBroker.Adapter.ReflectChangesInner()
at Unify.Product.IdentityBroker.Adapter.ReflectChanges()
at Unify.Product.IdentityBroker.AdapterAuditingDecorator.ReflectChanges()
at Unify.Product.IdentityBroker.AdapterNotifierDecorator.ReflectChanges()
at Unify.Product.IdentityBroker.ReflectAdapterOnChangeDueJob.<RunBase>b__9_0(IOperationalAdapter adapter).
Error details:
System.Exception: Max iteration count exceeded for base key discovery: ChildOrganisationUnits
at Unify.Product.IdentityBroker.KnownEntityBase`3.GetBaseKeysIter(TKey key)
at Unify.Product.IdentityBroker.KnownEntityBase`3.GetValueOriginInformation(TKey key)
at Unify.Product.IdentityBroker.EntityToEntityValueOriginsAdapter`2.<>c__DisplayClass3_0.<Transform>b__0(GroupedNameValueCollectionKey schemaKey)
at System.Linq.Enumerable.<SelectManyIterator>d__16`2.MoveNext()
at System.Linq.Enumerable.<SelectManyIterator>d__16`2.MoveNext()
at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
at Unify.Framework.Collections.ThreadsafeHashSet`1.UnionWith(IEnumerable`1 enumerable)
at Unify.Product.IdentityBroker.Adapter.WriteReflectionPageAdapterChanges(IAdapterEntityPartitionUpdatableContext adapterContext, IEntityValueOriginContext originContext, IDictionaryTwoPassDifferenceReport`4 report)
at Unify.Product.IdentityBroker.Adapter.ReflectChangesInner()
at Unify.Product.IdentityBroker.Adapter.ReflectChanges()
at Unify.Product.IdentityBroker.AdapterAuditingDecorator.ReflectChanges()
at Unify.Product.IdentityBroker.AdapterNotifierDecorator.ReflectChanges()
at Unify.Product.IdentityBroker.ReflectAdapterOnChangeDueJob.<RunBase>b__9_0(IOperationalAdapter adapter)

I'm assuming that it's because there's a forced upper limit on how many times it can loop through to find related entities and group - and i'm exceeding that limit with the children.

Would it be possible to get this limit configurable or similar? I'm unsure how many children the largest parent has, but some way around the upper limit would be fantastic.

When using v5.3 entity search table (either connector or adapter), i'm running into a couple of errors;

If I attempt to order a column (just by clicking on the header), and any entity in the set has a NULL value for that field, I get this error from datatables:

Error
System.AggregateException: One or more errors occurred. ---> Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
at Unify.Connect.Web.Client.EntityClient.<searchentitiesasync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connect.Web.Client.ProfiledEntityClient.<searchentitiesasync>d__4.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Unify.Connect.Web.IdentityBrokerEntitySearchController.CurrentEntities(EntityRetrievalInformation`1 information)
at Unify.Connect.Web.IdentityBrokerEntitySearchController.<entitydata>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at lambda_method(Closure , Task )
at System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass34.<begininvokeasynchronousactionmethod>b__33(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<invokeactionmethodfilterasynchronouslyrecursive>b__3c()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass45.<invokeactionmethodfilterasynchronouslyrecursive>b__3e()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<>c__DisplayClass28.<begininvokeaction>b__19()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<begininvokeaction>b__1b(IAsyncResult asyncResult)
---> (Inner Exception #0) HTTP Response: {"Message":"An error has occurred.","ExceptionMessage":"The entity does not contain a value for the XYZ field.","ExceptionType":"Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException",
"StackTrace":" at Unify.Product.IdentityBroker.EntityBase`3.GetValue[TValue](TKey key)\r\n 
at Unify.Connect.Web.ValueTypeIdentifierEntitySearchUtilityBase`4.<>c__DisplayClass1_0.<orderby>b__0(TEntity item)\r\n 
at System.Linq.EnumerableSorter`2.ComputeKeys(TElement[] elements, Int32 count)\r\n at System.Linq.EnumerableSorter`1.Sort(TElement[] elements, Int32 count)\r\n at System.Linq.OrderedEnumerable`1.<getenumerator>d__1.MoveNext()\r\n at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)\r\n at Unify.Product.IdentityBroker.EntityController.ApplySkipTake(IEnumerable`1 entities, Int32 skip, Int32 take)\r\n at Unify.Product.IdentityBroker.EntityController.ApplySearch(SearchInformation searchInformation, IEnumerable`1 entities, Guid partitionId)\r\n at Unify.Product.IdentityBroker.EntityController.SearchEntities(Guid partitionId, SearchInformation searchInformation)\r\n at lambda_method(Closure , Object , Object[] )\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<getexecutor>b__9(Object instance, Object[] methodParameters)\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.<invokeactionasynccore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ActionFilterResult.<executeasync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Filters.AuthorizationFilterAttribute.<executeauthorizationfilterasynccore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Dispatcher.HttpControllerDispatcher.<sendasync>d__1.MoveNext()"} Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
at Unify.Connect.Web.Client.EntityClient.<searchentitiesasync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connect.Web.Client.ProfiledEntityClient.<searchentitiesasync>d__4.MoveNext()<---</searchentitiesasync></searchentitiesasync></sendasync></executeauthorizationfilterasynccore></executeasync></invokeactionasynccore></getexecutor></getenumerator></orderby></begininvokeaction></begininvokeaction></invokeactionmethodfilterasynchronouslyrecursive></invokeactionmethodfilterasynchronouslyrecursive></begininvokeasynchronousactionmethod></entitydata></searchentitiesasync></searchentitiesasync>

Likewise, If I attempt to use the "Is-Null" filter on any column (whether they're all populated or not), I get the following error:

Error
System.AggregateException: One or more errors occurred. ---> Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
at Unify.Connect.Web.Client.EntityClient.<searchentitiesasync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connect.Web.Client.ProfiledEntityClient.<searchentitiesasync>d__4.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Unify.Connect.Web.IdentityBrokerEntitySearchController.CurrentEntities(EntityRetrievalInformation`1 information)
at Unify.Connect.Web.IdentityBrokerEntitySearchController.<entitydata>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at lambda_method(Closure , Task )
at System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass34.<begininvokeasynchronousactionmethod>b__33(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<invokeactionmethodfilterasynchronouslyrecursive>b__3c()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass45.<invokeactionmethodfilterasynchronouslyrecursive>b__3e()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<>c__DisplayClass28.<begininvokeaction>b__19()
at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<begininvokeaction>b__1b(IAsyncResult asyncResult)
---> (Inner Exception #0) HTTP Response: {"Message":"An error has occurred.","ExceptionMessage":"Could not find any resources appropriate for the specified culture or the neutral culture. Make sure \"Unify.Connect.Web.EntityResources.resources\" was correctly embedded or linked into assembly \"Unify.IdentityBroker.Entity.Api.Shared\" at compile time, or that all the satellite assemblies required are loadable and fully signed.","ExceptionType":"System.Resources.MissingManifestResourceException","StackTrace":" at System.Resources.ManifestBasedResourceGroveler.HandleResourceStreamMissing(String fileName)\r\n at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary`2 localResourceSets, Boolean tryParents, Boolean createIfNotExists, StackCrawlMark& stackMark)\r\n at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo requestedCulture, Boolean createIfNotExists, Boolean tryParents, StackCrawlMark& stackMark)\r\n at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents)\r\n at System.Resources.ResourceManager.GetString(String name, CultureInfo culture)\r\n at Unify.Connect.Web.IdentifierEntitySearchUtilityBase`2.GenerateSearchFunction(FilterInformation searchInformation, IDictionary`2 schema)\r\n at Unify.Product.IdentityBroker.EntityController.ApplyFilter(IEnumerable`1 entities, FilterInformation filterInformation, Guid partitionId)\r\n at Unify.Product.IdentityBroker.EntityController.<>c__DisplayClass8_0.<applyfilter>b__0(IEnumerable`1 filtered, FilterInformation filter)\r\n at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)\r\n at Unify.Product.IdentityBroker.EntityController.ApplySearch(SearchInformation searchInformation, IEnumerable`1 entities, Guid partitionId)\r\n at Unify.Product.IdentityBroker.EntityController.SearchEntities(Guid partitionId, SearchInformation searchInformation)\r\n at lambda_method(Closure , Object , Object[] )\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<getexecutor>b__9(Object instance, Object[] methodParameters)\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.<invokeactionasynccore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ActionFilterResult.<executeasync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Filters.AuthorizationFilterAttribute.<executeauthorizationfilterasynccore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Dispatcher.HttpControllerDispatcher.<sendasync>d__1.MoveNext()"} Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
at Unify.Connect.Web.Client.EntityClient.<searchentitiesasync>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connect.Web.Client.ProfiledEntityClient.<searchentitiesasync>d__4.MoveNext()<---</searchentitiesasync></searchentitiesasync></sendasync></executeauthorizationfilterasynccore></executeasync></invokeactionasynccore></getexecutor></applyfilter></begininvokeaction></begininvokeaction></invokeactionmethodfilterasynchronouslyrecursive></invokeactionmethodfilterasynchronouslyrecursive></begininvokeasynchronousactionmethod></entitydata></searchentitiesasync></searchentitiesasync>

I can seem to use other filters (such as "contains") on columns with null values no worries.

Replicated on both connector and adapter, on two machines. V5.3.0RTM

When you add a new Aurion connector (mainly with the Generic one, but applicable to others too), configuring the schema options automatically generates schema mappings. These display on the UI, but don't validate until the connector is run. Therefore you get an error on the first run of the connector regarding schema mappings, which means you have to run the save action on the connector to get it to persist to config.

It would be good if the schema mapping defaults could be persisted to configuration when the schema is modified or connector enabled - that way if they're not valid the connector isn't able to be enabled. 

Currently an issue against IDB v5.3