When an Aurion connector import operation runs while a connector export is already running the export fails with its session logged off:

14:30:00 baseline sync kicks off export:

Synchronization job started syncing 7977 changes on the 'Managed User > Aurion Security User' link from the locker to adapter. Job ID: 9f521182-e52e-4082-8685-899600d4456f",Normal

14:33:37 evidence of exports occurring:

Add entities [Count:3] to connector Aurion Security User reported 3 entities saved, 3 failed. Duration: 00:00:01.0862857",Normal

14:51:15 scheduled import operation runs:

20230531,14:51:15,UNIFYBroker,Change detection engine,Information,"Change detection engine import all items started.

Change detection engine import all items for connector Aurion Security User started.",Normal

14:52:54 hundreds of failed updates reported due to logoff (shared session with import?):

20230531,14:52:54,UNIFYBroker,Connector,Information,"Update entities to connector completed.

Update entities 6375 to connector Aurion Security User reported 6375 entities saved, 225 failed. Duration: 00:19:11.1067288",Normal

20230531,14:52:54,UNIFYBroker,EntitySaver,Error,The entity bcoff (2a12bf64-7aec-4101-a696-fa84054b0a0d) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal

20230531,14:52:54,UNIFYBroker,EntitySaver,Error,The entity kbair (603121f8-2724-4d34-a177-630bb718656b) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal

…

Either import and export operations should be able to run concurrently without interfering with each other, or otherwise the system should not permit them to run concurrently.

Matt wrote in email:

I have found some documentation that suggest long-lived TCP connections in Azure components are terminated after 4 minutes. Normally, you’d hope that a terminated TCP connection would immediately result in an exception such as “the underlying connection was closed”.

... I was able to reproduce the lack of response on this report in both the PowerShell harness and SoapUI. After some testing to force the connection to stay alive, I was able to find a way to successfully get this report to import on the test VM.

.... I’ll have to investigate the best way to solve this one long term. It might be something we need to set inside the connector, or inside UNIFYBroker, or inside the platform. I’m not sure if the problem is unique to Aurion or whether we can replicate it elsewhere – that’ll be part of the investigation to determine the best way to fix it. This command shouldn’t even be necessary – so we’d have to investigate the side effects of it and ensure that we won’t break something else by having this set.

Due to my customer having long running reports, could you please provide a fix for this issue?

Hi @Matthew Davis @Beau Harrison

A customer has asked which Aurion API function is used by the Aurion Person connector export functionality. I believe it’s EMP_UPDATE_PERS. Can you please confirm?

Thanks

The following error is occurring on both Full imports and Delta imports from Active Directory, in a customer UNIFYConnect environment:

20230512,02:28:25,UNIFYBroker,Change detection engine,Error,"Change detection engine import changes failed.
Change detection engine import changes for connector AD User failed with reason One or more errors occurred.. Duration: 00:00:02.9714687
Error details:
System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Unify.Connectors.AD.ADConnector.TransformEntry(ADAgent agent, SearchResultEntry searchResultEntry, Int64& uSNChangedToken)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.d__10`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.AuditEntityPollingAsyncConnectorDecorator.<>c__DisplayClass1_0.b__0(IEnumerable`1 entities)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass8_0`1.b__0(Task`1 t)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass5_0`1.b__0(Task`1 t)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.EventNotifierEntityPollingAsyncConnectorDecorator.d__1.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Unify.Product.IdentityBroker.ChangeDetectionEntityPollAsyncJob.RunBase()
   at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
   at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c__DisplayClass30_0.b__0()
   at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)
---> (Inner Exception #0) System.NullReferenceException: Object reference not set to an instance of an object.
   at Unify.Connectors.AD.ADConnector.TransformEntry(ADAgent agent, SearchResultEntry searchResultEntry, Int64& uSNChangedToken)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.d__10`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.AuditEntityPollingAsyncConnectorDecorator.<>c__DisplayClass1_0.b__0(IEnumerable`1 entities)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass8_0`1.b__0(Task`1 t)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass5_0`1.b__0(Task`1 t)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.EventNotifierEntityPollingAsyncConnectorDecorator.d__1.MoveNext()<---
",Normal

When a locker has inbound provisioning but not inbound deprovisioning it is sometimes desirable to manually delete an entity which is no longer wanted.  A button to do this would be helpful, because the current workaround is to suspend all outbound processing, delete all locker entities and perform a complete data reload.

After a link join is removed using the UI a locker entity retains attribute values that were previously contributed through the link.  This outcome is highly undesirable, since the locker now has information that should not be there (in my case an AD samaccountname value which is misleading, since the the AD user's adapter entity join was removed in order to allow the adapter entity to rejoin to a different locker record, but now both locker records have the same samaccountname value!)  There is no way to initiate removal of this out-of-date attribute data from the locker other than joining the locker to a different adapter entity (which may not be feasible) or performing a complete data reload from scratch (which is time-consuming and an outage).  Running a baseline sync on the link does not fix the problem.  The attribute values on the entity should be updated appropriately when the join is removed.

This issue is occurring in my customer's PROD environment, despite me having implemented the workaround of one Aurion agent for each Aurion connector and that workaround appearing to have worked correctly in their TEST environment (where these errors have not be observed):

Update entities 11 to connector Aurion Security User reported 11 entities saved, 11 failed. Duration: 00:00:57.4310747",Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (c71168bd-30e8-460e-832c-4e0983b47d6b) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (8d7ab2cf-fb2a-41c4-9700-fbba79ca1722) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (00a0802e-f57f-40e2-8667-7a9c5d7a7004) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (800e7663-b6b8-4e36-849b-477c69fb21c0) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (e55f8dbf-8d45-4f20-b561-08603923c0f0) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (ee80f7e3-7b8b-4cf1-bf12-3e58713ee29b) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (2435a9d0-263f-4c43-9e99-36ae99e239ae) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (dcfa3e64-be3a-41c4-a19b-a28fcef61700) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (c33745d2-dfd0-44f9-bbb9-456d2afdaac0) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (a5805bac-7b60-4ed9-9bae-449b481c094b) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal
20230404,22:52:59,UNIFYBroker,EntitySaver,Error,The entity XXX (3d3b1f15-bc78-4415-9419-4c782f956976) for the adapter Aurion Security User (f3c9eba8-ccd2-447b-ba37-67796af63171) failed to update for the following reasons: Aurion API error -1: You are not logged on to the Aurion web services application server. Use the LOGON operation.,Normal

All Aurion connectors are part of an exclusion connector group.

Could you please investigate and advise?

I renamed a locker field "HRISEmailAddress" to "EmailAddress" in the UNIFYBroker UI, and this stack dump error appeared:

System.Exception: Swagger Exception could not be parsed. SE response code: 500; SE response text: {"Message":"An error has occurred.","ExceptionMessage":"'The field EmailAddress could not be added to the schema","ExceptionType":"Unify.Framework.Schema.SchemaException","StackTrace":" at Unify.Framework.Schema.Schema`6.Add(TKey key, TFieldDef value)\r\n at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)\r\n at Unify.Product.IdentityBroker.EntitySchemaFactory.CreateComponent(IEntitySchemaConfiguration factoryInformation)\r\n at Unify.Product.Plus.LockerEngine.GenerateLockerPair(ILockerInformation lockerConfiguration)\r\n at Unify.Product.Plus.LockerEngine.LockerConfigurationChanged(Guid lockerId, Action`1 lockerAction)\r\n at Unify.Product.Plus.LockerEngine.<>c__DisplayClass33_0.<UpdateLockerSchemaRow>b__0()\r\n at Unify.Product.Plus.LockerEngine.<>c__DisplayClass49_0.<ConfigurationChanged>b__0()\r\n at Unify.Framework.ExtensionMethods.WaitOnMutex(Mutex mutex, Action work)\r\n at Unify.Product.Plus.LockerEngineAuditingDecorator.UpdateLockerSchemaRow(Guid lockerId, IEntitySchemaFieldDefinitionConfiguration entitySchemaRowConfiguration)\r\n at Unify.Product.Plus.LockerEngineNotifierDecorator.<>c__DisplayClass25_0.<UpdateLockerSchemaRow>b__0()\r\n at Unify.Framework.Notification.NotifierDecoratorBase.Notify(ITaskNotificationFactory notificationFactory, Action action)\r\n at lambda_method(Closure , Object , Object[] )\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClassc.<GetExecutor>b__6(Object instance, Object[] methodParameters)\r\n at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"An item with the same key has already been added.","ExceptionType":"System.ArgumentException","StackTrace":" at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)\r\n at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)\r\n at Unify.Framework.Schema.Schema`6.Add(TKey key, TFieldDef value)"}}; ---> Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).

When I tried to get back to the main locker UI page to select the affected locker and fix my mistake the following error now appears every time:

System.ArgumentException: An item with the same key has already been added.

Could you please review the config and fix it so the locker screen works again?

I have a solution with ~7000 managed AD user accounts.  To ensure any unauthorised changes to those accounts are reverted (continuous compliance) I run regular Baseline Sync operations on the outgoing link.

These Baseline Syncs take approximately 25 minutes to run, and during that time no other link synchonisations run.  This means urgent updates (such as SPOL user suspension functionality) is delayed.

What can I do to ensure fast response for urgent operations, while also having continuous compliance with a reasonable turn-around time (i.e. checked every hour or so - keeping in mind that an import all for my AD users only takes around 50 seconds to run).

The Remove Joines screen shows an error when invoked, in all dev/test UNIFYConnect environments.