When updating the Name configuration of a Chris21 connector, the following error appears:

An error has occurred: Controller Unify.Product.IdentityBroker.Chris21ConnectorController errored on action UpdateConnector with the following reason: Model state invalid for: connectorInformation.Extended.EaiFlagDeleteAttribute: The EaiFlagDeleteAttribute field is required.

This is the latest version of everything (UNIFYConnect 'demo.local' environment).

Given this config:

and target connector data like this:

I am seeing reflection fail (no entities processed) with this error logged:

20220121,07:27:22,UNIFYBroker,Adapter,Error,"Request to reflect change entities of the adapter.
Request to reflect change entities of the SPOL Azure-mastered AD Groups (928e6c08-48d0-48da-8330-08ce0df4ef55) adapter errored with message: One or more errors occurred.. Duration: 00:00:01.4730403
Error details:
System.AggregateException: One or more errors occurred. ---> Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
   at System.Linq.Parallel.AsynchronousChannelMergeEnumerator`1.MoveNextSlowPath()
   at System.Linq.Parallel.QueryOpeningEnumerator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at Unify.Product.IdentityBroker.Adapter.ReflectChangePage(IChangesRegisterKey[] changedPage, IEntityPartitionContext connectorContext, IAdapterEntityPartitionUpdatableContext adapterContext, ITransformation transformation, DuplicateDnDetector duplicateDnDetector)
   at Unify.Product.IdentityBroker.Adapter.ReflectChangesInner()
   at Unify.Product.IdentityBroker.Adapter.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterAuditingDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterNotifierDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.ReflectAdapterOnChangeDueJob.RunOnAdapter(IOperationalAdapter adapter)
---> (Inner Exception #0) Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---
---> (Inner Exception #1) Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---
",Normal

In terms of correct functionality, since onPremisesUserPrincipalName is the Reference field I would expect a join to a target connector entity where the onPremisesUserPrincipalName value is NULL to be ignored (i.e. a NULL onPremisesUserPrincipalName value wouldn't be populated into MemberADUPNs).

Note: this ticket might be related to https://voice.unifysolutions.net/en/communities/6/topics/4082-foreign-multivalued-group-transformation-reports-value-cannot-be-null-for-multivalued-attribute but that one is a different transform and looks like a subtly different manifestation, so maybe not.

After an AD record rename fails (in my case due to an invalid OU) the entire batch of all other AD updates is aborted:

20211201,02:01:06,UNIFYBroker,Connector,Warning,"Update entities to connector failed.
Update entities [Count:1336] to connector AD User failed with reason A task faulted. See inner exception for details.. Duration: 00:00:00.0903295
Error details:
System.Exception: A task faulted. See inner exception for details. ---> System.Exception: Received error code Other for item with dn CN=Customer User,OU=Location Name,OU=Customer Locations,OU=Users,OU=Customer Name,DC=customer,DC=com. Message: 00002089: UpdErr: DSID-031B0E6F, problem 5012 (DIR_ERROR), data 2
---> System.DirectoryServices.Protocols.DirectoryOperationException: An unknown error occurred.

Server stack trace:
at System.DirectoryServices.Protocols.LdapConnection.ConstructResponse(Int32 messageId, LdapOperation operation, ResultAll resultType, TimeSpan requestTimeOut, Boolean exceptionOnTimeOut)
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]:
at System.DirectoryServices.Protocols.LdapConnection.EndSendRequest(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connectors.AD.ADAgent.d__24`1.MoveNext()
--- End of inner exception stack trace ---
at Unify.Connectors.AD.ADAgent.ErrorCheckResponse(String dn, DirectoryResponse response, String operationName, Exception originalException)
at Unify.Connectors.AD.ADAgent.d__24`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Unify.Connectors.AD.ADAgent.d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Unify.Connectors.AD.ADConnector.d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Product.IdentityBroker.ConnectorToUpdatingAsyncConnectorBridge.d__8.MoveNext()
--- End of inner exception stack trace ---
at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass4_0.b__0(Task t)
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Product.IdentityBroker.EventNotifierUpdatingAsyncConnectorDecorator.d__3.MoveNext()",Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 143471 (b9e8dea3-2aa0-4edc-bc8f-b40ab0a95250) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145095 (a14daa52-6c57-4fd3-aa8a-f73be5d47301) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 142960 (297f0bbf-ec3a-46b8-a355-90cb4520af4b) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145766 (68f36cfa-0a5b-4211-8150-df9196331bbc) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145919 (1f57b3db-c2b2-4bd5-8d08-95083976e8f3) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 1883 (5b8886d7-ce77-4714-b634-e4175554c660) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145395 (602ca35a-d708-40e4-99a2-15b666810a8a) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144904 (95260bbc-9344-49e4-994d-8ca1fd1a3442) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144060 (f11080cc-95dc-4375-9f09-65b8f8c55227) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145692 (f7883a73-ab23-442d-b388-6b0006288506) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144767 (6603418a-e7c2-4b33-951b-3eb4417e1ac5) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 143176 (7fd6ee8f-43f2-42e8-a7a0-ea40cd1a0e56) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145666 (de6101ca-d184-4bf0-88b3-eea6c48edba7) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145112 (33b23a3f-f82c-46c0-bcaf-278c1a2e3a39) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 141673 (21a3cd55-616c-4559-8385-a4b407209d68) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145668 (e34a0f5e-18cc-40d0-bd44-027adbd49e1f) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal

(etc etc)

In this instance after the one AD record update fails all the other updates should still be attempted, since the error on this one record has no relevance to the other record updates which are likely to succeed.

This is impacting my customer since no changes to AD are currently being synchronised.  As a workaround I will correct the OU on the impacted user.

In my customer's UAT and PROD environments an adapter is retaining old entities when a connector import all operation completes with less entities (i.e. some entities have been deleted). This happens every time the import all operation is run.  It seems some updated/new entities may not be being processed either, but that's not as clear.  What is clear is that the adapter entity count increases to a number greater than the connector's entity after the import all.  There are no corresponding errors in the log.  Generate changes does not fix the problem, but deleting the adapter entities and then running generate changes does.

What information would you like to help debug this?

Version is v5.3.3 Revision #0
The connector is a PowerShell one

When the scheduler is disabled it's not possible to run Import or Sync operations.  During solution development, initial data load and remediation activities it is often necessary to perform manual import and sync operations in specific orders while all scheduled other activity is suspended.  To achieve this right now all connector and link schedules must be individually disabled, which can be time consuming for non-simple solutions, and comes with an increased risk of schedules being left disabled inadvertently.

Provide some way to suspended scheduled operations, while allowing manual operations to continue to work.

I suggest adding a "Manual Operations Only" flag (or similar) to the dashboard.

In my customer's TEST environment they are seeing AD sAMAccountName being updated for existing users where the join criteria are met.  The only place where the sAMAccountName is set is in the outgoing pre-provisioning task, which should not be called for existing joined entities.

I have also seen a second confirmation that the task is being called for pre-existing users: the "AD Account Creation" notifications being repeatedly sent for a large number of entities every time a Baseline Sync is performed.  The only place where that notification is sent is in that same outgoing pre-provisioning task.

When an adapter entity is deleted any persistent joins remain and block subsequent joins to remediated data.

Steps to reproduce
1. Configure a Link with outbound provisioning and persistent joins configured
2. Import an adapter entity that is intended to join a locker entity, but which is missing its join criteria field value
3. The link will provision a "duplicate" record (thereby creating an internal "Join" for the newly provisioned adapter entity)
4. To clean up the duplicate, delete the SoT for the "duplicate" adapter record and update the "intended" record to have a correct join criteria field value
5. Attempt to join the locker to the newly corrected adapter entity - it fails and re-provisions the "duplicate" record again

In my UNIFYConnect config the MOBILE_PHONE value isn't being written back to Aurion successfully.

The relevant connector config is:

In the customer's Aurion report the mobile phone field is called 'Contact_Mobile_Phone' so a mapping is necessary.  Import is working correctly, but the field is not updated in Aurion on export.

Here is the rest of the relevant config:

Adapter Transform (to remove underscores from the field name)

Link Mapping

Locker Entity

After a connector Import the Contact_Mobile_Phone field is set to the old value from Aurion as expected.  After a Baseline Sync on the link the value in the connector is changed to '0418 999 999' temporarily, but then reset back to the old value from Aurion after the next connector Import, showing that the value has not been updated in Aurion.

What is causing the writeback to fail and how do I get it to work?

This error is being written to the UNIFYBroker logs in multiple UNIFYConnect environments.  I deleted all transforms that referred to those fields, removed references to the fields from links, and then recreated them all, but it didn't stop the error from appearing.

In my customer's TEST I am seeing this Aurion error when a Baseline Sync runs, for 23 of ~500 entities:

20210803,01:01:15,UNIFYBroker,EntitySaver,Error,The entity mbishop (1100a4a9-3f58-4f13-9c39-480b36abbf41) for the adapter Aurion Security User (c5460bd3-0167-4290-a2a0-180f8632a474) failed to update for the following reasons: Aurion API error -1: User is already linked to an Employee and cannot be changed by this process,Normal

Looking at the data for this specific case, it appears the only update that needs to be sent to Aurion is Name (changing from "Matthew BISHOP" to "Matthew Bishop"). Other fields appear to be unchanged.

What do you suggest I do to debug the root cause?