Azure's SCIM implementation when creating a user who has a manager follows this approach:

1. Create the new SCIM user (without the manager reference)
2. Make sure the new user has been created
3. Update the new user's manager field

It seems the reason for this approach is because Azure wants to create all the users first, before it tries to add references to them via the manager field (which makes sense from a referential integrity perspective).

However, when using the UNIFYBroker SCIM gateway, step 2 is failing and Azure can't find the newly created user.  Azure first tries to get the new user object by ID, and then by username lookup, but neither works.  See the attached packet trace "SCIM User with manager creation.pcapng" for details.  The connector/adapter entity created has the same entity ID that was returned to Azure (9a9978e0-6179-42b5-8218-2173d7b6c0e5 in the packet trace) and the username field set correctly (adrian.corston@unifysolutions.net).

A later attempt by Azure to look up that entity by username (package trace "SCIM User lookup.pcapng") is successful (that lookup was followed by a PATCH which failed, but that is due to a different issue so please ignore that part of the trace).

SCIM User with manager creation.pcapng
SCIM User lookup.pcapng

This github ticket has information about the Azure SCIM pattern for setting manager: https://github.com/MicrosoftDocs/azure-docs/issues/11784 (comment from asmalser-msft)

Hi guys,

I am using the SCIM gateway in v5.3.2 and the department and manager attributes don't appear to work (i.e. the values sent by SCIM are not populated into the UNIFYBroker adapter fields).

Here's my configuration in Azure:

It seems possible that the problem might be related to the fact those two attributes are extension attributes and not part of the SCIM core user schema: https://datatracker.ietf.org/doc/html/rfc7643#section-4.1

As a separate request could you please add support for the givenName and familyName core user attributes?

I am setting up a SCIM gateway in UNIFYBroker, and it is returning a 401 (Unauthorized) error.

I have checked the Bearer token in the request header matches the Audience configuration in UNIFYBroker.

Do you have any suggestions what might be going wrong?

Obviously once this is working I will change the secret token shown above.

My customer has two adapters contributing data to the same locker field via two links.  I have set a priority sequence for the field, but when I run a Baseline sync on the link with the lower priority mapping the field in the locker is updated with the (wrong) value from that lower priority source. Running a Baseline Sync on the link with the higher priority mapping set the locker field back to the (right) value from the higher priority source.  The most recent Baseline sync to run always wins, regardless of the priority setting.

When updating the Name configuration of a Chris21 connector, the following error appears:

An error has occurred: Controller Unify.Product.IdentityBroker.Chris21ConnectorController errored on action UpdateConnector with the following reason: Model state invalid for: connectorInformation.Extended.EaiFlagDeleteAttribute: The EaiFlagDeleteAttribute field is required.

This is the latest version of everything (UNIFYConnect 'demo.local' environment).

Given this config:

and target connector data like this:

I am seeing reflection fail (no entities processed) with this error logged:

20220121,07:27:22,UNIFYBroker,Adapter,Error,"Request to reflect change entities of the adapter.
Request to reflect change entities of the SPOL Azure-mastered AD Groups (928e6c08-48d0-48da-8330-08ce0df4ef55) adapter errored with message: One or more errors occurred.. Duration: 00:00:01.4730403
Error details:
System.AggregateException: One or more errors occurred. ---> Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
   at System.Linq.Parallel.AsynchronousChannelMergeEnumerator`1.MoveNextSlowPath()
   at System.Linq.Parallel.QueryOpeningEnumerator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at Unify.Product.IdentityBroker.Adapter.ReflectChangePage(IChangesRegisterKey[] changedPage, IEntityPartitionContext connectorContext, IAdapterEntityPartitionUpdatableContext adapterContext, ITransformation transformation, DuplicateDnDetector duplicateDnDetector)
   at Unify.Product.IdentityBroker.Adapter.ReflectChangesInner()
   at Unify.Product.IdentityBroker.Adapter.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterAuditingDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterNotifierDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.ReflectAdapterOnChangeDueJob.RunOnAdapter(IOperationalAdapter adapter)
---> (Inner Exception #0) Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---
---> (Inner Exception #1) Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the onPremisesUserPrincipalName field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.RelationshipEntityProcessor.d__1.MoveNext()
   at System.Linq.Enumerable.d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.SimpleValueMultivalueGroupTransformation.TransformItem(IEntity leftSideEntity, ILookup`2 rightSideLookup, IEntitySchemaFieldDefinition primaryKeyField)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---
",Normal

In terms of correct functionality, since onPremisesUserPrincipalName is the Reference field I would expect a join to a target connector entity where the onPremisesUserPrincipalName value is NULL to be ignored (i.e. a NULL onPremisesUserPrincipalName value wouldn't be populated into MemberADUPNs).

Note: this ticket might be related to https://voice.unifysolutions.net/en/communities/6/topics/4082-foreign-multivalued-group-transformation-reports-value-cannot-be-null-for-multivalued-attribute but that one is a different transform and looks like a subtly different manifestation, so maybe not.

After an AD record rename fails (in my case due to an invalid OU) the entire batch of all other AD updates is aborted:

20211201,02:01:06,UNIFYBroker,Connector,Warning,"Update entities to connector failed.
Update entities [Count:1336] to connector AD User failed with reason A task faulted. See inner exception for details.. Duration: 00:00:00.0903295
Error details:
System.Exception: A task faulted. See inner exception for details. ---> System.Exception: Received error code Other for item with dn CN=Customer User,OU=Location Name,OU=Customer Locations,OU=Users,OU=Customer Name,DC=customer,DC=com. Message: 00002089: UpdErr: DSID-031B0E6F, problem 5012 (DIR_ERROR), data 2
---> System.DirectoryServices.Protocols.DirectoryOperationException: An unknown error occurred.

Server stack trace:
at System.DirectoryServices.Protocols.LdapConnection.ConstructResponse(Int32 messageId, LdapOperation operation, ResultAll resultType, TimeSpan requestTimeOut, Boolean exceptionOnTimeOut)
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]:
at System.DirectoryServices.Protocols.LdapConnection.EndSendRequest(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Connectors.AD.ADAgent.d__24`1.MoveNext()
--- End of inner exception stack trace ---
at Unify.Connectors.AD.ADAgent.ErrorCheckResponse(String dn, DirectoryResponse response, String operationName, Exception originalException)
at Unify.Connectors.AD.ADAgent.d__24`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Unify.Connectors.AD.ADAgent.d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Unify.Connectors.AD.ADConnector.d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Product.IdentityBroker.ConnectorToUpdatingAsyncConnectorBridge.d__8.MoveNext()
--- End of inner exception stack trace ---
at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass4_0.b__0(Task t)
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Product.IdentityBroker.EventNotifierUpdatingAsyncConnectorDecorator.d__3.MoveNext()",Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 143471 (b9e8dea3-2aa0-4edc-bc8f-b40ab0a95250) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145095 (a14daa52-6c57-4fd3-aa8a-f73be5d47301) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 142960 (297f0bbf-ec3a-46b8-a355-90cb4520af4b) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145766 (68f36cfa-0a5b-4211-8150-df9196331bbc) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145919 (1f57b3db-c2b2-4bd5-8d08-95083976e8f3) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 1883 (5b8886d7-ce77-4714-b634-e4175554c660) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145395 (602ca35a-d708-40e4-99a2-15b666810a8a) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144904 (95260bbc-9344-49e4-994d-8ca1fd1a3442) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144060 (f11080cc-95dc-4375-9f09-65b8f8c55227) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145692 (f7883a73-ab23-442d-b388-6b0006288506) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 144767 (6603418a-e7c2-4b33-951b-3eb4417e1ac5) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 143176 (7fd6ee8f-43f2-42e8-a7a0-ea40cd1a0e56) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145666 (de6101ca-d184-4bf0-88b3-eea6c48edba7) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145112 (33b23a3f-f82c-46c0-bcaf-278c1a2e3a39) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 141673 (21a3cd55-616c-4559-8385-a4b407209d68) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal
20211201,02:01:10,UNIFYBroker,EntitySaver,Error,The entity 145668 (e34a0f5e-18cc-40d0-bd44-027adbd49e1f) for the adapter AD User (c80b76c8-40e9-4e4e-a7ff-00c4cc5b919f) failed to update for the following reasons: A task faulted. See inner exception for details.,Normal

(etc etc)

In this instance after the one AD record update fails all the other updates should still be attempted, since the error on this one record has no relevance to the other record updates which are likely to succeed.

This is impacting my customer since no changes to AD are currently being synchronised.  As a workaround I will correct the OU on the impacted user.

In my customer's UAT and PROD environments an adapter is retaining old entities when a connector import all operation completes with less entities (i.e. some entities have been deleted). This happens every time the import all operation is run.  It seems some updated/new entities may not be being processed either, but that's not as clear.  What is clear is that the adapter entity count increases to a number greater than the connector's entity after the import all.  There are no corresponding errors in the log.  Generate changes does not fix the problem, but deleting the adapter entities and then running generate changes does.

What information would you like to help debug this?

Version is v5.3.3 Revision #0
The connector is a PowerShell one

When the scheduler is disabled it's not possible to run Import or Sync operations.  During solution development, initial data load and remediation activities it is often necessary to perform manual import and sync operations in specific orders while all scheduled other activity is suspended.  To achieve this right now all connector and link schedules must be individually disabled, which can be time consuming for non-simple solutions, and comes with an increased risk of schedules being left disabled inadvertently.

Provide some way to suspended scheduled operations, while allowing manual operations to continue to work.

I suggest adding a "Manual Operations Only" flag (or similar) to the dashboard.

In my customer's TEST environment they are seeing AD sAMAccountName being updated for existing users where the join criteria are met.  The only place where the sAMAccountName is set is in the outgoing pre-provisioning task, which should not be called for existing joined entities.

I have also seen a second confirmation that the task is being called for pre-existing users: the "AD Account Creation" notifications being repeatedly sent for a large number of entities every time a Baseline Sync is performed.  The only place where that notification is sent is in that same outgoing pre-provisioning task.