1. UNIFYBroker Forum
  2. UNIFYBroker/Plus
  3. Questions
0
Answered

Duplicate Key Calculating target to source look up

Tom Parker 7 years ago in UNIFYBroker/Plus • updated by anonymous 5 years ago • 9

Hi, I'm seeing this error when doing a baseline sync from a locker to AD.


This solution has previously had "An item with the same key has already been added." errors but I'm not sure what this error means.



Image 4424

Unify.Framework.UnifyDataException: Duplicate key calculating target to source id lookup: 138db3b0-4197-4bee-bd1a-010830bebd1d
   at Unify.Product.Plus.DeprovisioningExecutor`2.TargetIdToSourceIdLookupKeyClash(Guid key, Guid value, IConnection original)
   at Unify.Framework.Collections.EnumerableExtensions.ToDictionaryWithKeyClashError[TKey,TValue,TOriginal](IEnumerable`1 originalEnumerable, Func`2 keySelector, Func`2 valueSelector, Action`3 duplicateAction)
   at Unify.Product.Plus.DeprovisioningExecutor`2.Execute(IEnumerable`1 page)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.Plus.AdapterToLockerProvisioner.Execute(Func`3 generateAndMapTarget, IDictionary`2 changesDict)
   at Unify.Product.Plus.LinkSynchronizer`2.JoinAndMap(IEnumerable`1 filterResult, IDictionary`2 changesDict)
   at Unify.Product.Plus.Link.SynchronizeChanges[TSourceEntity,TTargetEntity](IEnumerable`1 changes, IEnumerable`1 syncTasks, Func`1 getTargetContextAccessor, IConnectionsContext connectionContext, ISynchronizationHelper`2 helper, IProvisioner`2 provisioner)
   at Unify.Product.Plus.Link.SynchronizeAdapterChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LinkNotifierDecorator.<>c__DisplayClass42_0.<SynchronizeAdapterChanges>b__0()
   at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
   at Unify.Product.Plus.LinkNotifierDecorator.SynchronizeAdapterChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LinkAuditingDecorator.SynchronizeAdapterChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.AdapterToLockerSynchronizationJob.RunBase()
   at Unify.Product.Plus.SynchronizationJobExecutor.<ThreadAction>d__8.MoveNext()",Normal


Answer

Answer
Curtis Lusmore 6 years ago

This issue is most commonly caused by "broken" connections - connections between locker entities and adapter entities where the adapter entity has since been deleted, and the connection wasn't properly deleted.

You can find such entities with a SQL query like

SELECT *  
FROM [Connections] [c]
LEFT JOIN [Entity] [e] ON [c].[AdapterEntityId] = [e].[EntityId]
WHERE [e].[EntityId] IS NULL
AND [c].[LinkId] = 'your-link-id'

Deleting any connections identified by that with the following query should resolve the issue.

DELETE  
FROM [Connections]
WHERE [ConnectionsKey] IN (
    SELECT [ConnectionsKey]
    FROM [Connections] [c]
    LEFT JOIN [Entity] [e] ON [c].[AdapterEntityId] = [e].[EntityId]
    WHERE [e].[EntityId] IS NULL
    AND [c].[LinkId] = 'your-link-id'
)

Alternatively, clearing the adapter context and re-generating the entities should clear up the connections.

We will need to investigate the cause of the broken connections.

Under review
Curtis Lusmore 7 years ago

Hi Tom,

This error indicates that there are multiple connections registered for the locker entity with id 138db3b0-4197-4bee-bd1a-010830bebd1d. If you recently deleted or otherwise cleaned up the duplicate adapter entities, these connections could be associated with those duplicates and are no longer valid.

Answered
Curtis Lusmore 7 years ago
Tom Parker 7 years ago

Hey so this has been reoccurring pretty constantly the last week. Before I clear out the latest instance is there anything I can do in terms of data gathering to help find a lasting solution?

+1
Curtis Lusmore 7 years ago

I don't think there's anything to capture now that it has happened but if this keeps reoccurring I would definitely suggest adding an Text Audit Writer. It should record details of entities which are updated and provisioned during synchronization.

Tom Parker 7 years ago

Thanks, I'll post more details when the error occurs again.

Huu Tran 6 years ago

Hi Tom, just wonder did you manage to clear the error message and what did you have to do. I am having the exactly same error and struggling to clear it.

Rgds,

Huu


Answer
Curtis Lusmore 6 years ago

This issue is most commonly caused by "broken" connections - connections between locker entities and adapter entities where the adapter entity has since been deleted, and the connection wasn't properly deleted.

You can find such entities with a SQL query like

SELECT *  
FROM [Connections] [c]
LEFT JOIN [Entity] [e] ON [c].[AdapterEntityId] = [e].[EntityId]
WHERE [e].[EntityId] IS NULL
AND [c].[LinkId] = 'your-link-id'

Deleting any connections identified by that with the following query should resolve the issue.

DELETE  
FROM [Connections]
WHERE [ConnectionsKey] IN (
    SELECT [ConnectionsKey]
    FROM [Connections] [c]
    LEFT JOIN [Entity] [e] ON [c].[AdapterEntityId] = [e].[EntityId]
    WHERE [e].[EntityId] IS NULL
    AND [c].[LinkId] = 'your-link-id'
)

Alternatively, clearing the adapter context and re-generating the entities should clear up the connections.

We will need to investigate the cause of the broken connections.

Adrian Corston 5 years ago

Now July 2019 and I am seeing this same problem.  It looks like the underlying root cause (connection not properly deleted) is still present in Broker/Plus v5.3.0.2.

Adrian Corston 5 years ago

I looks like this problem occurs when the connector and locker entities have both been cleared, but the adapter hasn't had pre-calculated entities cleared explicitly via the UI.  Clearing the connector entities appears to remove all the adapter entities, but I think there's still something there that doesn't show up in the UI.  When the next baseline sync happens it tries to recreate those same things (with all the same IDs) in the adapter, but they're already there so it fails with the duplicate key error.  When I clear pre-calculated entities on the adapter the problem doesn't occur.

Customer support service by UserEcho