0
Fixed

Duplicate Error on DN in PowerShell script

Daniel Walters 1 year ago in PowerShell connector updated by Beau Harrison (Senior Software Engineer) 1 year ago 10

I have a PowerShell connector that is pulling two attributes, DN and another one. DN is the key and it's throwing errors saying "An item with the same key has already been added". This isn't possible because DN is unique. Here is the Import All script:

import-module ActiveDirectory

$server = "Dcbr2wdc1"

$searchBase = "ou=Users,ou=dams,dc=internaldmz-dev,dc=local"

$users = get-aduser -Server $server -SearchBase $searchBase -Properties DistinguishedName, msDS-UserPasswordExpiryTimeComputed -Filter {PasswordNeverExpires -eq $false}

foreach($user in $users)

{

$entity=$entities.Create()

$entity["ADDN"] = $user.DistinguishedName

$entity["ComputedPasswordExpiry"] = $user.'msDS-UserPasswordExpiryTimeComputed'

$entity.Commit()

}

and here is the schema

New-Field 'ADDN' 'string' $true $false $true;

New-Field 'ComputedPasswordExpiry' 'string' $false $false $false;

# name type key read-only required

I tried writing the DNs to a file in the script and just got a list of DNs, no obvious duplicate. Attached the log and connector config.

Affected Versions:
Fixed by Version:

Writing the DNs to a file I get 408 entries and running the query external to broker also yields 408 entries. So all of the entries seem to go through $entity.Commit() OK and the error is being thrown after that.

I've manually checked there aren't any duplicate DNs and there aren't as expected.

Adding $users = $users[0] before the foreach loop gives the same duplicate error so even processing one user gives the duplicate error. There are 0 entities in the connector at this point. Also, this connector was working fine and then it stopped working and started throwing the error. Could this be DB corruption?

This is using Broker v5.2.0 RC2

This error is also appearing on other unrelated connectors that have not been altered since they were working. It seems it's not specific to this connector and may be something more global. Tried restarting the service to no effect.

There's a duplicate in the collection key table. Same text, same case. Will it fix it if I delete it?

Under review

Hi Daniel, you're correct. This is caused by the duplicate in that table. You can fix this issue by doing the following:

  1. Of the duplicates in the CollectionKey table, note the CollectionKeyId values for each. Pick one to keep.
  2. In the EntityValue, check the CollectionKeyId field for any instances of the duplicate captions id. Update these with the id of the caption you decided to keep.
  3. Repeat the above for the EntityValueOrigin table with the CollectionKeyId and SourceCollectionKeyId fields.
  4. Delete the duplicate entries in the CollectionKey table.

Thanks, this fixed it.