- UNIFYBroker Forum
-
Bugs
IDB 5.1 returning duplicate objects that only exist once in the Adapter/Connector
A full import ran overnight on an MA in MIM from Identity Broker returned 66 duplicate objects, on spot checking a number of the objects they only exist once in the Adapter and Connector. The EMPLID attribute (used as the CN attribute in the DN) is set as the Key on the Connector.
A subsequent full import (currently running) is exhibiting the same behaviour.
Answer
Is there any pattern to the entity counts, i.e. does MIM receive the same number entities as are in the adapter? Are the same duplicate entities appearing in both of the bad imports from the screenshots or are they different each import?
Will wait for the current running full import to complete before doing further analysis, however it appears that the duplicates reported on the current running import were not flagged as duplicates on the earlier run.
also file containing the duplicate DNs from each run, those in the 2nd run (smaller number) do not appear to have been flagged as duplicates in the first run and on checking a few of those users they havent been updated in the source system within the last week.
Confirmed that the entity counts are similar but not the same (connector imports are running on schedule every 15mins), so increases slightly every time. Also confirmed that there are no duplicate entries in the source database table which uses EMPLID as the primary key, which is the same key used within the Connector.
From the logs it looks like there's a lot going on at the same time. To rule out (or confirm suspicion), are you able to schedule a full import during a time that no other operations are running, e.g. overnight. I'm assuming there are already periods of inactivity scheduled for DB maintenance, etc, so after those tasks have finished would be ideal.
This wouldn't be the first time to see such numbers, but it is the first time to see changes to other connectors cause these duplicates. I'm not sure whether that's because others haven't raised it with us, or if it's because they schedule full imports to run on their own.
Identity Broker logs
| Timestamp | Count |
| 2016-10-27 13:54:45 | 545111 |
| 2016-10-28 01:14:49 | 545143 |
FIM runs
| Timestamp | Count | Duplicates |
| 2016-10-27 22:00:02 | 545045 total (543303 unchanged, 261 add, 1481 update) | 66 duplicates (=545111 total) |
| 2016-10-28 08:37:10 (unfinished) | (249838 unchanged, 32 add, 13 update) | 2 duplicates |
| 2016-10-28 08:37:10 (finished) | 545122 total (544978 unchanged, 73 add, 71 update) | 21 duplicates |
Will schedule a run tonight with everything else disabled (this is similar to the full import from overnight last night) for a full import on the broker connector followed by a full import on the management agent into FIM.
Can confirm that after running the full imports over the weekend in isolation that the issues appear to have resolved themselves.
Am going to tweak the Event Broker scheduling to try and prevent the scenario from occurring.
Customer support service by UserEcho
Can confirm that after running the full imports over the weekend in isolation that the issues appear to have resolved themselves.
Am going to tweak the Event Broker scheduling to try and prevent the scenario from occurring.