Identity Broker export performance issue for PowerShell connector

Bob Bradley 4 years ago in PowerShell connector • updated by anonymous 3 years ago 2

Export performance is likely to be a major bottleneck for Origin during the "initial load" sync process where FIM is writing back network account and email address to SuccessFactors (SAP HR). Current performance metrics from DEV are not a great guide due to the limited number of employees loaded into FIM for that environment (<50), however the last sizeable batch was 39 user updates in 00:09:50 (a rate of 1 every 15 seconds!!!). When it comes to a full set we are talking upwards of 33K users requiring updates - and at the above rate we will be looking at 8250 minutes, or 137.5 hours, or 5.7 days.

In an attempt to head this problem off in advance of it coming to the attention of the testers, I am thinking that we may have to rethink the way we are applying updates for the initial load.

See linked issue for idea as to how the PS connector architecture might be improved for exports in future.

Affected Versions:
Fixed by Version:

Hi Bob Bradley,

Anything added to the PowerShell connector would be to v5.x, as it would be pointless trying optimise in v4.1 where it's only able to export one at a time anyway.

The batch feature in v5.0 should help. Say you are exporting 10K items in 1K batches, that would mean you would be doing the connection setup and other overheads 10 times instead of 10K. That’s why I’d be hesitant to look at PRODUCT-380 (PowerShell idea to have start/export/end sections) without first seeing whether this is still going to be an issue.

Adam van Vliet - without getting a chance to test the 5.* feature on this Origin project, I will just take it from you that the new batch feature is effectively what I am asking for here. Let's just assume it is and that we have an issue we just have to live with for 4.* (for which I was trying to describe such a feature).