Ideas
Baseline Sync to revert target system changes takes a long time, even when there are no actual changes
My customer has 9,000 users in AD corresponding to 7,000 in a locker of which approximately 4,000 are joined (the discrepancy being due to terminated HR users who don't have or need to be provisioned with AD accounts).
When I run a Baseline Sync on my AD link, to revert any unauthorised changes that have been made to users in AD (i.e., disable any accounts that have been manually enabled when they shouldn't have) it takes upwards of 30 minutes to run, during which time all other system processing is blocked (since link operations are blocking). This doesn't meet customer expectations that unauthorised AD changes should be reverted within a 15-30 minute timeframe and that the system should always process low-latency operations (like manual suspensions using the Suspended Employees feature) with a delay not exceeding a few minutes.
Some possible ways to solve this problem would be to add a "true-up" operation, or allow parallel sync operations.
Customer support service by UserEcho
Hi Adrian,
Is the performance a problem with systems other than AD? Is the problem able to be replicated in a lower environment?
When you say a "true-up" operation, can you explain this a little more? What would be the difference between a true-up and a baseline sync?
Parallel sync operations are on our roadmap but are something we have to tread lightly about for data consistency purposes, and therefore wouldn't really help your scenario here (running a baseline sync on the AD context wouldn't allow other operations to run on the AD context at the same time, otherwise you risk overriding changes).
Is the performance a problem with systems other than AD?
Yes, I find Baseline Syncs to SharePoint online lists via a PowerShell connector or any other slow export connector are also slow and problematical.
Is the problem able to be replicated in a lower environment?
Yes, all environments show this behaviour.
When you say a "true-up" operation, can you explain this a little more?
A "true-up" operation is where a target field value has been changed and needs to be reverted to the value from the SOT. It is different to a normal change operation where the value change has been initiated in the SOT.
What would be the difference between a true-up and a baseline sync?
A "true-up" would compare the current connector/adapter state to the locker state (via the link mappings) and then only write the changed fields back to the target system, as opposed to a Baseline Sync which appears to re-export all field values to the target regardless of whether or not they have changed.
A "true-up" might also be implemented by identifying incoming changes on target fields during import and using that knowledge to drive the corrective export operations.
Parallel sync operations are on our roadmap but are something we have to tread lightly about for data consistency purposes, and therefore wouldn't really help your scenario here (running a baseline sync on the AD context wouldn't allow other operations to run on the AD context at the same time, otherwise you risk overriding changes).
Understood. Per-entity locking and change timestamping/sequencing would be needed to achieve that. But I figured I'd put it out there for consideration regardless.
Thanks Adrian - that explains it well.
There's not a lot we can do about the speed of the export, that's up to the end system. We do have a backlog item to review the calls that the AD connector is making, as we suspect it's not as efficient as it could be (we've had some issues with extra changes being reported too).
I've converted this to an idea, as we're in active development on the next version of Broker and so it's a perfect time to explore not only the support for a true-up operation, but also the ability for parallel processing. We'll keep you posted on how this progresses.
Two investigative backlog items added for 6.0 development:
- True-Up operation for Plus
- Improvements to, or addition of, parallel processing for critical operations.
Just an update to mention that my customer has raised this issue again - it's a high priority for them.