0
Completed

Decouple adapter import from entity transformations

Adam van Vliet 11 years ago updated by anonymous 7 years ago 17

The adapter transformation process should be decoupled from the adapter import.

Allow the adapter to be configured to allow for both modes - default to decoupled mode.

The benefits would be:

  • The appearance of faster imports into the identity management solution.
  • Searches on the adapter space would not have to wait for transformations.
  • Transformations could be performed as needed.

Tasks:

  • Add methods to adapter to perform transform only.
  • Create a job that can be added to the adapter configuration that runs the transformation after:
    • The base connector has performed an import.
    • A specified connector has performed an import.
    • A change is due in the changes register.
    • As per a schedule - regular timings.

IDB-151.png

Adapter methods done.

Next - the tasks and timings.

Resolved.

Reopen if further work required.

Matthew, please confirm.

Reopening.

Current configuration of Identity Broker does not allow reflection to happen. All adapters currently default to the AdapterImportSettings.ReflectOnRelationalConnectorPoll behaviour, with reflection only happening if:

  • The base connector is set to ReflectOnBaseConnectorImport, or
  • The relational connector is set to ReflectOnRelationalConnectorImport

A few changes need to be made:

  • The checks should be checking to see if the relevant flag is selected
  • Default behaviour needs to accommodate for this behaviour
  • The reflection behaviour of adapters needs to be configurable - in larger environments with large numbers of transformations, enabling reflection may not be desirable

Raising priority as this is now blocking the checking of IDB-299

The job on ReflectAdapterOnChangeDueJob runs every 2 minutes and will reflect any adapter changes, but only if the ReflectOnChangeDue flag is present. Hence none of the relevant flags are being set on the adapter on creation.

Possible UI is a set of multiple checkboxes under an "Advanced" screen on the main adapter information page, to add the flags to the final value.

Each reflection setting has been added to the UI, see attached screenshot. This option has introduced a major race condition, however, where it would now be possible for data to be missed due to the fact that the changes register may have been updated before the adapter context, and a delta import would pull the entity and clear the change before the context is updated. See IDB-299.

The race condition for deltas could be addressed here by extending the Changes Available check to account for the reflection job, and not just whether or not items are in the changes table. Full imports are also affected by this condition as they may be run before the adapter context has been updated - similar behaviour exists for v3 if imports are run before change detection processing completes, resulting in the "current state" of the adapter context being retrieved (minus the changes that are still pending on the job).

For CTP, the option to reflect may be set to the CoupledProcess option, and additional addressing of this issue addressed in Beta.

The UI should provide feedback on the reflection process occurring.

The reflection settings have been set to default to CoupledProcess and are hidden on the UI for CTP.

Adam, I believe this is as complete as it can be for CTP. Assigning to you for appropriate action.

Remaining work moved to Beta

Various reflection settings will need to be covered by the regression test document as they become available.

Lowering priority as currently out of scope