Investigate delta add behaviour with new adapter pre-populate functionality
Adam van Vliet 11 years ago • updated by anonymous 7 years ago • 17
Investigate delta add behaviour with new adapter pre-populate functionality.
CSODBB-113, it was highlighted that delta adds are a result of the entity not being present in the adapter context. (See Adapter.ProcessAttributeChangePage).
Please investigate the impact of the new processing model of pre-populating the adapter context, and come up with the possible options to work around this issue.
Customer support service by UserEcho
Check to see if the changes register can be used during change detection, keeping in mind that the delta file shows whether the item in an add or an update.
Raising priority - has the potential to introduce major changes
Currently, delta imports trigger direct changes to the adapter context, ignorant of the reflect settings. The current work on reflection therefore does not yet have any bearing on adapter delta imports, only full imports. However, if the impact of reflection is to be realized on deltas, the reflection process itself will need some sort of handle on the changes register, and some mechanism for establishing attribute level changes.
Deltas will require changes having been calculated at the same time that the adapter context is updated, which will now need to be on the reflection call of the adapter. All the logic already exists for each of the modification types (deletes, adds, updates, mod DN) in the attribute changes method, which is currently used for deltas. Deltas do not actually use the adapter context, nor do they actually need to, as it is the value of the changes that are important, and these are not persisted in the adapter context. What we are looking at for this issue is actually "changes reflection", where the changes context should be updated in a similar way to the adapter context during reflection, as the original state of the context is not persisted. There are a few options for how this can be done:
IDB-151for thoughts on the impacting race condition.
I think it would be useful to ensure that you layout what can be done under the following headings, to ensure that each criteria is met:
The only option where the above would have different storage is under the third option above. For the changes register hash option, it would be stored directly to the change in the changes table. For the third option:
I think the second option of storing the changes in the changes register may be the faster option and reduce database traffic around this operation, plus it takes care of the persistence problems.
After discussion with Adam, the following approach will be taken:
According to RFC-2849, the removal of a single value from a multivalue field requires the previous value. Alternatively, the entire multivalue could be replaced by the "replace" statement in LDIF, eg.
telephonenumber: +1 408 555 1234
telephonenumber: +1 408 555 5678
facsimiletelephonenumber: +1 408 555 9876
The case of group management highlights that the second approach is actually much more efficient in the long run (ie. not having to spit out 199 groups if 1 from 200 is deleted). However, this case means that we need to preserve the old value of the field following the change. Alternatives:
The first option would allow us to minimize the row count in the changes table, where the second would remove the need to access the entity value table at all. My preference would be the first as we will already be accessing the value to see the entity, and we will not need additional rows in the changes table.
Update: The Entity Value table should contain an Add/Update/Delete flag on items, where by default items marked as delete are ignored by the main adapter context. Deltas would see that an entity is an "update" change type in the Changes table, access an adapter context where all change types were present, and spit out all th eadd, update and delete values to the LDIF file.
The database, and relevant upgrade scripts, will need to be updated when this behaviour changes.
Adam, I believe this is as complete as it can be for CTP. Assigning to you for appropriate action.
Note: The save behaviour should be looked at once we fix the pre-populate functionality.
Delta imports do not appear to be working currently. Things do not seem to be properly marked as pending changes according to the adapter statistics, but a full import into FIM will bring in the change.
The repository objects when processing connector pages for change detection, and the change processor job was not running on the schedule. Delta imports are now working again for Identity Broker, meaning the work for Alpha is now completed.
Reducing priority of remaining work, and moving to Beta.
Work done in