Excessive MultiRelationalTransformationContribution execution time

Tony Sheehy 12 years ago updated by anonymous 9 years ago 8

The MultiRelationalTransformationContribution can take a very long time to evaluate, as the current method re-evaluates the repository for each entity in the page.

This is not a regression - I believe matt has some ideas about the correct way to do change detection with regards to membership list transformations.

Assigned to Matthew for comment (for Tony's comment).

My thoughts here was that the contribution actually needed to do more than it is - it essentially would need to recalculate the memberships to see which entities should be flagged as having changed - see IDB-750. Alternatively, you could flag every entity with a membership list as having changed for a relational connector change. The adapter import changes process already recalculates the memberships, and the import will only pull in items that are actually different from their last attempt.

This contribution had been recently updated to reflect support for any ValueType/Multivalued ValueType pair in the Membershiplist multivalue transformation. The operation used to replace the GetValue<Multivalue...> call was GetMultiValuesOrDefault, which returns an enumerable, where as the GetValue<MultiValue...> call would return a collection.

This enumerable was then used on every entity in the left entity repository, which meant it was re-evaluated for each entity. This method has been updated to evaluate the result before returning it.

Reassigned for confirmation of completion.

Testing with the file on IDB-739, a Membership Multivalue transformation with 10000 entities joining on a multivalue integer field ran successfully, and completed connector and adapter imports in a timely manner. Issue closed.

reopened for time.

closed following time.