High memory consumption when change detection affects large datasets
Matthew Clark 11 years ago • updated by anonymous 8 years ago • 4
QDET-194 shows that if a relational change affects a large data set, memory consumption can grow quite high and result in OutOfMemory exceptions. Changes should be made to the transformation contributions such that they do not load all potential matches immediately into memory.
Customer support service by UserEcho
A potential fix has been pushed to the v3.0.7 branch for the RelationalTransformationContribution. This will be uploaded to
QDET-194for confirmation of changes.
For v4.0, these fixes will need to be pulled over and added to the other relational contribution types as well, such as the multi source relational contribution.
Following feedback on
QDET-194, update the other transformation contribution types to use this paging algorithm.
Migrated fixes from v3.0. Fixed up other contributions.
Changes are present in v4.0, but we lack sufficient data to completely replicate and directly confirm this for v4.0. Issue closed.