Ability to generate hash on connectors for use as keys and relationships

Adam van Vliet 11 years ago updated by anonymous 7 years ago 4

From DEEWR-55, there were difficulties using multiple fields as relationships. These difficulties will be documented in IDB-352.

It could be of benefit to be able to generate a hash of any number of fields and have this included in the schema for the connector. Could be of benefit to allow this to be used as the key of the connector, and to be used for relationships.

Obviously the UI will have to be updated to accomodate, as would the config files and adapter/factory pairs. The schema provider may have to be updated to allow for these hashes to be suggested, depending on how the configuration is done.

The hash must always calculate the same for the same input, differently should the same values be swapped between two of the fields, and be resilient to clashes.

Consider adding method to context that can retrieve a page of data using a MultiKeyValue. This would require IDB-844, because the schema wouldn't be at the data context level. The method could use the entity value table only to find the entity ids, then use the regular method to retrieve the entities using the id. This method could replace both the connector lookup and transformation lookups.


  • for each key
    • retrieve matching entity id's
    • if first key add id's to hashset
    • if next key hashset equals AND of both sets.

Connector imports have been swapped over to the new methods. Next step is to do the same for transformations.

Fixed up the method slightly, it was failing for sets (1, a), (2, b), (1, b) - when selecting the first two the third would also be selected.

Migrated to VSO.