Feature request: Identity Broker 5.2 object filtering facility

For reference, the aforementioned powershell script is reproduced below:

# BrokerLoopback.psm1
#
# Written by Adrian Corston, UNIFY Solutions
#
# This module simplifies the creation of a powershell connector that extracts a filtered subset
# of objects from a Broker Adapter via LDAP.
#
# Example Broker Powershell Connector code to use this module:
#
# import-module E:\UNIFY\Adapters\BrokerLoopback
# foreach ($obj in Get-BrokerObjects `
#                  -adapterOU "AllOrganisationalUnits" `
#                  -attributes @( "UnitsID", "UnitsAbbreviation", "UnitsName") `
#                  -brokerUsername "MIM" -brokerPassword "xxx" `
#                  -objectFilter "(UnitsLevelId=LV05)")
# {
#     $entity = $entities.Create()
#     $entity['BusinessUnitCode'] = $obj.UnitsID
#     $entity['BusinessUnitName'] = $obj.UnitsName
#     $entity['BusinessUnitAbbreviation'] = $obj.UnitsAbbreviation
#     $entity.Commit()
# }
function Get-BrokerObjects
{
    param(
        [string] $adapterOU,
        [array] $attributes,
        [string] $objectFilter = "(objectClass=*)",
        [string] $brokerURL = "localhost",
        [string] $brokerUsername,
        [string] $brokerPassword,
        [bool] $secureConnection = $false
    )
    # LDAP connection object
    $null = [System.Reflection.Assembly]::LoadWithPartialName("System.DirectoryServices.Protocols")
    $broker = New-Object System.DirectoryServices.Protocols.LdapConnection $brokerURL
    $broker.AuthType = [System.DirectoryServices.Protocols.AuthType]::Basic
    $creds = New-Object "System.Net.NetworkCredential" -ArgumentList $brokerUsername, (ConvertTo-SecureString -String $brokerPassword -AsPlainText -Force)
    # Secure?
    if ($secureConnection)
    {
        $broker.SessionOptions.SecureSocketLayer = $true
        $broker.SessionOptions.ProtocolVersion = 3
    }
    # Bind to the LDAP server
    try
    {
        $ErrorActionPreference = 'Stop'
        $broker.Bind($creds)
        $ErrorActionPreference = 'Continue'
    }
    catch
    {
        throw ("Error binding to ldap: {0}" -f $_.Exception.Message)
    }
    # Retrieve objects
    try
    {
        $baseDN = "OU=$adapterOU,DC=IdentityBroker"
        $scope = [System.DirectoryServices.Protocols.SearchScope]::Subtree
        $query = New-Object System.DirectoryServices.Protocols.SearchRequest -ArgumentList $baseDN, $objectFilter, $scope, $null
        if ($attributes -ne $null)
        {
            $lowerAttributes = @()
            for ($i = 0; $i -lt $attributes.Count; $i++)
            {
                $attributes[$i] = $attributes[$i].ToLower();
            }
            $query.Attributes.AddRange($lowerAttributes)
        }
        $errorActionPreference = 'Stop'
        $response = $broker.SendRequest($query) 
        $errorActionPreference = 'Continue'
    }
    catch
    {
        write-host $_.Exception.Message
        throw ("Request failed: {0}" -f $_.Exception.Message)
    }
    if ($attributes -eq $null)
    {
        if ($response.Entries.Count -gt 0)
        {
            $attributes = $response.Entries[0].Attributes.AttributeNames
        }
    }
    $objects = @()
    foreach ($entry in $response.Entries)
    {
        $obj = @{}
        foreach ($a in $attributes)
        {
            $lowerA = $a.ToLower()
            if ($entry.Attributes.Contains($lowerA))
            {
                if ($entry.Attributes.$lowerA -ne $null)
                {
                    $obj.$a = $entry.Attributes.$lowerA.Item(0)   # weird syntax, sorry about that
                }
            }
        }
        $objects += $obj
    }
    return $objects
}
Export-ModuleMember -function Get-BrokerObjects

Hi Adrian,

Filtering entities is generally considered to be the concern of the connector. Which connector are you dealing with?

A custom REST API connector.  At this time he is not able to add filtering functionality.

It seems to me that it would be more efficient and sensible to add generic object filtering in the core Identity Broker product, rather than expecting every connector to implement it's own object filtering time and time again.

The reason filtering is generally considered a concern of the connector is that the connector is usually able to perform the filtering via targeted imports, pulling in only the required entities and reducing the size and duration of the import. Downstream identity management systems also typically have filtering capabilities, so we haven't yet found compelling business value in adding this to the product.

This feature is on our backlog, so I've added a reference to this ticket there so it can be prioritised appropriately.

Adrian, could you please supply a data model for what you're trying to achieve? There may be a more suitable set of transformations that will achieve it.

Hi Adam,

We get back a "jagged" (apparently that is the technical term for it) heirarchy of organisational unit objects as a flat data set where each node in the heirarchy has an ID and a reference to it's parent node via ID.  Each node has a level (e.g. "Business Unit", "Program", "Group", "Team").

Different parts of the tree have different parent level structures.  For example, one part of the heirarchy might be: Business Unit X -> Program Y -> Team Z, and another might be Business Unit X -> Group A -> Team B.  For any node, however, the path to the root of the heirarchy is fixed and consistent.
The heirarchy structure has some other complexities.  Firstly, the same ID may be used by different nodes at different levels.  So for example, Business Unit A -> Program B -> Team B (i.e. Program B and Team B use the same node ID).  Secondly, sometimes the parent of an object of a given type may be another object of the same type, e.g. Business Unit A -> Program B -> Program C -> Team D.
It's about as chaotic a heirarchy tree as you could ever hope for...

The key business requirement for this data is to automatically provision AD groups (with attributes that differ by object type, based on a set of rules) with criteria based membership.

Example:

For each Team, provision a AD group with these attributes:
* DisplayName = <BusinessUnitName> <TeamName>-DG
* Type = Distribution
* Members = All users matching the Team's <TeamCode>
* etc etc.

The difficulty here is that the BusinessUnitName that is required for the DisplayName comes from a parent up the "jagged" heirarchy above the Team object.  So for every Team (etc) object we have to walk the organisational object heirarchy (with all it's parent ID linking complexities) until we find the corresponding BusinessUnit object, then copy that object's BusinessUnitName attribute into our Team object.

This complex manipulation is really not something that Broker's joins are capable of doing :-)

Note: This ticket ONLY relates to the question of creating a container of object of a given type (level).  The bigger issue (of walking the organisational structure to extract the Business Unit Name) is not what I'm asking about - I have already coded a solution for that.

Hi Adrian,

This sounds extremely similar to the example underneath "Transformation Combinations" on Common Transformation Scenarios. Please take a look there to see if any techniques described might be useful to your data model.

Hi Curtis,

Please re-read my comment - the simple approach documented on that page isn't able to handle the complex details of the requirements.

Could you please clarify a few details that were glossed over a little in your comment?

1. The only level that must be exposed is Team (you are provisioning AD groups for each Team)?
2. Is there a limit to the depth of the hierarchy, or is it unbounded? i.e. Can you say definitively that the length of the path from a Team to its Business Unit will never be more than N, for any given value of N?
3. For each Team, the only piece of additional data you need to find is its BusinessUnitName? Do you need the full path of Programs/Groups?
4. A Team has exactly one Business Unit in its hierarchy, and you never need to look further up the hierarchy than the Business Unit? i.e. you only need to follow the parent until you reach an object of type Business Unit.
5. "Firstly, the same ID may be used by different nodes at different levels." Can this cause ambiguity? What if you start from Team A and its parent is B, but there is a Program B and a Group B? How do you know which is the parent?

1. No, all levels must be exposed as different objects

2. It would be reasonable to assume depth <= 6

3. Yes. No.

4. Yes.

5. Yes. The parent is the node with the largest level ID, unless that node is the current node.