Empty String behaviour change from IdB5 upgrade for JadeStar

Hi Bob

Can you provide your extensibility configuration for both 4.1 and 5.3?

How do these values appear in the adapter entity viewer and does this differ between versions?

Testing locally (not against JadeStar, of course) I cannot find an input value which results in the output you are seeing. Could you capture this request in a network trace?

Beau - after finally getting over all the script syntax issues in order to run a transform for all my attributes, I ended up back where I started - no net change in either MIM nor the LDP output.

Here's my PS transform in case you can spot any issue ... but no change in the result, and the logged event details follow.

Function Get-EmptyStringAsNull (
    [string]$testValue
) {
    if($testValue -ne $null -and $testValue.Length -eq 0) { 
        $testValue = $null 
    };
    return $testValue;
}

# Loop through each input entity
foreach ($entity in $entities)
{
    $msg = "";
    if (($entity['EmployeeNumber'].Value) -eq '500015') {
        $msg = "EmployeeNumber: $($entity['EmployeeNumber'].Value); "
        $msg += "Before: UserID IsNull: $($entity['UserID'].Value -eq $null); "
    }

    $entity['CellphoneCountryCode'] = (Get-EmptyStringAsNull -testValue ($entity['CellphoneCountryCode'].Value));
    $entity['CellphoneNumber'] = (Get-EmptyStringAsNull -testValue ($entity['CellphoneNumber'].Value));
    $entity['CellphonePrefix'] = (Get-EmptyStringAsNull -testValue ($entity['CellphonePrefix'].Value));
    $entity['DDiCountryCode'] = (Get-EmptyStringAsNull -testValue ($entity['DDiCountryCode'].Value));
    $entity['DDiNumber'] = (Get-EmptyStringAsNull -testValue ($entity['DDiNumber'].Value));
    $entity['DDiPrefix'] = (Get-EmptyStringAsNull -testValue ($entity['DDiPrefix'].Value));
    $entity['DepartmentCode'] = (Get-EmptyStringAsNull -testValue ($entity['DepartmentCode'].Value));
    $entity['DepartmentDescription'] = (Get-EmptyStringAsNull -testValue ($entity['DepartmentDescription'].Value));
    $entity['DepartmentJadeCode'] = (Get-EmptyStringAsNull -testValue ($entity['DepartmentJadeCode'].Value));
    $entity['DeskNumber'] = (Get-EmptyStringAsNull -testValue ($entity['DeskNumber'].Value));
    $entity['DivisionCode'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionCode'].Value));
    $entity['DivisionDescription'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionDescription'].Value));
    $entity['DivisionJadeCode'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionJadeCode'].Value));
    $entity['DivisionSubCode'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionSubCode'].Value));
    $entity['DivisionSubDescription'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionSubDescription'].Value));
    $entity['DivisionSubJadeCode'] = (Get-EmptyStringAsNull -testValue ($entity['DivisionSubJadeCode'].Value));
    $entity['EmailAddress'] = (Get-EmptyStringAsNull -testValue ($entity['EmailAddress'].Value));
    $entity['EmployeeStatus'] = (Get-EmptyStringAsNull -testValue ($entity['EmployeeStatus'].Value));
    $entity['EmployeeType'] = (Get-EmptyStringAsNull -testValue ($entity['EmployeeType'].Value));
    $entity['Extension'] = (Get-EmptyStringAsNull -testValue ($entity['Extension'].Value));
    $entity['FaxCountryCode'] = (Get-EmptyStringAsNull -testValue ($entity['FaxCountryCode'].Value));
    $entity['FaxNumber'] = (Get-EmptyStringAsNull -testValue ($entity['FaxNumber'].Value));
    $entity['FaxPrefix'] = (Get-EmptyStringAsNull -testValue ($entity['FaxPrefix'].Value));
    $entity['FirstName'] = (Get-EmptyStringAsNull -testValue ($entity['FirstName'].Value));
    $entity['Function'] = (Get-EmptyStringAsNull -testValue ($entity['Function'].Value));
    $entity['Initials'] = (Get-EmptyStringAsNull -testValue ($entity['Initials'].Value));
    $entity['JobFamily'] = (Get-EmptyStringAsNull -testValue ($entity['JobFamily'].Value));
    $entity['Level'] = (Get-EmptyStringAsNull -testValue ($entity['Level'].Value));
    $entity['LocationCode'] = (Get-EmptyStringAsNull -testValue ($entity['LocationCode'].Value));
    $entity['MiddleName'] = (Get-EmptyStringAsNull -testValue ($entity['MiddleName'].Value));
    $entity['OccupancyType'] = (Get-EmptyStringAsNull -testValue ($entity['OccupancyType'].Value));
    $entity['OID'] = (Get-EmptyStringAsNull -testValue ($entity['OID'].Value));
    $entity['OrganisationLevel'] = (Get-EmptyStringAsNull -testValue ($entity['OrganisationLevel'].Value));
    $entity['PositionCode'] = (Get-EmptyStringAsNull -testValue ($entity['PositionCode'].Value));
    $entity['PositionName'] = (Get-EmptyStringAsNull -testValue ($entity['PositionName'].Value));
    $entity['PositionOccupancyReportsToEmployee'] = (Get-EmptyStringAsNull -testValue ($entity['PositionOccupancyReportsToEmployee'].Value));
    $entity['PositionOccupancyReportsToName'] = (Get-EmptyStringAsNull -testValue ($entity['PositionOccupancyReportsToName'].Value));
    $entity['PreferredName'] = (Get-EmptyStringAsNull -testValue ($entity['PreferredName'].Value));
    $entity['ReportsToEmployee'] = (Get-EmptyStringAsNull -testValue ($entity['ReportsToEmployee'].Value));
    $entity['ReportsToName'] = (Get-EmptyStringAsNull -testValue ($entity['ReportsToName'].Value));
    $entity['ReportsToPositionCode'] = (Get-EmptyStringAsNull -testValue ($entity['ReportsToPositionCode'].Value));
    $entity['ReportsToPositionName'] = (Get-EmptyStringAsNull -testValue ($entity['ReportsToPositionName'].Value));
    $entity['StandardHoursFortnight'] = (Get-EmptyStringAsNull -testValue ($entity['StandardHoursFortnight'].Value));
    $entity['Surname'] = (Get-EmptyStringAsNull -testValue ($entity['Surname'].Value));
    $entity['TeamCode'] = (Get-EmptyStringAsNull -testValue ($entity['TeamCode'].Value));
    $entity['TeamDescription'] = (Get-EmptyStringAsNull -testValue ($entity['TeamDescription'].Value));
    $entity['TeamJadeCode'] = (Get-EmptyStringAsNull -testValue ($entity['TeamJadeCode'].Value));
    $entity['Title'] = (Get-EmptyStringAsNull -testValue ($entity['Title'].Value));
    $entity['UserID'] = (Get-EmptyStringAsNull -testValue ($entity['UserID'].Value));

    if (($entity['EmployeeNumber'].Value) -eq '500015') {
        $msg += "After: UserID IsNull: $($entity['UserID'].Value -eq $null); "
        Write-EventLog -LogName Application -Source 'FIMSynchronizationService' -EntryType Information -EventId 2004 -Message $msg
    }
}

Logged event for employee 500015:

EmployeeNumber: 500015; Before: UserID IsNull: False; After: UserID IsNull: False; 

Beau - the workaround has finally worked - my problem was the function I had declared was flawed in its parameter use.  The corrected function used a new internal $result variable:

Function Get-EmptyStringAsNull (
    [string]$testValue
) {
    $result = $testValue
    if($testValue -ne $null -and $testValue.Length -eq 0) { 
        $result = $null 
    };
    return $result;
}

I accept the above work-around is going to be satisfactory for our EOFY deadline, but I have concerns that this problem has only been identified now, several years after IdB v5.05 for MIM was first released (2016-04-07).

As I explained on the call today, I believe the IdB4 behaviour for this data was already 100% correct, and the IdB5 urgently needs to be brought into line with that.  While I can't imagine any use case where an empty string value should ever be imported to MIM in any way other than NULL.

I also appreciate that given this has potentially been in place for up to 3 years at some MIM sites, and therefore to change this behaviour without necessary impact assessment on these sites would be undermine our duty of care.  To overcome this problem a potential way of softening this behaviour would be to introduce a configuration flag in Broker v5 for MIM to support both options, including backwards compatibility for IdB5 (treat '' as '') as well as backwards compatibility for IdB4 (treat '' as null).

P.S. it is the original SQL MA which has the default behaviour checked to convert empty strings to nulls:

SQL MA Options

Thanks, Bob. Added this to the backlog

Here's a PowerShell transformation for correcting this behaviour on all string attributes of all entities, independent of attribute names:

function Optimize-BlankStringAttribute {
    param(
        $Entity,
        $AttributeName
    )

    if ($Entity[$attributeName].ValueType -eq "String" -and $Entity[$attributeName].Value -eq "") {
        $Entity[$attributeName] = $null
        #$Logger.LogInformation("Converting blank attribute $attributeName to NULL")
    }
}

function Optimize-Entity {
    param(
        $Entity
    )

    foreach ($attributeName in $Entity.Key.ValueName) {
        Optimize-BlankStringAttribute -Entity $Entity -AttributeName $attributeName
    }
}


foreach ($entity in $entities) {
    Optimize-Entity -Entity $entity
}