Blown away with how easy this is to work with - just needed a little push in the right direction and it worked a treat with minimal effort.  Endless possibilities here - particularly with scripted deployment!

Thanks guys!

We have staff details sourced from Oracle table example fields are EmployeeNumber, StartDate and LastUpdated. The requirement is to activate the staff account seven days before the start date. We calculate the AccountStatus in a PowerShell transformation i.e. Active or Inactive based on the StartDate. UNIFYBroker is configured to run full import every hour.

For example, a new staff member is added into the source system on 20-Dec-2019; and after 20-Dec-2019 the record is not updated in the source system, below is the state. 

As per the requirement staff should be enabled on 27-Dec-2019. On 2-Jan-2020 following was the state.

However, when we execute Advanced Operations --> Generate Changes manually AccountStatus was updated to ‘Active’.

It appears that if there is no change to the connector entity the adapter’s PowerShell transformation is not recalculated even on connector's full import. 

Is there a workaround?

Hi, 

Unify.IdentityBroker.FIMAdapter.dll is generating error during a Delta import, but works fine with a Full Import:

The extensible extension returned an unsupported error.
The stack trace is:

"Unify.Product.IdentityBroker.LdapOperationException: Found multiple entities with the distinguished name 'CN=00******,OU=Staff,DC=IdentityBroker'.
at Unify.Product.IdentityBroker.LdapConnection.SendRequest(ILdapRequest request)
at Unify.Product.IdentityBroker.LdapConnectionProxy.d__8.MoveNext()
at Unify.Product.IdentityBroker.ImportProxy.d__30.MoveNext()
at System.Linq.Enumerable.d__17`2.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.d__17`2.MoveNext()
at Unify.Product.IdentityBroker.ExtensionMethods.Take[TSource](IEnumerator`1 source, Int32 count, IList`1& items)
at Unify.Product.IdentityBroker.ExtensionMethods.d__3`1.MoveNext()
at Unify.Product.IdentityBroker.ImportProxy.Import(GetImportEntriesRunStep importRunStep)
at Unify.Product.IdentityBroker.UnifyLdapConnector.GetImportEntries(GetImportEntriesRunStep importRunStep)
Forefront Identity Manager 4.4.1749.0"

Each time the error occurs, it is a different user.

I have check in the UnifyBroker connector and adapter, there is only one entry for this user. I checked in ADSI, only one record as well.

There are 3 connectors plugged to UNIFYBroker, all connectors have already got this issue in the past.

The database is on a loadbalanced cluster.

Do you have any idea why this error occurs?

UNIFYBroker v.5.3.1 RC2

Unify.IdentityBroker.FIMAdapter.dll v5.3.0

Hi All,

Historically we have had this issue prior to upgrading the latest version of UNIFYBroker (on v5.0.0). However we are now on version 5.3.2, and didn't seen the errors for some time, though now they appear to be occurring again. The errors so far have only been seen occurring on every DIDS after a failing full import operations. As mentioned before UNIFYBroker is on the latest version and FIM this user the latest version on the Unify extension (v5.3.0).

The FIM operation currently has a page size of 1000 and an operation timeout of 1500 seconds. So it appears to get stuck for quite some time.

The UNIFYBroker Log doesnt have anything on the exact time of failure but it does appear to be surrounded by Timeout errors:

20191215,17:43:12,UNIFYBroker,Void AcquireReaderLockInternal(Int32),Warning,"mscorlib:
System.ApplicationException: This operation returned because the timeout period expired. (Exception from HRESULT: 0x800705B4)
at System.Threading.ReaderWriterLock.AcquireReaderLockInternal(Int32 millisecondsTimeout)
at System.Threading.ReaderWriterLock.AcquireReaderLock(TimeSpan timeout)
at Unify.Framework.Data.LinqQueryConversionProvider`5.ExecuteMethodCallExpression[TResult](MethodCallExpression methodCallExpression)
at Unify.Framework.Data.LinqQueryConversionProvider`5.Execute[TResult](Expression expression)
at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
at Unify.Product.IdentityBroker.AdapterStatisticsEngine.GetEntityCount(Guid adapterId)",Normal

Let me what other information you need, and I'll be happy to provide it.

Thanks

I am using the following PS Adapter transform to generate a "hrStatus" value for EVERY record:

foreach ($entity in $entities) {
    [string]$hrStatus = "Active"
    if (!($entity["uid"]) -or ($entity["uid"].Value -notlike "A*") -or ($entity["uid"].Value -like "*_*")) {
        $hrStatus = "Inactive"
    }
    $entity["hrStatus"] = $hrStatus
}

I declare the schema for the hrStatus property as follows:

New-Field 'hrStatus' 'string' $false $true $true;

I am then setting the DN to use this property as a next-level OU below OU=Employees.

However when I enable the Adapter I get the error below - does this mean that the REQUIRED property is being ignore?

System.Exception: Swagger Exception could not be parsed. SE response code: 500; SE response text: {"Message":"An error has occurred.","ExceptionMessage":"Error in adapter DotEE Employee distinguished name configuration: The DN component part 'OU=[hrStatus]' could not be executed as the field hrStatus is not required. An empty field would result in a DN of 'OU='.","ExceptionType":"Unify.Framework.UnifyConfigurationException","StackTrace":"   at Unify.Product.IdentityBroker.AdapterEngine.ValidateAdapterForEnabling(IOperationalAdapter adapter)\r\n   at Unify.Product.IdentityBroker.AdapterEngine.<>c__DisplayClass54_0.<EnableAdapter>b__0()\r\n   at Unify.Product.IdentityBroker.AdapterEngine.<>c__DisplayClass145_0.<ConfigurationChange>b__0()\r\n   at Unify.Framework.ExtensionMethods.WaitOnMutex(Mutex mutex, Action work)\r\n   at Unify.Product.IdentityBroker.AdapterEngineAuditingDecorator.EnableAdapter(Guid adapterId)\r\n   at Unify.Product.IdentityBroker.AdapterEngineNotifierDecorator.<>c__DisplayClass19_0.<EnableAdapter>b__0()\r\n   at Unify.Framework.Notification.NotifierDecoratorBase.Notify(ITaskNotificationFactory notificationFactory, Action action)\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClassc.<GetExecutor>b__6(Object instance, Object[] methodParameters)\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"The DN component part 'OU=[hrStatus]' could not be executed as the field hrStatus is not required. An empty field would result in a DN of 'OU='.","ExceptionType":"Unify.Framework.UnifyEngineException","StackTrace":"   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Validate(IEntitySchema schema)\r\n   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.Validate(IEntitySchema schema)\r\n   at Unify.Product.IdentityBroker.AdapterEngine.ValidateAdapterForEnabling(IOperationalAdapter adapter)"}}; ---> Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
   at Unify.Connect.Web.Client.AdapterClient.<ToggleAdapterEnabledAsync>d__117.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.Client.ProfiledAdapterClient.<ToggleAdapterEnabledAsync>d__124.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Framework.Web.UnifyController.<HandleRemoteException>d__42.MoveNext()
   --- End of inner exception stack trace ---
   at Unify.Framework.Web.UnifyController.InnerHandleSwaggerExceptionForApiCall(SwaggerException se, String messageTemplate, Action`1 handleMessage, Action`1 handleExtended)
   at Unify.Framework.Web.UnifyController.HandleSwaggerExceptionForApiCall(SwaggerException se, String messageTemplate)
   at Unify.Framework.Web.UnifyController.<HandleRemoteException>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.AdapterController.<ToggleEnableState>d__74.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.AdapterController.<EnableAdapter>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at lambda_method(Closure , Task )
   at System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass34.<BeginInvokeAsynchronousActionMethod>b__33(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3c()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass45.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3e()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<>c__DisplayClass28.<BeginInvokeAction>b__19()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<BeginInvokeAction>b__1b(IAsyncResult asyncResult)

Hi Guys,

We've had a couple of entities today reported not provisioning into FIM. So I did some investigation and I found that the entities have come into UNIFYBroker through a DB Connector and then added to the adapter. However I cannot see a change log entry in the DB to then flow in via a delta in FIM.

So far I have check the change log table as mentioned and the UNIFYBroker logs. I cannot see any errors around the time the record was created in the adapter nor can I see any errors referencing the connector/adapter combo.

Currently we are on the latest version of Broker v5.3.2.

As a workaround we are doing a Full Import to bring it into FIM but it takes quite a while as its a large MA, so it would be good to get to the bottom of this. If you need any more information or need me to do some more digging I'm be happy to help.

Thanks

The connector is able to push data to AD, but throws an error when performing a change/all import from AD.

Has anyone run into this issue on imports?

20191210,04:34:48,UNIFY Identity Broker,Change detection engine,Information,"Change detection engine import all items started.
Change detection engine import all items for connector Active Directory Connector started.",Normal
20191210,04:34:54,UNIFY Identity Broker,Change detection engine,Error,"Change detection engine import all items failed.
Change detection engine import all items for connector Active Directory Connector failed with reason One or more errors occurred.. Duration: 00:00:05.5141233
Error details:
System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Unify.Connectors.AD.ADConnector.TransformEntry(ADAgent agent, SearchResultEntry searchResultEntry, Int64& uSNChangedToken)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<actiononlast>d__10`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.AuditReadingAsyncConnectorDecorator.<>c__DisplayClass1_0.<getallentitiesasync>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass7_0`1.<createandsendlogentryasync>b__0(Task`1 t)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass5_0`1.<taskcontinuewithexceptionpassthough>b__0(Task`1 t)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.EventNotifierReadingAsyncConnectorDecoratorBase`1.<getallentitiesasync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.ChangeDetectionImportAllAsyncJob.<importallchangeprocess>d__6.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at Unify.Product.IdentityBroker.ChangeDetectionImportAllAsyncJob.RunBase()
   at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
   at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c__DisplayClass30_0.<run>b__0()
   at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)
---> (Inner Exception #0) System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Unify.Connectors.AD.ADConnector.TransformEntry(ADAgent agent, SearchResultEntry searchResultEntry, Int64& uSNChangedToken)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<actiononlast>d__10`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.AuditReadingAsyncConnectorDecorator.<>c__DisplayClass1_0.<getallentitiesasync>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass7_0`1.<createandsendlogentryasync>b__0(Task`1 t)
   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass5_0`1.<taskcontinuewithexceptionpassthough>b__0(Task`1 t)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.EventNotifierReadingAsyncConnectorDecoratorBase`1.<getallentitiesasync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Product.IdentityBroker.ChangeDetectionImportAllAsyncJob.<importallchangeprocess>d__6.MoveNext()
---> (Inner Exception #0) System.NullReferenceException: Object reference not set to an instance of an object.
   at Unify.Connectors.AD.ADConnector.TransformEntry(ADAgent agent, SearchResultEntry searchResultEntry, Int64& uSNChangedToken)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<actiononlast>d__10`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.AuditReadingAsyncConnectorDecorator.<>c__DisplayClass1_0.<getallentitiesasync>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()<---
<---
",Normal
</getallentitiesasync></actiononlast></importallchangeprocess></getallentitiesasync></taskcontinuewithexceptionpassthough></createandsendlogentryasync></getallentitiesasync></actiononlast></run></importallchangeprocess></getallentitiesasync></taskcontinuewithexceptionpassthough></createandsendlogentryasync></getallentitiesasync></actiononlast>

When switching from a PowerShell to C# connector for the same schema, the following error occurred when attempting to refresh the adapter (after Generate Changes):

Request to reflect change entities of the adapter.
Request to reflect change entities of the DotEE Company (1425b9d9-bdd2-4786-81ae-8c09272a0750) adapter errored with message: An error occurred retrieving the distinguished name component for field 'companyCode' of type CN. See inner exception for details.. Duration: 00:00:00.0937585
Error details:
Unify.Framework.UnifyDataException: An error occurred retrieving the distinguished name component for field 'companyCode' of type CN. See inner exception for details. ---> Unify.Framework.Collections.GroupedNameValueCollectionMissingFieldException: The entity does not contain a value for the companyCode field.
   at Unify.Product.IdentityBroker.EntityBase`3.GetValueEntry(TKey key)
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   --- End of inner exception stack trace ---
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__17`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.IO.DistinguishedName..ctor(IEnumerable`1 components)
   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.DistinguishedName(TEntity entity)
   at Unify.Product.IdentityBroker.Adapter.<>c__DisplayClass120_0.<ConvertPageAndUpdateContainers>b__1(IEntity entity)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at Unify.Product.IdentityBroker.Adapter.ConvertPageAndUpdateContainers(IEntity[] entities, Boolean updateContainers)
   at Unify.Product.IdentityBroker.Adapter.ReflectChangesInner()
   at Unify.Product.IdentityBroker.Adapter.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterAuditingDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.AdapterNotifierDecorator.ReflectChanges()
   at Unify.Product.IdentityBroker.ReflectAdapterOnChangeDueJob.<RunBase>b__9_0(IOperationalAdapter adapter)

In troubleshooting SAP connectivity I am repeatedly making changes to the Unify.Service.Connect.exe.config file and restarting Broker - but being careful not to start until the existing process has completely stopped - which usually takes about 30 seconds in my environment.

I got tired of doing this manually and wrote the following simple script - which must be run from an Administrator PS session:

cls
[bool]$isRunning = [bool](Get-Process "Unify.Service.Connect" -ErrorAction SilentlyContinue)
if ($isRunning) {
    net stop service.connect
}
[int]$counter = 0
while ($isRunning) {
    $counter++
    Write-Host "Waiting [$counter] seconds for service to shut down ..."
    Start-Sleep 1
    $isRunning = [bool](Get-Process "Unify.Service.Connect" -ErrorAction SilentlyContinue)
}
net start service.connect

I hope others find this useful too!

MIM has around 940 pending export deletes for HPPhoneNumber attributes, and they are being processed successfully by UNIFYBroker and the Aderant Expert connector (i.e. no export errors).  However, during a subsequent Full Import by MIM that attribute is being restored to the non-NULL value, meaning that the next Full Sync results in each of those objects having a new pending export, ad infinitum.