We've got a site where the adapter entities have been cleared and on a generate changes only one or two thousand entities get processed with one time the following error occurring:

"Transaction (Process ID 124) was deadlocked on lock resources with
another process and has been chosen as the deadlock victim. Rerun the
transaction.."

It looks like since the connector schedules haven't been disabled that they're running imports that are causing the database to deadlock, aborting the reflection process. I had a look through the documentation in the knowledge base but I couldn't find anything that confirms my suspicion.

What I'm looking for is confirmation that UNIFYBroker does hit a deadlock when a reflection and an import occur at the same on the same connector so I can advise the client on what to do in the future to avoid this (i.e., disable the schedules when doing a re-population).

Hello,

Not sure if this has been reported before, but couldn't see anything on my quick look. I am currently in an environment running Broker v5.3.1 Revision #4, and am experiencing what appears to be a UI bug where randomly the web page will begin to display "Service Unavailable" all over the page.

After several seconds to a minute or two, the page will then flick back to normal and continue in this pattern. Some days it will happen more often than others, with no increase in job frequency. Occasionally a service restart will improve the frequency of it occurring, however it will slowly return over time.

I've investigated a bit and it appears to not noticeably affect the current running jobs in UNIFYBroker, nor importing running from Broker into MIM. I have also see it happens more frequently when jobs are running in Broker, but not attributed to any one job in particular. There are also no errors in the Event Log, however there are some in the UNIFYBroker log, though they don't appear to be UI specific, nor can I say I have seen these on every occasion this happens.

The error I'm referring to in particular is:

"Unable to raise complete notification:

System.ApplicationException: This operation returned because the timeout period expired. (Exception from HRESULT: 0x800705B4)
at System.Threading.ReaderWriterLock.AcquireWriterLockInternal(Int32 millisecondsTimeout)
at System.Threading.ReaderWriterLock.AcquireWriterLock(TimeSpan timeout)
at Unify.Framework.Collections.ThreadsafeObjectQueueBase`2.Push(TItem item)
at Unify.Framework.Notification.TaskNotifier`1.Dispose()"

If you need any more information just let me know.

Thanks

At 17/03/2020 15:30:00 UNIFYNow attempted to run the Daily Full Cycle 2nd Step operation list, despite the presence of an exclusion group which also contained the Daily Full Cycle 1st Step operation list which was still running the last operation it contains at that time (MDR South Employees Management Agent Full Import Full Sync):

The issue was evidenced by the failed attempt to run the first operation of the 2nd Step operation list (MDR Master FIFS):

The MDR South FIFS (last operation of the 1st Step operation list) was definitely run by UNIFYNow, and not manually from the MIM Sync Service Manager.

Log and extensibility files will be attached to the next comment.

Even though a SCIM connection from Azure UNIFYBroker successfully created a new user in AD, it also logged an error.

Log and config attached.

create-success.pcapng

Extensibility.zip

UnifyLog20200212.zip

• Authenticating
  • What credentials?
  • Purpose of "Secret Token" in Azure portal
• Querying (used for test connection)
  • Null reference exception
•  HTTPS
  • Bad request returned from TLS1.2 Client Hello

Blown away with how easy this is to work with - just needed a little push in the right direction and it worked a treat with minimal effort.  Endless possibilities here - particularly with scripted deployment!

Thanks guys!

We have staff details sourced from Oracle table example fields are EmployeeNumber, StartDate and LastUpdated. The requirement is to activate the staff account seven days before the start date. We calculate the AccountStatus in a PowerShell transformation i.e. Active or Inactive based on the StartDate. UNIFYBroker is configured to run full import every hour.

For example, a new staff member is added into the source system on 20-Dec-2019; and after 20-Dec-2019 the record is not updated in the source system, below is the state. 

As per the requirement staff should be enabled on 27-Dec-2019. On 2-Jan-2020 following was the state.

However, when we execute Advanced Operations --> Generate Changes manually AccountStatus was updated to ‘Active’.

It appears that if there is no change to the connector entity the adapter’s PowerShell transformation is not recalculated even on connector's full import. 

Is there a workaround?

Hi All,

Historically we have had this issue prior to upgrading the latest version of UNIFYBroker (on v5.0.0). However we are now on version 5.3.2, and didn't seen the errors for some time, though now they appear to be occurring again. The errors so far have only been seen occurring on every DIDS after a failing full import operations. As mentioned before UNIFYBroker is on the latest version and FIM this user the latest version on the Unify extension (v5.3.0).

The FIM operation currently has a page size of 1000 and an operation timeout of 1500 seconds. So it appears to get stuck for quite some time.

The UNIFYBroker Log doesnt have anything on the exact time of failure but it does appear to be surrounded by Timeout errors:

20191215,17:43:12,UNIFYBroker,Void AcquireReaderLockInternal(Int32),Warning,"mscorlib:
System.ApplicationException: This operation returned because the timeout period expired. (Exception from HRESULT: 0x800705B4)
at System.Threading.ReaderWriterLock.AcquireReaderLockInternal(Int32 millisecondsTimeout)
at System.Threading.ReaderWriterLock.AcquireReaderLock(TimeSpan timeout)
at Unify.Framework.Data.LinqQueryConversionProvider`5.ExecuteMethodCallExpression[TResult](MethodCallExpression methodCallExpression)
at Unify.Framework.Data.LinqQueryConversionProvider`5.Execute[TResult](Expression expression)
at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
at Unify.Product.IdentityBroker.AdapterStatisticsEngine.GetEntityCount(Guid adapterId)",Normal

Let me what other information you need, and I'll be happy to provide it.

Thanks

I am using the following PS Adapter transform to generate a "hrStatus" value for EVERY record:

foreach ($entity in $entities) {
    [string]$hrStatus = "Active"
    if (!($entity["uid"]) -or ($entity["uid"].Value -notlike "A*") -or ($entity["uid"].Value -like "*_*")) {
        $hrStatus = "Inactive"
    }
    $entity["hrStatus"] = $hrStatus
}

I declare the schema for the hrStatus property as follows:

New-Field 'hrStatus' 'string' $false $true $true;

I am then setting the DN to use this property as a next-level OU below OU=Employees.

However when I enable the Adapter I get the error below - does this mean that the REQUIRED property is being ignore?

System.Exception: Swagger Exception could not be parsed. SE response code: 500; SE response text: {"Message":"An error has occurred.","ExceptionMessage":"Error in adapter DotEE Employee distinguished name configuration: The DN component part 'OU=[hrStatus]' could not be executed as the field hrStatus is not required. An empty field would result in a DN of 'OU='.","ExceptionType":"Unify.Framework.UnifyConfigurationException","StackTrace":"   at Unify.Product.IdentityBroker.AdapterEngine.ValidateAdapterForEnabling(IOperationalAdapter adapter)\r\n   at Unify.Product.IdentityBroker.AdapterEngine.<>c__DisplayClass54_0.<EnableAdapter>b__0()\r\n   at Unify.Product.IdentityBroker.AdapterEngine.<>c__DisplayClass145_0.<ConfigurationChange>b__0()\r\n   at Unify.Framework.ExtensionMethods.WaitOnMutex(Mutex mutex, Action work)\r\n   at Unify.Product.IdentityBroker.AdapterEngineAuditingDecorator.EnableAdapter(Guid adapterId)\r\n   at Unify.Product.IdentityBroker.AdapterEngineNotifierDecorator.<>c__DisplayClass19_0.<EnableAdapter>b__0()\r\n   at Unify.Framework.Notification.NotifierDecoratorBase.Notify(ITaskNotificationFactory notificationFactory, Action action)\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClassc.<GetExecutor>b__6(Object instance, Object[] methodParameters)\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()","InnerException":{"Message":"An error has occurred.","ExceptionMessage":"The DN component part 'OU=[hrStatus]' could not be executed as the field hrStatus is not required. An empty field would result in a DN of 'OU='.","ExceptionType":"Unify.Framework.UnifyEngineException","StackTrace":"   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Validate(IEntitySchema schema)\r\n   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.Validate(IEntitySchema schema)\r\n   at Unify.Product.IdentityBroker.AdapterEngine.ValidateAdapterForEnabling(IOperationalAdapter adapter)"}}; ---> Unify.Framework.Client.SwaggerException: The HTTP status code of the response was not expected (500).
   at Unify.Connect.Web.Client.AdapterClient.<ToggleAdapterEnabledAsync>d__117.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.Client.ProfiledAdapterClient.<ToggleAdapterEnabledAsync>d__124.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Framework.Web.UnifyController.<HandleRemoteException>d__42.MoveNext()
   --- End of inner exception stack trace ---
   at Unify.Framework.Web.UnifyController.InnerHandleSwaggerExceptionForApiCall(SwaggerException se, String messageTemplate, Action`1 handleMessage, Action`1 handleExtended)
   at Unify.Framework.Web.UnifyController.HandleSwaggerExceptionForApiCall(SwaggerException se, String messageTemplate)
   at Unify.Framework.Web.UnifyController.<HandleRemoteException>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.AdapterController.<ToggleEnableState>d__74.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Unify.Connect.Web.AdapterController.<EnableAdapter>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at lambda_method(Closure , Task )
   at System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass34.<BeginInvokeAsynchronousActionMethod>b__33(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3c()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass45.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3e()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<>c__DisplayClass28.<BeginInvokeAction>b__19()
   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<BeginInvokeAction>b__1b(IAsyncResult asyncResult)

Hi Guys,

We've had a couple of entities today reported not provisioning into FIM. So I did some investigation and I found that the entities have come into UNIFYBroker through a DB Connector and then added to the adapter. However I cannot see a change log entry in the DB to then flow in via a delta in FIM.

So far I have check the change log table as mentioned and the UNIFYBroker logs. I cannot see any errors around the time the record was created in the adapter nor can I see any errors referencing the connector/adapter combo.

Currently we are on the latest version of Broker v5.3.2.

As a workaround we are doing a Full Import to bring it into FIM but it takes quite a while as its a large MA, so it would be good to get to the bottom of this. If you need any more information or need me to do some more digging I'm be happy to help.

Thanks