Identity Broker Forum

Welcome to the community forum for Identity Broker.

Browse the knowledge base, ask questions directly to the product group, or leverage the community to get answers. Leave ideas for new features and vote for the features or bug fixes you want most.

0
Under review

Difference Report on Pending Changes for Full Sync

Oscar Ferne 7 months ago in UNIFYBroker/Plus • updated by Matthew Davis (Engineering Manager) 5 months ago 2

As part of an upgrade activity on an MA, we were required to deliver a difference report on the data as it would appear pre vs post synchronisation of the upgrade MA. This was done to better understand and review what attributes would be updated when a full sync of the upgraded MA would occur in PROD.

We were able to achieve this deliverable by exporting two csv's of the data pre & post synchronisation, and doing a data comparison in a third party app. This could be simplified if Identity Broker Plus could generate a difference report for full syncs to ensure that the MA update is producing clean data.

This report could vary in detail, but as a first pass being able to see a count of the new and updated identities and attributes would be preferable.

0
Declined

Baseline sync error: Execution Timeout Expired

Huu Tran 8 months ago in UNIFYBroker/Plus • updated by Matthew Davis (Engineering Manager) 6 months ago 6

Keep getting the below error for link's baseline outgoing sync to AD. I have tried to restart IdB service but no improvement. I don't have DB Admin right as it is PROD env and Shared SQL cluster. Just wonder what can I do to troubleshoot it?

Synchronization job failed syncing 40800 changes on the 'AD Link' link from the locker to adapter with the reason Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.. Job ID: e6165705-b8bf-4e86-953e-c1394ae692c8 Duration: 00:16:54.3542205
Error details:
System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult)
   at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
   at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
   at Unify.Framework.QueryableExtensions.<AutoStream>d__2`1.MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
   at Unify.Framework.QueryableExtensions.StreamToLookup[TKey,TElement](IOrderedQueryable`1 collection, Func`2 keySelector, Int32 pageSize)
   at Unify.Product.Plus.JoinExecutor`2.Execute(IEnumerable`1 sourceEntities, IQueryable`1 targetEntities)
   at Unify.Product.Plus.LinkSynchronizer`2.JoinAndMap(IEnumerable`1 filterResult, IDictionary`2 changesDict)
   at Unify.Product.Plus.Link.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
   at Unify.Product.Plus.LinkAuditingDecorator.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LockerToAdapterSynchronizationJob.RunBase()
   at Unify.Product.Plus.SynchronizationJobExecutor.<ThreadAction>d__8.MoveNext()
ClientConnectionId:e4b00f30-9b86-4cae-a54c-a96f2f4dc552
Error Number:-2,State:0,Class:11",Normal
20180904,07:19:04,UNIFY Identity Broker,"Void OnError(System.Data.SqlClient.SqlException, Boolean, System.Action`1[System.Action])",Error,".Net SqlClient Data Provider:
System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
Answer
Curtis Lusmore 6 months ago

Queuing a baseline synchronization job requires generating sync changes for all entities on both sides of the link, which again is a SQL-heavy operation. Do you know the hardware specifications of the SQL Server cluster or any of its configuration settings that might impact SQL performance? Do you know of any differences between how it's configured between this environment and the previous environment?

0
Answered

Error updating an AD user

Huu Tran 9 months ago in UNIFYBroker/Plus • updated 9 months ago 7

Error:

20180802,13:57:12,UNIFY Identity Broker,EntitySaver,Error,"The entity 603474 (6cd1989f-bfe8-4f1e-adb6-004af8cea53f) for the adapter AD User Adapter (9f73e5e5-30df-4142-b850-db3e31f0a931) failed to update for the following reasons: Received error code InvalidAttributeSyntax for item with dn CN=redacted,DC=au. Message: 00000057: LdapErr: DSID-0C090BD1, comment: Error in attribute conversion operation, data 0, v1772",Normal

It happens to both Add and Update. However, I changed Outgoing Filter to update only one user and AD Link only update 3 fields: company, department, title. The error still happens ...

TestHarness to CSV file works well ...

Answer
Curtis Lusmore 9 months ago

Via screenshare, the issue turned out to be the casing of the msExchHideFromAddressLists field, which was manually added to the connector as a boolean but requires uppercase. Changing it to string resolved the issue.

0
Not a bug

AD Link shows outgoing sync successful but entities werent provisioned

No error in Log either:

20180121,13:01:26,UNIFY Identity Broker,SyncEngine,Information,"Request to sync locker to adapter completed.
Synchronization job completed syncing 116 changes on the 'AD Link' link from the locker to adapter. Delayed: 0 Incomplete: 0 Denied: 0 Job ID: c5198353-498f-49ab-ad39-3f3ad154b57c Duration: 00:00:10.9188371",Normal
20180121,13:01:26,UNIFY Identity Broker,SyncEngine,Information,"Request to sync adapter to locker started.
Synchronization job started syncing 21057 changes on the 'AD Link' link from the adapter to locker. Job ID: 8ab397a7-93fc-484d-b25f-0f1faaa6e883",Normal
20180121,13:01:32,UNIFY Identity Broker,Change detection engine,Information,"Change detection engine unscheduled started.
Change detection engine unscheduled for connector AD Users started.",Normal

Link shows ougoing sync all good:






AD User Connector shows no entity is saved. AD User Adapter shows no entity is added. Test Mode is disabled.
0
Not a bug

Failed to export to AD

I have the below error when trying to export to AD. It may be due to the exporting volume.


20180121,00:00:00,UNIFY Identity Broker,Logging Engine,Information,Log file started.,Minimal
20180121,00:16:39,UNIFY Identity Broker,Link,Error,"Request to sync changes on link failed.
Request to sync changes on link AD Link (4a76f3ba-6c07-4d9a-9f96-c7dc14fff2e6) in direction incoming failed with message Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. [Count:158178]. Duration: 00:16:54.0665482
Error details:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
   at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult)
   at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
   at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
   at Unify.Product.Plus.LinkSynchronizer`2.<ProcessFilters>d__10.MoveNext()
   at Unify.Product.Plus.JoinExecutor`2.Execute(IEnumerable`1 sourceEntities, IQueryable`1 targetEntities)
   at Unify.Product.Plus.LinkSynchronizer`2.JoinAndMap(IEnumerable`1 filterResult, IDictionary`2 changesDict)
   at Unify.Product.Plus.Link.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
   at Unify.Product.Plus.LinkAuditingDecorator.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LockerToAdapterSynchronizationJob.RunBase()
   at Unify.Product.Plus.SynchronizationJobExecutor.<ThreadAction>d__8.MoveNext()
ClientConnectionId:d630c91c-5b93-44fe-ad1a-8e0b6dc2d624
Error Number:-2,State:0,Class:11",Normal
20180121,00:16:39,UNIFY Identity Broker,SyncEngine,Information,"Request to sync locker to adapter errored.
Synchronization job failed syncing 158178 changes on the 'AD Link' link from the locker to adapter with the reason Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.. Job ID: acf66fc4-57f8-47ae-8baf-fd121dee8efa Duration: 00:16:54.0665482
Error details:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
   at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult)
   at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
   at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
   at Unify.Product.Plus.LinkSynchronizer`2.<ProcessFilters>d__10.MoveNext()
   at Unify.Product.Plus.JoinExecutor`2.Execute(IEnumerable`1 sourceEntities, IQueryable`1 targetEntities)
   at Unify.Product.Plus.LinkSynchronizer`2.JoinAndMap(IEnumerable`1 filterResult, IDictionary`2 changesDict)
   at Unify.Product.Plus.Link.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
   at Unify.Product.Plus.LinkAuditingDecorator.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LockerToAdapterSynchronizationJob.RunBase()
   at Unify.Product.Plus.SynchronizationJobExecutor.<ThreadAction>d__8.MoveNext()
ClientConnectionId:d630c91c-5b93-44fe-ad1a-8e0b6dc2d624

Answer

If you've been performing a large number of changes and importing fresh data, the database is likely requiring some maintenance. See https://voice.unifysolutions.net/forums/7-identity-broker-knowledge/topics/2936-identity-broker-database-recommendations/

0
Under review

Viewing changes in IDB Plus

Huu Tran 1 year ago in UNIFYBroker/Plus • updated by Curtis Lusmore 1 year ago 1

It would be very useful to view the changes that are going to happen to verify data transformation.

Answer
Curtis Lusmore 1 year ago

Hi Huu,

Great idea, I've added this as an item to the backlog.

0
Answered

how to map Date to Timestamp from Adapter to Locker

Huu Tran 1 year ago in UNIFYBroker/Plus • updated by Curtis Lusmore 1 year ago 3

An vice versa? Is there any built in transformation or is has to be done by powershell task?

Answer
Curtis Lusmore 1 year ago

Hi Huu,

There is currently no way to do this directly. As you suggest, you could use a PowerShell task in the Synchronization stage to apply the mapping, or alternatively you could use a Time Offset Transformation in the adapter to generate a Timestamp field. We may look at easier ways to accomplish type conversions in the future, but it would more likely be by converting the type in an adapter transformation.

Out of curiosity, what is the use case?

0
Answered

How deprovisioning work in IDB Plus?

Huu Tran 1 year ago in UNIFYBroker/Plus • updated by Curtis Lusmore 1 year ago 1

It is outgoing provisioning and deprovisioning: Locker-AD Link-AD Adapter - AD User Connector - AD OU

Assume that Locker has 2000 users and there are 3000 users in AD OU--> 3000 in AD Adapter.

After Import All in AD User Connector and Baseline Sync in AD Link, 1500 users in Locker join 1500 in AD Adapter.

In this case, 500 new users will be created in AD because of outgoing provisioning. How about 1500 not-joined users in AD, will they be removed due to outgoing deprovisioning?

Answer
Curtis Lusmore 1 year ago

Hi Huu,

No, such entities shouldn't be deprovisioned during a baseline. A baseline effectively simulates a change to every entity on both sides of the link, but deprovisioning only occurs when an entity is removed from the source context (i.e. a change is registered against an entity that no longer exists in the context).

0
Under review

Health Check Uptime for IDaaS only shows past 24 hours

Shane Day (Chief Technology Officer) 1 year ago in UNIFYBroker/Plus • updated by Curtis Lusmore 1 year ago 1

This is probably fine for the customer facing thing - but I think we need to have something for our own purposes that gives a little more information than this.

0
Under review

Provisions in Last Month graph should be bar chart instead of line chart

Shane Day (Chief Technology Officer) 1 year ago in UNIFYBroker/Plus • updated by Curtis Lusmore 1 year ago 1

This graph is confusing - if it's the "last month" - where's the last month? I also think it would be better as a bar graph.