0
Under review

Baseline sync error: Execution Timeout Expired

Huu Tran 3 weeks ago in UNIFYBroker/Plus • updated by Adam van Vliet (Product Manager) 2 weeks ago 5

Keep getting the below error for link's baseline outgoing sync to AD. I have tried to restart IdB service but no improvement. I don't have DB Admin right as it is PROD env and Shared SQL cluster. Just wonder what can I do to troubleshoot it?

Synchronization job failed syncing 40800 changes on the 'AD Link' link from the locker to adapter with the reason Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.. Job ID: e6165705-b8bf-4e86-953e-c1394ae692c8 Duration: 00:16:54.3542205
Error details:
System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TrySetMetaData(_SqlMetaDataSet metaData, Boolean moreInfo)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult)
   at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries)
   at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
   at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
   at Unify.Framework.QueryableExtensions.<AutoStream>d__2`1.MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToLookup[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
   at Unify.Framework.QueryableExtensions.StreamToLookup[TKey,TElement](IOrderedQueryable`1 collection, Func`2 keySelector, Int32 pageSize)
   at Unify.Product.Plus.JoinExecutor`2.Execute(IEnumerable`1 sourceEntities, IQueryable`1 targetEntities)
   at Unify.Product.Plus.LinkSynchronizer`2.JoinAndMap(IEnumerable`1 filterResult, IDictionary`2 changesDict)
   at Unify.Product.Plus.Link.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
   at Unify.Product.Plus.LinkAuditingDecorator.SynchronizeLockerChanges(IEnumerable`1 changes)
   at Unify.Product.Plus.LockerToAdapterSynchronizationJob.RunBase()
   at Unify.Product.Plus.SynchronizationJobExecutor.<ThreadAction>d__8.MoveNext()
ClientConnectionId:e4b00f30-9b86-4cae-a54c-a96f2f4dc552
Error Number:-2,State:0,Class:11",Normal
20180904,07:19:04,UNIFY Identity Broker,"Void OnError(System.Data.SqlClient.SqlException, Boolean, System.Action`1[System.Action])",Error,".Net SqlClient Data Provider:
System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
Affected Versions:
Fixed by Version:
Under review

Hi Huu,

Are you noticing SQL timeouts for any other operation? Does a service restart have any effect? Do you know what maintenance tasks are being performed on the database (see UNIFYBroker SQL Server Database Recommendations)?

No. TImeout only happens to Link Sync. I restarted the service several times but it did not help. The database is just new, I just installed IdB plus, ported the solution, did import and now sync-ing.


I will run maintenance tasks now but I doubt it will help

 I ran the db maintenance task and tried to do manual baseline syncbut it is not running

It keeps queuing even though I kicked it off 25' ago

In the meanwhile, nothing else is running in IdB

 

Queuing a baseline synchronization job requires generating sync changes for all entities on both sides of the link, which again is a SQL-heavy operation. Do you know the hardware specifications of the SQL Server cluster or any of its configuration settings that might impact SQL performance? Do you know of any differences between how it's configured between this environment and the previous environment?