The Identity Broker for chris21 seems to take significantly long period of time to perform "Full Import" from chris21 system into Identity Broker Connector and then process/transform the data into the IdB Adapter.

I am not familar with the flow process and the analysing the log file content. Thus the below investigation may not be entirely correct. But the overall time taken for processing data does seems significantly long in Identity Broker for chris21.

The customer has 4999 records/users in chris21.

Analysis 1:

1 - When performing the "Import All Connectors" for all six (6) Connectors took approximately 32 minutes at other times it does seems to take significantly longer time, eg 45 minutes. During this time the CPU and disk accessing activities are at 100%.

Started at

20101215,07:22:14,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Address Connector started.,Normal

Finished at

20101215,07:54:15,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Org Unit Connector completed. Duration: 00:02:34.7343750,Normal

2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 10 minutes. At other times we believe it took significantly longer.

Started at

20101215,07:55:56,Adapter request to get attribute changes from adapter space.,Adapter,Information,Adapter request to get attribute changes from adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal

Finished at

20101215,08:05:46,Adapter get attribute changes completed.,Adapter,Information,Adapter get attribute changes for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 completed. Duration: 00:09:49.8125000,Normal

It has been mentioned that the "Import All Connectors" should take only a few minutes. Thus I would like to request a review of the IdB Connector and Adapter configuration files (attached) as to what could possibly create this behaviour. It is mentioned that the gtrChunkSize="1000" could be in appropriate for the volume of data in chris21.

Analysis 2:
Another example: When performing the "Import All Connectors" for all six (6) Connectors took approximately 14 minutes
Start at

20101213,07:01:22,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Person Connector started.,Normal

Finished at

20101213,07:15:17,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Placement Connector completed. Duration: 00:02:19.6875000,Normal

2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 27 minutes 30 seconds.

Started at

20101213,07:25:53,Request to get all entities for adapter space.,Adapter,Information,Request to get all entities for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal

Finished at

20101213,07:53:34,Adapter get all entities to adapter space completed.,Adapter,Information,Adapter get all entities to adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 returned 4999 entities. Duration: 00:27:41.0937500,Normal

Analysis 3:
"Import all connectors" in IdB took approximately 12 minutes. I do not have the log file
"Full Import" in FIM for chris21 MA took approximately 6 minutes.

Please help to review the Connector and Adapter configuration files and also advice if the above time is acceptable or not.

From Frontier, the logonapplication shouldn't be required when logging in.

Failing exports aren't being picked up as failures in FIM.
To easily reproduce, try exporting to a disabled adapter.

On the home-page, a Once-off timing sits on 1 second remaining after it has run.

Additionally, the expiry timing still shows next run information, however it correctly does not run.

The UI part for the Membership List MultiValue transformation doesn't filter the available left relationship key to only allow for strings.

Additionally the Right Side only supports MultiValued String valuetypes, which is not represented on the UI.

Either the UI or the transformation needs to be updated to reflect these requirements.

A binary values schema key field will fail with the following on import:

Change detection engine import all items failed.
Change detection engine import all items for connector binaryCSV failed with reason An error occurred while evaluating a task on a worker thread. See the inner exception details for information.. Duration: 00:00:00.4550782
Error details:
Unify.Framework.EvaluatorVisitorException: An error occurred while evaluating a task on a worker thread. See the inner exception details for information. ---> System.Data.SqlClient.SqlException: The data types varbinary(max) and image are incompatible in the equal to operator.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult)
at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries)
at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query)
at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
at Unify.Framework.Collections.EnumerableExtensions.ToDictionaryWithKeyClashError[TKey,TValue,TOriginal](IEnumerable`1 originalEnumerable, Func`2 keySelector, Func`2 valueSelector, Action`3 duplicateAction)
at Unify.Product.IdentityBroker.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId, IEnumerable`1 originalEntities)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32& index, Int32 entitiesProcessedSoFar, IEntityChangesReportGenerator`2 reportGenerator)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.<>c__DisplayClass3.<PerformChangeDetection>b__0(IEnumerable`1 page)
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate()
--- End of inner exception stack trace ---
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.CheckForException()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.QueuedConnectorExecutionProcessorEndDecorator.Run()

Adapter statistics on the adapter page seem to only update once an adapter operation (eg. full or delta import) has completed, and not as processing begins.

This cache is only cleared on Completed events for the Adapter jobs. This isn't a problem for smaller operations like Add Entity, Update Entity etc, and hasn't been a problem for Adapter import all due to the speed of LDIF generation in version 4.0.

All this will require is another notification factory to clear the cache on the AdapterEngine::ReflectPage method.

A relational transformation from a connector, with a field A, joined into an adapter and called B through the column mapper will not correctly register changes.

This is because change detection uses the connector context of the relational connector, and in this context a field B does not exist.

Namely, the context comes through the EntityChangeReportGenerator through the connectorEntityDictionary, which later finds itself on the validRightEntityPairs of the RelationalTransformationContribution, which request the non-existant RightKey.

When you attempt to promote or demote an adapter, the service attempts to clear the entity context. This is not necessary and introduces additional overhead, since the moved adapter will continue to use the same identifier for its entities.

Presently, an attempt to do this will result in the error seen in IDB-712 (and delete the adapter you were trying to move completely), but a delete should not be attempted at all in this case.

When an entity is successfully deleted via the adapter, the following appears in the log:

14/Aug/2012 13:59:34
Information
Connector
Request to delete entities from connector.
Request to delete entities Count:1 from connector MOSS 2007.
14/Aug/2012 13:59:43
Information
Connector
Delete entities from connector completed.
Delete entities Count:1 from connector MOSS 2007 deleted 0 entities. Duration: 00:00:08.8592715

The message makes it look like the delete did not succeed.