Identity Broker Forum
Welcome to the community forum for Identity Broker.
Browse the knowledge base, ask questions directly to the product group, or leverage the community to get answers. Leave ideas for new features and vote for the features or bug fixes you want most.
Unclear notification failure of import file generation
When the import file cannot be generated it is unclear what has happened.
Namely the following is what can be seen from the Event Viewer on the FIM side:
The extensible extension returned an unsupported error. The stack trace is: "System.Exception: Error encountered attempting import: System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:59990/IdentityBroker/FIMLDIFAdapter.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size) at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason) at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Unify.Product.IdentityBroker.IdentityBrokerLDIFAdapter.ILDIFAdapter.ImportAll(Guid adapterId) at Unify.Product.IdentityBroker.IdentityBrokerManagementAgentProxy.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData) at Unify.Product.IdentityBroker.IdentityBrokerManagementAgentProxy.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData) Forefront Identity Manager 4.0.3606.2"
whilst the following is what is available in the IDB logs
An exception has occured whilst performing a job for adapter 2656b225-d789-437d-bd32-03ae2f6ed07c job GetTransformedEntities (ParallelGate): System.Data.ConstraintException: Only standard components can be contained inside a multipart component. at Unify.Product.IdentityBroker.EntityMultiPartDistinguishedNameComponentGenerator`2.<>c__DisplayClassa.<GetDistinguishedNameComponents>b__9(IEntityDistinguishedNameComponentGenerator`2 generator) at System.Linq.Enumerable.<>c__DisplayClass12`3.<CombineSelectors>b__11(TSource x) at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() at System.Linq.Buffer`1..ctor(IEnumerable`1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) at Unify.Framework.IO.DistinguishedNameMultiPartComponent.ToString() at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext() at System.Linq.Buffer`1..ctor(IEnumerable`1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) at Unify.Framework.IO.DistinguishedName.GetStringValue() at Unify.Product.IdentityBroker.Repository.AdapterEntityContext.SetEntityRowDetails(__EntityInsertRow row, IAdapterEntity entity, EntityDataContext sourceContext, IEntityCollectionKeyUtility`1 keyUtility) at Unify.Product.IdentityBroker.Repository.KnownEntityContextBase`4.InsertItems(HashSet`1 addedItems, EntityDataContext sourceContext, SqlConnection connection) at Unify.Framework.Data.LinqContextConversionBase`4.SubmitChanges() at Unify.Framework.ParallelGate.ParallelGateJob.RunBase() at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)
Schema provider should mention that it won't override existing fields of the same name
Schema provider should mention that it won't override existing fields of the same name. As this behaviour could be confusing if it's not specified as a piece of text on the provider.
IdB for chris21 seems to take significantly long period of time to perform "Full Import" from chris21 system
The Identity Broker for chris21 seems to take significantly long period of time to perform "Full Import" from chris21 system into Identity Broker Connector and then process/transform the data into the IdB Adapter.
I am not familar with the flow process and the analysing the log file content. Thus the below investigation may not be entirely correct. But the overall time taken for processing data does seems significantly long in Identity Broker for chris21.
The customer has 4999 records/users in chris21.
Analysis 1:
1 - When performing the "Import All Connectors" for all six (6) Connectors took approximately 32 minutes at other times it does seems to take significantly longer time, eg 45 minutes. During this time the CPU and disk accessing activities are at 100%.
Started at
20101215,07:22:14,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Address Connector started.,Normal
Finished at
20101215,07:54:15,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Org Unit Connector completed. Duration: 00:02:34.7343750,Normal
2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 10 minutes. At other times we believe it took significantly longer.
Started at
20101215,07:55:56,Adapter request to get attribute changes from adapter space.,Adapter,Information,Adapter request to get attribute changes from adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal
Finished at
20101215,08:05:46,Adapter get attribute changes completed.,Adapter,Information,Adapter get attribute changes for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 completed. Duration: 00:09:49.8125000,Normal
It has been mentioned that the "Import All Connectors" should take only a few minutes. Thus I would like to request a review of the IdB Connector and Adapter configuration files (attached) as to what could possibly create this behaviour. It is mentioned that the gtrChunkSize="1000" could be in appropriate for the volume of data in chris21.
Analysis 2:
Another example: When performing the "Import All Connectors" for all six (6) Connectors took approximately 14 minutes
Start at
20101213,07:01:22,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Person Connector started.,Normal
Finished at
20101213,07:15:17,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Placement Connector completed. Duration: 00:02:19.6875000,Normal
2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 27 minutes 30 seconds.
Started at
20101213,07:25:53,Request to get all entities for adapter space.,Adapter,Information,Request to get all entities for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal
Finished at
20101213,07:53:34,Adapter get all entities to adapter space completed.,Adapter,Information,Adapter get all entities to adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 returned 4999 entities. Duration: 00:27:41.0937500,Normal
Analysis 3:
"Import all connectors" in IdB took approximately 12 minutes. I do not have the log file
"Full Import" in FIM for chris21 MA took approximately 6 minutes.
Please help to review the Connector and Adapter configuration files and also advice if the above time is acceptable or not.
UnifyLog20101213-extract.csv
UnifyLog20101215-extracted.csv
Make logonapplication optional
From Frontier, the logonapplication shouldn't be required when logging in.
Failing exports report success.
Failing exports aren't being picked up as failures in FIM.
To easily reproduce, try exporting to a disabled adapter.
Once off timing works but appears incorrectly on the UI
On the home-page, a Once-off timing sits on 1 second remaining after it has run.
Additionally, the expiry timing still shows next run information, however it correctly does not run.
Membership List MultiValue transformation incorrect key types.
The UI part for the Membership List MultiValue transformation doesn't filter the available left relationship key to only allow for strings.
Additionally the Right Side only supports MultiValued String valuetypes, which is not represented on the UI.
Either the UI or the transformation needs to be updated to reflect these requirements.
Cannot have a binary valued schema key
A binary values schema key field will fail with the following on import:
Change detection engine import all items failed. Change detection engine import all items for connector binaryCSV failed with reason An error occurred while evaluating a task on a worker thread. See the inner exception details for information.. Duration: 00:00:00.4550782 Error details: Unify.Framework.EvaluatorVisitorException: An error occurred while evaluating a task on a worker thread. See the inner exception details for information. ---> System.Data.SqlClient.SqlException: The data types varbinary(max) and image are incompatible in the equal to operator. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning() at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlDataReader.ConsumeMetaData() at System.Data.SqlClient.SqlDataReader.get_MetaData() at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result) at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method) at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method) at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior) at System.Data.Linq.SqlClient.SqlProvider.Execute(Expression query, QueryInfo queryInfo, IObjectReaderFactory factory, Object[] parentArgs, Object[] userArgs, ICompiledSubQuery[] subQueries, Object lastResult) at System.Data.Linq.SqlClient.SqlProvider.ExecuteAll(Expression query, QueryInfo[] queryInfos, IObjectReaderFactory factory, Object[] userArguments, ICompiledSubQuery[] subQueries) at System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Execute(Expression query) at System.Data.Linq.DataQuery`1.System.Collections.Generic.IEnumerable<T>.GetEnumerator() at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() at System.Linq.Buffer`1..ctor(IEnumerable`1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator() at Unify.Framework.Collections.EnumerableExtensions.ToDictionaryWithKeyClashError[TKey,TValue,TOriginal](IEnumerable`1 originalEnumerable, Func`2 keySelector, Func`2 valueSelector, Action`3 duplicateAction) at Unify.Product.IdentityBroker.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId, IEnumerable`1 originalEntities) at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32& index, Int32 entitiesProcessedSoFar, IEntityChangesReportGenerator`2 reportGenerator) at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.<>c__DisplayClass3.<PerformChangeDetection>b__0(IEnumerable`1 page) at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate() --- End of inner exception stack trace --- at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.CheckForException() at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit() at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities) at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess() at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase() at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run() at Unify.Product.IdentityBroker.QueuedConnectorExecutionProcessorEndDecorator.Run()
Adapter statistics update at the end of an operation
Adapter statistics on the adapter page seem to only update once an adapter operation (eg. full or delta import) has completed, and not as processing begins.
This cache is only cleared on Completed events for the Adapter jobs. This isn't a problem for smaller operations like Add Entity, Update Entity etc, and hasn't been a problem for Adapter import all due to the speed of LDIF generation in version 4.0.
All this will require is another notification factory to clear the cache on the AdapterEngine::ReflectPage method.
Changes are not picked up on relational transformations where names are changed.
A relational transformation from a connector, with a field A, joined into an adapter and called B through the column mapper will not correctly register changes.
This is because change detection uses the connector context of the relational connector, and in this context a field B does not exist.
Namely, the context comes through the EntityChangeReportGenerator through the connectorEntityDictionary, which later finds itself on the validRightEntityPairs of the RelationalTransformationContribution, which request the non-existant RightKey.
Customer support service by UserEcho
