When attempting a confirming delta import delta sync after a successful export, the following exception is thrown in the new Aurion demo environment (FIM, using Windows 2k8). This is thrown as a stopped-extension-dll-exception in FIM. According to Patrick, this error has not been seen before in other environments. It has been remedied by performing a confirming full import full sync instead.

The extensible extension returned an unsupported error.
The stack trace is:

"System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:59990/IdentityBroker/RemoteXmlAdapter.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
— End of inner exception stack trace —
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
— End of inner exception stack trace —

Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at 0:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Unify.Framework.ILDIFAdapter.ImportChanges(Guid adapterId)
at Unify.Framework.IdentityBrokerManagementAgentProxy.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData)
at Unify.Framework.IdentityBrokerManagementAgent.GenerateImportFile(String fileName, String connectTo, String user, String password, ConfigParameterCollection configParameters, Boolean fFullImport, TypeDescriptionCollection types, String& customData)
Forefront Identity Manager 4.0.3531.2"

WCF logging should be enabled to diagnose the true error.

All rows in the view have a value in EMPLID - The attribute used for the DN.

Warning - The following occurred in module: Adapter

Adapter import all entities for adapter failed.
Adapter import all entities for adapter e14dde8a-73ff-4723-acbd-b34f5cb38ac0 failed with reason One or more errors occurred.. Duration: 00:00:01.0661066
Error details:
System.AggregateException: One or more errors occurred. ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: attributeValue
   at Unify.Framework.IO.DistinguishedNameComponent..ctor(DNAttributeType attributeType, String attributeValue)
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Unify.Framework.IO.DistinguishedName..ctor(IEnumerable`1 components)
   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.DistinguishedName(TEntity entity)
   at Unify.Product.IdentityBroker.DistinguishedNameGeneratorTransformation.Transform(IEntity sourceValue)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
   at System.Linq.Parallel.QueryTaskGroupState.QueryEnd(Boolean userInitiatedDispose)
   at System.Linq.Parallel.AsynchronousChannelMergeEnumerator`1.MoveNextSlowPath()
   at System.Linq.Parallel.QueryOpeningEnumerator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at Unify.Framework.AggregateArrayedValueAdapter`2.Transform(TElement[] baseValue)
   at System.Linq.Enumerable.<>c__DisplayClass12`3.<CombineSelectors>b__11(TSource x)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.ActionOnExceptionEnumerator`1.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at Unify.Framework.Collections.ActionOnExceptionEnumerator`1.MoveNext()
   at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at System.Linq.Enumerable.<ConcatIterator>d__71`1.MoveNext()
   at Unify.Framework.IO.LDIFComponentFileGenerator`1.GenerateFile(TextWriter writer, IEnumerable`1 entries)
   at Unify.Product.IdentityBroker.LDIFAdapterBase.<>c__DisplayClass14`1.<CreateLDIFComponentStream>b__13(Stream stream)
   at Unify.Framework.IO.LazyEvaluationStream.Evaluate(Object obj)
---> (Inner Exception #0) System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: attributeValue
   at Unify.Framework.IO.DistinguishedNameComponent..ctor(DNAttributeType attributeType, String attributeValue)
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Unify.Framework.IO.DistinguishedName..ctor(IEnumerable`1 components)
   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.DistinguishedName(TEntity entity)
   at Unify.Product.IdentityBroker.DistinguishedNameGeneratorTransformation.Transform(IEntity sourceValue)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---

---> (Inner Exception #1) System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: attributeValue
   at Unify.Framework.IO.DistinguishedNameComponent..ctor(DNAttributeType attributeType, String attributeValue)
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Unify.Framework.IO.DistinguishedName..ctor(IEnumerable`1 components)
   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.DistinguishedName(TEntity entity)
   at Unify.Product.IdentityBroker.DistinguishedNameGeneratorTransformation.Transform(IEntity sourceValue)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---

---> (Inner Exception #2) System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: attributeValue
   at Unify.Framework.IO.DistinguishedNameComponent..ctor(DNAttributeType attributeType, String attributeValue)
   at Unify.Product.IdentityBroker.FieldTemplateDistinguishedNameComponentExecutor`2.Get(TEntity entity)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Unify.Framework.IO.DistinguishedName..ctor(IEnumerable`1 components)
   at Unify.Product.IdentityBroker.TemplateDistinguishedNameExecutor`2.DistinguishedName(TEntity entity)
   at Unify.Product.IdentityBroker.DistinguishedNameGeneratorTransformation.Transform(IEntity sourceValue)
   at System.Linq.Parallel.PartitionedDataSource`1.ListContiguousIndexRangeEnumerator.MoveNext(T& currentElement, Int32& currentKey)
   at System.Linq.Parallel.PipelineSpoolingTask`2.SpoolingWork()
   at System.Linq.Parallel.SpoolingTaskBase.Work()
   at System.Linq.Parallel.QueryTask.BaseWork(Object unused)
   at System.Threading.Tasks.Task.Execute()<---

Hi Product team,

Import changes are not being processed to adapters when the adapter is disabled.
Should this be processing regardless of if the adapter is enabled/disabled?

Happy to provide screenshots and logs if required

Hi Gents,

Interesting one - we're consistently seeing the below error while running delta imports from IDB into FIM, on the LDAP Groups partition.

 "System.InvalidOperationException: Sequence contains more than one element
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.ImportProxy.ChangesToAttributeChange(ObjectModificationType modType, IGrouping`2 changes, AttributeType desiredType)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at Unify.Product.IdentityBroker.ExtensionMethods.Visit[TSource](IEnumerable`1 source, Action`1 visitor)
   at Unify.Product.IdentityBroker.ImportProxy.EntryToDeltas(SearchResultEntry searchEntry)
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at Unify.Product.IdentityBroker.ExtensionMethods.Take[TSource](IEnumerator`1 source, Int32 count, IList`1& items)
   at Unify.Product.IdentityBroker.ExtensionMethods.<Page>d__0`1.MoveNext()
   at Unify.Product.IdentityBroker.ImportProxy.Import(GetImportEntriesRunStep importRunStep)
Forefront Identity Manager 4.1.3646.0"

Note this is the same error noted in UNINWCSTL-1, however this is not a DN issue.

Nothing is noted in the IDB logs around this, although I have attached a Diagnostic log taken while this occurred.

In investigating, i noted that initially the delta's seemed to be working fine, and then started failing from a specific point - notably 2nd Oct, 2:05 am.

I looked at the IDB ChangeLog table for changes on this adapter around this time, abd found some interesting entries:

ChangeLogKey, AdapterId, ChangeType, ChangeTimestamp, TargetDistinguishedName, NewDistinguishedName, Changes
1868584, 65F663DB-076A-4630-92C2-6FE82BD5FE95, 2, 2015-10-01 16:14:14.1352367, UID=e5453054-7ba4-460b-8222-a0cc83dc0cca,OU=LDAPGroups,DC=IdentityBroker, NULL, LDAPGroupObjectClass:-;LDAPGroupCn:-;LDAPGroupGidNumber:-;LDAPGroupObjectDN:-;LDAPGroupDescription:-
1868610, 65F663DB-076A-4630-92C2-6FE82BD5FE95, 3, 2015-10-02 01:20:09.8420583, UID=b3ac6d1e-025c-46cb-ab31-56d3169af7dc,OU=LDAPGroups,DC=IdentityBroker, NULL, memberUid:+ c1234580
1868611, 65F663DB-076A-4630-92C2-6FE82BD5FE95, 3, 2015-10-02 01:20:09.8420583, UID=243a5553-04c1-4eec-92f6-dab5b2f7a9d5,OU=LDAPGroups,DC=IdentityBroker, NULL, memberUid:+ c1234580
1888631, 65F663DB-076A-4630-92C2-6FE82BD5FE95, 2, 2015-10-02 16:14:45.9843075, UID=e8fbc6c2-3b28-42e7-8943-a148a8351af2,OU=LDAPGroups,DC=IdentityBroker, NULL, LDAPGroupObjectClass:-;LDAPGroupCn:-;LDAPGroupGidNumber:-;LDAPGroupObjectDN:-;LDAPGroupDescription:-

I sampled around either side, but if you note the 2 entries in the middle are both just before the delta's started failing, and are adding a value to a multivalue field. This seems consistent with the error text aswell.

I've attached full config aswell, let me know if more is provided.

When the Identity Broker is accessed, all the Connectors are currently Red and the errors in the log are as follows:

Change detection engine import all items failed.
Change detection engine import all items for connector Chris21 POS failed with reason Line type is recognized but not valid Command. Line: GTR:cbr="logon",logonapplication="breoption",user="REDACTED",password="REDACTED",credentialsrequired="Y",error01="BRE029:Unable to access interface. Contact your Security Officer.",error02="BRE103:Access has been denied.",status="fail". Duration: 00:00:01.4218568
Error details:
System.IO.InvalidDataException: Line type is recognized but not valid Command. Line: GTR:cbr="logon",logonapplication="breoption",user="REDACTED",password="REDACTED",credentialsrequired="Y",error01="BRE029:Unable to access interface. Contact your Security Officer.",error02="BRE103:Access has been denied.",status="fail"
at Unify.Product.IdentityBroker.Chris21Agent.CheckLoginResult(IChris21Record loginResultRecord, Action`1 pre)
at Unify.Product.IdentityBroker.Chris21Agent.GetLoginToken(IChris21ConnectorInformation connectorRequest)
at Unify.Product.IdentityBroker.Chris21Agent.List(IEnumerable`1 keyColumnNames, IChris21ConnectorInformation connectorRequest)
at Unify.Product.IdentityBroker.Chris21ConnectorBase.GetAllEntities(IStoredValueCollection storedValueState)
at Unify.Product.IdentityBroker.ConnectorToReadingConnectorBridge.GetAllEntities(IStoredValueCollection storedValueState)
at Unify.Product.IdentityBroker.EventNotifierReadingConnectorDecoratorBase`1.GetAllEntities(IStoredValueCollection storedValues)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c_DisplayClass29.<Run>b_27()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)

Looks like a permissions issue at this point with the BRE, but we need to know where we should be looking.

When doing a full import on a placeholder connector with no entities, the following error is returned. This should probably be captured, reported and not classed as an error (its not actually broken, its just a fact of data).

Timestamp Severity Source Module Message
14/01/2011 9:33:00 AM Warning Change detection engine import all items failed. Change detection engine "Change detection engine import all items for connector Organisation Unit mapping failed with reason An error occurred while evaluating a task on a worker thread. See the inner exception details for information.. Duration: 00:00:01.8906734
Error details:
Unify.Framework.EvaluatorVisitorException: An error occurred while evaluating a task on a worker thread. See the inner exception details for information. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
at Unify.Repository.EntityExpressionQueryVisitor`3.CreateMultiValueRangeExpression(IMultiKey`1 multiKey, IEnumerable`1 sourceValueList, IQueryable`1 sourceQueryable)
at Unify.Repository.EntityExpressionQueryVisitor`3.VisitMethodWhereWithMultiValueEnumerableContains(MethodCallExpression expression)
at Unify.Repository.EntityExpressionQueryVisitor`3.VisitMethodCall(MethodCallExpression m)
at Unify.Framework.ExpressionVisitorBase`11.Visit(Expression exp)
at Unify.Framework.LinqContextConversionExpressionVisitorBase`5.ConvertExpression(Expression partitionExpression)
at Unify.Framework.LinqQueryConversionProvider`5.GetOrderedQuery(Expression businessExpression, IQueryable`1 sourceQueryable, TContext sourceContext)
at Unify.Framework.LinqWhereQuery`5.EvaluateQuery(TContext context)
at Unify.Framework.LinqWhereQuery`5.GetEnumerator()
at System.Linq.Enumerable.ToDictionaryTSource,TKey,TElement(IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Linq.Enumerable.ToDictionaryTSource,TKey,TElement(IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
at Unify.Repository.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId, IEnumerable`1 originalEntities)
at Unify.Repository.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId)
at Unify.Framework.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32 index)
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate()
— End of inner exception stack trace —
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.CheckForException()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`2 visitor, Int32 maxThreads)
at Unify.Framework.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Framework.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Framework.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.MutexJobDecorator.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)"

I'm building a Powershell based connector that integrates with a system with a few thousand records. Each record has a User Name which must be unique, however not all users are to be imported into FIM. The IdB Powershell MA has to be provisioned to, so as part of that a unique User Name has to be generated.

This has lead me down the route of developing extension code to perform an LDAP lookup on the Identity Broker 5 LDAP interface to check whether a user name is available or not (as I can't compare with the metaverse since the end system isn't fully represented there). I've found that the string attributes exposed by the Identity Broker 5 LDAP interface have an Equality rule of 'caseExactMatch'. I can't guarantee what the case is of data in that end system unless I manipulate it on import to Identity Broker (have my Powershell MA cast to Upper case for example). As a result, any query against the LDAP interface is going to be complicated by the fact that I can't merely search for objects with (USNAME=Svcfimadmin) since that won't give me a result is the USNAME is 'svcfimadmin'. I would need to use (USNAME=svcfimadmin) to get any result. Instead I'll likely need to do something like get the first character and import all objects with that name, grab all the User Names and manipulate them in code.

That could be avoided if I could set the comparison to be case insensitive. I can't really think of any cases at all where I'd want a comparison to be case sensitive, and indeed on review of the issue it appears that Active Directory and other systems typically have comparisons set as case insensitive (a few attributes are sensitive). Many end systems will ignore the case for attributes like Account Names etc too.

I imagine the case sensitive could make more difficult in other situations where someone wants to make use of the LDAP interfaces to perform searches and the like.

I'm not sure how feasible it would be to add a 'case insensitive' string type, or provide a flag that could be set on an attribute basis to determine how searches via the LDAP interface are handled.

While the next run information is always correct, the CheckDue function of exclusion timings starts returning true if an exclusion period ends, rather than at the appropriate next run time. This is because the internal timing's NextRun property is not updated when an exclusion period is in effect, and as such will retain a NextRun value in the past, causing it to run on the next CheckDue call outside of the exclusion.

The definition about "console" is a bit unclear. I had no clue until Tony explained what it is referring to,
"it writes to the console if you're debugging Identity Broker through the console at:

C:\Program Files\UNIFY Solutions\IdentityBroker\Services\Unify.Service.Connect.Debug.exe"

I think it could be a good idea to add some detailed information on the page just to make it clear for everyone.

Cheers,
Sam