Hi Gents,

Found an interesting one i believe. We have been seeing the following behavior in the PROD environment out at QDET.

1. A delta import from IDB fails with due to the configured operation timeout being exceeded.

2. After the failure, no subsequent delta imports are triggered via EVB (using the IDB changes plugin as a trigger).

3. Manually triggering a delta import (with an extended timeout setting), clears the issue once the import completes. Further imports are triggered ok via EVB. (Running a Full Import also works here)

I believe the following is occurring:

1. Delta fails due to timeout in FIM.

a. The following is in the windows event log:

The extensible extension returned an unsupported error.

The stack trace is:

"Unify.Product.IdentityBroker.LdapOperationException: Operation timed out.

at Unify.Product.IdentityBroker.LdapConnection.SendRequest(ILdapRequest request)

at Unify.Product.IdentityBroker.LdapConnectionProxy.<SearchRequestPaged>d__6.MoveNext()

at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()

at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()

at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()

at Unify.Product.IdentityBroker.ExtensionMethods.Take[TSource](IEnumerator`1 source, Int32 count, IList`1& items)

at Unify.Product.IdentityBroker.ExtensionMethods.<Page>d__0`1.MoveNext()

at Unify.Product.IdentityBroker.ImportProxy.Import(GetImportEntriesRunStep importRunStep)

Forefront Identity Manager 4.1.3627.0"

b. IDB log shows the LDAP connection, followed by a connection aborted error after the timeout.

2. For some reason, the delta operation is not actually terminated within IDB. Thus when EVB tries to check for changes, the operation returns false as it thinks an import is in progress.

This is evidenced by the following. On manually running a delta import from FIM, IDB showed on the adapter that the operation had a duration of 15 hours:

I was able to trace this time back to the exact time when the first delta import operation had timed out:

So essentially it seems that because IDB thinks it's still going, it stops the IDB changes plugin from initiating subsequent operations.

We have a workaround for now - obviously increase timeouts, and run manual operations when needed.

Was running a bulk export of 966 users to Office 365 using the Graph API connector, the MIM MA finished running in approx 2mins 30secs however the Identity Broker save entities process continued running for an additional 20mins.

MIM received an ma-extension-error for each users object with unexpected error has occurred however all the user objects were successfully created in Office 365. Using version 5.0.4 for both IDB and the FIM connector

Found the following error entries in the IDB logs which are timestamped approx 20secs after the MIM MA finished its run.

Entry 1

Handling of LDAP Bulk Update request.
Handling of LDAP Bulk Update request received from user admin on connection 127.0.0.1:55046 failed with error "Cannot access a disposed object.
Object name: 'System.Net.Sockets.NetworkStream'.". Duration 00:01:44.1525241.

Entry 2

An error occurred on client from 127.0.0.1:55046. More details:
Internal Server Error #11: System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Sockets.NetworkStream'.
at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)
at Unify.Product.IdentityBroker.LDAPConnection.<RespondToMessageAsync>d__33.MoveNext()

I'm currently running another full sync to generate an additional bulk export to test further.

With IDB5.0.3 configured for the Extensible Connectivity 2 Management Agent using the dll from the Unify Identity Broker for Microsoft FIM 5.0.3 installer.

Create an export run profile with an object limit.

Run the export run profile

All pending export objects are exported and when the run completes, the status message indicates that object limit was reached. See screenshots.

When performing a delta import, the MA performs a change log request for the last seen change log entry. If the change log entry has been cleared or truncated so the last seen change log entry no longer exists the delta operation stops and the following exception message is logged by FIM

The extensible extension returned an unsupported error.
 The stack trace is:
 
 "System.Exception: Changelog corruption detected. Could not find changelog entry with change number 48.
   at Unify.Product.IdentityBroker.LdapConnectionProxy.PartitionDeltaRequestPaged(String partitionDN, Int64 lastChangeNumber, Int32 pageSize) in s:\HG\Product\FIMMA\Working\Source\Unify.IdentityBroker.FIMAdapter\LdapConnectionProxy.cs:line 189
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at Unify.Product.IdentityBroker.ExtensionMethods.Take[TSource](IEnumerator`1 source, Int32 count, IList`1& items)
   at Unify.Product.IdentityBroker.ExtensionMethods.<Page>d__0`1.MoveNext()
   at Unify.Product.IdentityBroker.ImportProxy.Import(GetImportEntriesRunStep importRunStep) in s:\HG\Product\FIMMA\Working\Source\Unify.IdentityBroker.FIMAdapter\ImportProxy.cs:line 113
Forefront Identity Manager 4.1.3599.0"

The MA should perform some kind of discovery procedure at this point. Also ensure that a last seen change number of 0 (changeLogKey auto increment starts at 1) is covered.

Generating an FIM Packaged Management Agent for an adapter and selecting a Save Location of FIM Instance under the Save Options the IdB creates three new folders

.\UIShell\XMLs\PackagedMAs

under the path where the packaged MA file should be created

%Program Files%\Microsoft Forefront Identity Manager\2010\Synchronization Service\UIShell\XMLs\PackagedMAs

which in the test environment resulted in the xml being placed in the following directory:

C:\Program Files\Microsoft Forefront Identity Manager\2010\Synchronization Service\UIShell\XMLs\PackagedMAs\UIShell\XMLs\PackagedMAs

I've created a single Powershell Connector and Adapter as per the attached documents. If I attempt to create a new MA in FIM 2010 R2, I get the following error after entering my credentials and attempting to proceed from the connectivity portion of the FIM MA config:

Unable to retrieve schema. Error: An anchor attribute defined by the extension must not be of type Reference or Boolean. A multivalued attribute defined by the extension must not be of type Boolean.

In the event log I see:

Log Name: Application
Source: FIMSynchronizationService
Event ID: 6801
Level: Error


The extensible extension returned an unsupported error.
 The stack trace is:
 
 "System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Unify.Product.IdentityBroker.SchemaEntryToAttributeTypeAdapter.Transform(String schemaEntry)
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
   at Unify.Product.IdentityBroker.LdapConnectionProxy.get_Schema()
   at Unify.Product.IdentityBroker.UnifyLdapConnectorTypeProxy.GetSchema(KeyedCollection`2 configParameters)
Forefront Identity Manager 4.1.3627.0"

My DN is defined as UID=UFYGuid

That UFYGuid is not one of the Boolean attributes (it is type GUID).

If I change the schema of the connector so that the two Boolean attributes are of type string, then I can proceed to create the MA without any other changes.

The following is the exception thrown when the schema of an adapter does not match the generated XMA schema from FIM.

The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveAdapterBase`1.<>c__DisplayClassf.<ConvertValues>b__b(<>f__AnonymousType0`2 item)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveChangeAdapter.GetConvertedValuesFromSchema(String objectClass, IEnumerable`1 values)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveChangeAdapter.<Transform>d__3.MoveNext()
   at Unify.Product.IdentityBroker.LDIFAdapterBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)
   at SyncInvokeExportChanges(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
   at Unify.Product.IdentityBroker.IdentityBrokerManagementAgentProxy.ExportEntry(ModificationType modificationType, String[] changedAttributes, CSEntry csentry)
Forefront Identity Manager 4.0.3606.2"

While it is often useful to be able to filter records at the connector level, it would be handy to be able to do this at the adapter level as well.

An example is one which occurred at DEEWR where I needed to exclude all CLAIM objects from the Claims adapter where the IsDerived flag was set to 1. Since this was a single adapter/single connector configuration, this was easily achieved at the connector level. However, the side-effect was that this same connector was also being used in another adapter with a different base connector ... to derive group membership style reference properties for a PERSON object. There was one such transformation that needed ALL claims objects (i.e. inclusive of the IsDerived=1 claims) in order to calculate the membership. In my case I had already decidedd to discontinue using the group transformation and achieve the requirement a different way, but this could have forced me down the path of multiple connectors (10s of 1000s of rows) for the same data source.

If this is not already achievable (without configuring multiple connectors for the same data source), please consider this as a feature request.

In Framework Core:

A better error message could be provided when a null value has been set in an entity at the connector level. For example when the value is taken from the entity repository and that value turns out to be null.

Error appears in:
Unify.Framework.IO.LDAP\LDIF\LDIFSafeStringFilter.IsSafe
as an argument null exception.

However the value is retrieved in:
Unify.Framework.Adapter.LDIF\AdapterEntityToLDIFAttrvalRecordAdapter.GetLDIFAttrvalSpec

Because of the use of the GetObjectClass method which uses an object's DN to work out which schema it should use, new objects are not found for added entities exported to the adapter. The following error is thrown:

For adds, the adapter should be checking for the presence of the objectClass attribute exclusively (which is present in the LDIF).