Hi guys!

Is there any way to override the default queries the Cisco UCM IdB connector presents to the Cisco AXL service? There appears to be two modes of operation:
1. sqlTrust=True communicator option set results in the executeSQLQuery AXL request being sent with "select * from <object>".
2. sqlTrust=False communicator option set results in an AXL "get<object>" request being sent for the specified object. SQL queries are performed internally based on the object requested.

Option 1 returns too few attributes, option 2 is too verbose and when executed on End User objects takes as much time to process 1 user as operation mode 1 does in processing all users in my test environment.

If we could pass any SQL query to the AXL service via the executeSQLQuery AXL request the connector could be customised to return exactly what we need.

The attached files demonstrate the functionality and what is actually being processed by the AXL service when these requests come in. sqlTrust=False causes a heap of unnecessary queries being made to the SQL database by the AXL service, slowing down response.

Exporting to an adapter with a space at the end of its object class results in the following exception:

The extensible extension returned an unsupported error.
 The stack trace is:
 
 "System.Exception: Error occurred when attempting to save entity with distinguished name

CN=2

Error:

The given key was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveAdapterBase`1.ConvertValues(String objectClass, IEnumerable`1 convertedValues)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveChangeAdapter.GetConvertedValuesFromSchema(String objectClass, IEnumerable`1 values)
   at Unify.Product.IdentityBroker.LDIFToAdapterEntitySaveChangeAdapter.<Transform>d__3.MoveNext()
   at Unify.Product.IdentityBroker.LDIFAdapterBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)
   at SyncInvokeExportChanges(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
   at Unify.Product.IdentityBroker.IdentityBrokerManagementAgentProxy.ExportEntry(ModificationType modificationType, String[] changedAttributes, CSEntry csentry)
Forefront Identity Manager 4.0.3606.2"

A determination needs to be made about whether FIM adheres to the standard in this regards, specifically as to whether we should validate on the IDB-side for adapter object classes with whitespace.

The logging of change detection is a little bit confusing. Consider the following excerpt:

The Post Processing information is correct, but it makes it look like that no changes should exist, even though the one change has been detected as a difference when processing the page. Perhaps it could be clarified by putting page information in the same format, and/or clarifying what post processing means. Since post processing happens afterwards, it looks like the earlier change was not found to be a match.

When attempting to clear a DN value from an entity on an export, the following error appears:

Attempted export:

delete,Manager,reference,"CN=102994,DC=CHRIS21DEMODC",

When attempting to run a full import on Employee connector the following error is produced:

20110221,03:50:29,Logging Engine,Logging Engine,Information,Log file started.,Minimal
20110221,03:50:29,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Employee Connector started.,Normal
20110221,03:50:31,Change detection engine import all items failed.,Change detection engine,Warning,"Change detection engine import all items for connector Employee Connector failed with reason An error occurred attempting to connect to the SAP system, received : SAP.Connector.RfcLogonException: Incomplete logon data.
at SAP.Connector.Connection.Open()
at Unify.Communicators.SapHRCommunicator.SapHrCommunicator.Open(). Duration: 00:00:01.7187500
Error details:
Unify.Communicators.SapHRCommunicator.SapHrCommunicatorException: An error occurred attempting to connect to the SAP system, received : SAP.Connector.RfcLogonException: Incomplete logon data.
at SAP.Connector.Connection.Open()
at Unify.Communicators.SapHRCommunicator.SapHrCommunicator.Open()
at Unify.Communicators.SapHRCommunicator.SapHrCommunicator.Open()
at Unify.Communicators.SapHRCommunicator.SapHrCommunicator.GetEmployees(String statusType, DateTime startDate, DateTime endDate, String infoType, IEnumerable`1 subTypes, IEnumerable`1 employeeIdRecords)
at Unify.Connectors.SapHrEmployeeConnector.ProcessGetEntities(IEnumerable`1 employeeRecords)
at Unify.Connectors.SapHrEmployeeConnector.GetAllEntities(IStoredValueCollection storedValueState)
at Unify.Framework.ConnectorToReadingConnectorBridge.GetAllEntities(IStoredValueCollection storedValueState)
at Unify.Framework.EventNotifierReadingConnectorDecoratorBase`1.GetAllEntities(IStoredValueCollection storedValues)
at Unify.Framework.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Framework.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.MutexJobDecorator.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal

Branched from Help Desk item 290 - http://development.unifysolutions.net:8080/project/index.php?m=helpdesk&a=view&item_id=290. When baselining a solution, and using a delta import as the first confirming import against an Identity Broker v3 system, a NullReferenceException is thrown across the WCF boundary, stating that a parameter is null. The inner exception in WCF logging reveals a seemingly different parameter is null each time. Temporary work around is to perform a full import for the first confirming import. Refer to help desk item 290 for more history of issue.

Event Viewer gives "A connection was forcibly closed" error.

WCF logging reveals stack trace:

• WCF logging yields stack trace: System.NullReferenceException, Object reference not set to an instance of an object.

Unify.Framework.ValueToLDIFAttrvalRecordAdapter`1.CreateAttrvalSpecs(TKey key, IValue value) in S:\SVN\Framework\Core\Release\v3.0.0\Source\Adapter\Unify.Framework.Adapter.LDIF\ValueToLDIFAttrvalRecordAdapter.cs:line 74
Unify.Framework.AdapterEntityChangeIndicatorToLDIFComponentAdapter.GetLDIFModSpec(IAdapterEntityAttributeChangeValue value) in S:\SVN\Framework\Core\Release\v3.0.0\Source\Adapter\Unify.Framework.Adapter.LDIF\AdapterEntityChangeIndicatorToLDIFComponentAdapter.cs:line 131
System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
Unify.Framework.LDIFChangeModify.WriteComponentDetails(TextWriter writer) in S:\SVN\Framework\Core\Release\v3.0.0\Source\IO\Unify.Framework.IO.LDAP\LDIF\LDIFChangeModify.cs:line 49
Unify.Framework.LDIFChangeRecordBase.WriteComponent(TextWriter writer) in S:\SVN\Framework\Core\Release\v3.0.0\Source\IO\Unify.Framework.IO.LDAP\LDIF\LDIFChangeRecordBase.cs:line 46
Unify.Framework.LDIFComponentFileGenerator`1.GenerateFile(TextWriter writer, IEnumerable`1 entries) in S:\SVN\Framework\Core\Release\v3.0.0\Source\IO\Unify.Framework.IO.LDAP\LDIF\LDIFComponentFileGenerator.cs:line 49
Unify.Framework.LDIFAdapter.c_DisplayClass5`1.b_4(Stream stream) in S:\SVN\Framework\Core\Release\v3.0.0\Source\Adapter\Unify.Framework.Adapter.Remoting\LDIFAdapter.cs:line 169
Unify.Framework.LazyEvaluationStream.Evaluate(Object obj) in S:\SVN\Framework\Core\Release\v3.0.0\Source\IO\Unify.Framework.IO\LazyEvaluationStream.cs:line 119

There is currently a workaround in place for IDB-731 due to the expression visitor handling when GetMultiKeyValue is called on fields that are not populated. This needs to be eventually updated so that it doesn't throw a null reference exception in this case.

A null reference exception can also be produced by calling doing an "entity.GetMultiKeyValue(key) != null" check. This can be reproduced by readding this check to EntityCompositeKeyDistinguishedNameStaticRelationValueAdapterBase.Transform and reattempting the scenario described in IDB-731.

The PS script for importing all users from 0365 sometimes errors with the following:

Import all entities from connector failed.
Import all entities from connector Office 365 Staff Licenses failed with reason An unexpected error occurred.. Duration: 00:00:06.8594919
Error details:
Microsoft.Online.Administration.Automation.MicrosoftOnlineException: An unexpected error occurred.
at Unify.Product.IdentityBroker.PowerShellConnector.<GetEntitiesInScript>d__a.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.Collections.ActionOnExceptionEnumerator`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ProduceAutoPages>d__a`1.MoveNext()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c_DisplayClass29.<Run>b_27()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)

Original discussion with product team suggested that since that function returns IEnumerable result that if the connection were to drop part way through the call would not complete. Modified script to define an array for the result and populate that. Sometimes the array populates sometimes it fails. Normally failure stops the script, but sometimes it gets seen as an empty result and the 45000 users are wiped from the IdB connector - which flows deletes to FIM CS. Next successful or partially successful load puts them back and they rejoin, but this should not be happening.

The portion of the PS script that connects and gets users has been run as a standalone from the server and did not drop out or fail, but running from IdB seems to be consistently flakey.

I understand that this is more likely to be an MSOL or PS issue, but would appreciate any assistance around how to troubleshoot the unexpected errors or any suggestions for possible workarounds.

There is a race condition in Identity Broker that could cause incorrect deletes on adapter delta imports.

Imagine I have done an Import All on a connector which returned 20 changes.

I immediately follow with a clear all operation, which clears the connector and adapter context, as well as any processed changes.

Imagine that 10 changes have not yet been processed (possible with very large change sets).

These changes will then be picked up by the change processor, and registered as changes.

If I follow up with a Delta import from FIM, IDB will calculate these 10 orphaned changes as deletes.

We can handle the currently unprocessed changes by clearing the remaining untouched changes.

For the changes in memory, either the count of changes processed on each cycle will need to be throttled, or a conditional might be added to wrap each cycle, or this potential race condition might just need to be highlighted on the UI/documentation.

Relational transformations with optional DN generators can break on change detection if the optional DN generator is configured.

The factory being used for this does not add a contribution for the target of the DN Generator attribute mapper on ApplyChainedChangeDetection, which can result in a NullReferenceException during change detection.

This appears in the logs as

Changes register item processing on failed.
Changes register item processing on connector CSV Connector failed with reason Object reference not set to an instance of an object.. Duration: 00:01:47.7557745
Error details:
System.NullReferenceException: Object reference not set to an instance of an object.
at Unify.Product.IdentityBroker.ChainedTransformationChangeProcessor.PublishChange(IEnumerable`1 changedEntities, DateTime changeProcessTime, ICollection`1 changeRecords) in S:\hg\Product\IdentityBroker\Master-Changes\Source\Adapter\Unify.IdentityBroker.Adapter.EntityTransformation\ChainedTransformationChangeProcessor.cs:line 132
at Unify.Product.IdentityBroker.ChainedTransformationChangeProcessor.ProcessChangeReport(IDictionaryTwoPassDifferenceReport`4 changesReport, DateTime changeProcessTime) in S:\hg\Product\IdentityBroker\Master-Changes\Source\Adapter\Unify.IdentityBroker.Adapter.EntityTransformation\ChainedTransformationChangeProcessor.cs:line 109
at Unify.Product.IdentityBroker.ChangeReportProcessor.<>c__DisplayClass12.<ProcessCurrentReport>b__11(ITransformationChangeProcessor processor) in S:\hg\Product\IdentityBroker\Master-Changes\Source\ChangeDetection\Unify.IdentityBroker.ChangeDetection\ChangeReportProcessor.cs:line 159
at Unify.Framework.Visitor.<>c__DisplayClass1`1.<Visit>b__0(T item, Int32 index) in S:\hg\Framework\Core\a\Source\DesignPatterns\Unify.Framework.DesignPatterns\Visitor.cs:line 23
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor) in S:\hg\Framework\Core\a\Source\DesignPatterns\Unify.Framework.DesignPatterns\Visitor.cs:line 47
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`1 visitor) in S:\hg\Framework\Core\a\Source\DesignPatterns\Unify.Framework.DesignPatterns\Visitor.cs:line 23
at Unify.Product.IdentityBroker.ChangeReportProcessor.ProcessCurrentReport(IEnumerable`1 adapterTransformationProcessors, IDictionaryTwoPassDifferenceReport`4 differenceReport, DateTime changeTime) in S:\hg\Product\IdentityBroker\Master-Changes\Source\ChangeDetection\Unify.IdentityBroker.ChangeDetection\ChangeReportProcessor.cs:line 158
at Unify.Product.IdentityBroker.ChangeReportProcessor.CreateAndProcessReport[T](IEnumerable`1 adapterTransformationProcessors, IEnumerable`1 sourceEnumerable, DateTime changeTime, Action`2 addAction) in S:\hg\Product\IdentityBroker\Master-Changes\Source\ChangeDetection\Unify.IdentityBroker.ChangeDetection\ChangeReportProcessor.cs:line 147
at Unify.Product.IdentityBroker.ChangeReportProcessor.ProcessReport(IChangeReportProcessingRequest request) in S:\hg\Product\IdentityBroker\Master-Changes\Source\ChangeDetection\Unify.IdentityBroker.ChangeDetection\ChangeReportProcessor.cs:line 118

Additionally the Time Relational transformation adds a contribution for the target twice, resulting in a argument exception.