The image node for a composite adapter renders the image node for any contained adapters redundant. However, when the adapter node is excluded from the adapter xml the Identity Broker service fails to start - and an error message is displayed stating that the node is mandatory. When an empty node is added the service still fails to start, but another exception is raised instead. The service will only start when a non-empty image node is included in the configuration.

I noticed the presence of these images (which are not displayed in the Identity Broker Management Studio) when building xsl stylesheets to document the Identity Broker configuration for DEEWR. Not only do they add unnecessary bulk to the configuration, but they can lead to irrelevant images persisting and being accidentally deployed (the images that I found were actually carried over from another project).

As a work-around I have generated a dummy binary string from the smallest PNG file I could find, and used that for all of the adapters that make up my composite adapter.

I believe the image node should only be mandatory for an adapter if it is NOT part of a composite adapter. Given that composite adapters are now likely to be the norm rather than exception, certainly when used with FIM, then this issue is likely to affect more deployments. I doubt whether anyone who has deployed a composite adapter actually realizes what images lay hidden in the nested adapter configurations they have deployed.

After deploying UNIFY Identity Broker for Microsoft FIM v3.0.0 (x86).msi from https://unifysolutions.jira.com/wiki/display/SUBIDBFIM/Downloads and completing my IdB 3.0.6 DEEWR configuration, I was ready to create an instance of the IdB FIM xMA ...

After installing using the default options, I found that:

(a) PDF files cannot normally be opened on a server - we might want to think of an alternative format that can say be opened in Wordpad which is (almost) guaranteed to be there ... I got around this by mailing myself the file from my DEEWR email account which was the only way I could get hold of the file over a VPN. Of course I could have installed this to my XP laptop ...

I've noticed that in general the exception reporting is very good at identifying the cause of a problem, however in the following scenarios it is not:

• when 2 field nodes in the same entitySchema have the same name (obvious error but easy to make when hand crafting xml) the exception raised in the Application Event log (when the Identity Broker fails to start) is simply "The parameter is incorrect". There is no evidence as to what was the problem, nor whether it was an adapter or connector issue;
• when configuring a Relation.Group.Composite transformation, I accidentally included a key reference to a column of the base connector in a dnComponent, instead of a column defined by the RelationshipConnectorID - in this case the IdB service started OK, but when attempting an Adapter entity search an exception "Adapter get all entities for adapter xxx failed with reason 'Specified argument was out of the range of valid values. Parameter name: attributeValue'". Of course there is no such parameter "attributeValue" exposed in the adapter config, so I presume this is internal to IdB. While the text makes sense once you know the problem is with your DN, trying to track this problem down in a composite adapter with many adapters configured is quite problematic.

I'm sure if I was diligent in logging more of these in JIRA I would come up with a few more, so maybe we can keep reusing this JIRA item in the future ... but right now the above 2 are a good start

Export performance is likely to be a major bottleneck for Origin during the "initial load" sync process where FIM is writing back network account and email address to SuccessFactors (SAP HR). Current performance metrics from DEV are not a great guide due to the limited number of employees loaded into FIM for that environment (<50), however the last sizeable batch was 39 user updates in 00:09:50 (a rate of 1 every 15 seconds!!!). When it comes to a full set we are talking upwards of 33K users requiring updates - and at the above rate we will be looking at 8250 minutes, or 137.5 hours, or 5.7 days.

In an attempt to head this problem off in advance of it coming to the attention of the testers, I am thinking that we may have to rethink the way we are applying updates for the initial load.

See linked issue for idea as to how the PS connector architecture might be improved for exports in future.

Experiencing issues clearing out the start/end dates defined in the schema of my WSS list. FIM export flow rule is set to "allow nulls" which means any existing values in the WSS list should be cleared - but there is no change and I get the dreaded "exported-not-reimported" error. Have confirmed that WSS list fields allow nulls.

Removing schedule returns an error.

Can you tell me if the CSV connector is now rolled into the std broker install (like the placeholder connector). I tried installing the one in IDB-74 and it said a newer version was already installed.

If not, let me know where to get it.

Thanks

At QDET in a mirror production environment we recently saw an issue around the changes register of Identity Broker. A full import was running on a large connector (500000~ users) where 40 changes in the target system were present. The Identity Broker Changes Plug-in detected a change during the import process and kicked off two delta imports into FIM. Possibly due to poor infrastructure or heavy database server load, the delta imports failed - logs below:

Subsequent delta imports into FIM were successful, however, the import returned 0 results. This may suggest that the changes register is cleared regardless of the return state of a delta import. A full import into FIM was required to pick up the changed users.

Attempts to replicate this behaviour have so far been unsuccessful - the database is no longer timing out.

This is not a pressing issue as we have been unable to replicate yet (and may not be able to), but it would be worth investigating (for this or future versions) to prevent this from occurring again as the behaviour does incur operational intervention.

In IDaaS the full import of a chris21 connector is receiving an error as logged in the log file as follows. It seems to point to a problem with SQL Error code 40174.

The cause was identified as a result of throttling on Azure DB.

Solution: Increasing the DTU capacity on the DB in Azure.

Connector Processing started for connector chris21 - POS Form (page 15)",Normal
20151023,04:20:58,UNIFY Identity Broker,List,Information,Next chunk is [1000] sub total [19000]. Duration: 00:00:10.2657290,Normal
20151023,04:21:00,UNIFY Identity Broker,Change detection engine,Information,"Started processing changes register items.
Started processing changes register items for connector chris21 - POS Form.",Normal
20151023,04:21:01,UNIFY Identity Broker,Change detection engine,Information,"Changes register item processing completed.
Changes register item processing on connector chris21 - POS Form completed.  Duration: 00:00:01.1719116",Normal
20151023,04:21:07,UNIFY Identity Broker,Connector Processor,Information,"Connector processing failed.
Connector Processing page 15 for connector chris21 - POS Form failed with reason The service has encountered an error processing your request. Please try again. Error code 40174.
A severe error occurred on the current command.  The results, if any, should be discarded.. Duration: 00:00:09.7625532. 
Error details:
System.Data.SqlClient.SqlException (0x80131904): The service has encountered an error processing your request. Please try again. Error code 40174.
A severe error occurred on the current command.  The results, if any, should be discarded.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
   at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
   at System.Data.SqlClient.SqlDataReader.Read()
   at System.Data.Linq.SqlClient.ObjectReaderCompiler.ObjectReaderBase`1.Read()
   at System.Data.Linq.SqlClient.ObjectReaderCompiler.ObjectReader`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId, IEnumerable`1 originalEntities)
   at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32& index, Int32 entitiesProcessedSoFar, IEntityChangesReportGenerator`2 reportGenerator)
   at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.<>c__DisplayClass3.<PerformChangeDetection>b__0(IEnumerable`1 page)
   at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate()
ClientConnectionId:abc9d8a7-32a0-402b-9273-7c87fc4223c4
Error Number:40197,State:3,Class:20",Normal
20151023,04:21:08,UNIFY Identity Broker,Connector Processor,Information,"Connector Processing started.
Connector Processing started for connector chris21 - POS Form (page 16)",Normal
20151023,04:21:08,UNIFY Identity Broker,List,Information,Next chunk is [1000] sub total [20000]. Duration: 00:00:10.5594588,Normal
20151023,04:21:17,UNIFY Identity Broker,List,Information,Next chunk is [1000] sub total [21000]. Duration: 00:00:08.7187876,Normal
20151023,04:21:23,UNIFY Identity Broker,Change detection engine,Error,"Change detection engine import all items failed.
Change detection engine import all items for connector chris21 - POS Form failed with reason An error occurred while evaluating a task on a worker thread.  See the inner exception details for information.. Duration: 00:06:12.2925941
Error details:
Unify.Framework.EvaluatorVisitorException: An error occurred while evaluating a task on a worker thread.  See the inner exception details for information. ---> System.Data.SqlClient.SqlException: The service has encountered an error processing your request. Please try again. Error code 40174.
A severe error occurred on the current command.  The results, if any, should be discarded.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
   at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
   at System.Data.SqlClient.SqlDataReader.Read()
   at System.Data.Linq.SqlClient.ObjectReaderCompiler.ObjectReaderBase`1.Read()
   at System.Data.Linq.SqlClient.ObjectReaderCompiler.ObjectReader`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Framework.Data.LinqWhereQuery`5.GetEnumerator()
  at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Unify.Product.IdentityBroker.EntityRepositoryExtensions.ConvertConnectorEntitiesWithRepositoryEntities(IEnumerable`1 connectorEntities, IMultiKey`1 schemaKey, IQueryable`1 sourceEntities, Guid connectorId, IEnumerable`1 originalEntities)
   at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32& index, Int32 entitiesProcessedSoFar, IEntityChangesReportGenerator`2 reportGenerator)
   at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.<>c__DisplayClass3.<PerformChangeDetection>b__0(IEnumerable`1 page)
   at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate()
   --- End of inner exception stack trace ---
   at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.CheckForException()
   at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.WaitForAvailableThread()
   at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
   at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
   at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
   at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
   at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
   at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c__DisplayClass29.<Run>b__27()
   at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal
20151023,04:21:34,UNIFY Identity Broker,Connector Processor,Information,"Connector processing success.
17625 entites in cumulative total. Current processing of page 16 for connector chris21 - POS Form processed 1175 entities, finding 1175 differences. Duration: 00:00:26.1862477. ",Normal
20151023,04:21:35,UNIFY Identity Broker,Change detection engine,Information,"Started processing changes register items.
Started processing changes register items for connector chris21 - POS Form.",Normal
20151023,04:21:37,UNIFY Identity Broker,Change detection engine,Information,"Changes register item processing completed.
Changes register item processing on connector chris21 - POS Form completed.  Duration: 00:00:02.0317620",Normal

In order to aid diagnosis of failing processes, I think it would be a good idea for there to be a configurable option to provide detailed diagnosis information at every interface boundary within Identity Broker.

By that, I mean every Identity Broker process that has interfaces should be able to have a decorator inserted to provide details information about the methods and data provided at each step of a process. This would prevent the kinds of Product Support issues where: "I can't really tell what's going on so I suspect something is wrong with Identity Broker".