It is more like a theoretical question but happen to Monash's implementation;

In Adaptor in IdB, we have 2 tables name A and B. We need to join A.x = B.y to get the attribute B.z.

However, x is integer and y is string and hence they do not match.

Is the any convert function for this?

In one of the table connector, there are conditions
BEGDA "less than or equal"

It is expected that all records with Begin Date <= current date <= End Date will be return. However it returns none.

Did a APAB tracing with Monash SAP SME and it turns out

is parsed to the wrong format. In the tracing, the value that ABAP code received is "1/23/2014 1:52:41 PM" while the expected value is "20140123"

We did set a bench mark for 10,000 exports/hour – which translates to about 8 exports per second.
The current speeds for exports are at 5-7 users per minute. We would expect out peak student intake to take ~24 hours to process.
I did call out very early in the piece that performance was important, but tricky to get right. No consulting has been done with Monash on the export process or results shared of IdB internal benchmarks. As it stands, this is a show stopping issue.

1a. Unable to delete a multivalued reference attribute from calendar.
So when we try to delete a member’s attribute (either readers, writers, owners etc.) in calendar, it fails to do so.

1b. No error in FIM GUI, but error is detected in Event Viewer

Monash needs a transformation in adaptor to join two tables and group the resolved values (type of String) to a multi-value string attribute.

I looked at GROUP transformation but it only groups values to multi-value DN.

Please suggest the alternative if any or request additional transformation

The XML Blob for organization in objectclass user is not working as expected. Having exported-changes-not-imported issues

Expo

Hi

Looking for a migration guide on how to migrate IdB Settings / MA etc across environments.

Regards

We use the SQL agent in IdB to read and write from a simulated SAP data source for testing. When exporting changes to the database via IdB, after approximately 100 exports, IdB starts reporting the following exception. When the IdB is stopped and restarted, a further 100 can be exported before the error reappears.

System.InvalidOperationException: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)   at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)   at System.Data.SqlClient.SqlConnection.Open()   at Unify.Framework.Data.DbConnectionExtensions.OpenIfClosed(IDbConnection connection)   at Unify.Framework.Data.SqlServerDataControl.CreateComponent(Boolean factoryInformation)   at Unify.Framework.Data.AdoNetDataControl.ExecuteDataSetQuery(DataSet resultSet, String commandText, IEnumerable`1 parameters, String[] tableNames, CommandType commandType)   at Unify.Framework.Data.AdoNetDataControl.ReturnDataSetQueryT(String commandText, IEnumerable`1 parameters, String[] tableNames, CommandType commandType)   at Unify.Product.IdentityBroker.DirectCommunicatorBase`1.GetEntitiesDataTable(ICollection`1 keyList)   at Unify.Product.IdentityBroker.DirectCommunicatorBase`1.UpdateEntities(IEnumerable`1 entities)   at Unify.Product.IdentityBroker.DirectReadWriteConnectorBase`1.UpdateEntities(IEnumerable`1 entities)   at Unify.Product.IdentityBroker.ConnectorToUpdatingConnectorBridge.UpdateEntities(IEnumerable`1 entities)   at Unify.Product.IdentityBroker.EventNotifierUpdatingConnectorDecorator.UpdateEntities(IEnumerable`1 entities)   at Unify.Product.IdentityBroker.Adapter.UpdateEntities(IEnumerable`1 entities, EntityToConnectorEntityBridge[]& connectorEntities)   at Unify.Product.IdentityBroker.Adapter.UpdateEntities(IEnumerable`1 entities)   at Unify.Product.IdentityBroker.Adapter.UpdateEntity(IAdapterEntity entity)   at Unify.Product.IdentityBroker.AdapterNotifierDecoratorBase`1.UpdateEntity(IAdapterEntity entityToSave)   at Unify.Product.IdentityBroker.CompositeAdapter.UpdateEntity(IAdapterEntity entity)   at Unify.Product.IdentityBroker.AdapterNotifierDecoratorBase`1.UpdateEntity(IAdapterEntity entityToSave)   at Unify.Product.IdentityBroker.LDIFAdapterBase.HandleExportUpdate(IAdapter adapter, IAdapterEntitySaveChange pendingUpdate)   at Unify.Product.IdentityBroker.LDIFAdapterBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)   at Unify.Product.IdentityBroker.LDIFAdapterServiceHostDecoratorBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)   at SyncInvokeExportChanges(Object , Object[] , Object[] )   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)

Request to import all entities from connector gappsCalendarConnector.",Normal
20150618,04:51:49,UNIFY Identity Broker,Connector,Warning,"Import all entities from connector failed.
Import all entities from connector gappsCalendarConnector failed with reason A Google Data exception was thrown for call CalendarResources.RetrieveAllCalendarResources with message ""Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/"". See inner exception for details. Processing continued: False. Response: Version 3.0 is not supported.. Duration: 00:00:07.2319590
Error details:
System.Exception: A Google Data exception was thrown for call CalendarResources.RetrieveAllCalendarResources with message ""Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/"". See inner exception for details. Processing continued: False. Response: Version 3.0 is not supported. ---> Google.GData.Client.GDataRequestException: Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/ ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at Google.GData.Client.GDataRequest.Execute()
— End of inner exception stack trace —
at Google.GData.Apps.AppsPropertyService.QueryExtendedFeed(Uri uri, Boolean shouldGetAllPages)
at Google.GData.Apps.CalendarResourceService.RetrieveAllCalendarResources()
at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetryTResult(String logEvent, Boolean primaryCall, Func`1 request, TResult& result, Int32 retries)
— End of inner exception stack trace —
at Unify.Product.IdentityBroker.GoogleAgent.<>c_DisplayClassab`1.<BackoffRetry>b_a5()
at Unify.Product.IdentityBroker.GoogleAgent.ThrowIfPrimaryCall(Boolean primaryCall, Action throwException)
at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetryTResult(String logEvent, Boolean primaryCall, Func`1 request, TResult& result, Int32 retries)
at Unify.Product.IdentityBroker.GoogleAgent.<CalendarGetPages>d__23.MoveNext()
at Unify.Product.IdentityBroker.GoogleCalendarConnector.<GetAllEntities>d__15.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.Collections.ActionOnExceptionEnumerator`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ProduceAutoPages>d__a`1.MoveNext()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`2 visitor, Int32 maxThreads)
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`1 visitor, Int32 maxThreads)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c_DisplayClass29.<Run>b_27()
at Unify.Framework.ActionJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal
20150618,04:51:49,UNIFY Identity Broker,Change detection engine,Error,"Change detection engine import all items failed.
Change detection engine import all items for connector gappsCalendarConnector failed with reason A Google Data exception was thrown for call CalendarResources.RetrieveAllCalendarResources with message ""Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/"". See inner exception for details. Processing continued: False. Response: Version 3.0 is not supported.. Duration: 00:00:07.2739485
Error details:
System.Exception: A Google Data exception was thrown for call CalendarResources.RetrieveAllCalendarResources with message ""Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/"". See inner exception for details. Processing continued: False. Response: Version 3.0 is not supported. ---> Google.GData.Client.GDataRequestException: Execution of request failed: https://apps-apis.google.com/a/feeds/calendar/resource/2.0/ga-staff-dev.monash.edu/ ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at Google.GData.Client.GDataRequest.Execute()
— End of inner exception stack trace —
at Google.GData.Apps.AppsPropertyService.QueryExtendedFeed(Uri uri, Boolean shouldGetAllPages)
at Google.GData.Apps.CalendarResourceService.RetrieveAllCalendarResources()
at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetryTResult(String logEvent, Boolean primaryCall, Func`1 request, TResult& result, Int32 retries)
— End of inner exception stack trace —
at Unify.Product.IdentityBroker.GoogleAgent.<>c_DisplayClassab`1.<BackoffRetry>b_a5()
at Unify.Product.IdentityBroker.GoogleAgent.ThrowIfPrimaryCall(Boolean primaryCall, Action throwException)
at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetryTResult(String logEvent, Boolean primaryCall, Func`1 request, TResult& result, Int32 retries)
at Unify.Product.IdentityBroker.GoogleAgent.<CalendarGetPages>d__23.MoveNext()
at Unify.Product.IdentityBroker.GoogleCalendarConnector.<GetAllEntities>d__15.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.Collections.ActionOnExceptionEnumerator`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ActionOnLast>d__19`1.MoveNext()
at Unify.Framework.Collections.EnumerableExtensions.<ProduceAutoPages>d__a`1.MoveNext()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`2 visitor, Int32 maxThreads)
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`1 visitor, Int32 maxThreads)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c_DisplayClass29.<Run>b_27()
at Unify.Framework.ActionJob.RunBase()
at Unify.Framework.JobBase.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal

We are seeing that googleapps MA is reporting errors in FIM in async mode.

Wanted to get more insight on why this is happening because as per our understanding there should be no error reporting in async mode.

Attached Unify Log from yesterday with loads of errors..

Attaching screenshot of errors in FIM

The error from one of them (reported in FIM)

System.Exception: Error occurred when attempting to save entity with distinguished name

CN=75fd882f-a487-4e03-9a77-5d3af811a8e1,OU=users

Error:

A Google API exception was thrown for call Users.Update with message "Google.Apis.Requests.RequestError
Entity already exists. [409]
Errors [
	Message[Entity already exists.] Location[ - ] Reason[duplicate] Domain[global]
]
". See inner exception for details. Processing continued: False.
   at Unify.Product.IdentityBroker.GoogleAgent.<>c__DisplayClassb1`1.<BackoffRetry>b__a9()
   at Unify.Product.IdentityBroker.GoogleAgent.ThrowIfPrimaryCall(Boolean primaryCall, Action throwException)
   at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetry[TResult](String logEvent, Boolean primaryCall, Func`1 request, TResult& result, Int32 retries)
   at Unify.Product.IdentityBroker.GoogleAgent.UserUpdate(IConnectorEntity[] entities, IEntitySchema schema, IDictionary`2 matchingEntities, DictionaryCache`2 renamedEntities, Boolean singleUpdate)
   at Unify.Product.IdentityBroker.GoogleAgentNotificationDecorator.UserUpdate(IConnectorEntity[] entities, IEntitySchema schema, IDictionary`2 matchingEntities, DictionaryCache`2 renamedEntities, Boolean singleUpdate)
   at Unify.Product.IdentityBroker.GoogleUserConnector.ModifyAnchor(MultiKeyValue oldKey, MultiKeyValue newKey)
   at Unify.Product.IdentityBroker.EventNotifierModifyAnchorConnectorDecorator.ModifyAnchor(MultiKeyValue oldKey, MultiKeyValue newKey)
   at Unify.Product.IdentityBroker.Adapter.<>c__DisplayClass1e.<UpdateEntities>b__12(IEntity providedEntity)
   at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)
   at Unify.Product.IdentityBroker.Adapter.UpdateEntities(IEnumerable`1 entities, EntityToConnectorEntityBridge[]& connectorEntities)
   at Unify.Product.IdentityBroker.Adapter.UpdateEntities(IEnumerable`1 entities)
   at Unify.Product.IdentityBroker.AdapterNotifierDecoratorBase`1.UpdateEntity(IAdapterEntity entityToSave)
   at Unify.Product.IdentityBroker.AdapterNotifierDecoratorBase`1.UpdateEntity(IAdapterEntity entityToSave)
   at Unify.Product.IdentityBroker.LDIFAdapterBase.HandleExportUpdate(IAdapter adapter, IAdapterEntitySaveChange pendingUpdate, IDictionary`2 getEntityCache)
   at Unify.Product.IdentityBroker.LDIFAdapterBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)
   at SyncInvokeExportChanges(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
   at Unify.Product.IdentityBroker.IdentityBrokerManagementAgentProxy.ExportEntry(ModificationType modificationType, String[] changedAttributes, CSEntry csentry)