I am seeing a small number (16) repeating errors in one of my MAs - with exports failing for some users with this error.

System.Exception: Status: 400 Bad Request
   at Unify.IdentityBroker.xxxx.Agent.DefaultCommunicator.SendCommand(String urlPath, Method method, Object data)
   at Unify.IdentityBroker.xxxx.Agent.DefaultCommunicator.Add(AgentEntity entity)
   at Unify.IdentityBroker.xxxx.Connector.Connector.AddEntity(IConnectorEntity entity, ISaveEntityResults`2 results, DefaultCommunicator communicator)

Three of them are for a known data error (malformed email addresses) but I cannot find anything to tell me more about why the others are failing.

I see the same error in Eventviewer for each failed user.  In IdB I see this in the log - it reports 16 entities saved then reports that 0 were successful.  Any help in interpreting this log or the issue would be appreciated

Trying to request schema for an HPE Content Manager connector and I get the following error:

An error has occurred: Method not found: 'Unify.Product.IdentityBroker.IEntitySchemaConfigurationUtility Unify.Product.IdentityBroker.IMultiKeyedConnectorFactoryInformation.get_SchemaConfigurationUtility()'.

Is this because the connector doesn't support schema retrieval, or have I done something wrong?

The Agent test connection succeeds, and that's as far as I've got.

I am seeing timeout issues while trying to perform a delta import from Identity Broker.  MIM just reports stopped-extension-dll-exception with no other detail, but in event viewer I see event id 6803.

The management agent "XXXXXX" failed on run profile "DI" because the server encountered errors.

Then event ID 6801:

The extensible extension returned an unsupported error.
 The stack trace is:
 
 "Unify.Product.IdentityBroker.LdapOperationException: Error during processing of SearchRequest targetting cn=changelog: Operation timed out while waiting for message queue with id of 14. ---> System.OperationCanceledException: Operation timed out while waiting for message queue with id of 14.
   at Unify.Product.IdentityBroker.LdapConnection.GetMessage(Int32 messageId)
   at Unify.Product.IdentityBroker.SearchRequest.Send(Func`2 send, Func`2 recv)
   at Unify.Product.IdentityBroker.LdapConnection.SendRequest(ILdapRequest request)
   --- End of inner exception stack trace ---
   at Unify.Product.IdentityBroker.LdapConnection.SendRequest(ILdapRequest request)
   at Unify.Product.IdentityBroker.LdapConnectionProxy.<SearchRequestPaged>d__8.MoveNext()
   at Unify.Product.IdentityBroker.ImportProxy.<GetChangedEntriesPaged>d__30.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
   at Unify.Product.IdentityBroker.ExtensionMethods.Take[TSource](IEnumerator`1 source, Int32 count, IList`1& items)
   at Unify.Product.IdentityBroker.ExtensionMethods.<Page>d__3`1.MoveNext()
   at Unify.Product.IdentityBroker.ImportProxy.Import(GetImportEntriesRunStep importRunStep)
Forefront Identity Manager 4.3.2266.0"

The corresponding time in IDB log has

26/Sep/2018 10:11:17 
Information
LDAP engine Handling of LDAP unbind request.
Handling of LDAP unbind request received on connection 127.0.0.1:61732 to connect as user ******** completed successfully. Duration: 00:00:00.

Is there a setting somewhere that will let me increase IDB LDAP timeouts?  I could not find one, but it has been a few years since I used the product.

I was reading through the migration guide and it didn't mention the circumstance I'm in.

Are there any considerations that need to be made regarding the database's data when migrating between environments causes an adapter to be removed? As in does UNIFY Broker have the ability to detect that an adapter and connector has been removed and delete the entities that were in it from the database when you're migrating by replacing the extensibility folder?

Attempting to post the following to AddSCIMGateway

{

"DisplayName":"SCIM Gateway",

"Comment":"",

"Extended":{

"Address":"http://40.118.23.253:59991/IdentityBroker",

"Audience":"",

"Tenant":"https://unifyb2cworkshop.onmicrosoft.com/",

"UserIdLookupField":"upn",

"UsersMappings":{

"AdapterId":"df97e04e-4d4c-475e-bf89-8a6c3f1b66d3",

"Mappings":{}

},

"GroupsMappings":{

"AdapterId":"e7db372f-a14d-4fdc-909b-2406b8b3f874",

"Mappings":{}}

}

}

Receive the following Error Response. Thanks in advance!

{
  "Message": "An error has occurred.",
  "ExceptionMessage": "Response status code does not indicate success: 404 (Not Found).",
  "ExceptionType": "System.Net.Http.HttpRequestException",
  "StackTrace": "   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()\r\n   at Microsoft.Owin.Security.ActiveDirectory.WsFedMetadataRetriever.GetSigningKeys(String metadataEndpoint, TimeSpan backchannelTimeout, HttpMessageHandler backchannelHttpHandler)\r\n   at Microsoft.Owin.Security.ActiveDirectory.WsFedCachingSecurityTokenProvider.RetrieveMetadata()\r\n   at Microsoft.Owin.Security.ActiveDirectory.WsFedCachingSecurityTokenProvider..ctor(String metadataEndpoint, ICertificateValidator backchannelCertificateValidator, TimeSpan backchannelTimeout, HttpMessageHandler backchannelHttpHandler)\r\n   at Owin.WindowsAzureActiveDirectoryBearerAuthenticationExtensions.UseWindowsAzureActiveDirectoryBearerAuthentication(IAppBuilder app, WindowsAzureActiveDirectoryBearerAuthenticationOptions options)\r\n   at Microsoft.SystemForCrossDomainIdentityManagement.WebApplicationStarter.ConfigureApplication(IAppBuilder applicationBuilder)\r\n   at Microsoft.Owin.Hosting.Engine.HostingEngine.Start(StartContext context)\r\n   at Microsoft.SystemForCrossDomainIdentityManagement.Service.Start(Uri baseAddress)\r\n   at Unify.Product.IdentityBroker.SCIMGateway.StartGateway()\r\n   at Unify.Product.IdentityBroker.GatewayBase.Start()\r\n   at Unify.Product.IdentityBroker.GatewayNotifierDecorator.Start()\r\n   at Unify.Product.IdentityBroker.GatewayRepository.AddAndStart(IOperationalGateway gateway)\r\n   at Unify.Product.IdentityBroker.GatewayEngine.<>c__DisplayClass31_0.<ConfigurationChange>b__0()\r\n   at Unify.Framework.ExtensionMethods.WaitOnMutex(Mutex mutex, Action work)\r\n   at Unify.Framework.Notification.NotifierDecoratorBase.Notify(ITaskNotificationFactory notificationFactory, Action action)\r\n   at Unify.Product.IdentityBroker.GatewayEngineNotifierDecorator.Add(IGatewayConfiguration gateway)\r\n   at Unify.Product.IdentityBroker.GatewayEngineAuditingDecorator.Add(IGatewayConfiguration gateway)\r\n   at Unify.Product.IdentityBroker.GatewayController.InnerAddGateway[T](GatewayApiInformation`1 gatewayInformation, Guid gatewayId, XElement extended)\r\n   at Unify.Product.IdentityBroker.GatewayController.AddSCIMGateway(SCIMGatewayApiInformation gatewayInformation)\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()"
}

When attempting to connect to the LDAP gateway from One Identity's LDAP connector, One Identity is throwing an error regarding it:

2018-07-13 00:50:51.1156 FATAL UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Error parsing condition.
syntax error!
Value "" was found, but one of the following values expected.

Unfortunately it's not a very helpful error.

The full logs of what One Identity is doing are as follows:

2018-07-13 00:50:46.7972 TRACE UFY-1IM-WEB01\UFYAdmin (SqlLog ) : -- Connection 1 switched from Working to Available 
2018-07-13 00:50:50.8968 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Simple LdapSearch BaseDN: '', SearchScope: 'Base', Filter: '(objectclass=*)', RequestAttributes: 'subschemaSubentry' 
2018-07-13 00:50:50.9594 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : LdapSearchResult code: 'Success' entries: '1' 
2018-07-13 00:50:50.9594 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Schema DN is 'cn=schema' 
2018-07-13 00:50:50.9594 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Simple LdapSearch BaseDN: 'cn=schema', SearchScope: 'Base', Filter: '(objectclass=*)', RequestAttributes: 'ldapSyntaxes,attributeTypes,matchingRules,matchingRuleUse,objectClasses' 
2018-07-13 00:50:51.0062 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Got 16 elements of type 'ldapsyntaxes' 
2018-07-13 00:50:51.0843 TRACE UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Got 34 elements of type 'matchingrules' 
2018-07-13 00:50:51.1156 FATAL UFY-1IM-WEB01\UFYAdmin (SystemConnector ) : Error parsing condition.
syntax error!
Value "" was found, but one of the following values expected.

In the logs, we can see that it's requesting certain attributes from Broker:

'ldapSyntaxes,attributeTypes,matchingRules,matchingRuleUse,objectClasses'

And this can also be seen from a wireshark trace:

But when Broker responds, we're only sending back 4 attributes:

I'm unsure if that's the cause of the issue, as One Identity doesn't provide any more information regarding the connection. But it's the only discrepancy that I can see.

The pcap file is also attached for reference.

Output.pcap

When generating a DN with a non-keyed field in an adapter, if duplicate DNs are generated, a reflection error is thrown regarding the duplicate.

However, if the DN field being used has case-insensitive duplicates, reflection runs without issues, but an error is thrown on the LDAP gateway while attempting to perform a delta import:

An error occurred for gateway LDAP Gateway (6210ccad-9e16-419e-85aa-b3bf94bfacfd) on client from 127.0.0.1:56636. More details:
Internal Server Error #11: System.Exception: A task faulted. See inner exception for details. ---> System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at Unify.Product.IdentityBroker.CachedAdapterContext.GetEntitiesByKeyValues(IEnumerable`1 values)
at Unify.Product.IdentityBroker.ChangeLogRequestHandler.<>c__DisplayClass9_3.<normalsearch>b__3(IGrouping`2 group)
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)
at Unify.Product.IdentityBroker.ChangeLogRequestHandler.<>c__DisplayClass9_2.<normalsearch>b__1()
at System.Lazy`1.CreateValue()
at System.Lazy`1.LazyInitValue()
at Unify.Product.IdentityBroker.ChangeLogToLDAPEntryConverter.EntryUuidAttributeValue(IChangeLogItem sourceValue, IDictionary`2 partialAttributes)
at Unify.Product.IdentityBroker.ChangeLogToLDAPEntryConverter.Transform(IChangeLogItem sourceValue)
at Unify.Product.IdentityBroker.ChangeLogRequestHandler.<normalsearch>d__9.MoveNext()
at Unify.Product.IdentityBroker.ForwardLookingEnumerator`1.MoveNext()
at Unify.Product.IdentityBroker.LDAPEngineExtensions.<takefromenumerator>d__1`1.MoveNext()
at Unify.Product.IdentityBroker.SearchRequestHandlerBase.<finalizesearchresults>d__12.MoveNext()
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)
at Unify.Product.IdentityBroker.SearchRequestHandlerBase.HandleRequest(IRfcLdapMessage message, CancellationToken token, Action`1 postAction)
at Unify.Product.IdentityBroker.ChangeLogRequestHandler.<handlerequest>d__4.MoveNext()
--- End of inner exception stack trace ---
at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass4_0.<taskcontinuewithexceptionpassthough>b__0(Task t)
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Unify.Product.IdentityBroker.LDAPConnection.<respondtomessageasync>d__35.MoveNext()</respondtomessageasync></taskcontinuewithexceptionpassthough></handlerequest></finalizesearchresults></takefromenumerator></normalsearch></normalsearch></normalsearch>

It would be good if, upon DN generation, a case-insensitive comparison was done to ensure that no duplicates are present (since case sensitive DN's are not treated as different objects in consuming LDAP applications).

Service will not run in a Windows Kubernetes container without this capability

Group ID's are being duplicated on connectors when modifying group membership, and not being removed on deletion.

Reproduction steps:

1. Create multiple connectors
2. Create connector group
3. Add a few connectors to group. Save group
   1. At this point, note the configuration file adds the group ID to the connector configs
4. Modify group again, adding another connector
   
   1. At this point, note that the original connectors added to the group now have two entries for group ID in the connector config. The new connector has one entry for group ID.
5. Modify group again, removing one of the original connectors
   1. At this point, note that only one of the ID's is removed from the configuration
6. Modify group again
   1. At this point, note that the connector removed in the previous step is still marked as being a member of the group
7. Delete group
   1. At this point, note that group ID's do not get removed from the connector configuration.

When attempting to create and save a join transformation, the following error presents when attempting to save the transform without any attributes mapped:

If I edit the configuration to remove the mappings, the transformation performs as expected, which confirms that they are not actually required by the service.
The transformation should let you configure and save without any mapped attributes.