Duplicate address error when adding members to Google Groups

Hi Hayden

After discussing this, it was decided that as a temporary solution we would implement the changes we talked about last week. However as I've been doing this work, I've noticed some things that do not add up.

Firstly, my understanding it that this issue was caused by adding an account or email being added to a group when another account has that email as their recovery address. I have not been able to replicate this. I've tried both internal and external members, as well as adding the recovery address both before and after the other account. Perhaps there is a setting somewhere for disallowing this that I don't have enabled in my dev environment? Can you try and find more information about this events that cause this issue? Is it actually a recovery relationship, and if so what order are the members added and is one of them an external member or are both internal members?

The other dependency is the claim that the duplicate member errors are causing other accounts to not be placed into groups if they would have been processed after the failing one. Last week I incorrectly stated that that group member processing would halt if an error occurred part way though. Digging into the code this week I've realised this is not the case, the error is just logged. Sorry for the confusion. Are there any documented cases of members not being correctly added to group without a logged error, or is this aspect of the issue based on the bad information I provided?

Finally, while debugging, I came across an issue that could cause group connector imports not properly populate the internal membership fields in certain situations. Unsure if it could have contributed to this issue, but here's a patch anyway.

Unify.IdentityBroker.Communicator.Google.dll

Hi Beau,

Sorry for the delay in testing this patch, this one appears to have brought in a large number of changes to external members. Looking at a few it appears to be delete/adding a bunch of records (Not all) based on cases. For example: deleting "Test.User@test.com" and adding "test.user@test.com". When exporting these changes IDB provides us with the following error:

Unify.IdentityBroker.Communicator.Google:
System.Exception: A Google API exception was thrown for call Members.External.Insert with message "Google.Apis.Requests.RequestError
Member already exists. [409]
Errors [
Message[Member already exists.] Location[ - ] Reason[duplicate] Domain[global]
]
". See inner exception for details. Processing continued: True. Retries: 1. ---> Google.GoogleApiException: Google.Apis.Requests.RequestError
Member already exists. [409]
Errors [
Message[Member already exists.] Location[ - ] Reason[duplicate] Domain[global]
]

at Google.Apis.Requests.ClientServiceRequest`1.Execute()
at Unify.Product.IdentityBroker.GoogleAgent.BackoffRetry[TResult](String logEvent, Boolean throwExceptions, Func`1 request, Action newClient, TResult& result, Int32 retries, Boolean retryOnInvalidValue)
--- End of inner exception stack trace ---
at Unify.Product.IdentityBroker.GoogleAgent.<>c__DisplayClass69_4`1.b__2()
at Unify.Product.IdentityBroker.GoogleAgent.ThrowIfPrimaryCall(Boolean primaryCall, Action throwException)

Looking at the data further, it appears only some groups performing delete/adds are failing with the above error. Other groups seem to be delete/adding in the same way as above and not throwing errors. I'll start the export again and see what happens if I just let it run. So far I stopped it at about 500/2500 updates. At that stage there was about 50 erroring groups.

I have also just tried clearing the connector and adapter as it is seemingly complaining about a duplicate key. And also manually searching the database for a duplicate key value and after trying both, the error still shows and I couldn't find anything in the DB.

Hi Hayden

If I recall correctly, this was the patch to allow group membership changes to continue processing even when one fails, so I'm not surprised you're seeing more activity. From what I can tell, the API error is a result of two issue:

• Google return a success response to the remove group member request, but the actual processing of this change takes time in their back-end
• We consider the same email with different case to be not equal, but Google does.

Basically, we try to remove a group member, Google says it's removed, and then we immediately try to add the same (according to Google) member back, except the original one is still there because the removal process takes time in their back-end.

As you saw, rerunning the export fixed this, as by that time Google processed the removal requests. Whether this is an actual issue depends on what happens if we add a group member and Google changes the case. ie we add "test.user@test.com" and Google changes this to "Test.User@test.com". This would result in a constant cycle of removing and re-adding.

The other "same key" error can be ignored. It's a known issue with the Broker portal where that error is thrown making requests to the Broker web service, seemingly at random. Minimise the effect by closing the portal when it's not in use, which I'd recommend doing anyway to avoid the overhead of its status update requests.

Hi Beau,

Thanks for clarifying that. A re-occurrence hasn't happened since the ticket was raised and since the patch has been in things have been looking good.

On the "same key" error, it seems to be happening consistently for the last week on the two user connector for both the client and myself. Even after restarting the service and doing an iisreset. Basically I just want to make sure there isn't a possibility of a duplicate key or anything. Is it safe to say that there isn't a duplicate key in these connectors base on what you've seen previously?

Thanks

Sorry, I misread your last post. Somehow missed the part where it happened on the entity search and jumped to the wrong conclusions.

The good news is it's not related to the connector entities, nor is it related to the GA patch. It's a bug in the entity search API when filtering a connector/adapter whose schema has more than one field with the same name. The only valid scenario where this is possible should be with grouped names (i.e. GroupA:Name, GroupB:Name) as Broker shouldn't even start otherwise. Is this the case for the Google connectors causing this issue?

It doesn't look like the patch is being loaded. Check that:

• the dll has been placed in <installDir>\Identity Broker\Services
• that folder doesn't contain any other version of that file, such as an older patch, even if it's been renamed
• the previous point is also true of the <installdir>\Identity Broker\Services\Patches directory
• no error messages are being logged when the service starts
  

Sorry Beau, I thought this file needed to go in the Web/bin directory. I've put it in the Services directory now and its working great now. Thanks for your help.