0
Declined

PowerShell sync task can't connect to AD

Adrian Corston 11 months ago in UNIFYBroker Service updated by Matthew Davis (Technical Product Manager) 11 months ago 5

Sometimes I see errors in my customer's production environment when the birthright group provisioning PowerShell task is unable to connect to AD.  This is happening immediately after a successful connection to AD has provisioned the user account.  There are two types of errors that are returned, as below:

UnifyLog20230517.csv:1629:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.1916629",Normal
UnifyLog20230518.csv:57203:Add entities [Count:3] to connector AD User reported 3 entities saved, 0 failed. Duration: 00:00:01.2855075",Normal
UnifyLog20230521.csv:219718:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:01:00.0326068",Normal
UnifyLog20230521.csv:219982:20230521,15:41:05,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:220247:20230521,15:43:05,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:234120:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0287518",Normal
UnifyLog20230521.csv:234384:20230521,16:41:05,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:234683:20230521,16:43:05,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:248409:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0017363",Normal
UnifyLog20230521.csv:248672:20230521,17:41:06,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:248971:20230521,17:43:06,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:262577:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0161713",Normal
UnifyLog20230521.csv:262840:20230521,18:40:50,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:263093:20230521,18:42:50,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:276860:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0138167",Normal
UnifyLog20230521.csv:277241:20230521,19:40:52,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:277494:20230521,19:42:52,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:291308:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0170553",Normal
UnifyLog20230521.csv:291572:20230521,20:41:01,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:291845:20230521,20:43:01,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:305473:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0058264",Normal
UnifyLog20230521.csv:305736:20230521,21:40:51,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:305989:20230521,21:42:51,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:319874:Add entities [Count:2] to connector AD User reported 2 entities saved, 2 failed. Duration: 00:01:00.0170181",Normal
UnifyLog20230521.csv:320142:20230521,22:41:00,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:320398:20230521,22:43:00,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: The operation returned because the timeout limit was exceeded.",Normal
UnifyLog20230521.csv:331465:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:00:01.1680608",Normal
UnifyLog20230521.csv:331472:20230521,23:30:44,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aau,OU=Standard
Users,OU=X,DC=local in AD: Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.",Normal
UnifyLog20230521.csv:331477:20230521,23:30:47,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=jball,OU=Standard
Users,OU=X,DC=local in AD: Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.",Normal
UnifyLog20230523.csv:10338:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.0310308",Normal
UnifyLog20230523.csv:32001:Add entities [Count:3] to connector AD User reported 3 entities saved, 0 failed. Duration: 00:00:01.2325114",Normal
UnifyLog20230523.csv:32955:Add entities [Count:8] to connector AD User reported 8 entities saved, 0 failed. Duration: 00:00:02.0700229",Normal
UnifyLog20230523.csv:34211:Add entities [Count:4] to connector AD User reported 4 entities saved, 0 failed. Duration: 00:00:01.0275391",Normal
UnifyLog20230526.csv:12458:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.1577689",Normal
UnifyLog20230531.csv:11081:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.1605492",Normal
UnifyLog20230531.csv:11851:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.1728933",Normal
UnifyLog20230602.csv:2129:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.1171594",Normal
UnifyLog20230602.csv:2138:20230602,01:09:42,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=dtai2,OU=Standard
Users,OU=X,DC=local in AD: Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.",Normal
UnifyLog20230604.csv:21979:Add entities [Count:3] to connector AD User reported 3 entities saved, 0 failed. Duration: 00:00:01.5039543",Normal
UnifyLog20230605.csv:7281:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.1775722",Normal
UnifyLog20230605.csv:24280:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:00:01.4059702",Normal
UnifyLog20230606.csv:18238:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.2258243",Normal
UnifyLog20230606.csv:20135:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.2077376",Normal
UnifyLog20230606.csv:20865:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.2686140",Normal
UnifyLog20230609.csv:11402:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.1731736",Normal
UnifyLog20230611.csv:21783:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:01.2460368",Normal
UnifyLog20230611.csv:21786:20230611,15:38:16,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=spoyn,OU=Standard
Users,OU=X,DC=local in AD: Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.",Normal
UnifyLog20230612.csv:24388:Add entities [Count:7] to connector AD User reported 7 entities saved, 0 failed. Duration: 00:00:01.7047121",Normal
UnifyLog20230612.csv:24410:20230612,15:40:07,UNIFYBroker,PowerShellTask,Error,"Birthright group assignment cannot be performed due to provision failure - error searching for user CN=aeasl,OU=Standard
Users,OU=X,DC=local in AD: Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running.",Normal
UnifyLog20230612.csv:24465:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:00:00.1749234",Normal
UnifyLog20230613.csv:11991:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.2090408",Normal
UnifyLog20230613.csv:25059:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:00:01.2958574",Normal
UnifyLog20230614.csv:17864:Add entities [Count:1] to connector AD User reported 1 entities saved, 0 failed. Duration: 00:00:00.1804325",Normal
UnifyLog20230614.csv:29130:Add entities [Count:2] to connector AD User reported 2 entities saved, 0 failed. Duration: 00:00:01.2327871",Normal

The birthright group provisioning is a critical event-driven call and it must succeed.  Can you please investigate why it failed like this and see if there's some way to improve it's reliability?

Under review

Hi Adrian,

The PowerShell code executed is outside the control of the product. Given that the connector is successfully able to connect to AD without problems, it would appear at first glance to be an issue with the powershell code. What investigation has been done to rule the code out as being a problem? Is the issue reproducible in any other environments?

The first lot of errors suggest that the command being run is being executed but hitting the default AD module timeout of 2 minutes. Have you tried making the scope of the query smaller to see if it exhibits the same behaviour?

Hi Matt,
Investigation is as per https://voice.unifysolutions.net/helpdesks/9/tickets/5317-scheduled-job-randomly-logs-messages-from-previous-invocations
I haven't seen the issue in other environments (including this customer's test instance).
The scope of the Get-ADUser cmdlet is one user (i.e., the one that has just been created in AD - noting that when the user creation hasn't completed yet the error seen is different).

No worries, thanks. There doesn't appear to be much we can go on then to support at this stage - if it's unable to be reproduced in another environment, and one of the product developers was also unable to reproduce, then it would point towards being a script or environmental problem. 

More than happy to help investigate if there's any evidence that would point towards it more likely being a problem with the product.

Thanks Matt.  You may as well close this ticket then.