EvB service will not start

Mark,

Thank you for logging this support request. I have asked for it to be addressed as a matter of urgency due to the nature of the issue.

Hi Mark,

This error message indicates an error thrown via the WMI interface used to negotiate with ILM. There may be a more descriptive error message in the Windows Event Viewer around the time you are seeing this error. Some things to consider to further diagnose what issue may be occurring here:

• Ensure the ILM service is running correctly (which it sounds like it is)
• Review any changes to the environment or service account at the time this error started occurring, as I believe this error has not been seen before (although this would need to be confirmed by yourself or Nick)
• Ensure the Event Broker service account has been correctly configured according to https://unifysolutions.jira.com/wiki/display/EB223/Prerequisites

Please let us know if you encounter any difficulties. Any other additional information on the issue would also be useful.

Regards,

Matt

Atempted to start Event Broker Service. It took longer than usual but while the bar was filling up a Full Sync on AD was started by the Event Broker Service Account. Got an error message saying the service couldn't start.

Event Viewer showed only this at the same time.
CAPI2
Failed extract of third-party root list from auto update cab at: <http://www.download.windowsupdate.com/msdownload/update/v3/static/trustedr/en/authrootstl.cab> with error: A required certificate is not within its validity period when verifying against the current system clock or the timestamp in the signed file.

I don't know if that error has anything to do with Event Broker.

In the services window it reported starting and couldn't be started or stopped. I ended the process in Task Manager.

Tried to run the service again:
The service didn't start: Unify Event Broker service started then stopped some services stop if they are not in use.

Cannot start service
UNIFYNowEngine.EventBrokerStartupException: Exception of type 'UNIFYNowEngine.EventBrokerStartupException' was thrown.
at UNIFYNowEngine.EventBrokerEngine.ProcessStartup(Boolean doInternalProcessDbLog)
at UNService.UNService.DoStart()
at UNService.UNService.OnStart(String[] args)

There's also an IdmNotificationService - I don't know what this is.

WMI tracing must not be enabled because nothing has been logged.

Event Broker runs a full sync on AD when you attempt to start it but apart from that it looks like the last time it was running normally was on Friday at about 2pm.

I didn't find alot of useful information.

Mark: Were there any changes made to your servers around 1-2pm on Friday? Are there any permissions that the Event Broker service account may have lost that is required - shown in here https://unifysolutions.jira.com/wiki/display/EB223/Prerequisites

Matt: Does that Event Viewer error give you any ideas?

Hey Dan, a quick search looks like that first error message is a Windows Server issue, and should not be affecting Event Broker's ability to start. It sounds like you are trying to start the service with Startup/Shutdown options enabled - if these are disabled, does the service succeed in starting?

Yup it started straight away with that turned off. I'll watch it for a while and see what it does.

Good news. Something worth doing is verifying that the AD Full Import/Full Sync profile is configured correctly in the start up operation, and returns successfully. Let us know if there are any further problems here.

The startup operations look ok to me. There are some new errors now. Exporting to AD there are intermittent exch2010 errors so it's likely that something has changed to prevent the remote powershell calls that FIM fires at the exchange server either being sent, getting there or being recieved. There is also still no confirming delta imports for AD. These are seperate issues though, and I think this issue as a blocker can probably be resolved except that I still don't know why this was happening.

The password is bad for the GroupPopulator MA. The startup list must have been crashing when it hit this error. Mark can you please check the credentials entered into the Group Populator MA and ensure that the account used has read access to the Group Populator database.

This is likely the reason you are receiving a Startup exception. From Event Broker v3.0, startup failures do not cause the service to fail to start. Please confirm if this resolves the issue.

OK, checked MA and made sure pwd for the account was correct and ran it. Ran OK but came up with a bunch of sync errors.

Since the beginning of this issue the ONLY way I have been able to get new users provisioned is to manually enter the user details for HR Import connector, manually run a preview and commit it and then do an AD export.

Updates seem to be going through OK now, but there are still major issues with our account creation process now that werent there a month ago.

Thanks for all the help thus far though Daniel

No worries Mark. That emoticon looks like it had a stroke. I'll look into these issue some more today - at least we have event broker up and running again and we also know why it stopped it working.

Hi Mark,

I've been looking at this with Paul today. We've been looking first at the export errors from AD. It reports that a user trying to be provisioned already exists and this uniqueness test uses samAccountName. However the join rule is a simple samAccountName to samAccountName so with provisioning turned off a full import and full sync should join these users back together but it isn't. We think this might be because FIM isn't looking at the correct containers.To change or even check which containers the AD MA is looking at we need the password for svc_miis. Could you please email it to Paul or me.

Thanks for sending that. After looking through the containers it looks like for at least some of the users the reason is that FIM is looking at only some of the OU's and users have been moved out of those OU's. Peter Jeffrey, samaccountname = jeffreyp, is an example of this.

Do you think there have been users moved around enough to account for those 470 or so failed exports?

OK I've looked through about 10 of them and that's the problem with all of them. The user has been moved from the OU that they were in so FIM is trying to reprovision OR there are users that have been manually created in AD in an OU that FIM isn't looking at and then FIM generates that same SAMAccountName for a new user because it doesn't know that SAMAccount exists because it's not looking at that OU.

Do either of these explanations seem plausible to you Mark?

Definite possibility. I guess I can try doing a disconnect of the AD connector on some of these accounts, but I have an example of 1 this doesnt work for. Check out GUESTS - Samantha Guest. Break the AD connector and runa preview - it comes back all good, but then wont provision. The AD connector that gets created points to the Pending OU, but she is not in that OU.

Yes that all sounds correct. The AD connector at that point is a pending export - not until the export is run would she be created in AD. The reason that she can't be created is that there is another account with the samaccountname equal to 'guests'. I checked this with an ldap query through adsiedit.

So why is FIM trying to do something this dumb? Because it's being told to look at only some OU's in the domain. So if there is a user that is in one of the OUs that FIM isn't looking at that has the same samaccount name as a user coming in from the HR system the FIM can't provision the user from HR to AD because the samaccountname has to be unique across the domain and FIM doesn't know about the user in the other OU so has no chance to add a suffix to the samaccountname.

The only fix I can see is to tick the entire OU structure in the containers menu so that the FIM metaverse retains a complete list of used SAMAccountNames, otherwise any accounts added manually (or that exist by default as with the guests account) will create a potential for conflict with FIM. The downside of doing this is that it will bloat the AD connectorspace and the provisioning code will also need to be updated to cater for this.

If this seems like the best way forward I think I'll have to ask Nick about changes like this. I'm going to resolve this issue because the original problem of EvB not starting is resolved.

Recap - EvB service wasn't starting because it had Startup processes enabled and this included a full import/full sync on the Group Populator. The credentials in the Group Populator MA were out of date so the full import/full sync couldn't run, preventing the startup process from completing and this crashed the service before it could start. Reset the credentials and issue is resolved.

Closing issue.