Hi Adam

http://googleappsupdates.blogspot.com/2015/09/upcoming-change-to-google-calendar-url.html

Is this going to have any impact to IDB or the API?

Thanks

It seems that an operation called "Merge Attributes" doesn't follow all of the same rules in the driver in Novell Identity Manager.

I am seeing that when such an operation occurs (after an update to NIM) the subsequent update sent to Identity Broker fails. I have no idea why the merge operation behaves differently, If I change an attribute (such as email address) in eDirectory, the update flows and Identity Broker returns success (excerpt of trace at end);

Please not there is no point trying to correlate these errors by using the date/timestamp as the NIM server keeps on randomly assigning itself a new time... I can reproduce these errors (& successful transactions) though.

Here are the errors (warnings actually) that appear in the Identity Broker Console:

Timestamp Severity Source Module Message
26/07/2011 4:16:44 AM Warning An entity failed validation. Adapter The entity 5887c669-9f2c-4dfc-b507-009585afb47e on connector 34b83581-377c-41b5-afb9-2a705076285f failed validation 1 times for the following reasons: detnumber is a required field and is not present.

26/07/2011 4:16:44 AM Warning Adapter request to save entity to adapter space failed. Adapter "Adapter request to save entity 5887c669-9f2c-4dfc-b507-009585afb47e to adapter space 53e85508-7648-409c-bd3a-0737028eba29 failed with reason 1 items failed schema validation during Adapter operation. Check log for validation errors.. Duration: 00:00:00
Error details:
Unify.Framework.AdapterSchemaException: 1 items failed schema validation during Adapter operation. Check log for validation errors.
at Unify.Framework.Adapter.SaveEntities(IEnumerable`1 entities, Boolean reflect)
at Unify.Framework.Adapter.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Framework.AdapterNotifierDecorator.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Adapters.NovellIdentityManagerIdentityBrokerDriverAdapter.PerformSubscribeModifyAction(XElement actionNode, IAdapter adapter)"

Here is what I believe to be the relevent portion of the NIM Trace File:

26/07/2011 14:17:18.05v: 0 CHRIS21-IdB L3 PT:Performing operation modify for \SAWWFT\RES\Users\User\JAMYBSM1.
26/07/2011 14:17:18.05v: 0 CHRIS21-IdB L3 PT:Modifying entry \SAWWFT\RES\Users\User\JAMYBSM1.
26/07/2011 14:17:18.15v: 0 CHRIS21-IdB L3 PT:Scheduling update of application with eDirectory values.
26/07/2011 14:17:18.15v: 0 CHRIS21-IdB L3 PT:
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.5.10.20070918 ">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<modify class-name="User" event-id="5" from-merge="true" qualified-src-dn="O=RES\OU=Users\OU=User\CN=JAMYBSM1" src-dn="\SAWWFT\RES\Users\User\JAMYBSM1" src-entry-id="36598">
<association>MYBSM1</association>
<modify-attr attr-name="mobile">
<remove-all-values/>
</modify-attr>
<modify-attr attr-name="Facsimile Telephone Number">
<remove-all-values/>
</modify-attr>
<modify-attr attr-name="Internet EMail Address">
<remove-all-values/>
</modify-attr>
<modify-attr attr-name="Telephone Number">
<remove-all-values/>
</modify-attr>
</modify>
</input>
</nds>

>
>Some irrelevent Detail removed from here
>

26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:UnifySubscriptionShim: execute end
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:SubscriptionShim.execute() returned:
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:
<nds dtdversion="1.0" ndsversion="8.5">
<output>
<response event-id="1" level="error">
Unify.Framework.AdapterSchemaException: 1 items failed schema validation during Adapter operation. Check log for validation errors.
at Unify.Framework.Adapter.SaveEntities(IEnumerable`1 entities, Boolean reflect)
at Unify.Framework.Adapter.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Framework.AdapterNotifierDecorator.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Adapters.NovellIdentityManagerIdentityBrokerDriverAdapter.PerformSubscribeModifyAction(XElement actionNode, IAdapter adapter)
</response>
</output>
</nds>
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Applying input transformation policies.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Applying policy: Read All Attributes.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST: Applying to response #1.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Policy returned:
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:
<nds dtdversion="1.0" ndsversion="8.5">
<output>
<response event-id="1" level="error">
Unify.Framework.AdapterSchemaException: 1 items failed schema validation during Adapter operation. Check log for validation errors.
at Unify.Framework.Adapter.SaveEntities(IEnumerable`1 entities, Boolean reflect)
at Unify.Framework.Adapter.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Framework.AdapterNotifierDecorator.SaveEntity(IAdapterEntity entity, Boolean reflect)
at Unify.Adapters.NovellIdentityManagerIdentityBrokerDriverAdapter.PerformSubscribeModifyAction(XElement actionNode, IAdapter adapter)
</response>
</output>
</nds>
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Applying schema mapping policies to input.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Applying policy: CHRISMappingRules.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Resolving association references.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Processing returned document.
26/07/2011 14:17:19.60v: 0 CHRIS21-IdB L3 ST:Processing operation <response>
for .

Here is what the trace looks like for a successful transaction:

17/05/2011 15:38:56.17v: 0 Level 3 Chris21 ST:Start transaction.
17/05/2011 15:38:56.17v: 0 Level 3 Chris21 ST:Processing events for transaction.
17/05/2011 15:38:56.17v: 0 Level 3 Chris21 ST:
<nds dtdversion="3.5" ndsversion="8.x">
<source>
<product version="3.5.10.20070918 ">DirXML</product>
<contact>Novell, Inc.</contact>
</source>
<input>
<modify cached-time="20110517053856.135Z" class-name="User" event-id="W2K3-R2-001-NDS#20110517053856#1#1" qualified-src-dn="O=RES\OU=Users\OU=User\CN=AM102983" src-dn="\SAWWFT\RES\Users\User\AM102983" src-entry-id="38266" timestamp="1305610736#1">
<association state="associated">f3b39b95-57ce-40b9-a5bd-f89dcc7d9aed</association>
<modify-attr attr-name="Internet EMail Address">
<add-value>
<value timestamp="1305610736#1" type="string">totally@amazing@sawater.sa.gov.au</value>
</add-value>
</modify-attr>
</modify>
</input>
</nds>
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST:Applying event transformation policies.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST:Applying policy: Updates to Chris.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST: Applying to modify #1.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST:Policy returned:
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST:

> some repeated stuff removed from here

17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST:Applying policy: Event Transformation.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST: Applying to modify #1.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST: Evaluating selection criteria for rule 'Veto all other changes'.
17/05/2011 15:38:56.22v: 0 Level 3 Chris21 ST: (if-class-name equal "User") = TRUE.
17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST: (if-association not-associated) = FALSE.
17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST: Rule rejected.
17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST:Policy returned:
17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST:

> some stuff repeated removed from here

17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST:Subscriber processing modify for \SAWWFT\RES\Users\User\AM102983.
17/05/2011 15:38:56.25v: 0 Level 3 Chris21 ST:No command transformation policies.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:Filtering out notification-only attributes.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:Fixing up association references.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:Applying schema mapping policies to output.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:Applying policy: CHRISMappingRules.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST: Mapping attr-name 'Internet EMail Address' to 'CC-Email'.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST: Mapping class-name 'User' to 'person'.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:No output transformation policies.
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:Submitting document to subscriber shim:
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:

> some stuff repeated removed from here

17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: execute start
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: createWebServiceInterface: URL: http://192.168.0.230:59990/IdentityBroker/NIMDriver.svc?wsdl
17/05/2011 15:38:56.26v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: createWebServiceInterface: Creating service
17/05/2011 15:38:56.32v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: createWebServiceInterface: Creating interface
17/05/2011 15:38:56.34v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: createWebServiceInterface: Returning port
17/05/2011 15:38:56.71v: 0 Level 3 Chris21 PT:UnifyPublicationShim: polling
17/05/2011 15:38:56.71v: 0 Level 3 Chris21 PT:UnifyPublicationShim: createWebServiceInterface: URL: http://192.168.0.230:59990/IdentityBroker/NIMDriver.svc?wsdl
17/05/2011 15:38:56.71v: 0 Level 3 Chris21 PT:UnifyPublicationShim: createWebServiceInterface: Creating service
17/05/2011 15:38:56.89v: 0 Level 3 Chris21 PT:UnifyPublicationShim: createWebServiceInterface: Creating interface
17/05/2011 15:38:56.91v: 0 Level 3 Chris21 PT:UnifyPublicationShim: createWebServiceInterface: Returning port
17/05/2011 15:39:00.48v: 0 Level 3 Chris21 ST:UnifySubscriptionShim: execute end
17/05/2011 15:39:00.48v: 0 Level 3 Chris21 ST:SubscriptionShim.execute() returned:
17/05/2011 15:39:00.48v: 0 Level 3 Chris21 ST:
<nds dtdversion="1.0" ndsversion="8.5">
<output>
<response event-id="W2K3-R2-001-NDS#20110517053856#1#1" level="success"/>
</output>
</nds>

PJ: When new entities are created, the entity id is being returned to NIM but should be updated to return the key.

Hi,

I notice that it is very clearly stated that the Sharepoint List connector is for Sharepoint 2007. I don't see a 2010 equivalent.

Is there an expectation that it will work with Sharepoint 2010? If so, how does the configuration need to differ? I notice that there are distinct differences between the 2010 communicator and the 2007 communicator for the other connectors

This is an item that has been referenced in a couple of other issues but requires its own issue so it can be tracked specifically. I have identified what I believe is a major problem with the Sharepoint Connector with Identity Broker. It relates to the fact that a case difference in the domain name (and perhaps the account name as well - unsure).

For example if the case changes or appears to change from say CN=T00000003539,DC=cad to CN=T00000003539,DC=CAD or the other way around, Identity Broker triggers a migration of the user profile. The result is that the Sharepoint Profile is deleted. We then ended up with what looked like duplicates in ILM - both with the same account name but one entry having the DN as the GUID rather than the correct account name - probably because Broker had the old value but was not able to confirm the new or something similar - not pretty.

Working with Peter Sullivan, we replicated the problem in a vanilla Sharepoint Install by doing the following:

1. Created a user in AD and running a Profile Import from MOSS against AD. Added some profile information against the user
2. Searched for the user and confirmed that they are in fact present
3. Running a profile migration as follows (BTW - you could have any combination of case including the same case):
stsadm -o migrateuser -oldlogin DEV\peter -newlogin dev\peter -ignoresidhistory
4. Searched for the user again - they were no longer present in MOSS
5. Ran another profile import against AD and the user re-appeared but with no profile information against them

Hence it appears as if this could be a major problem. It is no doubt a bug with the Sharepoint API's but it has catastrophic implications for the management of profiles. I managed to destroy over 700 profiles this morning in production when this happenned to occur.

Can you please investigate and attempt to reproduce? In the meantime I may revert the solution back to an account that does not have the priveleges to run a migration in order to stop this ocurring in production from now on, unless you have a tactical recommendation.

I have attached the logfile from Broker as well as the screenshot of the ILM view of things.

thanks,
Craig

It appears the organization profile connector currently only works in environments where OUs are added sequentially with sequential IDs.

SharePoint has a “RecordId” field which is a unique identifier for an organization profile. This is also used to manage the hierarchy, with organization profiles having a parent property which contains the RecordId of the parent. However, this field is readonly in SharePoint, meaning that we have no control over its value, and to ensure the hierarchy is correct, the parent values would need to be updated for the organization profiles after an initial export. Currently, the mapping is only correct if SharePoint happens to generate an ID which is the same as the one from the source system.

The connector must be revisited to handle this behaviour.

Current work is around testing the addition of two fields to SharePoint and the FIM metaverse for handling this mapping as follows, making use of the export control of reference values. From an earlier email:

Unable to Import from the 2 of the SharePoint Connectors since provisioning 37K+ Parents (Error is below).

In order for the SharePoint Export to work the Title attribute in the External Users list was changed from Required = True to False.

C:\Program Files\UNIFY Solutions\Services\Extensibility\ConnectorEngine.extensibility.config.xml.
Old <field name="Title" required="True" validator="string"/>
New <field name="Title" required="False" validator="string"/>

The Export was successful and parents appear in the IdB Connector and SharePoint List. However I now cannot Import back from SahrePoint.
I have changed the Title requirement back to true, this was not successful.

Error details:
System.Xml.XmlException: '=' is an unexpected token. The expected token is ';'. Line 1, position 56.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos)
at System.Xml.XmlTextReaderImpl.ParseAttributeValueSlow(Int32 curPos, Char quoteChar, NodeData attr)
at System.Xml.XmlTextReaderImpl.ParseAttributes()
at System.Xml.XmlTextReaderImpl.ParseElement()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlLoader.ParsePartialContent(XmlNode parentNode, String innerxmltext, XmlNodeType nt)
at System.Xml.XmlLoader.LoadInnerXmlElement(XmlElement node, String innerxmltext)
at Unify.Communicators.Moss2007ListWebServiceCommunicatorBase.<GetListItems>d__6.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.EnumerableExtensions.<ActionOnFirst>d__1c`1.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.ActionOnExceptionEnumerator`1.MoveNext()
at Unify.Framework.EnumerableExtensions.<ActionOnLast>d__16`1.MoveNext()
at Unify.Framework.EnumerableExtensions.<ProduceAutoPages>d__9`1.MoveNext()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`2 visitor, Int32 maxThreads)
at Unify.Framework.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Framework.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Framework.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.MutexJobDecorator.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal

Currently, the only way for us to create group memberships is to take a connector that contains a unique list of groups and do a relational.dn join to get its members.

Eg,
class connector contains class 001,
student enrolment connector contains a relationship between Student1 and 001
student enrolment connector contains a relationship between Student2 and 001

relational join on class:student1,student2

However, in some cases we may wish to create groups based on aggregate values (ie, group by) within a single connector - without a relationship connector.

Eg, Student connector that has Name, academicYearLevel and homeRoomClass

It would be great if i could create attribute-based groups based off of these values without a second connector that contains a unique list of academicYearLevels to join against.

Eg:

This would create an adapter that contains a unique list of academic years with a multi-value field that has DN's for the students

This would create an adapter that has attribute based groups for both academicYearLevel AND studentID... This is essentially 'union' functionality so that you don't need a seperate adapter for every type of attribute-based group. Fields are "groupkey" and "studentID". Note the prefix allows you to distinguish between the two groups (is pre-pended to attribute value)

Is equivalent to SQL group by school,academicYearLevel. Would return a group for each yearLevel at each school. The columns returned would be groupkey (containing school:academicYearLevel) and "studentID" (multi-value list of DN's... column name matches

When deleting a user, what will occur if the user has resources (eg a document) checked out in SharePoint. When using the standard web interface it fails. Can you describe the Broker process (will it just fail and return an error?)

When attempting to retrieve user profiles from SharePoint 2010, the following error was thrown on the SharePoint side. Note that this is from a WCF trace on the SharePoint side - the error message thrown to Identity Broker is not at all helpful:

There was an error while trying to serialize parameter http://www.unifysolutions.net/IdentityBroker/SharePoint:GetProfilePageResult. The InnerException message was 'Maximum number of items that can be serialized or deserialized in an object graph is '65536'. Change the object graph or increase the MaxItemsInObjectGraph quota. '.  Please see InnerException for more details.

Error thrown to Identity Broker:

Error occurred in module: Change detection engine

Change detection engine poll for connector SharePoint 2010 User Profile Connector failed with reason An error occurred while receiving the HTTP response to http://c21sharepoint/_vti_bin/unify/userprofile.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.. Duration: 00:01:13.9003906
Error details:
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://c21sharepoint/_vti_bin/unify/userprofile.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace: 
   at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Unify.Communicators.UserProfileManager.ISharePoint2010UserProfileService.GetProfilesByName(String[] accountNames)
   at Unify.Connectors.SharePoint2010UserProfileConnector.GetEntities(IEnumerable`1 entityIds)
   at Unify.Framework.ConnectorToPollingConnectorBridge.GetEntities(IEnumerable`1 entityIds)
   at Unify.Framework.EventNotifierReadingConnectorDecoratorBase`1.GetEntities(IEnumerable`1 entityIds)
   at Unify.Framework.ChangeDetectionPollJob.RunBase()
   at Unify.Framework.MutexJobDecorator.Run()
   at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
   at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)

The workaround is to decrease the number of users retrieved from SharePoint at a time using the bulkPageSize property of the communicator. However, the above attribute should be changed programmatically in the WCF service itself. SharePoint 2010 allocates a dynamic schema to all custom WCF services - see http://msdn.microsoft.com/en-us/library/ff521586.aspx.