Write-back for SharePoint List Connector fails for nulls
Bob Bradley 8 years ago in UNIFYBroker/Microsoft SharePoint • updated by anonymous 8 years ago • 38
Experiencing issues clearing out the start/end dates defined in the schema of my WSS list. FIM export flow rule is set to "allow nulls" which means any existing values in the WSS list should be cleared - but there is no change and I get the dreaded "exported-not-reimported" error. Have confirmed that WSS list fields allow nulls.
Network Trace - No Patch.log
Network Trace - Patch Applied.log
Network Trace - Soap Exception.log
Customer support service by UserEcho
Hi Adam van Vliet - this only became apparent with a different set of test data, and the need to clear out values. Not dead urgent, as most records should have a start date, but it is expected that lots will have an open (null) end date, or that end dates can change/be cleared.
Hi Bob Bradley,
Have you performed a trace to see what Identity Broker is sending out? A potential cause could be that a few releases of Identity Broker v4.1.x there were issues removing previous values (see
Hi Bob Bradley, were you able to check up on this issue?
Adam van Vliet - not yet, but we're in the middle of a large SIT deployment and if there is any issue it is sure to come up here.
Adam van Vliet - I am seeing this again now in both DEV and Production. I am too stretched right now to deal with this one issue alone, but I will need to come back to this very soon as it will become a blocker. If there's anything you already know about this problem for which a patch may already be available, I'm willing to try it first because I am way out of troubleshooting time .
Richard Green - perhaps if you have some bandwidth you may be able to help me diagnose this issue in DEV today - with Adam van Vliet if he happens to be able to spare time to assist?
In DEV the SARa MA is in an infinite loop for 2 users where the attempt by FIM to delete the end date property is failing in the connector (but not reporting an error back to FIM).
Thanks Bob Bradley. Richard Green, perhaps try with Unify.IdentityBroker.Adapter.LDIF.dll from
BCE-356. If it still fails then you may need to trace the call being made to SharePoint - IDBSP41:Enabling Network logging (there's a chance it's still configured, but commented out by Bob).
Looks like it still has the same issue with the patch from
BCE-356in place. Before I rule this out though, I tested it with the patch in the services directory as indicated on the link, however I notice there is also a patches directory in the services folder. Is there a difference to files placed in this folder as opposed to the base services directory?
Hi Richard Green, dll's are loaded from both directories. The patches directory just gets cleared out on upgrade of Identity Broker (after v5.0), so it's ideal for this sort of stuff.
Thanks Adam, that's good to know
As discussed, I've attached network traces of the export transaction with the
BCE-356patch applied and removed. There are 2 updates going out in each case.
Following are some details on the connectors and fields involved in the transactions:
Connector (ID = 435e6a2e-3cfa-41d4-aa8e-cba1f3e8d124, Name = SARa)
Adapter (ID = 87653f10-7115-4fd6-bf46-04addfab67b3, Name = SARa User Access)
Key Field: ID ;Type: Integer
Connector Field: End_x0020_Date ;Type: Date
Adapter Field: Endx0020Date
DN: CN=Bob Willis
Endx0020Date: Export Null (Old value: 2015-04-16T00:00:00.000)
DN: CN=Don Bradman
Endx0020Date: Export Null (Old value: 2015-04-16T00:00:00.000)
I'll also attach the connector and adapter config files for good measure.
Thanks Richard Green. Your trace showed that the connector isn't sending out the field being nulled out, which from memory was the right thing to do and was tested. However, I did come across http://sharepoint.stackexchange.com/questions/3009/updatelistitems-wont-let-you-reset-field-value-to-empty which suggested the field should be sent but empty. I've attached Unify.Connectors.Microsoft.SharePoint.zip that does just that.
Please let me know how it goes.
Does this still require the previous patch installed also? Or did the traces show no difference?
I don't think so, the trace didn't appear any different.
Woohoo - confirming that fix has worked Thanks Adam.
Does this mean a connector repackaging?
Thanks Adam van Vliet - Richard Green just reported this is now working! Will leave this for Richard Green to mark as resolved once deployed to DEV/TEST.
I'll need to schedule a release. Thanks.
Rolled back patch in TEST due to following error which occurred for all 1000 export attempts:
Restored DLLs allowed exports to work as before - but with the fix for nulling of the date fields still pending again
Released v4.1.1 RC3 at SUBIDBSP:Downloads. Please let me know how it goes.
Just got this patch installed in the Origin DEV environment. Testing some export adds produced the following exception. The id field mentioned is the key field - this is not provided on export, and I believe is generated by SharePoint (I will confirm this with Bob).
Yes it's generated by SharePoint, but will fail with this error if string.Empty is provided. Could you check the logic to make sure that's not happening?
Confirming that an empty string is not being exported in this field. I've also rolled back to the original connector and confirmed the record exported as is with that version.
I see the issue. It was relying on some logic higher up to return null, but it was converting to string.Empty. Sorry about that. New version available.
Actually, I think I may have broken the fix to be able to clear out values. Please wait whilst I fix that up.
Okay, good to go now. Thanks.
Woohoo - a new error to replace the last one. And this looks like a really useful one
Let me know if you need me to provide a Trace to assist with this one.
Yes please Richard Green, a trace would be helpful.
Attached a Network Trace of the Soap Exception.
Thanks Richard Green, unfortunately there's not much more detail in the trace. Have any settings on the list changed? Also, I noticed that the connector is now exporting a bunch of new fields, is there anything special about them (e.g. read-only)? If so, I can try removing them by only exporting non-read-only fields?
Recent traced export:
Previous traced export:
Adam van Vliet - knowing what I do about SharePoint lists, yes that is very much going to be the problem here.
The values in the first trace appear to be the ones that correspond to FIM export flow rules - the other are indeed going to be internal to SharePoint and should not be set.
Of the following, the ones I have marked with an X (XName) are the ones that I believe we should (continue to be) treating as read only.
I expect that some special cases above will need to be set on insert (ID=0, GUID?), but not on modify.
I have a change hanging over my head here to change the DN/anchor for this connector to something unique - was thinking of ID but now thinking I need to use GUID (because ID is going to be 0 I think for all new provisioned items). I am using the FIM guid for the GUID property on create.
Thanks Bob Bradley. I'll update the connector to only include items that are not marked as read-only in the connector schema, as well as the ID field and any special cases we come up with.
Adam van Vliet - it really comes down to only writing those attributes that FIM is trying to change isn't it (rather than the complete set)? I think we had a similar challenge with the PowerShell connector.
That's true, but it's not something we have direct access to at the connector layer. We have the current state, optionally the previous state (from the connector store), and optionally the previous state (directly from SharePoint). From that we have to figure out what the desired behaviour is. The read-only setting is probably the easiest way to do it, as it becomes a configuration exercise instead of having to do anything more complicated/slower/error prone (retrieving from either IdB or SharePoint).
I've done another release. I did it as a DEV release this time though as I want to get it right before I do up another candidate.
Latest version (27/5/2015) is looking good so far, Adam van Vliet and Richard Green . Exported all items just now in DEV, including null values for dates, and am not getting the dreaded exported-not-imported error. Will confirm when we are good for a final release version.
Adam van Vliet - please create a release candidate for Origin and alert Richard Green and André van der Westhuizen. The latest one works well thanks, and we could deploy to Origin SIT/UAT/PROD with that, but I suspect you want us to use a new one?
Bob Bradley - released v4.1.1 RTM.
Thanks Adam van Vliet. Richard Green - have just copied this down to DEV (C:\FIM Installers) ready to install when you are.