0
Fixed
SharePoint 2010 List connector failing scheduled polling imports and large lists
Paul Barratt 13 years ago
in UNIFYBroker/Microsoft SharePoint
•
updated by anonymous 9 years ago •
65
Manual polling import started but doesn't seem to finish and Management Studio doesn't show any activity on that connector. Also, all three SharePoint connectors have polling imports scheduled every 10 mins and there is no polling import activity on any of these connectors that I would expect.
In summary:
- Manually started polling import doesn't seem to finish
- Scheduled polling imports on any of the SharePoint connectors doesn't seem to be occurring before or after the manual polling import on the SharePoint External Users connector was initiated.
ConnectorEngine.extensibility.config.zip
Externals connector Configs.zip
IdB_network - Copy.zip
Log.zip
Logs.zip
Network.zip
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.Interfaces.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Connectors.Microsoft.SharePoint.dll
Unify.DLLs.zip
Unify.DLLs.zip
UnifyLog20120606.zip
UnifyLog20120607.csv
UnifyLog20120607.csv
UnifyLog20120611.zip
UnifyLog20120613.csv
UnifyLog20120613.zip
UnifyLog20120614.zip
UnifyLog20120625.zip
Customer support service by UserEcho
The SharePoint External Users connector is throwing the following error
After looking at SoapExceptions thrown when attempting to import from list connectors troubleshooting page the URI and list looks correct.
Will change the
communicator timeout="00:30:00"
to
communicator timeout="02:00:00"
due to the number of objects in this list to ensure it is not timing out before completing an polling or full import.
Hi Adam,
I have attached the latest logs after changing the timeout mentioned in the previous post?
The SoapServerException error is still occuring on the SharePoint External Users connector, but most of the polling imports on this connector work but always seem to be reporting 37k+ changes everytime.
Cheers
Paul
Without seeing the trace the exception isn't much help. I have attached a new version of Unify.Communicators.Moss2007List.dll that pages the requested items slightly differently. Failing that, please trace the failing call, as there may be more detail to the exception.
Hi Paul,
I have attached Unify.Connectors.Microsoft.SharePoint.dll and Unify.Communicators.Moss2007List.dll. These will log additional details for the type of exception that is occurring.
Thanks.
Adam,
Finally captured the error. Please see zipped logs attached.
Sorry it's taking so long, it's a little tricky as I can't reproduce here to ensure that the code will be exactly as needed. I have uploaded a new version of Unify.Communicators.Moss2007List.dll that will log the details properly.
Thank you for your patience.
Adam,
Log files attached using latest dll.
Fantastic, thank you, that is exactly what we are after.
You may have seen in the message the following line:
I had a quick look around and came up with the following resource - http://technet.microsoft.com/en-us/library/cc262787.aspx. There is the message of List view threshold. Are you able to take a look at this setting and see if you are able to set it?
Thanks.
Hi Boyd,
Can you take a look at the link provided by Adam and see if the List View Threshold value can be changed. Although it looks like there is an limit of 20,000 object even for an administrator.
I also find this
http://blogs.technet.com/b/speschka/archive/2009/10/27/working-with-large-lists-in-sharepoint-2010-list-throttling.aspx
which may provide some more information.
Cheers
Paul
List view threshold maxes out at 20,000 objects as Paul pointed out. It can be disabled, however this is not recommended for the reasons listed in this blog and on technet
Although this is a configurable default, we strongly recommend that you keep it. If poorly performing queries are used on lists with more than 5,000 items, overall throughput can significantly decrease when you increase this limit.
Boyd had scalability concerns about the solution reaching 100,000 + Users.
Located the 'SharePoint Server 2010 capacity management: Software boundaries and limits'
Items: 30,000,000 per list. This has increased dramatically since SP 2007. http://technet.microsoft.com/en-us/library/cc262787.aspx
100,000 items shouldn't be too much of a stretch.
One solution might be to implement some type of paging. An example can be found here
Hi Matthew,
The connector already utilises server side paging. This is set using the rowLimit setting on the communicator element.
Thanks.
rowLimit is set to 100 as can be seen in XML I am about to post. It would appear that rowLimit is not being used if we are getting the list view threshold exceeded error.
Additionally, I am unable to get this solution working in dev on DvAP501. The DLLs are the same. The XML config appears to match to me and is valid XML. I will attach XML for comparison.
Error in dev is as follows:
"SharePoint List connector requires a single key of name ID."
Error occurs when starting IdB.
The error with the wrong key is because there is a version mismatch between the two environments. That issue was fixed in v3.0.5.3. If you urgently require that environment to match the other, then I would copy the dll's across. If not, then I would wait until we get this issue sorted and are able to release a candidate v3.0.5 that can be installed correctly.
The row limit is definitely sent through with the request, whether it is the one that is set in config I couldn't say (as there are very many layers between the config and the actual web service request). To prove that the correct value is being sent through there are two options and I'll leave that choice up to you:
a) Trace the web service traffic as you have done previously.
b) I can create a new version of the dll that logs a message right before the web service call that shows the values that are about to be sent via the web service (including the row limit).
I copied all the DLLs from Prod to Dev already. I will upload the DLL zip that I copied across for you to review.
Edit: removed dev information as it is now a new issue.
Latest error and trace can be found in UnifyLog20120614.zip
20120614,04:54:29,Change detection engine import all items failed.,Change detection engine,Warning,"Change detection engine import all items for connector SharePoint External Users Connector failed with reason Value cannot be null.
Parameter name: source. Duration: 00:00:00.4843719
Error details:
System.ArgumentNullException: Value cannot be null.
Parameter name: source
at Unify.Framework.EnumerableExtensions.<ProducePages>d__2`1.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.<SelectManyIterator>d__14`2.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.EnumerableExtensions.<ActionOnFirst>d__1c`1.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Unify.Framework.ActionOnExceptionEnumerator`1.MoveNext()
at Unify.Framework.EnumerableExtensions.<ActionOnLast>d__16`1.MoveNext()
at Unify.Framework.EnumerableExtensions.<ProduceAutoPages>d__9`1.MoveNext()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Framework.Visitor.VisitEvaluateOnThreadPoolT(IEnumerable`1 visitCollection, Action`2 visitor, Int32 maxThreads)
at Unify.Framework.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Framework.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Framework.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.MutexJobDecorator.Run()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)",Normal
20120614,04:58:23,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector started.,Normal
20120614,04:58:23,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
Is the above error related to this issue or the other one that Paul created?
On this issue, I plan on making a change to the connector so that it will set the maximum changed timestamp on the connector full import. That way the connector will not have such a large task to perform during the polling import.
Attached updated Unify.Connectors.Microsoft.SharePoint.dll. This version will save the highest Modified value from a full connector import. This will mean that the delta shouldn't get anywhere near the number of items if a full import has previously succeeded (with this new version).
installed DLL, restarted service. Run full import on SharePoint External Users Connector .
Same error, Logs attached.
Attached new version of Unify.Communicators.Moss2007List.dll. A previous fix for the delta broke the full import. This should work for both and allow the full import to set the maximum change timestamp.
Deployed Dll to both Dev(DVAP501) and Prod(PNAP501). DVAP501 is now processing a delta import every 10 mins for SharePoint
Running full import on SharePoint External connector in IdB. Once this is done delta imports should use high-water mark from the full import.
Added log files after doing a full import on the SharePoint External user connector.
Adam, As you can see from the latest log files I manually started a full import on the SharePoint External Users connector which finish successfully at 04:36:10 UTC, then a polling import started and then failed with the
error. It looks like the query on a polling import is still exceeding the list limit.
Boyd,
Can you send the contents of c:\Program Files\Common Files\Microsoft Shared\web server extensions\14\LOGS from PNWB450 for today. As the polling import is running the same query as the polling but the full import is not hitting this limit. The contents of these logs should shed some light on why these queries differ.
Paul, I have had another read of http://technet.microsoft.com/en-us/library/cc262813.aspx and that had me think about the Modified field, which I had thought was indexed by default.
From our chat before, it is apparent this field is not indexed. For an index to be added, you'll probably have to organise with Boyd a time to increase the list view threshold large enough to allow you to add an index on the Modified field.
Thanks.
Email sent to Boyd
Hi Paul
Our SharePoint administrator is away today. To circumvent this I am deleting records to drop below the 5000 items from the list which can then be re-exported by FIM.
Boyd
From: Paul Barratt (JIRA) jira@unifysolutions.jira.com
Sent: Monday, 25 June 2012 5:03 PM
To: Boyd Bostock
Subject: JIRA (
IDBSP-52) SharePoint 2010 List connector failing scheduled polling imports and large listshttps://unifysolutions.jira.com/s/en_USxu6vbx/733/89/_/jira-logo-scaled.png
https://unifysolutions.jira.com/secure/useravatar?ownerId=paul.barratt&avatarId=10831Paul Barratt<https://unifysolutions.jira.com/secure/ViewProfile.jspa?name=paul.barratt> commented on Defect
IDBSP-52<https://unifysolutions.jira.com/browse/IDBSP-52>SharePoint 2010 List connector failing scheduled polling imports and large lists<https://unifysolutions.jira.com/browse/IDBSP-52>
Email sent to Boyd
Hi Boyd,
As you have probably seen from Adams comments on
{72C1C85B-1D2D-4A4A-90DE-CA74A7808184}IDBSP-52the list needs an index adding for the modify attribute on the External Users as this list exceeds the view list threshold and the indexing operation will take some time we will have to organise a time to temporarily increase or disable the threshold (see http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?PageType=4&ListId=&pID=420<http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?PageType=4&ListId=%7b72C1C85B-1D2D-4A4A-90DE-CA74A7808184%7d&pID=420>).
I have added an index to DEV External Users which didn’t require the threshold to be changed or disabled.
Can you organise a time to complete this work and inform us when it is complete?
Regards
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators<https://unifysolutions.jira.com/secure/ContactAdministrators!default.jspa>.
For more information on JIRA, see: http://www.atlassian.com/software/jira
– Disclaimer: The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Brisbane Catholic Education. The content, including attachments, is a confidential communication between the sender or stated author and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
Brisbane Catholic Education does not warrant or represent that this email is free from viruses, malicious software or defects.
1. External Users list has been dropped to below 4000 entries.
2. Index added to Modified field.
3. Full Import done on all 3 SharePoint lists.
4. Full Import on SharePoint MA performed and Full Synchronization started to regenerate SharePont Exports. However read below because I think more testing is required before the 5000 item threshold is broken again.
Synchronisation Import processes without errors in the IdB log
However initial testing of changes to the list do not seem to be detected. I have tried
1. Adding an entry to the External Users list – User did not appear after a Sync Import, it did after a Full Import
2. Deleting an entry from the External Users list – User was not removed after a Sync Import, it did after a Full Import (Not actually required as date ranges are used)
3. Modifying an email address in the list – Address was unchanged after a Sync Import, it did after a Full Import.
I am happy for you to test by creating dummy records in the Production lists providing it is removed afterwards.
From: Paul Barratt (JIRA) jira@unifysolutions.jira.com
Sent: Monday, 25 June 2012 5:03 PM
To: Boyd Bostock
Subject: JIRA (
IDBSP-52) SharePoint 2010 List connector failing scheduled polling imports and large listshttps://unifysolutions.jira.com/s/en_USxu6vbx/733/89/_/jira-logo-scaled.png
https://unifysolutions.jira.com/secure/useravatar?ownerId=paul.barratt&avatarId=10831Paul Barratt<https://unifysolutions.jira.com/secure/ViewProfile.jspa?name=paul.barratt> commented on Defect
IDBSP-52<https://unifysolutions.jira.com/browse/IDBSP-52>SharePoint 2010 List connector failing scheduled polling imports and large lists<https://unifysolutions.jira.com/browse/IDBSP-52>
Email sent to Boyd
Hi Boyd,
As you have probably seen from Adams comments on
{72C1C85B-1D2D-4A4A-90DE-CA74A7808184}IDBSP-52the list needs an index adding for the modify attribute on the External Users as this list exceeds the view list threshold and the indexing operation will take some time we will have to organise a time to temporarily increase or disable the threshold (see http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?PageType=4&ListId=&pID=420<http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?PageType=4&ListId=%7b72C1C85B-1D2D-4A4A-90DE-CA74A7808184%7d&pID=420>).
I have added an index to DEV External Users which didn’t require the threshold to be changed or disabled.
Can you organise a time to complete this work and inform us when it is complete?
Regards
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators<https://unifysolutions.jira.com/secure/ContactAdministrators!default.jspa>.
For more information on JIRA, see: http://www.atlassian.com/software/jira
– Disclaimer: The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Brisbane Catholic Education. The content, including attachments, is a confidential communication between the sender or stated author and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
Brisbane Catholic Education does not warrant or represent that this email is free from viruses, malicious software or defects.
Hi Adam,
I have recreated this DEV as well. Added a user to the list before this polling import occurred and no changes are detected.
20120626,06:26:20,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector started.,Normal
20120626,06:26:20,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120626,06:26:23,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 0 changes. Duration: 00:00:02.4843750,Normal
20120626,06:26:23,Change detection engine poll completed.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector returned 0 possible changes. Duration: 00:00:02.6250000,Normal
Then I run a full import on the connector
20120626,06:27:36,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector SharePoint External Users Connector started.,Normal
20120626,06:27:36,Request to get all entities from connector.,Connector,Information,Request to get all entities from connector SharePoint External Users Connector.,Normal
20120626,06:28:21,Get all entities from connector completed.,Connector,Information,Get all entities from connector SharePoint External Users Connector return 323 entities. Duration: 00:00:44.5625000,Normal
20120626,06:28:21,Connector Processing started.,Connector Processor,Information,Connector Processing started for connector SharePoint External Users Connector (page 1),Normal
20120626,06:29:04,Connector processing success.,Connector Processor,Information,"Processing page 1 for connector SharePoint External Users Connector processed 323 entities, finding 1 differences. Duration: 00:00:43.3437500",Normal
20120626,06:29:04,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector SharePoint External Users Connector completed. Duration: 00:01:28.2031250,Normal
20120626,06:29:05,Started processing changes register items.,Change detection engine,Information,Started processing changes register items for connector SharePoint External Users Connector.,Normal
20120626,06:29:07,Changes register item processing completed.,Change detection engine,Information,Changes register item processing on connector SharePoint External Users Connector completed. Duration: 00:00:02.3125000,Normal
Do you want me to run a test and leave to see if a polling import detects the change on subsequent polls?
Hi Paul,
From my look at the connector and a bit of research about the modified timestamp, it's a bit difficult to say what the exact cause is.
If you were able to trace just the polling import call when there is a change ready, then I would be able to see if there is something wrong with what is being requested.
Thanks.
.Net Packet capture attached.
Completed .NET network packet capture which contains the requests you are looking for.
Hi Paul,
I found an issue with the time formatting string and have fixed that up in the latest Unify.Connectors.Microsoft.SharePoint.dll. It doesn't really explain why the connector didn't at least attempt to use the query for modified items though. Has a get all succeeded on this connector?
Thanks.
I have deployed the dll in Dev. The change detection only seems to work on a polling import if a full import and then polling import are run consecutively. I ran:
If anything else happens in between the full import and the polling import changes are not detected.
For that entity that you changed, are you able to look in the connector entity search and find the Modified field's value?
Would also be a good idea to confirm that the StoredValueCollection is being correctly updated, could you please find the row that matches the StoredValueCollectionId with the id of the connector?
Thanks.
Deltas not picking up changes
Ran Sync Import
Kylie Andrew Modified 28/06/2012 04:04:20AM
Leah Anglish Modified 31/05/2012 06:50:20AM
Values: <?xml version="1.0" encoding="utf-16"?><StoredValueCollection xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Unify.Framework"><StoredValueCollectionId>adfe8da2-96dc-11e0-8c5c-51b84724019b</StoredValueCollectionId><ValueStore xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays"><d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K><d2p1:Key><ValueName>changeTimestamp</ValueName><_GroupName>adfe8da2-96dc-11e0-8c5c-51b84724019b</_GroupName><_ValueName>changeTimestamp</_ValueName></d2p1:Key><d2p1:Value xmlns:d4p1="http://www.w3.org/2001/XMLSchema" i:type="d4p1:dateTime">2012-06-28T04:04:20Z</d2p1:Value></d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K></ValueStore></StoredValueCollection>
Changed Kylie Andrew to Kylie Andrews, run sync import
no change detected,Modified stamp not changed
Leah Anglish to Peter Anglish, run sync import
No change detected, Modified stamp not changed
Values: <?xml version="1.0" encoding="utf-16"?><StoredValueCollection xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Unify.Framework"><StoredValueCollectionId>adfe8da2-96dc-11e0-8c5c-51b84724019b</StoredValueCollectionId><ValueStore xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays"><d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K><d2p1:Key><ValueName>changeTimestamp</ValueName><_GroupName>adfe8da2-96dc-11e0-8c5c-51b84724019b</_GroupName><_ValueName>changeTimestamp</_ValueName></d2p1:Key><d2p1:Value xmlns:d4p1="http://www.w3.org/2001/XMLSchema" i:type="d4p1:dateTime">0001-01-01T00:00:00</d2p1:Value></d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K></ValueStore></StoredValueCollection>
Sync Import
Kylie Andrew Modified 28/06/2012 04:26:51AM
Leah Anglish Modified 28/06/2012 04:32:32AM
Both Changes detected
<?xml version="1.0" encoding="utf-16"?><StoredValueCollection xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Unify.Framework"><StoredValueCollectionId>adfe8da2-96dc-11e0-8c5c-51b84724019b</StoredValueCollectionId><ValueStore xmlns:d2p1="http://schemas.microsoft.com/2003/10/Serialization/Arrays"><d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K><d2p1:Key><ValueName>changeTimestamp</ValueName><_GroupName>adfe8da2-96dc-11e0-8c5c-51b84724019b</_GroupName><_ValueName>changeTimestamp</_ValueName></d2p1:Key><d2p1:Value xmlns:d4p1="http://www.w3.org/2001/XMLSchema" i:type="d4p1:dateTime">0001-01-01T00:00:00</d2p1:Value></d2p1:KeyValueOfGroupedNameValueCollectionKeyanyTypebtyk_P19K></ValueStore></StoredValueCollection>
Trace of Sync Import attached.
Network Trace
Attached new version of Unify.Connectors.Microsoft.SharePoint.dll. On closer inspection of the storedvaluecollection, I could see that the timestamp was changed to a default value which was wrong. It would work when at least one item was returned during the polling import, but if it ever retrieved none it would fail.
Are you able to run the connector get all import? As the value was stuffed up.
This version should now at least send a correct value to the web service. As discussed though, I may need to alter the time due to a possible SharePoint issue.
Thanks.
Seems to be detecting new changes properly. Will need more testing. Looking at the logs below, One manually run sync import appears to run the sync several times. Is this the case?
There are some places that get logged a few times on the way through, but I hadn't thought the entries would look so similar like this.
Can you monitor and let me know if it happens again?
It appears to loop through each time the Connector sync import runs. Logs below. IdB SharePoint connectort is not detecting changes on Sync import today.
20120702,01:24:02,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector started.,Normal
20120702,01:24:02,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120702,01:24:05,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 0 changes. Duration: 00:00:02.5000000,Normal
20120702,01:24:05,Change detection engine poll completed.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector returned 0 possible changes. Duration: 00:00:02.5156250,Normal
20120702,01:24:05,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint School Roles Connector started.,Normal
20120702,01:24:05,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint School Roles Connector.,Normal
20120702,01:24:05,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint School Roles Connector reported 0 changes. Duration: 00:00:00.2500000,Normal
20120702,01:24:05,Change detection engine poll completed.,Change detection engine,Information,Change detection engine poll for connector SharePoint School Roles Connector returned 0 possible changes. Duration: 00:00:00.2656250,Normal
20120702,01:24:05,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Roles Connector started.,Normal
20120702,01:24:05,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Roles Connector.,Normal
20120702,01:24:05,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Roles Connector reported 0 changes. Duration: 00:00:00.2500000,Normal
20120702,01:24:05,Change detection engine poll completed.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Roles Connector returned 0 possible changes. Duration: 00:00:00.2656250,Normal
20120702,01:24:26,Adapter request to get attribute changes from adapter space.,Adapter,Information,Adapter request to get attribute changes from adapter space 85c3f1c8-970a-11e0-8262-b8ee4724019b.,Normal
20120702,01:24:27,Adapter get attribute changes completed.,Adapter,Information,Adapter get attribute changes for adapter space 85c3f1c8-970a-11e0-8262-b8ee4724019b completed. Duration: 00:00:00.0312500,Normal
20120702,01:25:21,Change detection engine poll started.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector started.,Normal
20120702,01:25:21,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120702,01:25:24,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 1 changes. Duration: 00:00:02.5156250,Normal
20120702,01:25:24,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120702,01:25:24,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 0 changes. Duration: 00:00:00.2343750,Normal
20120702,01:25:24,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120702,01:25:24,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 0 changes. Duration: 00:00:00.2500000,Normal
20120702,01:25:24,Request to get entities from connector.,Connector,Information,Request to get entities Count:0 from connector SharePoint External Users Connector.,Normal
20120702,01:25:24,Get entities from connector completed.,Connector,Information,Get entities Count:0 from connector SharePoint External Users Connector returned 0 entities. Duration: 00:00:00,Normal
20120702,01:25:24,Request to get changes from connector.,Connector,Information,Request to get changes from connector SharePoint External Users Connector.,Normal
20120702,01:25:24,Get changes from connector completed.,Connector,Information,Get changes from connector SharePoint External Users Connector reported 0 changes. Duration: 00:00:00.2343750,Normal
20120702,01:25:24,Change detection engine poll completed.,Change detection engine,Information,Change detection engine poll for connector SharePoint External Users Connector returned 0 possible changes. Duration: 00:00:03.3437500,Normal
20120702,01:26:47,Adapter request to get attribute changes from adapter space.,Adapter,Information,Adapter request to get attribute changes from adapter space 85c3f1c8-970a-11e0-8262-b8ee4724019b.,Normal
20120702,01:26:47,Adapter get attribute changes completed.,Adapter,Information,Adapter get attribute changes for adapter space 85c3f1c8-970a-11e0-8262-b8ee4724019b completed. Duration: 00:00:00.0312500,Normal
Re the flood of imports, I just remembered something similar happening with the polling import. It was only happening for Tony, and we were never able to see what was causing it. I recalls having to restart Identity Broker Management Studio.
Regarding the changes coming through, are you able to run through a similar process that you did on 28/Jun/12 3:00PM? Following that, if all the information has been fixed up, then a trace of one polling import will show that the request is correct.
Thanks.
Attached:
Unify.Connectors.Microsoft.SharePoint.dll
Unify.Communicators.Moss2007List.dll
Unify.Communicators.Moss2007List.Interfaces.dll
Matthew if you set a new attribute on the list connector you can control the timestamp that is requested for the polling import:
<communicator pollingChangeTokenOffset="-1.00:00:00" />
This may result in false returns, but I suspect it is better than not functioning at all.
When you do get time, however, please also look at the previous comment and trace the calls so that we can prove that the connector is making the correct request to SharePoint.
Installed DLLs and new attribute
ran Full Import,
Made change in Sharepoint, then run Sync Import in IdB, followed by delta import in FIM. repeated several times. Delta worked initially, then stopped. Logs below.
If the above was accomplished by having that offset of 1 day, then it would be useful to know the state of the storedvaluecollection and the trace of the web service call for each run. After
BCE-187obviously.@boyd: I believe we have a fix for this issue. Can you please let me know when a suitable time to update and test on PNAP501 might be?
Thanks,
Matthew
Now is fine.
From: Matthew Woolnough (JIRA) jira@unifysolutions.jira.com
Sent: Thursday, 5 July 2012 11:21 AM
To: Boyd Bostock
Subject: JIRA (
IDBSP-52) SharePoint 2010 List connector failing scheduled polling imports and large listshttps://unifysolutions.jira.com/s/en_US-hijllw/734/89/_/jira-logo-scaled.png
https://unifysolutions.jira.com/secure/useravatar?avatarId=10152Matthew Woolnough<https://unifysolutions.jira.com/secure/ViewProfile.jspa?name=matthew.woolnough> commented on Defect
IDBSP-52<https://unifysolutions.jira.com/browse/IDBSP-52>SharePoint 2010 List connector failing scheduled polling imports and large lists<https://unifysolutions.jira.com/browse/IDBSP-52>
@boyd: I believe we have a fix for this issue. Can you please let me know when a suitable time to update and test on PNAP501 might be?
Thanks,
Matthew
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators<https://unifysolutions.jira.com/secure/ContactAdministrators!default.jspa>.
For more information on JIRA, see: http://www.atlassian.com/software/jira
– Disclaimer: The content of this e-mail is the view of the sender or stated author and does not necessarily reflect the view of Brisbane Catholic Education. The content, including attachments, is a confidential communication between the sender or stated author and the intended recipient. If you are not the intended recipient, any use, interference with, disclosure or copying of this e-mail, including attachments, is unauthorised and expressly prohibited. If you have received this e-mail in error please contact the sender immediately and delete the e-mail and any attachments from your system.
Brisbane Catholic Education does not warrant or represent that this email is free from viruses, malicious software or defects.
Spoke with boyd, deltas look as though they are functioning properly.
Boyd is going to start exporting accounts into SharePoint in greater numbers now.
Resolved.
Matthew, this is available in the v3.0.5.8 RC on SUBIDBSP:Downloads.
Please close if satisfactory.
Thanks.