Connector full import synchronization aborts encompassing transaction against large number of deletes (> 500,000 entities)

Tony Sheehy 13 years ago updated by anonymous 9 years ago 4

A connector full import/sync that results in a large number of deletions (> 500,000) will abort its respective encompassing TransactionScope.

The following was additionally observed when the TransactionScope was observed to abort:

  • When debugging, the TransactionScope itself is registered as complete. (TransactionScope::complete = true)
    • From this, it can be potentially inferred that the problem itself happens either on the MSDTC or SQL Server end - as far as the IdB application is concerned, the TransactionScope completed.
  • The exception was thrown during IDB ChangeDetection.
  • If debugging, ensure that break on all exceptions is enabled.
  • During the import, in the respective SQL traces, a substantial number of entries were logged with the following EventClass and ApplicationName.
    EventClass: Lock:Timeout
    ApplicationName: .Net SqlClient Data Provider

The TransactionScope was timing out because all deletions would be pulled in at once (~500,000 entities in test import). This was compounded by the fact that although the TransactionScope timeout was equal to ~2.7 hours, the Maximum available Transaction timeout defaults to 10minutes and is only configurable in the respective host machine.config. This configuration needs to be set in the following way:

	<machineSettings maxTimeout="<MAX_TIMEOUT: e.g. 2:00:00>"/>
	<defaultSettings timeout="<DEFAULT_TIMEOUT: e.g. 00:03:00>"/>

Additionally the following can be set to enable editing from the application side:

<sectionGroup name="system.transactions" type="System.Transactions.Configuration.TransactionsSectionGroup, System.Transactions, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null">
	<section name="defaultSettings" type="System.Transactions.Configuration.DefaultSettingsSection, System.Transactions, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null"/>
	<section name="machineSettings" type="System.Transactions.Configuration.MachineSettingsSection, System.Transactions, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089, Custom=null" allowDefinition="MachineOnly" allowExeDefinition="MachineOnly"/>

This however is only a workaround, and the entire Full import paging mechanism will need to be replaced such that entities are paged/deleted in separate transactions.

Reassigned for confirmation of completeness.

Covered by 6.0 Performance in the regression test document