Questions
IdB for chris21 seems to take significantly long period of time to perform "Full Import" from chris21 system
The Identity Broker for chris21 seems to take significantly long period of time to perform "Full Import" from chris21 system into Identity Broker Connector and then process/transform the data into the IdB Adapter.
I am not familar with the flow process and the analysing the log file content. Thus the below investigation may not be entirely correct. But the overall time taken for processing data does seems significantly long in Identity Broker for chris21.
The customer has 4999 records/users in chris21.
Analysis 1:
1 - When performing the "Import All Connectors" for all six (6) Connectors took approximately 32 minutes at other times it does seems to take significantly longer time, eg 45 minutes. During this time the CPU and disk accessing activities are at 100%.
Started at
20101215,07:22:14,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Address Connector started.,Normal
Finished at
20101215,07:54:15,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Org Unit Connector completed. Duration: 00:02:34.7343750,Normal
2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 10 minutes. At other times we believe it took significantly longer.
Started at
20101215,07:55:56,Adapter request to get attribute changes from adapter space.,Adapter,Information,Adapter request to get attribute changes from adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal
Finished at
20101215,08:05:46,Adapter get attribute changes completed.,Adapter,Information,Adapter get attribute changes for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 completed. Duration: 00:09:49.8125000,Normal
It has been mentioned that the "Import All Connectors" should take only a few minutes. Thus I would like to request a review of the IdB Connector and Adapter configuration files (attached) as to what could possibly create this behaviour. It is mentioned that the gtrChunkSize="1000" could be in appropriate for the volume of data in chris21.
Analysis 2:
Another example: When performing the "Import All Connectors" for all six (6) Connectors took approximately 14 minutes
Start at
20101213,07:01:22,Change detection engine import all items started.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Person Connector started.,Normal
Finished at
20101213,07:15:17,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Chris21 Placement Connector completed. Duration: 00:02:19.6875000,Normal
2 - When performing the "Full Import" in FIM for chris21 MA it took approximately 27 minutes 30 seconds.
Started at
20101213,07:25:53,Request to get all entities for adapter space.,Adapter,Information,Request to get all entities for adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3.,Normal
Finished at
20101213,07:53:34,Adapter get all entities to adapter space completed.,Adapter,Information,Adapter get all entities to adapter space d5251d7d-05cc-4d35-916e-e99e06b9ffd3 returned 4999 entities. Duration: 00:27:41.0937500,Normal
Analysis 3:
"Import all connectors" in IdB took approximately 12 minutes. I do not have the log file
"Full Import" in FIM for chris21 MA took approximately 6 minutes.
Please help to review the Connector and Adapter configuration files and also advice if the above time is acceptable or not.
UnifyLog20101213-extract.csv
UnifyLog20101215-extracted.csv
Customer support service by UserEcho
Configure the chunk size setting.
It is currently set to 1000, which means 5 GTR requests are required to get the data. If you look at the log during a full import of the person connector, you will notice each chunk takes longer and longer to get the requested data.
To optimise this setting, try getting all records in one chunk (gtrChunkSize="5000") and compare this time to the current results (gtrChunkSize="1000"). This will give you a range comparison.
Next step is try the GTR request with 2 calls (gtrChunkSize="2500"). This should then be compared to the previous results.
Note that their current list is 4999, so the results of setting "5000" will be invalid, when they add 2 more users (5001). Due to this, use a setting of "6000" if one call is the optimal. Try 2 calls at "3000", then attempt 3 calls at "2000", the results of these should be enough for the setting to be optimised.
When creating the issue I could not find a mean to attached the log files mentioned. They are attached now for referrencing.
Hi Rodney,
Now that you highlighted the timing for each sub-total I be see that for each subsequence 1000 chunk the time does take significant longer (ranging from 20 seconds to 40 seconds) and yes they do add up.
I will advise the customer (AHG) to try the combination of setting the gtrChunkSize="3000". It will be a while yet before they hit the 6000 mark.
My understanding is there no similar setting for the IdB Adapter configuration file when performing "Full Import" from FIM chris21 MA. Thus the duration will depend on machine capability, load, network etc.
Is this correct?
Please help to clarify the Adapter part of the configuration in regarding to the duration it may takes.
Adapter performance is nothing to do with Identity Broker for Frontier chris21 - please ask appropriate question in Identity Broker project if you want to know more about adapter configuration.
Answer provided.