0
Fixed

Connector scheduled import not clearing old records

Boyd Bostock (BCE) 10 years ago updated by anonymous 6 years ago 18

It has come to our attention that the Relief Staff Adaptor/Connector is (on the majority) adding new relief assignments, but is not dropping them from groups on the next day when they are not scheduled. It appears to only be a problem with the groups, not an issue with eMinerva permissions.

We have confirmed the Oracle View is reading correctly. If we clear the connector, then run an import it is reading correctly.

I also had an issue where one staff member was not processed on a Delta Import, it took a Full Import to bring it through to ILM.

This (non-scheduled staff not removed) happened last week as well. You will not be able to do testing until tomorrow as I have cleared the connector already. Dev is not suited to this testing as it will not contained Relief Schedules for today.


.jpg
Connector Search showing 7 Relief Staff @ School 401.jpg
UnifyLog20120314.csv
Affected Versions:
Fixed by Version:

Hi Adam, I asked Shane to deal with this on Wednesday ... I believe he is sick today. This looks like an IdB issue ... Can you look at it or assign it to someone and get back to Boyd on the status. I would but I am on leave today.

Hi Boyd,

Could you please confirm the connector and adapter ids that you are referring to?

Does an import into Identity Broker (without first clearing it) not resolve the issue?

Thanks.

Connector = Talent 2 Relief Staff Planner Connector
Adapter = Relief Staff Composite Adapter

Full imports are scheduled every 10 minutes or so. A Synchronise import is not possible with Talent/Oracle.

I have started running Full Imports from ILM every half hour instead of deltas, this ensure that every relief teacher is given access. This resolves the issue where they were not given access but did not resolve the issuse with taking access away. Previously on one Full Import was done per day at around 4am.

The way to idenity them would be to clear the connector, repopulate and Run a Full Import Sync in ILM. There should generally not be any access removed until the first run after midnight.

Hi Boyd,

Could you please let me know the times of the two imports that show first the missing change, and then it coming through?

Are you referring to group membership for a Relief Staff being removed? Or the user itself? If it's the membership, is it for PortalMembers or ReliefDistributionGroup?

As mentioned above, does an import into Identity Broker (without first clearing it) not resolve the issue?

Thanks.

Hi Adam
I have not done anything outside of the scheduled imports and ILM actions. Below is an example that was logged today.

As at 12:30 there were 212 Relief teachers scheduled to work in BCE Schools. Below are the details of 2 users (similar name) that are not scheduled today but still have access.

School 402

Name: Diane Mackenzie
Title: Teacher - U MT GRAV'T (S)
Employee ID: 021579
It is suspected that the school has timesheeted the wrong Di Mackenzie, which has resulted in Diane (021579) recieving email from school 402. However she is no longer timesheeted at the school and should have been removed from the groups. We believe this was entered on the 7th.

AD Groups
Still a member of IM_STAFF_402_RELIEF_Distribution & IM_STAFF_402_RELIEF_Portal
Still has eMinerva permissions to school 402

Name: Diana MacKenzie
Title: Relief Teacher - Teacher - RELIEF
Employee ID: 021822

Member of AD Groups & has eMinerva Permissions. Was correctly Time sheeted but she is not scheduled to work today.

I had to sort this out as it was impacting staff.

1. Run a Full import on Connecter
2. Could not see user in Adaptor
3. Search in ILM confirmed she still had a Relief Staff connecter
4. Run a Full Sync in ILM, no change
5. Run a Full Import and Full Sync, connector was deleted and exports generated.

I assume this was due to the Full Import on the Connector but it is my understanding Full Imports are all that is supported with T2/Oracle.

Hi Boyd,

So the full import into Identity Broker, and then into ILM has resolved this issue?

As the scenario was for a user being removed from Talent2, this required a full import against the Talent2 connector.

The Talent2 connector only supports Full Imports into Identity Broker. Whereas Identity Broker can generate Delta Imports for all exposed connectors into ILM.

Thanks.

Those were the steps I took this morning and it resolved the issue.

However because the following steps below are scheduled to occur I don't understand how my actions this morning triggered it to remove previous assignments

  • Connector is scheduled to run ever 10 minutes (I think) which are Full Imports because Delta's are not possible
  • ILM Full Import and Full Sync runs at 4:00AM every day
  • ILM Full Import and Delta Sync scheduled every 30 minutes.

I have had a look at this issue today, and this is what appears to be happening:

Identity Broker is performing a "get all entities" every 15 minutes for the "Relief Staff Planner" connector. However, this doesn't seem to change the contents of the connector.

I have uploaded a coy of the relevant log-file which appears to show that some differences are identified during the period of the log-file.

I have uploaded a copy of the UMC showing the incorrect connector search results (whereby 7 entries appear n the list for school 40).

I have uploaded a copy of the UMC showing no entries for relief staff for school 401 after executing a full-import from UMC.

If a "Full Import" is executed from the UNIFY Management Console, the connector entries are updated in the connector entity search.

This doesn't seem to be the case when an automatic full import is executed.

Screenshot after "Full Import" from UMC... now shows no relief staff at School 401

Identity Broker Log-file

To save me some time, could you please point out the lines for the two different imports?

Thanks.

Here is the Manual Import:

20120314,05:48:12,Request to get all entities from connector.,Connector,Information,Request to get all entities from connector Talent2 Relief Staff Planner Connector.,Normal
20120314,05:48:12,Get all entities from connector completed.,Connector,Information,Get all entities from connector Talent2 Relief Staff Planner Connector return 161 entities. Duration: 00:00:00,Normal
20120314,05:48:12,Connector Processing started.,Connector Processor,Information,Connector Processing started for connector Talent2 Relief Staff Planner Connector (page 1),Normal
20120314,05:48:12,Connector processing success.,Connector Processor,Information,"Processing page 1 for connector Talent2 Relief Staff Planner Connector processed 161 entities, finding 0 differences. Duration: 00:00:00.4999520",Normal
20120314,05:48:13,Change detection engine import all items completed.,Change detection engine,Information,Change detection engine import all items for connector Talent2 Relief Staff Planner Connector completed. Duration: 00:00:01.2655035,Normal

The rest are automatic ones

I can demonstrate a case of this occurring within the UNIFY lab. (I think).

Connect to the following computer: 192.168.16.230 (Administrator / P@ssw0rd);

The CSV connector is currently configured to perform full imports every minute;

These full imports only appear to update if you add a value to the CSV file. If you remove a value from the CSV file it isn't updated in the connector search until you perform a "full import" from the console.

Updated IdB deployed to UNIFY Test Lab which has proven to be successful. Thanks Adam, love your work.

Will need to run through some more extensive testing & await release of update prior to deployment at BCE... but at least this issue appears resolved.

Tested in environment that was exhibiting the same behaviour.

Resolved.

To be confirmed in regression test.

Added 3.1.8 and 3.1.9 for QDET-150 to v4 regression test document.