I have an intermittent problem with particular PowerShell connectors that intermittently hang on the Polling import - in that the connector displays as running the polling import for days, from the logs nothing is happening, and the only way to stop it is to restart the IDB service.

I have three connectors that connect to Exchange (two different Exchange environments), and we have seen the problem on all three connectors, in all three environments (dev, test, prod). I have other PowerShell connectors that do not have this problem. We have also never seen the problem on the Import All.

The three connectors run the same script, just with different parameters. I have added detailed logging for Polling runs and can't find a pattern - the log files stop at different places. Sometimes it's while collecting data from Exchange, but just as often it's after the script has closed the connection to Exchange and is looping through updating the entities in IDB.

Is there any way to enforce a timeout in the Powershell connector?

I suddenly have an error with a connector that is one of three identical Powershell connectors (same underlying scripts, just different parameters specifying target domain). The data gathering part of the script is working fine. The script is also getting through the entire entity creation loop (I have dropped detailed logs), but is then failing after that (ie at the end of the script) with 0 entities created.

This is the error reported in the IdB log:

Change detection engine import all items failed.
Change detection engine import all items for connector PowerShell HomeFolder Protected failed with reason One or more errors occurred.. Duration: 00:08:50.4965198
Error details:
System.AggregateException: One or more errors occurred. ---> System.ComponentModel.Win32Exception: The network path was not found
--- End of inner exception stack trace ---
at Unify.Product.IdentityBroker.PowerShellConnector.d__30.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at Unify.Product.IdentityBroker.AuditReadingConnectorDecorator.GetAllEntities(IStoredValueCollection storedValues, CancellationToken cancellationToken)
at Unify.Product.IdentityBroker.EventNotifierReadingConnectorDecoratorBase`1.GetAllEntities(IStoredValueCollection storedValues, CancellationToken cancellationToken)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c__DisplayClass30_0.b__0()
at Unify.Framework.AsynchronousJobExecutor.PerformJobCallback(Object state)
---> (Inner Exception #0) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #1) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #2) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #3) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #4) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #5) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #6) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #7) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #8) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

---> (Inner Exception #9) System.ComponentModel.Win32Exception (0x80004005): The network path was not found<---

I have just upgraded IdB in TEST to 5.2, and migrated in the Connector and Adapter files from Dev. Dev was already on 5.2 and all connectors are working.

In TEST all of the new Connectors (names "PowerShell HomeFolder*" and "PowerShell MemberOf*") are failing with the error below. The "Powershell Exchange*" connectors work fine (though they already pre-existing in IdB 5.1 before I upgraded to 5.2).

The Connector config file is the same as the one I sent with the previous question. While the error looks very similar to that one it can't be the same - that was a duplicate schema mapping in the connector config, but Powershell connectors don't have schema mapping.

My Import scripts drop a full log of all entity values before running the $entity.Create ... $entity.Commit loop. There are no duplicate sAMAccountNames.

Note that when I did the IdB database upgrade in Test I removed the final lines from the script as told to do here. I don't seem to have had any problems with this in Dev, but thought it worth mentioning.

Change detection engine import all items failed.
Change detection engine import all items for connector PowerShell HomeFolder NMI failed with reason An error occurred while evaluating a task on a worker thread. See the inner exception details for information.. Duration: 00:01:37.5326976
Error details:
Unify.Framework.EvaluatorVisitorException: An error occurred while evaluating a task on a worker thread. See the inner exception details for information. ---> System.ArgumentException: An item with the same key has already been added.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
at Unify.Product.IdentityBroker.Repository.EntityLinqQueryConverterUtilitiesBase`4.GetCollectionKeyData(TEntityKey key, EntityDataContext sourceContext)
at Unify.Product.IdentityBroker.Repository.EntitySingleValueDataUtilityBase`2.CreateEntityValue(TEntityKey key, IValue value, IEntityCollectionKeyUtility`1 collectionKeyUtility, EntityDataSet set, __EntityInsertRow row, EntityDataContext sourceContext)
at Unify.Product.IdentityBroker.Repository.KnownEntityContextBase`4.ConvertEntityValueToDataValue(KeyValuePair`2 entityValueAndKey, __EntityInsertRow row, EntityDataSet entityDataSet, EntityDataContext sourceContext)
at Unify.Product.IdentityBroker.Repository.KnownEntityContextBase`4.<>c__DisplayClass31_0.<convertitemtovalues>b__0(KeyValuePair`2 entityValueAndKey)
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.<selectmanyiterator>d__16`2.MoveNext()
at Unify.Framework.Visitor.Visit[T](IEnumerable`1 visitCollection, Action`2 visitor)
at Unify.Product.IdentityBroker.Repository.KnownEntityContextBase`4.InsertItems(ISet`1 addedItems, EntityDataContext sourceContext, SqlConnection connection)
at Unify.Framework.Data.LinqContextConversionBase`4.SubmitChanges()
at Unify.Product.IdentityBroker.SaveChangedEntitiesTransformationUnit.Transform(IDictionaryTwoPassDifferenceReport`4 input)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetectionOnConnectorEntityPage(IEnumerable`1 connectorEntities, Int32& index, Int32 entitiesProcessedSoFar, IEntityChangesReportGenerator`2 reportGenerator, IHashSet`1 seenKeys)
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.<>c__DisplayClass11_0.<performchangedetection>b__0(IEnumerable`1 page)
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.ThreadsafeItemEvaluator.Evaluate()
--- End of inner exception stack trace ---
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.CheckForException()
at Unify.Framework.Visitor.ThreadsafeVisitorEvaluator`1.Visit()
at Unify.Product.IdentityBroker.RepositoryChangeDetectionWorkerBase.PerformChangeDetection(IEnumerable`1 connectorEntities)
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.ImportAllChangeProcess()
at Unify.Product.IdentityBroker.ChangeDetectionImportAllJob.RunBase()
at Unify.Framework.DefinedScopeJobAuditTrailJobDecorator.Run()
at Unify.Product.IdentityBroker.ConnectorJobExecutor.<>c__DisplayClass30_0.<run>b__0()
at Unify.Framework.<span class="redactor-selection-marker" id="selection-marker-1"></span>AsynchronousJobExecutor.PerformJobCallback(Object state)
</run></performchangedetection></selectmanyiterator></convertitemtovalues>

I have been struggling with an odd issue. I have a number of PowerShell connectors in my solution, and quite a few of them have Boolean attributes (in addition to String). I have created a new connector which is very similar to the existing ones, and uses very similar scripts. Here is what is happening:

- When I comment out my import step for the Boolean attribute (so it just imports the String attributes) the full import works

- When I add the Boolean attribute in the import fails with "Access Denied". I know the correct value is actually being generated as I'm dropping a debug log file which shows all the values it's trying to put into the entity.

- When I run a Polling import (where I can list account names in a text file and it just imports those) it works correctly, including bringing in the Boolean value. This is also weird as it's the same Powershell script - the differences are in the initial collection of data, but the part where it updates and commits the entities is identical for both full and polling imports.

I thought it might be something to do with $null values but have made sure that the script always sets a default value of $false. This is the error message - it is definitely happening during the loop where I go through the data I've collected committing the entities.

I've also upgraded IdB to 5.2, but no change from 5.1 on this issue.

Change detection engine import all items for connector PowerShell HomeFolder failed with reason Access is denied. Duration: 00:01:19.9160765

Error details:

System.UnauthorizedAccessException: Access is denied ---> System.ComponentModel.Win32Exception: Access is denied

   --- End of inner exception stack trace ---

   at Unify.Framework.Auditing.AuditingExtensions.<>c__DisplayClass4_0.<TaskContinueWithExceptionPassthough>b__0(Task t)

   at System.Threading.Tasks.Task.Execute()",Normal

Been searching the doco and other Voice questions but I can't find this one - how do I access a multivalue string attribute in a PowerShell connector script?

I have tried $entity["attrib"].Values, but that doesn't work.

Using $entity["attrib"].Value seems to return a long string with all the values joined by semi-colons - at least that's how it gets written to the logger. I tried splitting on semi-colon but got "Method invocation failed because [Unify.Framework.StringValue] does not contain a method named "Split"."

It would be helpful if there were examples for different data types including multi-value on this page: https://voice.unifysolutions.net/knowledge-bases/7/articles/2912-powershell-connector-entities.

I am developing a Powershell connector. It uses exactly the same script for Full and Polling imports, just with a "-RunType Delta" switch for polling. The difference is all about how it detects how many users to look at - once it gets to creating the IdB entities the script is identical.

I have four new objects in the external system. They are correctly identified by the script on a Polling import, however the new entities are not created in IdB. When I run a Full import the entities are created. (I have done searches to confirm this)

The following log excerpt shows the four entities that should be created, but the changes reported are 0:

20171211,04:47:22,UNIFY Identity Broker,Logging,Information,Exchange Protected Import: Creating 4 entities,Normal
20171211,04:47:23,UNIFY Identity Broker,Logging,Information,"
Key   : DN
Value : CN=Supressed1\, Changed1,OU=ACT,OU=Users,OU=Accounts,OU=DEV,DC=domain
Key   : PersonNumber
Value : 20523
Key   : TerminationAutoReply
Value : False
Key   : Status
Value : ACTIVE
Key   : HiddenFromGAL
Value : False
",Normal
20171211,04:47:23,UNIFY Identity Broker,Logging,Information,"
Key   : DN
Value : CN=Supressed2\, Changed2,OU=ACT,OU=Users,OU=Accounts,OU=DEV,DC=domain
Key   : PersonNumber
Value : 11831
Key   : TerminationAutoReply
Value : False
Key   : Status
Value : ACTIVE
Key   : HiddenFromGAL
Value : False
",Normal
20171211,04:47:23,UNIFY Identity Broker,Logging,Information,"
Key   : DN
Value : CN=Supressed3\, Changed3,OU=ACT,OU=Users,OU=Accounts,OU=DEV,DC=domain
Key   : PersonNumber
Value : 73564915
Key   : TerminationAutoReply
Value : False
Key   : Status
Value : ACTIVE
Key   : HiddenFromGAL
Value : False
",Normal
20171211,04:47:23,UNIFY Identity Broker,Logging,Information,"
Key   : DN
Value : CN=Supressed4\, Changed4,OU=ACT,OU=Users,OU=Accounts,OU=DEV,DC=domain
Key   : PersonNumber
Value : 18582
Key   : TerminationAutoReply
Value : False
Key   : Status
Value : ACTIVE
Key   : HiddenFromGAL
Value : False
",Normal
20171211,04:47:23,UNIFY Identity Broker,Connector,Information,"Request to import changes from connector.
Request to import changes from connector PowerShell Exchange PROTECTED.",Normal
20171211,04:47:23,UNIFY Identity Broker,Connector,Information,"Import changes from connector completed.
Import changes from connector PowerShell Exchange PROTECTED reported 0 changes. Duration: 00:00:00",Normal
20171211,04:47:23,UNIFY Identity Broker,Change detection engine,Information,"Change detection engine import changes completed.
Change detection engine import changes for connector PowerShell Exchange PROTECTED returned 0 possible changes. Duration: 00:00:04.8439784",Normal

The part of the script the creates the entities is as follows. When I generated that log above I had the two lines uncommented that log the full $entity:

###
### Create/update entities
###
if ($ManagedUsers.count -gt 0) 
{
    $logger.LogInformation("$LogPrefix Creating {0} entities" -f $ManagedUsers.count.ToString())
    foreach ($user in $ManagedUsers)
    {
        $entity = $entities.Create()
        $entity["PersonNumber"] = $user.employeeNumber
        $entity["DN"] = $user.DistinguishedName
        $entity["Status"] = $user.extensionAttribute13
        if ($Mailboxes.ContainsKey($user.DistinguishedName))
        {
            $mb = $Mailboxes.($user.DistinguishedName)
            $entity["DirectPermissions"] = $mb.DirectPermissions
            $entity["HiddenFromGAL"] = $mb.HiddenFromGAL
            $entity["MailboxType"] = $mb.MailboxType
            $entity["PrimaryEmailAddress"] = $user.mail
            $entity["ProxyEmailAddresses"] = $mb.EmailAddresses
            $entity["TerminationAutoReply"] = $mb.TerminationAutoReply
            $entity["TerminationMailCount"] = $mb.TerminationMailCount
        }
        else
        {
            $entity["DirectPermissions"] = $null
            $entity["HiddenFromGAL"] = $false
            $entity["MailboxType"] = $null
            $entity["PrimaryEmailAddress"] = $null
            $entity["ProxyEmailAddresses"] = $null
            $entity["TerminationAutoReply"] = $false
            $entity["TerminationMailCount"] = $null
        }
        #[string]$str = $entity | fl | out-string
        #$logger.LogInformation($str)
        $entity.Commit()
    }
}

I'm on v5.1.0 Revision #1. I should be able to upgrade but it's a bit of a process to get software into the environment, so is there anything else I should be looking at?

With the Powershell connector I add lots of logging into my scripts. When troubleshooting I want to bump the log level up to Verbose so I can see my Information logs - however IdB UI search logging also seems to run at this level. So if I put "My Powershell script" as a search in the IdB Logs UI it fills up with lots of logging about that particular search string, making it hard for me to track my own logs. Could the IdB search logging be moved to a Diagnostic setting?

I currently have a PowerShell Connector that reads in image binary data utilizing the FIMLDIFAdapter service (http://localhost:59990/IdentityBroker/FIMLDIFAdapter.svc). I currently use the ImportAll function provided by this service and it works providing the maximum message size quota for incoming messages is set fairly high (3040032000 bytes to be specific). If i try a lower value (i.e. 2040032000 bytes, this will cause the connector to fail with an exception "The maximum message size quota for incoming messages (2040032000) has been exceeded.").

I was wanting to know if there was a more efficient way to read from this service (another function I've possibly missed?), other than just increasing this value, as the more users that have images in the adapter the larger this quota needs to be.

Thankyou

Import All and Import Changes both fail with the same error message: "Change detection engine import all items for connector Staff Advanced Attribute Precedence Connector failed with reason You cannot call a method on a null-valued expression.. Duration: 00:04:36.5985550"

Checked the data the script should be pulling in and it appears to be fine. Logging in powershell script suggests it finishes its part just fine.

Have tried clearing entities from the IdB connector.

Each import fails at varying entity counts, sometimes round numbers like 10000 or 11500 other times 10781.

The PowerShell Connector treats DN.multi attributes with a single value differently to how it treats string.multi attributes with a single value. This can be overcome in the script however unnecessarily complicates import the script.

Error if handled the same as a string.multi (appears to take first char rather than first array entry):

DN.multi Error.txt

Script that will handle the single value: StudentParentRelationshipsImport.ps1

DN.multi attribute: ParentRelationshipDNs

String.multi attribute: ParentIDLIST

Identity Broker: v5.0.5