Identity Broker Forum

Welcome to the community forum for Identity Broker.

Browse the knowledge base, ask questions directly to the product group, or leverage the community to get answers. Leave ideas for new features and vote for the features or bug fixes you want most.

Under review

Add SFTP functionality to the CSV connector and/or agent

Aneesh Varghese 2 years ago in CSV connector • updated by anonymous 1 year ago 1

Add the ability for the FTP agent to connect to SFTP servers. Also update the CSV connector if required to be able to use the updates to the connector.


Please support write/export operation for CSV Connector using FTP agent.

Aneesh Varghese 2 years ago in CSV connector • updated by Adam van Vliet (Product Manager) 2 months ago 6

CSV Connector using FTP agent currently doesn't support write/export operation but only import. Understood, it is a bit more complicated than updating a local file but can we look to see what we can do or suggest a workaround to cater write operation.

Some HR Systems like "PayGlobal" currently doesn't have a Web Service interface and the only option for integration is CSV/file extracts. Systems like PayGlobal is also remotely hosted on the Vendors Cloud environment and does require remote file transfer.




Strange behaviour of CSV connector export

Matthew Clark 6 years ago in CSV connector • updated by anonymous 2 years ago 7

The CSV connector is exhibiting some strange behaviour when exporting. See below a file that was written to. Telephone is a multivalue string field in the connector schema - the rest are strings.

13,Matthew Clark,1337,,Unify.Framework.StringValue[]
19,Anthony Sheehy,1339,,Unify.Framework.StringValue[]

Telephone and email have seemingly switched value types. I am assuming this is why the string value array writing has appeared.

CSV Export User.png

CSV Connector fails export

Tony Sheehy 6 years ago in CSV connector • updated by anonymous 2 years ago 21

Exports to the CSV Connector are failing with the following:

Save entities [Count:1] to connector CSV Connector 2 failed with reason The process cannot access the file 'C:\Shared\Identity Broker 4\CSV Test\Talent2 Person Connector5.csv' because it is being used by another process.. Duration: 00:00:00
Error details:
System.IO.IOException: The process cannot access the file 'C:\Shared\Identity Broker 4\CSV Test\Talent2 Person Connector5.csv' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.StreamWriter.CreateFile(String path, Boolean append)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding)
at System.IO.File.AppendAllLines(String path, IEnumerable`1 contents)
at Unify.Product.IdentityBroker.CsvConnector.AddEntities(IEnumerable`1 entities)
at Unify.Product.IdentityBroker.ConnectorToWritingConnectorBridge.AddEntities(IEnumerable`1 entities)
at Unify.Product.IdentityBroker.EventNotifierWritingConnectorDecorator.AddEntities(IEnumerable`1 entities)
at Unify.Product.IdentityBroker.Adapter.AddEntities(IEnumerable`1 entities, Boolean reflect)
at Unify.Product.IdentityBroker.Adapter.AddEntity(IAdapterEntity entity)
at Unify.Product.IdentityBroker.AdapterNotifierDecoratorBase`1.AddEntity(IAdapterEntity entity)
at Unify.Product.IdentityBroker.LDIFAdapterBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)
at Unify.Product.IdentityBroker.LDIFAdapterServiceHostDecoratorBase.ExportChanges(ExportedLDIFForAdapter exportedLdifForAdapter)
at SyncInvokeExportChanges(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

There were a series of improvements made to the CSV connector that will need to be re-evaluated and completely regression tested. These changes were made in attempts to remove the likelihood of locks on a file being breached on the same thread by reducing the number of reads/writes to the file.

These included:

  • Preventing the collection of IConnectorEntities being completely evaluated before being written back.
  • Reducing the reads on the same file to 1 instead of 3 on Add entities.

CSV Connector does not support binary values

Tony Sheehy 6 years ago in CSV connector • updated by anonymous 2 years ago 4

A CSV connector configured to import a binary value fails with:

Change detection engine import all items for connector CSV Connector Example failed with reason Line 1 column 1 value of 010011010110000101101110 is not valid. Duration: 00:00:00.1406250

This was tried with a byte value, byte array value, binary value and base64 encoded string; all resulting with a message similar to the one provided above.

  • Either the validator should be updated to handle string values
  • Or an intermediate step is included where the CSV connector tries to parse the values retrieved based on the field ValueType, before creating with the Validator
  • Otherwise, a more descriptive exception should be provided denoting that this connector doesn't support that value type.

CSV Connector does not seem to pick up multivalue fields

Matthew Clark 6 years ago in CSV connector • updated by anonymous 2 years ago 4

The field "Telphone" is multivalue, and the following line is being imported:

13,Matthew Clark,1337,"0413371337,0412123123",

However, the Telephone value is still being sent to FIM as a single valued field:

dn: CN=859cbd1d-fe77-4cd7-b88d-998f12aae491
objectClass: csvPerson
IdBID: 859cbd1d-fe77-4cd7-b88d-998f12aae491
Key: 13
Name: Matthew Clark
EmployeeID: 1337
Telephone: 0413371337,0412123123


Improve CSV Connector edge case for empty target file.

Tony Sheehy 5 years ago in CSV connector • updated by anonymous 2 years ago 1

The current message thrown when the target CSV file is empty is the following:

Save entities to connector failed.
Save entities [Count:40] to connector Test CSV - C:\log\AurionGeneric.csv failed with reason Non-negative number required.
Parameter name: value. Duration: 00:00:00.0040002
Error details:
System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: value
at System.IO.FileStream.set_Position(Int64 value)
at Unify.Product.IdentityBroker.CsvConnector.<>c__DisplayClass2a.<AddEntities>b__28(Stream old, Stream new)
at Unify.Product.IdentityBroker.FileStreamCommunicator.Shift(UpdateFile fileUpdate)
at Unify.Product.IdentityBroker.CsvConnector.AddEntities(IEnumerable`1 entities)
at Unify.Product.IdentityBroker.EventNotifierAddingConnectorDecorator.AddEntities(IEnumerable`1 entities)
at Unify.TestHarness.Connector.ConnectorTestHarness.<>c__DisplayClass6.<Add>b__5(IEntitySchema schema, IEnumerable`1 entities)
at Unify.TestHarness.Connector.ConnectorTestHarness.Shift(Guid sourceId, Action`2 entityHandler)
at Unify.TestHarness.Connector.ConnectorTestHarness.Add(Guid sourceId, Guid targetId)
at Unify.TestHarness.Connector.ConnectorTestHarnessServiceController.Add(StandardPostData postData)

Instead, the schema field names should be written as the header, with an informational event logged stating that the file was updated with the correct heading.


IdB4: CSV Connector schema - cannot specify key

Peter Wass 6 years ago in CSV connector • updated by anonymous 2 years ago 12

When specifying the schema for a CSV file, you cannot change what is the required / readonly fields to generate keys. The CSV file format is not always controllable as it can be system produced. Can you please allow the modification of the schema settings.

Incorrec time error.jpg

CSV Connector file path cannot be viewed

Tony Sheehy 6 years ago in CSV connector • updated by anonymous 2 years ago 3

Scrolling to the right will rubber-band back to the left for the CSV file path.


CSV connector display issues

Matthew Clark 6 years ago in CSV connector • updated by anonymous 2 years ago 4

There are some display issues with the CSV connector.

  • Some label names are still garbled
  • Some FTP settings display regardless of the type of communicator (File and FTP)
  • A validation message does not display if the file path is cleared
  • There is a ")" character after "Ignore Certificate Errors"