I have Unicode data in a locker field, sourced from a PowerShell connector, which looks file in the UNIFYBroker UI:

However when I export it to a CSV connector, the non-latin1 character doesn't appear to be correct:

(viewed using Notepad++)

Does the CSV connector support Unicode characters?

In UNIFYConnect when attempting to retrieve a log file via the API it usually works fine but once I saw this error.  This is a very low priority issue for me.

20221214,20:00:16,UNIFYBroker,SyncEngine,Information,"Request to baseline synchronize link completed.
Request to queue a baseline synchronization job for the 'Employee > AD User' link completed. Duration: 00:00:12.2592180",Normal
20221214,20:00:19,UNIFYBroker,Logging engine,Warning,"Request to download log file.
Request to download log file for 12/14/2022 failed with message The process cannot access the file 'C:\app\Services\Logs\UnifyLog20221214.csv' because it is being used by another process.. Duration: 00:00:00.1899900
Error details:
System.IO.IOException: The process cannot access the file 'C:\app\Services\Logs\UnifyLog20221214.csv' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Unify.Framework.Logging.CsvLogReaderWriter.DownloadLogFile(DateTime date)
at Unify.Framework.Logging.LoggingEngine`1.DownloadLogFile(DateTime date)
at Unify.Framework.Notification.NotifierDecoratorBase.Notify[TResult](ITaskNotificationFactory notificationFactory, Func`1 function)
at Unify.Framework.Logging.LoggingEngineNotifierDecorator.DownloadLogFile(DateTime date)
at Unify.Framework.Logging.LoggingController.DownloadLogFile(Int32 year, Int32 month, Int32 day)
at lambda_method(Closure , Object , Object[] )
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.b__9(Object instance, Object[] methodParameters)
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ApiControllerActionInvoker.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__1.MoveNext()",Normal

When I put an Entity Id into the search field on the Entity Search screen, UNIFYBroker doesn't search for the entity.

This is particularly problematical when searching by ID for one entity in a large partition, since the only way to find it is to order by Entity Id and browse result pages one by one, matching Entity Ids visually until the required entity is found.  Even after selecting "Show 100 entries" for a customer with a large number of entities this takes many minutes and dozens of click-wait-for-page-load-look-and-scroll cycles.  Sometimes this process needs to be repeated for many Entity Ids and it becomes frustrating and wastes time.

When multiple filters are applied on an entity search the operator applied is by default an AND.  A checkbox option(s) similar to say the MIM Portal entity search filtering options would be ideal, but at a minimum the ability to OR filters would be handy to assist test scenarios and troubleshooting.

Following successful initial data load and a number of normal HRIS/AD connector import cycles an outbound link to AD has somehow ended up with a join to a non-existent adapter entity (in the case above, eeb1ac7e-fda0-4f64-a509-2af2a88ac22f).

Could this be 4306 Non-existant adapter joins when outgoing baseline syncs fail back to haunt me again?  :(

Priority: Critical
Impact: Use of transformation renders whole UNIFYConnect environment unusable

A join transformation on a single valued field is causing high CPU, Memory and Database resource consumption. 

Source adapter has 7000 entities. Connector being joined to has 7100 entities. 

May be of interest: The field being joined to in the connector  has some values as "empty" rather than null.

Priority: Low/Medium
Impact: Higher than necessary UNIFYConnect resource consumption

With a connector and adapter configured in UNIFYBroker, and the adapter has multiple transformations registered (such as joins and future dated changes), a change appears to be registered in the change table for each combination of entity + transformation. 

This can result in 8-10 changes being registered for the same entity. While this isn't impactful from a change perspective (the latter computations of changes result in no actual changes being made to the adapter entity), a connector import of 7000 entities is resulting in upwards of 60,000 changes being registered which results in unnecessary delays of processing and computation on the database.

In UNIFYConnect very occasionally Change Sync won't run (either from a schedule or when invoked manually from the UI) when there are pending changes on the link.  Service restart doesn't help, but running a Baseline Sync does.  I have no idea what causes it to get into this state I'm afraid.

The error Changes register item processing on connector TechOne Person failed with reason Cannot process a DateTime of type 'Unspecified' is logged for an adapter which has a join to another connector with a Timestamp field that was populated by PowerShell when that adapter's base connector imports a new or updated entity, even when that Timestamp field has DateTime kind 'Utc' or 'Local'.

My config has two PowerShell connectors, TechOne Person and TechOne Position.  The Position connector has two fields StartTimestampUTC and EndTimestampUTC which are set to valid DateTime values, of kind 'Utc'.  There is one adapter, TechOne Person, for which the Person connector is a base, and the Position connector is a Join transform with StartTimestampUTC and EndTimestampUTC fields both mapped into adapter fields.

When a new record is present in the Person connector import, reflection causes the above error message to be logged, but the adapter record is created correctly nevertheless.  When a subsequent Generate Changes is run on the adapter, the error is not logged.  If the record is reimported with updated values, the error is logged.

Hi Team,

A customer has reported high CPU usage for the UnifyBroker service coming to me initial with the following report:

"In the past month we have been getting High CPU (90%) usage of the UnfyBroker service. We are using version v5.3.3. There has been no change in the environment other than the normal Microsoft patches once a month. It is occurring on both our prod and uat environments."

Confirm some further details with the customer I got the following spec information on the hosts and details about the behaviour:

"Specs on the servers.

Name Memory vCPU OS

PRD 16Gb 4 2016

UAT          14Gb 4 2016

Processor type - Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz, 2295 Mhz, 1 Core(s), 1 Logical Processor(s)

The high cpu is more often on the uat server which processes the same data as prod for new accounts but the changes during the day are very minimal. The cpu usage does come down after

The utilization in some instances has cleared itself. Some have been by stopping the service. Timing is from 10 mins to an hour for uat. Interestingly on the prd server the time has been for several hours."

Some initial adjustments were made to the scheduler to ensure nothing is overlapping, though this wasn't really happening much anyway. These adjustments did not see any improvement to the situation.

Additionally the customer reported back the entity counts in the connectors to get a gauge for the size of the environment:

"Connector Prod Uat

Employee CSV 10792 10792

Position CSV 10792 10792

Teams 14805 1035

AD 45683 51356

Adapters

Person 10792 10792

Position 10792 10792"

Looking at the extensibility for the setup there are a number of powershell transform in the environment, which may explain some high cpu usage while connectors or running, though doesn't seem to explain why the high cpu usage continues afterwards.  

Finally the following resource monitor screenshots were provided to see what is consuming the resources for the service while no connectors are running:

I have confirmed no out of bounds scripts are contacting the service. And that there aren't a large number of WebUI sessions open that could be causing issues. So I am looking for the next steps in troubleshooting this one. Are you able to provide any guidance in figuring out what could be consuming the resources like this?

Let me know if there is any further information I can provide you.

Thanks in advance