Performance Tuning in Precision Bridge

Precision Bridge gives several options to allow users to maximize throughput when migrating records.

Many of these rely on concurrent processing, where multiple commands are being sent to the source/target server at the same time. Most servers will support this.

The options that define this are on the Performance Tuning tab of the Execution Options.

Setting the Maximum Number of Connections that will be used Concurrently

These settings allow Precision Bridge to use more of the server resources than would be possible with a single connection. 

There are three settings:

• Max Source Connections - the maximum number of connections to the source server that Precision Bridge will ever use concurrently during an execution.
• Max Target Connections - the maximum number of connections to the target server that Precision Bridge will ever use concurrently during an execution.
• Max External Lookup Server Connections - where an additional server is defined to provide a source of external data, the maximum number of connections to this server that Precision Bridge will ever use concurrently during an execution.

Leaving these as '1' will mean that Precision Bridge will be restricted to a single server operation at a time, no matter how many retrieve/submit threads are configured for the mapping and no matter how many form mappings are configured to run in parallel. If you have configured multiple threads and multiple parallel form mappings, you will need to increase the number of connections to see significant performance improvements.

Setting the Maximum Retrieve / Submit  Threads

A significant amount of the processing that Precision Bridge does is retrieving the source records for migration and submitting them for create/update on the target server. Both of these operations are 'multi-threaded' to allow Precision Bridge to request multiple batches of records from the source server at the same time and to allow Precision Bridge to submit multiple batches of records to the target server at the same time.

The maximum number of threads configurable is 20. Note that if the number of Connections is lower than the number of threads, the additional threads will not improve performance, and could actually reduce it.

Setting the Maximum Source Deletion Threads

This setting applies to the archiving use case, where the target is a database and the project is configured to delete source records following deletion.

The delete operation, as well as any validation checking that occurs on both source and target before records are deleted can take some time to complete. This operation can also be multi threaded, with a maximum of 50 batches of records being processed at a time. Note that if the number of source and target connections is lower than the number of threads, the additional threads may not improve performance.

Setting the Maximum Number of Parallel Form Mappings

By setting the Max Parallel Form Mapping Executions to a higher number, Precision Bridge can run form mappings concurrently. For example, if your project is migrating several different CMDB classes, and there is no dependency between these, Precision Bridge can start them concurrently, up to the maximum number defined. This will make better use of the server resources on source and target.

Dependency Checking in Parallel Form Mapping Executions

It is important to understand that parallel execution may have an impact on the migration results, particularly where there are complex dependencies between form mappings.

Where multiple form mappings are running concurrently, the form mappings will not necessarily run in the order that they are shown in the project. This is something to be aware of if you have complex and interdependent mappings. In some cases, parallel form mapping execution may not be suitable for your project. 

Precision Bridge will identify dependencies between form mappings and attempt to make sure that a dependent form mapping does not run before the mapping it is dependent on. This dependency check will prevent:

1. Form Mappings with an inclusion filter that references an earlier (parent) form mapping being executed before the parent has completed
2. Form Mappings that include a reference field mapping that references an earlier form mapping being run before the earlier form mapping has completed.
3. Form mappings that have a target lookup field mapping, where the lookup is querying a target form that is the target of an earlier form mapping being started before the earlier form mapping has completed.
4. Form mappings that have an ID List field mapping where the target form for the id list values matched the target of an earlier form mapping being started before the earlier form mapping has completed. [Service Now -> Service Now migrations only]
5. Form mappings that have an ID Replace mapping being run before any earlier mapping has completed. [Service Now -> Service Now migrations only]
6. Form mappings where source deletion is defined being run before the form mappings for any cascade deleted tables where the cascade delete action is not 'Delete Source Records' [Service Now -> Database archiving use case only] This is to prevent errors if 'child' records are not migrated (and deleted from source) before the parents.
7. Any form mappings where the 'Force Execution in Sequence' option on the Options->Advanced tab of the form mapping is checked being executed before all earlier form mappings have completed.

The Force Execution in Sequence Option

Sometimes, in a project, there may be a dependency between form mappings that Precision Bridge does not detect. To allow parallel migrations to be performed in this situation, you can use the 'Force Execution in Sequence' option.

This option can be found on the Options->Advanced tab and will prevent the form mapping from executing until all those before it has completed.

Related Articles

Recommendations for Performance Tuning