Optimising Migration Throughput

There are a number of factors that can affect the throughput of migrations. 

1. Bandwidth.

The network bandwidth between the source server and the Precision Bridge client and between the Precision Bridge client and the target server should be as high as possible.  A low bandwidth for either of these will have a big impact on performance.  Ideally the Precision Bridge client should be installed as close as possible to the source server, if the source server is on premise rather than cloud based.

2. Indexing.  

Focus effort on source tables where there are >20K existing records and target tables where there are >20K records are being migrated or where there are >20K target records in total. 

If you are getting a particularly slow throughput for a given table, particularly if it spends a lot of time in the 'transform' stage (say > 10 seconds per 100 records), consider indexing the fields for any lookup mappings that will be searching a large table. If ID mapping is slow, this will likely be due to the key mapping fields not being indexed on the target table. If retrieval is slow ensure that the fields involved in the source filter are indexed.  Often, if one field in the lookup/key mapping/source filter is indexed, that is sufficient, though in huge tables where multiple fields are used in the query performance might be further improved by adding an index including all fields involved. This index can be removed once migration is complete.

3. Use Multithreading (if available) I

If the source server supports multithreading (for example Service Now) increase the number of threads that can be used when retrieving data to a maximum of 2 per Service Now Node. See section Multi-threading for ServiceNow data extraction

3. Import Set Tables (ServiceNow target)

It is best to break down the migration so that a max of 1m records for each table are migrated each time. If the speed of the 'update' stage starts to slow down,  the advice would be to try clearing down the import set tables. Don't however do this while the migration is running. You can use the ServiceNow Utility to do this (System Import Sets->Import Set Tables->Cleanup)

4. Attachments

Performance of throughput is affected by the number and size of attachments.  If you don't need all the attachments you can exclude these as part of the migration configuration.  You can also limit the size of attachment that gets migrated, leaving particularly large ones to be migrated in a different way. For Service Now attachment migration, use the Server->Server attachment migration method if possible. This is implemented as a direct 'pull' of the attachment from the source server. This is more efficient than  downloading to the client, then uploading to the target. See section Defining Advanced Execution Options

5. Business Rules

Having business rules enabled can slow down migrations for ServiceNow targets.  Unless you need to have the business rules run, you can speed up performance by selecting the migration option Import Set, No Business Rules either when you initiate an execution or in the options for individual table mappings.

6. Lookup Mappings

Lookup mappings can slow-down the performance of the migration. Reference Mappings can be used instead to improve performance, particularly if:

• Many records being migrated have the same source value.
• The id mapping for the source value has always already been identified in an earlier (enabled) form mapping.

If reference mappings are not an option, consider setting the Null Parameter Action on the Lookup Definition to either Return Null, Return No Match or Ignore.  These options will prevent the Lookup from running if one or more parameters are Null.

Additionally, if the table you are searching on in the lookup definition is very large, check that the fields you are searching on are indexed (see point 2 above).

7. Proximity to the Source/Target Server

If either the source or target server is on premise, install Precision Bridge as geographically close to the on-premise server as possible. This will reduce round trip delays in communicating with the server.