Sometimes, when migrating data from a database into a target system, you might want to include attachments. Often the path to the attachments exists, or can be identified from metadata about attachments in the database itself.
In Precision Bridge, you can configure how the attachment metadata should be used to download the actual attachment content. Typically, this option is added to the mapping for the parent table holding the records with which the attachments are associated.
This configuration is located on the options tab. Select the Attachment Migration Option 'Attachments from URL' or 'Attachments from Binary'. Additional configuration options will then be displayed.
Note that for a BMC Remedy target, the Attachment Migration Option will instead be 'Attachments from URL/Binary fill Selected Attachment Fields' and you will also need to select the target attachment fields to populate. See Migrating Attachments into a BMC Remedy Server for more details.
The Attachment Source options identify the source forms (database tables) and fields (representing columns on these tables) that will provide the attachment metadata used in migrating attachments.
If configured correctly, any attachments related to the records migrated should now be downloaded and attached to the target record when the project executes. Note that attachments will not be migrated for those records that reported an error. The configuration options are detailed below.
CSV Attachment Configuration Tab
| Option | Description |
| Attachment Form | Required: This is the identifier of the source table that contains the attachment record metadata (including the filenames/URL to use for downloading the attachment) |
| Parent Form ID Field | Required: This is the field on the Source Form for this table mapping that uniquely identifies the record. |
| Authentication Method |
Optional: This option is used to configure authentication when downloading the attachment from a URL. Basic and Token based authentication is currently supported, as well as a specific use case for Attachment URLs referencing a Manage Engine server. If basic authentication is selected, you will need to enter a username/password to be used to authenticate the attached download request in the Attachment Execution Options Likewise, if Token based authentication is selected, you will need to enter the token details in the Attachment Execution Options See the article Manage Engine Authentication for Attachment Migration for more information on this specific use case. Footprints authentication methods applies to the Database source use case only. |
Configure Attachment Table Fields Tab
| Option | Description |
| Parent ID |
This is the field on the Attachments table (defined on the Attachment Source tab) that identifies the id of the record that the attachment is associated with. It is required, unless a filter condition is defined on the advanced tab. |
| Content URL |
If the Attachment is downloaded from a URL, this is the field on the Attachment Form that holds the URL used to download the attachment file. This configuration option is required, unless the Content Path format is defined on the Advanced tab, and the content path format does does not include a marker for the content path. If the the Attachment is downloaded from a binary field (eg BLOB) in the same table, this is the name of that field. |
| File Name | This is the field on the Attachment Form that holds the file name of the attachment. This configuration option is mandatory, unless a File Name format is defined on the advanced tab. |
| Content Type |
This is the field on the Attachment Form that holds the mime content type of the attachment file. This configuration option is optional. If not defined, Precision Bridge will use the extension of the file name to identify the content type. If the extension is not defined, the file type will be obtained by content analysis after download of the binary. |
| Size |
This is the field on the Attachment Form that holds the size (in bytes) of the attachment file. This configuration option is optional. If not defined, Precision Bridge will not be able to replace existing attachments of the same name but different size when performing 'update' migrations. Precision Bridge will also be unable to perform a check when the attachment has been downloaded to see if it has the correct number of bytes (an incorrect number of bytes may indicate a download error). |
Advanced Tab
These advanced options may be used if the filename needs to be extracted from a longer string within the content URL for download, have an extension added, or transformed into a longer string. In many cases, these options will not be required.
| Option | Description |
| Content Path Format |
Only applies if a URL is used to download content. This can be used to construct the content path (file path or url) from where the attachment content can be downloaded. It may use one of several markers, delimited by square brackets to identify values that should be inserted at execution time. The supported markers are: [content-path_url_encoded] - same as for [content_path] but the value will be URL encoded to be suitable to be used as a URL parameter value. [file_name] will be replaced at execution time by the value for the defined file_name field. Note that any File Name regex expression defined will be applied first. [file_name_url_encoded] - same as for [file_name] but the value will be URL encoded to be suitable to be used as a URL parameter value. [parent_id] - will be replaced at execution time by the correct parent id for the attachment (from the source record). Where Footprints Authentication is used, these markers can also be used: |
| File Name Format |
This can be used to construct a file name. There are 2 supported markers that can be used, these will be replaced at execution time: [file_name] will be replaced at execution time by the value for the defined file_name field. Note that any File Name regex expression defined will be applied first. [ext] will be replaces with the extension derived from the mime type of the file (this will be derived from binary content if the file type field is not configured) for example: if the file name field provides the value '10001', the file mime type is image/jpg and the following file name format is used: attachment_[file_name][ext] the final file name will be attachment_10001.jpg Note that if the [file name] marker is not used, all attachments will be given the same name. This is useful for ServiceNow migrations into an image field, where the attachment name must be the name of the target image field. |
| File Name Regex | This can be used to extract the actual file name from a longer string (e.g. a url) The value must be a valid regex string with at least one group defined. The value extracted will be that for group 1 of the match. |
| Filter Condition | This can be used to give an alternate SQL condition that will be used when extracting the attachment metadata records for a given parent record. It must be formatted as a valid SQL condition for the attachment table, and must use the [parent_id] marker, which will be replaced at execution time when the query is run for each migrated source record. |
Attachment Migration at Execution Time
Precision Bridge will process the attachments as follows:
- Get the identifier for the parent record (using the Parent Form ID Field to identify the id column)
- Get all attachment records where the value for the parent table column matches the source form name and the value for the parent id column matches the identifier retrieved above.
- If configured, use the advanced options File Name Regex to update the filename for the attachment (this will be used in building the URL, if required).
- If configured, use the advanced option content path URL to calculate the URL for the attachment, replacing markers for content_path and file_name).
- If configured, use the advanced options File Name Format to calculate the filename for the attachment, replacing the [file_name] marker with the current file name value. Note that the [ext] marker will not be replaced yet.
- Download these attachments (unless they are larger than the configured maximum attachment size, or already exist on the target with the same filename and size) If an authentication method has been selected, authentication will be applied to the download request.
- If the file name terminates with the [ext] marker, use the file mime type to identify the correct file extension.
- Upload the attachment to the target.
Errors or Warnings will be reported if:
- Error: The download fails due to an invalid URL or invalid/missing authentication
- Error: The Upload to the target fails
- Error: The attachment is larger than the configured maximum attachment size (see Defining Attachment Options)
- Warning: The downloaded attachment size is different to the size declared in Attachment table (if recorded).
Note that Attachment migration can typically take a significant time.
Related Articles:
CSV To Service Now Attachment Migration - Overview
Manage Engine Authentication for Attachment Migration
Comments
0 comments
Please sign in to leave a comment.