Configuring a Databricks Server Connection

Overview

Databricks can be used as Target Server (subject to licencing).

Note that currently Precision Bridge requires all records to have a primary key field, which acts as an identifier. Since Databricks does not support primary keys, you will need to nominate the field(s) that will act as the primary key on your tables. More information on this later in the article.

To use Databricks as the Target Server, select Databricks DB from the Target Server Type dropdown.

Pre-requisites:

• Databricks JDBC Driver - first check that you have setup the path to the corresponding JDBC driver. See Configuring Access to a Databricks Server for more details.

 

Details

The Databricks connection details will be displayed on selection of this server type (see below)

These must be completed as follows:

Host Name:

The server Hostname of your Databricks instance. This can be found in the connection details of your Databricks SQL Warehouse.

Http Path:

Again, this can be found in the connection details of your Databricks SQL Warehouse.

Schema / DB Name:

The name of the schema/database you want to connect to. If this is in your default catalog, you can just give the name of the database/schema. If it is in another catalog, you must use the format <catalog name>.<database name>. See the example above - the database tk-test is located in the pbridge catalog.

Port:

The SSL Port configured for Databricks on the server. The default is 443.

Key field Names:

As stated above, all tables must have a field that can be identified as a unique 'primary key'. For tables generated using the Precision Bridge Table Generator, the field pb_uid can be used for this purpose. For tables mirroring ServiceNow instances, the field, sys_id can be used. In more complex situations, where different tables have different primary key fields, you can specify a comma separated list of all the fields that can be considered to be a primary key field candidate. For example, if some of your tables use the field recId as the primary key and others use instance_id, you can enter recId,instance_id

Only tables containing one of these fields will be supported as target tables in Precision Bridge.

Personal Access Token:

This is an access token tied to a specific user generated within Databricks. To generate from your Databricks workspace, open your user settings and go to User->Developer. In the Access Tokens section, click Manage to create a new one.

Further Information:

If you want to retain the personal access token for each instance, select the Save Authentication option.  After selecting this option, the personal access token is stored and do not need to be entered again the next time you open the same project from the same client machine.  Otherwise, you will need to re-enter it each time you open the project.

In order to check that the relevant servers can be accessed using the credentials provided, click on the Test button on the right for each connection.

If the selected server is reachable then a confirmation message will be displayed.  If the server is not reachable then a failure/timeout message will be displayed.

Related Articles

Configuring Access to a Databricks Server

The Precision Bridge Archiving Wizard