Get an Azure Active Directory token using a service principal

• 12/01/2020
• 4 minutes to read

In this article

This article describes how a service principal defined in Azure Active Directory (Azure AD) can also act as a principal on which authentication and authorization policies can be enforced in Azure Databricks. Service principals in an Azure Databricks workspace can have different fine-grained access control than regular users (user principals).

A service principal acts as a client role and uses the OAuth 2.0 code grant flow to authorize to Azure Databricks resources.

You can manage service principals using the Databricks SCIM API (ServicePrincipals) API or use the following procedure in Azure portal.

You can also use the Azure Active Directory Authentication Library (ADAL) to programmatically get an Azure AD access token for a user. See Get an Azure Active Directory token using Azure Active Directory Authentication Library.

Provision a service principal in Azure portal

1. Log in to Azure portal.

2. Navigate to Azure Active Directory > App Registrations > New Registrations. You should see a screen similar to this:

   

3. Click Certificates & secrets and generate a new client secret.

   

4. Copy and store that secret in a secure place as this secret is the password for your application.

5. Click Overview to look at details like Application (client) ID and Directory (tenant) ID.

Use an app identity to access resources covers how you can provision an application (service principal) in Azure AD.

Get an Azure Active Directory access token

To access the Databricks REST API with the service principal, you get an Azure AD access token for the service principal. You can use the client credentials flow to get an access token (with the AzureDatabricks login application as the resource).

Replace the following parameters in the curl request:

curl -X GET -H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id=<client-id>&resource=<azure_databricks_resource_id>&client_secret=<application-secret>' \
https://login.microsoftonline.com/<tenant-id>/oauth2/token

The response should look like:

{
  "token_type": "Bearer",
  "expires_in": "599",
  "ext_expires_in": "599",
  "expires_on": "1575500666",
  "not_before": "1575499766",
  "resource": "2ff8...f879c1d",
  "access_token": "ABC0eXAiOiJKV1Q......un_f1mSgCHlA"
}

The access_token in the response is the Azure AD access token.

Use the service principal’s Azure AD access token to access the Databricks REST API

In the following examples, replace <databricks-instance> with the per-workspace URL of your Azure Databricks deployment.

API access for service principals that are Azure Databricks workspace users and admins

Use the access token as the Bearer token.

Example

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
https://<databricks-instance>/api/2.0/clusters/list

API access for service principals that are not workspace users

Use the authentication process described in this section if:

• The service principal is not defined as a user (whether admin or not) in the Azure Databricks workspace.
• The service principal belongs to the workspace, but you want to add it automatically as an admin user.

Otherwise, see API access for service principals that are Azure Databricks workspace users and admins.

Requirements

To log in as a non-workspace user:

• The service principal must be in a Contributor or Owner role on the workspace resource in Azure.
• You must provide the X-Databricks-Azure-Workspace-Resource-Id header, which contains the ID of the workspace resource in Azure. You construct the ID using the Azure subscription ID, resource group name, and workspace resource name.
• You must provide a management access token for the Azure Resource Management endpoint.

Get the Azure Management Resource endpoint token

Replace the following parameters in the curl request:

curl -X GET -H 'Content-Type: application/x-www-form-urlencoded' \
-d 'grant_type=client_credentials&client_id=<client-id>&resource=<management-resource-endpoint>&client_secret=<application-secret>' \
https://login.microsoftonline.com/<tenantid>/oauth2/token

Sample response:

{
  "token_type": "Bearer",
  "expires_in": "599",
  "ext_expires_in": "599",
  "expires_on": "1575500666",
  "not_before": "1575499766",
  "resource": "https://management.core.windows.net/",
  "access_token": "LMN0eXAiOiJKV1Q......un_f1mSgCHlA"
}

The access_token in the response is the management endpoint access token.

Use the management endpoint access token to access the Databricks REST API

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
-H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \
https://<databricks-instance>/api/2.0/clusters/list

Sample request:

curl -X GET \
-H 'Authorization:Bearer ABC0eXAiOiJKV1Q......un_f1mSgCHlA' \
-H 'X-Databricks-Azure-SP-Management-Token: LMN0eXAiOiJKV1Q......un_f1mSgCHlA' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/3f2e4d...2328b/resourceGroups/Ene...RG/providers/Microsoft.Databricks/workspaces/demo-databricks' \
https://<databricks-instance>/api/2.0/clusters/list