Skip to content

Azure ADLS Data Server

This topic covers integration of Azure Data Lake Storage (ADLS) with the Privacera Platform using Privacera Data Access Server.

Prerequisites

Ensure that the following prerequisites are met:

  • You have access to an Azure Storage account along with required credentials.
    For more information on how to set up an Azure storage account, refer to Azure Storage Account Creation.

  • Get the values for the following Azure properties: Application (client) ID, Client secrets

CLI Configuration

  1. Go to the privacera-manager folder in your virtual machine. Open the config folder, copy the sample vars.dataserver.azure.yml file to the custom-vars/ folder, and edit it.

    cd ~/privacera/privacera-manager
cp config/sample-vars/vars.dataserver.azure.yml config/custom-vars/
vi custom-vars/vars.dataserver.azure.yml

  2. Edit the Azure-related information. For property details and description, click here.

    1. If you want to use Azure CLI, use the following properties:

      ENABLE_AZURE_CLI: "true"
AZURE_GEN2_SHARED_KEY_AUTH: "true"
AZURE_ACCOUNT_NAME: "<PLEASE_CHANGE>"
AZURE_SHARED_KEY: "<PLEASE_CHANGE>"

    2. If you want to access multiple Azure storage accounts with shared key authentication, use the following properties:

      AZURE_GEN2_SHARED_KEY_AUTH: "true"
AZURE_ACCT_SHARED_KEY_PAIRS: "<PLEASE_CHANGE>"

      Note

      Configuring AZURE_GEN2_SHARED_KEY_AUTH property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal.

    3. If you want to access multiple azure storage account with OAuth application based authentication, use the following properties:

      AZURE_GEN2_SHARED_KEY_AUTH: "false"

AZURE_TENANTID: "<PLEASE_CHANGE>"
AZURE_SUBSCRIPTION_ID: "<PLEASE_CHANGE>"
AZURE_RESOURCE_GROUP: "<PLEASE_CHANGE>"

DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST:
 - index: 0
   clientId: "<PLEASE_CHANGE>"
   clientSecret: "<PLEASE_CHANGE>"
   storageAccName: "<PLEASE_CHANGE>"

      Note

      Configuring AZURE_GEN2_SHARED_KEY_AUTH property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal.

    Note

    You can also add custom properties that are not included by default. See Dataserver.

  3. Run the following command.

    cd ~/privacera/privacera-manager
./privacera-manager.sh update

Configuration Properties

Property Name Description Example

ENABLE_AZURE_CLI

 

 

 

Uncomment to use Azure CLI.

The AZURE_ACCT_SHARED_KEY_PAIRS property wouldn't work with this property. So, you have set the AZURE_ACCOUNT_NAME and AZURE_SHARED_KEY properties.

 

true

 

 

 

AZURE_GEN2_SHARED_KEY_AUTH

For AZURE_GEN2_SHARED_KEY_AUTH property, use shared key authentication. Set it to true.

To use multiple Azure storage accounts with shared key authentication, then set this property to true, along with AZURE_ACCT_SHARED_KEY_PAIRS.

To use multiple Azure storage accounts with OAuth authentication, then set this property to false, along with DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST.

true

AZURE_ACCOUNT_NAME

Azure ADLS storage account name

company-qa-dept

AZURE_SHARED_KEY

Azure ADLS storage account shared access key

=0Ty4br:2BIasz>rXm{cqtP8hA;7|TgZZZuTHJTg40z8E5z4UJ':roeJy=d7*/W"
AZURE_ACCT_SHARED_KEY_PAIRS 

Comma-separated multiple storage account names and its shared keys.

The format must be ${storage_account_name_1}:${secret_key_1},${storage_account_name_2}:${secret_key_2}

accA:sharedKeyA, accB:sharedKeyB
AZURE_TENANTID To get the value for this property, Go to Azure portal > Azure Active Directory > Properties > Tenant ID  5a5cxxx-xxxx-xxxx-xxxx-c3172b33xxxx
AZURE_APP_CLIENT_ID Get the value by following the Pre-requisites section above.  8c08xxxx-xxxx-xxxx-xxxx-6w0c95v0xxxx
AZURE_SUBSCRIPTION_ID To get the value for this property, Go to Azure portal > Select Subscriptions in the left sidebar > Select whichever subscription is needed > Click on overview > Copy the Subscription ID 27e8xxxx-xxxx-xxxx-xxxx-c716258wxxxx
AZURE_RESOURCE_GROUP To get the value for this property, Go to Azure portal > Storage accounts > Select the storage account you want to configure > Click on Overview > Resource Group privacera 
DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST:
 - index: 0
   clientId: "<PLEASE_CHANGE>"
   clientSecret: "<PLEASE_CHANGE>"
   storageAccName: "<PLEASE_CHANGE>"

Configure multiple OAuth Azure applications and the storage accounts mapped with the configured client id.

**Note**: The ‘clientSecret’ property must be in BASE64 format in the YAML file. 

DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST:
 - index: 0
   clientId: "8c08xxxx-xxxx-xxxx-xxxx-6w0c95v0xxxx"
   clientSecret: "WncwSaMpleRZ1ZoLThJYWpZd3YzMkFJNEljZGdVN0FfVAo="
   storageAccName: "storageAccA,storageAccB"
 - index: 1
   clientId: "5d37xxxx-xxxx-xxxx-xxxx-7z0cu7e0xxxx"
   clientSecret: "ZncwSaMpleRZ1ZoLThJYWpZd3YzMkFJNEljZGdVN0FfVAo="
   storageAccName: "storageAccC"

Validation

All-access or attempted access (Allowed and Denied) for Azure ADLS resources will now be recorded to the audit stream. This Audit stream can be reviewed in the Audit page of the Privacera Access Manager.  Default access for a data repository is 'Denied' so all data access will be denied.

To verify Privacera Data Management control, perform the following steps:

  1. Login to Privacera Portal, as a portal administrator, open Data Inventory: Data Explorer, and attempt to view the targeted ADLS files or folders. The data will be hidden and a Denied status will be registered in the Audit page.

  2. In Privacera Portal, open Access Management: Resource Policies.  Open System 'ADLS' and 'application' (data repository) 'privacera_adls'.  Create or modify an access policy to allow access to some or all of your ADLS storage.

  3. Return to Data Inventory: Data Explorer and re-attempt to view the data as allowed by your new policy or policy change.  Repeat step 1.

    You should be able to view files or folders in the account, and an Allowed status will be registered in the Audit page.

To check the log in the Audit page in Privacera Portal, perform the following steps:

  1. On the Privacera Portal page, expand Access Management and click the Auditfrom the left menu.

  2. The Audit page will be displayed with Ranger Audit details.