Azure ADLS Data Server
This topic covers integration of Azure Data Lake Storage (ADLS) with the Privacera Platform using Privacera Data Access Server.
Prerequisites
Ensure that the following prerequisites are met:
-
You have access to an Azure Storage account along with required credentials.
For more information on how to set up an Azure storage account, refer to Azure Storage Account Creation. -
Get the values for the following Azure properties: Application (client) ID, Client secrets
CLI Configuration
-
Go to the privacera-manager folder in your virtual machine. Open the config folder, copy the sample vars.dataserver.azure.yml file to the custom-vars/ folder, and edit it.
cd ~/privacera/privacera-manager cp config/sample-vars/vars.dataserver.azure.yml config/custom-vars/ vi custom-vars/vars.dataserver.azure.yml
-
Edit the Azure-related information. For property details and description, click here.
-
If you want to use Azure CLI, use the following properties:
ENABLE_AZURE_CLI: "true" AZURE_GEN2_SHARED_KEY_AUTH: "true" AZURE_ACCOUNT_NAME: "<PLEASE_CHANGE>" AZURE_SHARED_KEY: "<PLEASE_CHANGE>"
-
If you want to access multiple Azure storage accounts with shared key authentication, use the following properties:
AZURE_GEN2_SHARED_KEY_AUTH: "true" AZURE_ACCT_SHARED_KEY_PAIRS: "<PLEASE_CHANGE>"
Note
Configuring
AZURE_GEN2_SHARED_KEY_AUTH
property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal. -
If you want to access multiple azure storage account with OAuth application based authentication, use the following properties:
AZURE_GEN2_SHARED_KEY_AUTH: "false" AZURE_TENANTID: "<PLEASE_CHANGE>" AZURE_SUBSCRIPTION_ID: "<PLEASE_CHANGE>" AZURE_RESOURCE_GROUP: "<PLEASE_CHANGE>" DATASERVER_AZURE_APP_CLIENT_CONFIG_LIST: - index: 0 clientId: "<PLEASE_CHANGE>" clientSecret: "<PLEASE_CHANGE>" storageAccName: "<PLEASE_CHANGE>"
Note
Configuring
AZURE_GEN2_SHARED_KEY_AUTH
property allows you to access the resources in the Azure accounts only through the File Explorer in Privacera Portal.
Note
You can also add custom properties that are not included by default. See Dataserver.
-
-
Run the following command.
cd ~/privacera/privacera-manager ./privacera-manager.sh update
Configuration Properties
Property Name | Description | Example |
---|---|---|
ENABLE_AZURE_CLI
|
Uncomment to use Azure CLI. The
|
true
|
AZURE_GEN2_SHARED_KEY_AUTH |
For
To use multiple Azure storage accounts with shared key authentication, then set this property to
To use multiple Azure storage accounts with OAuth authentication, then set this property to |
true |
AZURE_ACCOUNT_NAME |
Azure ADLS storage account name |
company-qa-dept |
AZURE_SHARED_KEY |
Azure ADLS storage account shared access key |
=0Ty4br:2BIasz>rXm{cqtP8hA;7|TgZZZuTHJTg40z8E5z4UJ':roeJy=d7*/W" |
AZURE_ACCT_SHARED_KEY_PAIRS |
Comma-separated multiple storage account names and its shared keys. The format must be ${storage_account_name_1}:${secret_key_1},${storage_account_name_2}:${secret_key_2} |
accA:sharedKeyA, accB:sharedKeyB |
AZURE_TENANTID | To get the value for this property, Go to Azure portal > Azure Active Directory > Properties > Tenant ID | 5a5cxxx-xxxx-xxxx-xxxx-c3172b33xxxx |
AZURE_APP_CLIENT_ID | Get the value by following the Pre-requisites section above. | 8c08xxxx-xxxx-xxxx-xxxx-6w0c95v0xxxx |
AZURE_SUBSCRIPTION_ID | To get the value for this property, Go to Azure portal > Select Subscriptions in the left sidebar > Select whichever subscription is needed > Click on overview > Copy the Subscription ID | 27e8xxxx-xxxx-xxxx-xxxx-c716258wxxxx |
AZURE_RESOURCE_GROUP | To get the value for this property, Go to Azure portal > Storage accounts > Select the storage account you want to configure > Click on Overview > Resource Group | privacera |
|
Configure multiple OAuth Azure applications and the storage accounts mapped with the configured client id. **Note**: The ‘clientSecret’ property must be in BASE64 format in the YAML file. |
|
Validation
All-access or attempted access (Allowed and Denied) for Azure ADLS resources will now be recorded to the audit stream. This Audit stream can be reviewed in the Audit page of the Privacera Access Manager. Default access for a data repository is 'Denied' so all data access will be denied.
To verify Privacera Data Management control, perform the following steps:
-
Login to Privacera Portal, as a portal administrator, open Data Inventory: Data Explorer, and attempt to view the targeted ADLS files or folders. The data will be hidden and a Denied status will be registered in the Audit page.
-
In Privacera Portal, open Access Management: Resource Policies. Open System 'ADLS' and 'application' (data repository) 'privacera_adls'. Create or modify an access policy to allow access to some or all of your ADLS storage.
-
Return to Data Inventory: Data Explorer and re-attempt to view the data as allowed by your new policy or policy change. Repeat step 1.
You should be able to view files or folders in the account, and an Allowed status will be registered in the Audit page.
To check the log in the Audit page in Privacera Portal, perform the following steps:
-
On the Privacera Portal page, expand Access Management and click the Auditfrom the left menu.
-
The Audit page will be displayed with Ranger Audit details.