Skip to content

Spark Configuration Table Properties#

Fine-grained Access Control#

Property Value Description
spark.databricks.isv.product privacera To specify partnership with Privacera. This is required to set via Spark Config UI only.
spark.databricks.delta.formatCheck.enabled FALSE Databricks Recommended.
spark.driver.extraJavaOptions -javaagent:/databricks/jars/privacera-agent.jar To enable code injection. This is required to set via Spark Config UI only.
spark.executor.extraJavaOptions -javaagent:/databricks/jars/privacera-agent.jar To enable Code Injection. This is required to set via Spark Config UI only.
spark.sql.extensions org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension This is to enable Privacera Ranger Spark SQL Authorization.
spark.databricks.repl.allowedLanguages sql,python,r Allowed Languages. Scala is excluded since Scala queries run as root user and can bypass Privacera.
spark.databricks.cluster.profile serverless Databricks Specific. This should be used to enable multi-concurrency.
privacera.custom.current_user.udf.names current_user
privacera.spark.view.levelmaskingrowfilter.extension.enable TRUE To enable View Level Access Control (Using Data_admin feautre), View Level Column Masking, and View Level Row Filtering
spark.databricks.pyspark.enableProcessIsolation TRUE This should be enabled for additional security. It does the following- i. Remove dbfs mounting on the cluster, ii. Blocks User directly trying to access IAM Role, etc.
spark.databricks.pyspark.enablePy4JSecurity TRUE Blocks python libraries which could bypass security
spark.databricks.pyspark.dbconnect.enableProcessIsolation TRUE
spark.databricks.clusterUsageTags.clusterName Cluster Name which can be seen in Privacera Portal Audits
spark.privacera.spark.clusterType databricks Default is databricks. Can be emr, kubernetes, etc
JWT Related Props
privacera.jwt.token The file which contains the JWT Secret token
privacera.jwt.token.str The actual jwt token in string format. Either above property or this can be used
privacera.jwt.token.issuer Issuer in the JWT Token Payload
privacera.jwt.token.subject Subject in the JWT Token Payload
privacera.jwt.oauth.enable Enable usage of JWT token. If jwt.token is not set, authz will fall back on the user
privacera.jwt.token.parser.type JWT Parsery Type. Default is PING_IDENTITY. Other supported values are KEYCLOAKS
privacera.jwt.oauth.use.privacera.token Required in case of long running jobs where JWT Token might expire
privacera.jwt.token.secret If the jwt token has been encrypted using secret, use the property to set the secret.
privacera.jwt.token.publickey Public Key which was used to create JWT Token
privacera.jwt.token.groupKey Property to define a unique group key from the token payload. Value of this key will be used for authorization check as ranger groups
privacera.jwt.token.userKey Property to define a unique user key from the token payload. Value of this key will be used for authorization check as ranger user

Object-level Access Control#

Property Value Description
spark.databricks.isv.product privacera To specify partnership with Privacera.
spark.driver.extraJavaOptions -javaagent:/databricks/jars/privacera-agent.jar To enable code injection for Privacera authorization. This is required to set via Spark Config UI only.
spark.executor.extraJavaOptions -javaagent:/databricks/jars/privacera-agent.jar To enable code injection for Privacera authorization. This is required to set via Spark Config UI only.
spark.hadoop.privacera.ds.host 10.xx.xx.xx Dataserver Hostname
spark.hadoop.privacera.ds.port 8181 Dataserver Port
spark.hadoop.signed.url.enable TRUE To enable signed URL.
spark.hadoop.fs.s3.impl com.databricks.s3a.PrivaceraDatabricksS3AFileSystem Enable usage of Privacera File system which is responsible for authorization of any user trying to access S3.
spark.hadoop.fs.s3n.impl com.databricks.s3a.PrivaceraDatabricksS3AFileSystem Enable usage of Privacera File system responsible for authorization of any user trying to access S3.
spark.hadoop.fs.s3a.impl com.databricks.s3a.PrivaceraDatabricksS3AFileSystem Enable usage of Privacera File system responsible for authorization on any user trying to access s3
JWT Related Props
privacera.jwt.oauth.enable Enable usage of JWT token. If jwt.token is not set, authorization will fall back on the user.
privacera.jwt.token The file which contains the JWT Secret token
privacera.jwt.token.str The actual jwt token in string format. Either above property or this can be used

Last update: October 13, 2021