Skip to content

Discovery#

This topic allows you to set up the AWS configuration for installing Privacera Discovery in a Docker and Kubernetes (EKS) environment.

IAM Policies#

To use the Privacera Discovery service, ensure the following IAM policies are attached to the Privacera_PM_Role role to access the AWS services.

Policy to create AWS resources

This policy gives permissions to Privacera Manager to create AWS resources like DynamoDB, Kinesis, SQS, and S3 using terraform.

  • ${AWS_REGION}: AWS region where the resources will get created.
{
"Version":"2012-10-17",
"Statement":[
    {
        "Sid":"CreateDynamodb",
        "Effect":"Allow",
        "Action":[
            "dynamodb:CreateTable",
            "dynamodb:DeleteTable",
            "dynamodb:DescribeTable",
            "dynamodb:ListTables",
            "dynamodb:TagResource",
            "dynamodb:UntagResource",
            "dynamodb:UpdateTable",
            "dynamodb:UpdateTableReplicaAutoScaling",
            "dynamodb:UpdateTimeToLive",
            "dynamodb:DescribeTimeToLive",
            "dynamodb:ListTagsOfResource",
            "dynamodb:DescribeContinuousBackups"
        ],
        "Resource":"arn:aws:dynamodb:${AWS_REGION}:*:table/privacera*"
    },
    {
        "Sid":"Kinesis",
        "Effect":"Allow",
        "Action":[
            "kinesis:CreateStream",
            "kinesis:DeleteStream",
            "kinesis:ListStreams",
            "kinesis:UpdateShardCount"
        ],
        "Resource":"arn:aws:kinesis:${AWS_REGION}:*:stream/privacera*"
    },
    {
        "Sid":"S3BucketCreate",
        "Effect":"Allow",
        "Action":[
            "s3:CreateBucket",
            "s3:ListAllMyBuckets",
            "s3:GetBucketLocation",
            "s3:DeleteBucket"
        ],
        "Resource":[
            "arn:aws:s3:::*"
        ]
    },
    {
        "Sid":"ManageSQSMessages",
        "Effect":"Allow",
        "Action":[
            "sqs:*"
        ],
        "Resource":[
            "*"
        ]
    }
]
}
Policy to access AWS services

This policy gives permissions to store Privacera's configuration and meta-data in DynamoDB, Kinesis, SQS, and S3.

  • ${PRIVACERA_BUCKET}: AWS Bucket used by Privacera to store its configuration files.

  • ${ACCOUNT_ID}: AWS account where the installation is being done.

  • ${AWS_REGION}: AWS region where resource will get created.

{
"Version":"2012-10-17",
"Statement":[
    {
        "Sid":"Dynamodb",
        "Effect":"Allow",
        "Action":[
            "dynamodb:BatchGet*",
            "dynamodb:DescribeStream",
            "dynamodb:DescribeTable",
            "dynamodb:Get*",
            "dynamodb:Query",
            "dynamodb:Scan",
            "dynamodb:BatchWrite*",
            "dynamodb:Delete*",
            "dynamodb:Update*",
            "dynamodb:PutItem"
        ],
        "Resource":"arn:aws:dynamodb:${AWS_REGION}:*:table/privacera*"
    },
    {
        "Sid":"Kinesis",
        "Effect":"Allow",
        "Action":[
            "kinesis:Get*",
            "kinesis:DescribeStreamSummary",
            "kinesis:ListStreams",
            "kinesis:PutRecord",
            "kinesis:AddTagsToStream",
            "kinesis:DecreaseStreamRetentionPeriod",
            "kinesis:DescribeLimits",
            "kinesis:DescribeStream",
            "kinesis:DescribeStreamConsumer",
            "kinesis:DescribeStreamSummary",
            "kinesis:GetShardIterator",
            "kinesis:IncreaseStreamRetentionPeriod",
            "kinesis:ListShards",
            "kinesis:ListStreamConsumers",
            "kinesis:ListStreams",
            "kinesis:ListTagsForStream",
            "kinesis:MergeShards",
            "kinesis:PutRecord",
            "kinesis:PutRecords",
            "kinesis:GetRecords",
            "kinesis:RegisterStreamConsumer"
        ],
        "Resource":"arn:aws:kinesis:${AWS_REGION}:*:stream/privacera*"
    },
    {
        "Sid":"S3BucketRead",
        "Effect":"Allow",
        "Action":[
            "s3:Head*",
            "s3:List*",
            "s3:Get*"
        ],
        "Resource":[
            "arn:aws:s3:::${PRIVACERA_BUCKET}"
        ]
    },
    {
        "Sid":"S3ObjectAll",
        "Effect":"Allow",
        "Action":[
            "s3:PutObject",
            "s3:PutObjectAcl",
            "s3:GetObject",
            "s3:GetObjectAcl",
            "s3:DeleteObject",
            "s3:GetObjectVersion",
            "s3:DeleteObject",
            "s3:DeleteObjectVersion"
        ],
        "Resource":[
            "arn:aws:s3:::${PRIVACERA_BUCKET}/*"
        ]
    },
    {
        "Sid":"S3GlobalRead",
        "Effect":"Allow",
        "Action":[
            "s3:HeadBucket",
            "s3:ListAllMyBuckets"
        ],
        "Resource":[
            "arn:aws:s3:::${PRIVACERA_BUCKET}"
        ]
    },
    {
        "Sid":"ManageSQSMessages",
        "Effect":"Allow",
        "Action":[
            "sqs:DeleteMessage",
            "sqs:ReceiveMessage"
        ],
        "Resource":[
            "arn:aws:sqs:${AWS_REGION}:${ACCOUNT_ID}:privacera*"
        ]
    },
    {
        "Sid":"ListSQSQueues",
        "Effect":"Allow",
        "Action":"sqs:ListQueues",
        "Resource":[
            "arn:aws:sqs:${AWS_REGION}:${ACCOUNT_ID}:privacera*"
        ]
    }
]
} 
Policy to scan S3 buckets

This policy allows to access the S3 buckets that contains data to be scanned.

  • ${CUSTOMER_REALTIMESCAN_BUCKET}: Bucket that will be monitored for files to be scanned in real-time.

  • ${CUSTOMER_LANDING_BUCKET}: Bucket that will be monitored for files to be scanned.

  • ${CUSTOMER_QUARANTINE_BUCKET}: Bucket to which files will be quarantined.

  • ${CUSTOMER_TRANSFER_BUCKET}: Bucket to which files will be transferred if they do not have any sensitive tags.

  • ${CUSTOMER_ARCHIVE_BUCKET}: Bucket to which a copy of the original files will be transferred if they contain any sensitive tags.

{
"Version":"2012-10-17",
"Statement":[
    {
        "Sid":"S3ObjectAll",
        "Effect":"Allow",
        "Action":"s3:*",
        "Resource":[
            "arn:aws:s3:::${CUSTOMER_LANDING_BUCKET}",
            "arn:aws:s3:::${CUSTOMER_LANDING_BUCKET}/*",
            "arn:aws:s3:::${CUSTOMER_REALTIMESCAN_BUCKET}",
            "arn:aws:s3:::${CUSTOMER_REALTIMESCAN_BUCKET}/*",
            "arn:aws:s3:::${CUSTOMER_QUARANTINE_BUCKET}",
            "arn:aws:s3:::${CUSTOMER_QUARANTINE_BUCKET}/*",
            "arn:aws:s3:::${CUSTOMER_TRANSFER_BUCKET}",
            "arn:aws:s3:::${CUSTOMER_TRANSFER_BUCKET}/*",
            "arn:aws:s3:::${CUSTOMER_ARCHIVE_BUCKET}",
            "arn:aws:s3:::${CUSTOMER_ARCHIVE_BUCKET}/*"
        ]
    }
]
}

Configuration#

  1. SSH to the Privacera Host VM.

  2. Configure your environment.

    • Configure Discovery for a Kubernetes environment. You need to set the Kubernetes cluster name. For more information, see Discovery (Kubernetes Mode)

    • For a Docker environment, you can skip this step.

  3. Run the following commands.

    cd ~/privacera/privacera-manager
    cp config/sample-vars/vars.discovery.aws.yml config/custom-vars/
    vi config/custom-vars/vars.discovery.aws.yml
    
  4. Edit the following properties. For property details and description, click here.

    DISCOVERY_BUCKET_NAME: "<PLEASE_CHANGE>"
    
  5. (Optional) If you want to customize Discovery configuration further, you can add custom Discovery properties. For more information, refer to Discovery Custom Properties.

    For example, by default, the username and password for the Discovery service is padmin/padmin. If you choose to change it, refer to Add Custom Properties.

  6. Run the following commands.

    cd ~/privacera/privacera-manager
    ./privacera-manager.sh update
    

Enable Realtime Scan#

An AWS SQS queue is required, if you want to enable realtime scan on the S3 bucket.

After running the PM update command, an SQS queue will be created for you automatically with the name, privacera_bucket_sqs_{{DEPLOYMENT_ENV_NAME}}, where {{DEPLOYMENT_ENV_NAME}} is the environment name you set in the vars.privacera.yml file. This queue name will appear in the list of queues of your AWS SQS account.

If you have an SQS queue which you want to use, add the DISCOVERY_BUCKET_SQS_NAME property in the vars.discovery.aws.yml file and assign your SQS queue name.

If you want to enable realtime scan on the bucket, click here.


Last update: July 23, 2021