aws-cdk-py-s3-01

Using AWS CDK Python to create S3 bucket

Prerequisite

• Node.js & NPM

• Python - pre-installed with Ubuntu

• PIP

$ apt install python3-pip

• Python virtual environment

$ apt install python3-virtualenv

• AWS CLI

• Install WS CDK

$ npm install -g aws-cdk
$ cdk --version
$ cdk --help

• Configure CDK credentials

  • Configure CDK default region in the ~/.aws/config file

[default]
region=us-west-2

• Configure CDK credential ~/.aws/credential file. The IAM user credential to be used for deploying to AWS. Also see "Set Policy" below, if the user does not belong to the Administrator group.

[default]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

Setup CDK

• This is a one time setup for the AWS Account; it's called CDK Bootstrap;

• Use the AWS S3 console to check whether the CDK Bootstrap has been activated, by looking for the following S3 bucket:

cdk-<UNIQUE-ID>-assets-<ACCOUNT-NUMBER>-<REGION>

• Otherwise, run the CDK Bootstrap, as follow:

// Get acct-number from AWS console or
$ aws sts get-caller-identity
// Get the default region for the profile
$ aws configure get region

$ cdk bootstrap aws://ACCOUNT-NUMBER/REGION

Build and Deploy

• [Skip Create Project] This repo contains artifacts generated by the following codes:

$ cd ~/projects
$ mkdir aws-cdk-py-s3-01
$ cd aws-cdk-py-s3-01
$ cdk init sample-app --language python

• Create python virtual environment

$ virtualenv .venv
$ source .venv/bin/activate

(.venv) $ pip install -r requirements.txt

• Set Policy; the following policies are needed minimally to cdk deploy, and they are needed for CloudFormation & S3.
  • Unless, the IAM user (configured above in the ~/.aws/credential and aws_access_key_id key) belongs to the Administrators group.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "cloudformation:DescribeStacks",
                "cloudformation:CreateChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:ExecuteChangeSet",
                "cloudformation:DescribeStackEvents",
                "cloudformation:DeleteChangeSet",
                "cloudformation:DeleteStack",
                "cloudformation:GetTemplate",
                "s3:CreateBucket"
            ],
            "Resource": "*"
        }
    ]
}

• Build

(.venv) $ cdk synth

• Deploy

(.venv) $ cdk deploy

# to speed up the deployment time 
(.venv) $ cdk deploy --hotswap

Test

• Install testing packages

(.venv) $ cd ~/projects/aws-cdk-py-s3-01

(.venv) $ pip install -r requirements-dev.txt

• The test file: ./tests/unit/test_aws_cdk_py_s3_01_stack.py

• Test cases validate that:

  • the S3 bucket is created
  • the bucket is setup with removal_policy = RemovalPolicy.DESTROY
  • the bucket is setup with auto_delete_objects = True
  • the bucket is setup with block_public_access=s3.BlockPublicAccess.BLOCK_ALL

• Test Results as of 2022, July 2

(.venv) $ cd ~/projects/aws-cdk-py-s3-01
(.venv) $ pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/<user>/projects/aws-cdk-py-s3-01
collected 4 items

tests/unit/test_aws_cdk_py_s3_01_stack.py ....                                                                                  [100%]

========================================================== 4 passed in 6.21s ==========================================================
(.venv) $

CLEANUP

• Note: during cleanup, the default mechanism is to not delete the S3 bucket (removalPolicy: cdk.RemovalPolicy.RETAIN); only delete other services created during deployment, i.e., the cloudformation. However, for this tutorial, the S3 bucket has been setup to delete the S3 bucket during cleanup, as defined in the ./aws-cdk-py-s3-01/aws_cdk_py_s3_01_stack.py:

    # create bucket: no encryption, auto delete objects in the bucket, and
    # the bucket during dismantling
    bucket = s3.Bucket(self, "MyCdkSample01Bucket-20220624-dzong",
              removal_policy = RemovalPolicy.DESTROY,
              auto_delete_objects = True,
              block_public_access=s3.BlockPublicAccess.BLOCK_ALL)

• Cleanup all artifacts created by CDK

(.venv) $ cdk destroy

Output:

✨  Synthesis time: 10.26s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
┌───┬───────────────────────────────────┬────────┬───────────────────────────────────┬───────────────────────────────────┬───────────┐
│   │ Resource                          │ Effect │ Action                            │ Principal                         │ Condition │
├───┼───────────────────────────────────┼────────┼───────────────────────────────────┼───────────────────────────────────┼───────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCust │ Allow  │ sts:AssumeRole                    │ Service:lambda.amazonaws.com      │           │
│   │ omResourceProvider/Role.Arn}      │        │                                   │                                   │           │
├───┼───────────────────────────────────┼────────┼───────────────────────────────────┼───────────────────────────────────┼───────────┤
│ + │ ${MyCdkSample01Bucket-20220624-dz │ Allow  │ s3:DeleteObject*                  │ AWS:${Custom::S3AutoDeleteObjects │           │
│   │ ong.Arn}                          │        │ s3:GetBucket*                     │ CustomResourceProvider/Role.Arn}  │           │
│   │ ${MyCdkSample01Bucket-20220624-dz │        │ s3:List*                          │                                   │           │
│   │ ong.Arn}/*                        │        │                                   │                                   │           │
└───┴───────────────────────────────────┴────────┴───────────────────────────────────┴───────────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬───────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┐
│   │ Resource                                                      │ Managed Policy ARN                                             │
├───┼───────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${Custom::S3AutoDeleteObjectsCustomResourceProvider/Role}     │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role │
│   │                                                               │ /AWSLambdaBasicExecutionRole"}                                 │
└───┴───────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Do you wish to deploy these changes (y/n)? y
aws-cdk-py-s3-01: deploying...

• Additional cleanup not done by the the cdk destroy

  • CloudWatch Log group; go to AWS Console "CloudWatch > Logs > Log groups" and delete two log groups:
    • /aws/lambda/<stack-name>-CustomCDKBucketDeployment<unique-id>
    • /aws/lambda/<stack-name>-CustomS3AutoDeleteObjectsCustomResourcePr-<unique-id>

• Exit virtualenv

(.venv) $ deactivate
$

Useful commands

• cdk ls list all stacks in the app
• cdk synth emits the synthesized CloudFormation template
• cdk deploy deploy this stack to your default AWS account/region
• cdk diff compare deployed stack with current state
• cdk docs open CDK documentation