Add infra templates (#45)

* updating broken links, documentation and terraform code

* correct the constraints version in requirements.txt

---------

Co-authored-by: ramadu <[email protected]>

blueprints/README.md

@@ -2,7 +2,7 @@
 
 ## Description
 
-This is a collection of getting started blueprints for using Amazon Managed Workflows for Apache Airflow (MWAA). Below
+This is a collection of getting started blueprints for building pipelines that run in Amazon Managed Workflows for Apache Airflow (MWAA). Below
 is the high level structure and the key files
 
 ```sh
@@ -32,7 +32,6 @@ is the high level structure and the key files
 │   │   ├── dags
 │   │   └── image
 └── infra
-    ├── cdk
     ├── cloudformation
     └── terraform
 ```
@@ -47,20 +46,23 @@ is the high level structure and the key files
 - **examples:** This folder has a collection of technology specific DAGs organized into specific subfolders. Review the
   subfolders for details
 
-- **infra:** This folder has the infrastructure setup needed t o run the examples. Infrastructures are based
-  on ```cloudformation```, ```cdk``` and ```terraform```.
+- **infra:** This folder has the infrastructure as code samples for creating example MWAA environment
+
 
-## Badges
 
 ## Installation
 
 ### CDK
-This example cretes MWAA environment and has the DAGs to create an EKS cluster. 
+This example creates MWAA environment along with permissions related to EKS. 
 Setup Environment and execute examples [cdk](examples/EKS/README.md)
 
 ### Terraform
 
-Access [terraform](infra/terraform/README.md)
+Access [terraform](../infra/terraform/README.md)
+
+### AWS CloudFormation
+
+Access [CloudFormation](../infra/cloudformation/README.md)
 
 #### Examples
 

blueprints/examples/EKS/README.md

@@ -2,18 +2,9 @@
 
 ## Description
 
-```
-The EKS cluster needed to run the pods needed by mwaa can be creaed via mwaa (using the EKSPodOperator) 
-or can be created separately as a dedicated cluster. This POC creates the cluster using the mwaa operators
-
-```
+The set of sample files in the folder creates an MWAA environment with permission to create/delete EKS cluster and run a job in EKS Pod.
 
----
-
-```
-```
-
-#### Setup
+## Setup
 
 **_These steps setup CDK in the development or the build environment_**
 
@@ -36,53 +27,49 @@ These commands provision the infrastructure needed to run the workload
 - S3 Bucket to store the artifacts needed for MWAA
 - An empty MWAA environment
 
-4. Setup the infrastructure using the command
+1. Setup the infrastructure using the command
 
 ```shell
       pip install -r ./infra/cdk/requirements.txt
-      make AWS_PROFILE=[your aws_profile] cdk-deploy-infra
+      make AWS_PROFILE=[your aws_profile]       
 ```    
 
-5. If you are creating the EKS cluster using the MWAA DAG, it needs the role for the node group to be pre-created
+2. If you are creating the EKS cluster using the MWAA DAG, it needs the role for the node group to be pre-created
 
 ```shell
       make AWS_PROFILE=[your aws_profile] cdk-setup-eks-role
 ```
-6. Copy the output **_NODEGROUPROLEARN_**. This is the role_arn used for EKS and will be needed later
-7. Deploy the example DAGS and the requirements.txt file into the S3 bucket
+3. Copy the output **_NODEGROUPROLEARN_**. This is the role_arn used for EKS and will be needed later
+4. Deploy the example DAGS and the requirements.txt file into the S3 bucket
 
 ```shell
     make AWS_PROFILE=[your aws_profile] cdk-deploy-to-bucket
 ```
 
-7. Update the configurations for the mwaa environment
+5. Update the configurations for the mwaa environment
 
-**Many customers have existing existing MWAA environments. If you have an existing environment, you can skip the encreation above. This change will need a restart to the MWAA environment**
+**Many customers have existing existing MWAA environments. If you have an existing environment, you can skip the creation above. This change will need a restart to the MWAA environment**
 
-![edit env](../../images/edit_environment.png)
+![edit env](images/edit_environment.png)
 
 
 Add the path to requirements.txt file in the mwaa environment
 
-![requirements](../../images/requirements.png)
-
-Add a variable cdk.nodegroup_role with the value of the NODEGROUPROLEARN created above (Note: This step will be automated in the next release)
-
-![nodegroup_role](../../images/nodegroup_variable.png)
+![requirements](images/requirements.png)
 
-8. This completes the setup. The rest of the steps are performed in the MWAA environment
+Add a variable cdk.nodegroup_role with the value of the NODEGROUPROLEARN created above 
 
-9. Initialize the variables by initialize_vars DAG. (The other DAGs will show as failed. They are dependent on the variables. Running this will fix the issue) 
+![nodegroup_role](images/nodegroup_variable.png)
 
-![initialize](../../images/initialize_vars.png)
+6. This completes the setup. The rest of the steps are performed in the MWAA environment
 
-10. Run the create_eks_cluster_nodegroup DAG to create the EKS cluster
+7. Run the create_eks_cluster_nodegroup DAG to create the EKS cluster
 
-11. Run the eks_run_pod to execute a sample pod using EksPodOperator
+8. Run the eks_run_pod to execute a sample pod using EksPodOperator
 
-12. Once done, Run delete_eks_cluster_nodegroup to delete the cluster
+9. Once done, Run delete_eks_cluster_nodegroup to delete the cluster
 
-13. [BONUS]: To update or add a DAGs, add them to examples/EKS/cdk/dags and deploy
+10. [BONUS]: To update or add a DAGs, add them to examples/EKS/cdk/dags and deploy
 
 ```shell
     make AWS_PROFILE=[your aws_profile] cdk-deploy-to-bucket

blueprints/examples/EKS/dags/requirements.txt

@@ -1,4 +1,3 @@
---constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.7.txt"
-apache-airflow-providers-amazon==3.0.0
+--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.5.12/constraints-3.10.txt"
 apache-airflow-providers-cncf-kubernetes==3.0.0
 

blueprints/examples/EMR/README.md

@@ -21,10 +21,6 @@ _Note: If you do not have running MWAA environment, deploy it from the root of t
 
 Clone the repository
 
-```sh
-git clone <repo>
-```
-
 Navigate into one of the example directories and run `make` by passing MWAA environment related arguments
 
 ```sh

infra/cloudformation/README.md

@@ -0,0 +1,32 @@
+
+# Welcome to your MWAA Cloudformation project!
+
+Welcome to Amazon MWAA Blueprints !
+
+This repository contains a collection of code that aim to make it easier and faster for customers to adopt Amazon MWAA. It can be used by AWS customers, partners, and internal AWS teams to configure and manage complete MWAA environment that are fully bootstrapped with the operational software that is needed to deploy and operate workloads.
+
+
+
+## Getting Started
+
+### Prerequisites
+
+First, ensure that you have installed the following tools locally.
+
+1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
+2. [MWAA deployer Permissions](https://docs.aws.amazon.com/mwaa/latest/userguide/access-policies.html#full-access-policy)
+
+### Deployment Steps
+1. To create MWAA environment with VPC with NAT/IGW
+https://docs.aws.amazon.com/mwaa/latest/userguide/quick-start.html
+2. To create MWAA environment with no NAT/IGW but with VPCEndpoints
+
+    - Run the command below after replacing your_bucket_name with the S3 Bucket where DAGs are present
+```
+aws cloudformation create-stack --stack-name mwaa-environment-private-network --template-body file://template.yaml --parameters ParameterKey=S3Bucket,ParameterValue=your_bucket_name --capabilities CAPABILITY_IAM
+
+```
+### Cleanup
+```
+aws cloudformation delete-stack --stack-name mwaa-environment-private-network 
+```