diff --git a/doc/architecture/Architecture.md b/doc/architecture/Architecture.md index 4ffe961c..3940ddee 100644 --- a/doc/architecture/Architecture.md +++ b/doc/architecture/Architecture.md @@ -2,24 +2,18 @@ --- -## TREEHOOSE (TRE) +This document explains the high level architecture of +the Trusted Research Environment that would be deployed +on AWS Cloud following the installation +steps in this repository. ---- - -TREEHOOSE is the Trusted Research Environment (TRE) implementation -that will be deployed for each research project. -Deploying the solution with the **default parameters** -builds the following environment in the AWS Cloud. - -![TREEHOOSE Architecture](../../res/images/TREEHOOSE-architecture.png) - -### Overview +## Overview --- The TREEHOOSE solution is formed of [Service Workbench on AWS](https://aws.amazon.com/government-education/research-and-technical-computing/service-workbench/) -and a data lake that together provides the building blocks +and a data lake that together provide the building blocks for the Trusted Research Environment (TRE) capability. [AWS Control Tower](https://aws.amazon.com/controltower/) provides the scalable multi-account setup for managing TRE implementations at scale in AWS Cloud. @@ -31,29 +25,69 @@ provides optional add-on components to enable - Workspace backups - Budget controls -### Solution Overview +TREEHOOSE is the Trusted Research Environment (TRE) implementation +that will be deployed for each research project. +Deploying the solution with the **default parameters** +builds the following environment in AWS Cloud. + +![TREEHOOSE Architecture](../../res/images/TREEHOOSE-architecture.png) + +The solution uses Infrastructure as Code for deployment. +Additional sections in this document provide additional details about each component. Below is a brief explanation +of the numbered steps in the diagram. + +1. TRE Data Managers use AWS Management console to upload + data to the TRE Data Lake to be used for research. +1. IT Administrators use the Service Workbench web application + to administer resources in the TRE environment. +1. The budget controls component is used to set budget limits for the TRE + project. IT Administrators can set the budget and any actions + to be taken when the budget thresholds are breached. +1. Backup functionality for research workspaces can also be + enabled. IT Admins can monitor + these through AWS Backup. +1. Data Managers and IT Administrators can work together to provide researchers with access to relevant + data sets from the data lake. +1. Researchers can create and connect to approved workspaces through the Service Workbench web application. + They get secure access to compute resources using + Amazon AppStream 2.0. +1. On research completion the researcher can request egress of + research results. +1. The egress request is processed through a Data Egress App add-on + with a comprehensive review process with multiple approvers + before the data is available for download. +1. Egress requests that are approved can be downloaded by Data Egress Managers + and shared with the Researcher who requested the data egress. + There is a configurable limit to the number of downloads which can be made. +1. Audit & Compliance teams get full visibility into all + user activities resulting in AWS API calls through centralised + CloudTrail logs. Additionally, they get breakglass + access to all TRE projects/accounts in the TRE through + a Lambda function role in the Audit account. + +## Component Overview --- -#### *AWS Control Tower* +### *AWS Control Tower* --- -Using TREEHOOSE implemenation a user should be able to run multiple -isolated projects and trusted research environment in parallel -and scale according the organisation's research needs. +Using the TREEHOOSE implementation allows a user to run multiple isolated +TRE projects in parallel and to scale according to the organisation's research needs. -To enable TREEHOOSE implementation to support scalable research workloads -, meet the organization’s security and auditing requirements, and evolve with business requirements -it uses AWS Control Tower to set up and govern a secure, +The TREEHOOSE TRE implementation supports scalable research workloads, +aims to meet an organization’s security and auditing requirements, +and can evolve with the business demands. +To meet this goal, an AWS Control Tower provides the setup to govern a secure, multi-account AWS environment, called a landing zone. -Below is the high-level Organization Unit and Account Structure +Below is the high-level Organizational Unit and Account Structure that will be setup by using the TREEHOOSE solution. ![Multi-account structure](../../res/images/multi-account-setup.png) -#### *Service Workbench on AWS Solution* +### *Service Workbench on AWS Solution* --- @@ -73,14 +107,12 @@ Key Components : (more services as desired; this is customisable by providing Service Catalog templates). - For the secure access environment: AWS AppStream 2.0 -#### *Datalake* +### *Data Lake* --- -TREEHOOSE uses a data lake setup that -uses [AWS Lake Formation](https://aws.amazon.com/lake-formation/) -under the hoods for creating a secure and scalable -data store for storing research data. +TREEHOOSE uses a data lake setup that leverages AWS Lake Formation +to create a secure and scalable data store for storing research data. A data lake is a centralized, curated, and secured repository that stores all your data, both in its original form and prepared for analysis. It creates a pre-configured data lake to be used for TRE data pipelines. @@ -88,9 +120,9 @@ This is a mandatory add-on. Key Components : -- AWS Lake Formation +- AWS Lake Formation, Amazon S3, AWS KMS, AWS Glue, Amazon Athena -#### *Data Egress Application* +### *Data Egress Application* --- @@ -98,7 +130,7 @@ This add-on provides a data egress approval workflow for researchers to take out data from TRE with the permission of multiple parties (data manager, research IT, etc.). The add-on is hosted as a web application supported by -backend infrastrucutre. Each add-on installation is tied +backend infrastructure. Each add-on installation is tied to a specific TRE project. The add-on provides a streamlined @@ -115,9 +147,9 @@ Key Components : - For the UI: AWS Amplify - For the backend: AWS Step Functions, Amazon EFS, - AWS Lambda, Amazon DynamoDB, Amazon SES, Amazon S3, Amazon SNS, Amazon Cognito + AWS Lambda, Amazon DynamoDB, Amazon SES, Amazon S3, AWS KMS, Amazon SNS, Amazon Cognito, AWS AppSync -#### *Workspace backup* +### *Workspace backup* --- @@ -131,12 +163,15 @@ researchers to select whether they want to enable periodic workspace backups when creating the workspace. Only TRE administrators can control the backup frequency -and back retention periods. Also, any restore operations +and retention periods. Also, any restore operations need to be performed by admins. -This add-on uses [AWS Backup](https://aws.amazon.com/backup/) for backing up block storage attached to -[Amazon EC2](https://aws.amazon.com/ec2/) based compute workspaces while it uses a be-spoke -implementation to backup [Amazon SageMaker Notebook Instances](https://docs.aws.amazon.com/sagemaker/latest/dg/nbi.html) +This component uses: + +- [AWS Backup](https://aws.amazon.com/backup/) for backing up block storage attached to + [Amazon EC2](https://aws.amazon.com/ec2/) based compute workspaces +- a be-spoke + implementation to backup [Amazon SageMaker Notebook Instances](https://docs.aws.amazon.com/sagemaker/latest/dg/nbi.html) Below diagrams explain how the backup solution works for @@ -152,23 +187,23 @@ for Key Components: - For the backend: AWS Step Functions, - AWS Lambda, Amazon CloudWatch Events, AWS CloudForamtion, AWS Backup, Amazon S3 + AWS Lambda, Amazon CloudWatch Events, AWS CloudFormation, AWS Backup, Amazon S3 -#### *Budget controls* +### *Budget controls* --- Budget controls is an optional -add-on that allows administrators and finance stakeholders +component that allows administrators and finance stakeholders of the TRE to stay on top of project finances. -This add-on can optionally be deployed for +This component can optionally be deployed for each TRE project and allows to - **Monitor** : set thresholds for sending budget alerts - **Report** : sending notification on budget usage -- **Repond** : automate actions to avoid over-spending +- **Repsond** : automate actions to avoid over-spending -The add-on uses [AWS Budgets](https://aws.amazon.com/aws-cost-management/aws-budgets/) +The component uses [AWS Budgets](https://aws.amazon.com/aws-cost-management/aws-budgets/) to plan and set expectations around TRE project costs. Key Components: diff --git a/doc/architecture/Cost.md b/doc/architecture/Cost.md index e6541c13..23c4c016 100644 --- a/doc/architecture/Cost.md +++ b/doc/architecture/Cost.md @@ -4,9 +4,100 @@ You are responsible for the cost of the AWS services used to run this solution. As of January 2022, the cost for running this solution with the default settings -in the EU West (Ireland) AWS Region is approximately $X for TRE account with all add-ons. +in the EU West (London) AWS Region is approximately **$30** for TRE account with all add-ons. Prices are subject to change. For full details, see the pricing page for each AWS service used in this solution. > **_NOTE:_** Many AWS Services include a Free Tier – a baseline amount of the service that customers can use at no charge. > Actual costs may be more or less than the pricing examples provided. + +The baseline cost is just for spinning up the infrastructure. +As the solution is based on Serverless architecture, you only +pay for what you use when you use. + +Following factors will contribute to incremental costs for an actively used deployment or TRE account: + +- Compute resources used by researchers in the form + of EC2 instances +- Volume of data stored in S3 buckets in the Data Lake account +- AppStream resources used by researchers to interact + with their research workspace +- Volume of backup data stored by AWS Backup + +The cost of using and maintaining an AWS Control Tower +environment can be found [here](https://aws.amazon.com/controltower/pricing/). + +The best place to calculate the cost of using this solution +is by using [AWS Pricing Calculator](https://calculator.aws/#/) +and putting in the correct usage information. + +## Example cost table + +--- + +The following table provides an example cost breakdown for deploying this +solution with the default settings in EU West (Ireland) AWS Region. + +### Base Installation + +An installation of TRE without any workspaces and users. + +|AWS Service|Monthly cost| +|----|----| +|Networking services|$11| +|KMS|$6| +|Config|$4| +|CloudTrail|$3.5| +|EC2-other|$1.5| +|DynamoDB|$6| +|Service Catalog|$1| +|Step Functions|$0.09| +|Lambdas|$0.003| +|CloudFront|$0.0002| +|CloudWatch|$0.0003| +|Total|$33.0935| + +### EC2 Usage + +Below example is based on on-demand +pricing. +A researcher uses a workspace for 730 hours. + +Example - 1 +|AWS Service|Monthly cost| +|----|----| +|EC2 - t3.large|$66.58 | +|EBS - 10GB| $1.10| +|Total|$67.68| + +Example - 2 +|AWS Service|Monthly cost| +|----|----| +|EC2 - m6g.8xlarge|$1,004.48 | +|EBS - 80GB| $8.80| +|Total|$1,013.28| + +### SageMaker Usage + +A researcher use sagemaker notebook +for 730 hours on a project. + +Example +|AWS Service|Monthly cost| +|----|----| +|SageMaker - notebook - ml.c5.large| $82.80| +|Total|$82.80| + +### S3 storage + +A researcher works on a 1 TB data study +and produces a 10 GB output to download. + +Example +|AWS Service|Monthly cost| +|----|----| +|S3 - study data| $23.58 | +|Data Egress| $0.90| +|Total|$24.48| + +All cost examples provided above are indicative. diff --git a/doc/architecture/Design-Considerations.md b/doc/architecture/Design-Considerations.md index 28465e69..47e1143f 100644 --- a/doc/architecture/Design-Considerations.md +++ b/doc/architecture/Design-Considerations.md @@ -8,8 +8,8 @@ Below are the key design considerations for TREEHOOSE --- -- All the core infrastructure is deployed using IaC (Infrastructure as Code). -- Maximised the use of AWS Serverless services for ease of operability and scalability. +- All the core infrastructure is deployed using [IaC (Infrastructure as Code)](https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-aws/infrastructure-as-code.html). +- The solution is based on Serverless Architecture for ease of operability and scalability. ## Audit @@ -19,6 +19,8 @@ Below are the key design considerations for TREEHOOSE and the logs centralised for Auditing. - [AWS Config](https://aws.amazon.com/config/) is enabled in all AWS accounts and the config records centralised for Auditing. +- [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/) is used for + log aggregation and metrics for each TRE project/AWS account. ## Security @@ -27,4 +29,25 @@ Below are the key design considerations for TREEHOOSE - Use [AWS KMS](https://aws.amazon.com/kms/) for encryption at-rest. - Encryption in-transit is enabled for all AWS services where applicable and also enabled for all API calls. -- For all [AWS IAM](https://aws.amazon.com/iam/) policies principle of least privilege has been followed. +- For all [AWS IAM](https://aws.amazon.com/iam/) policies the principle of least privilege has been followed. +- [AWS Accounts](https://aws.amazon.com/account/) provide well-defined billing and security boundaries. + Hence each research project should be hosted in a separate AWS account. + +## Considerations for End Users + +--- + +These are some additional decisions that the end user of +TREEHOOSE should make based on their functional and +non-functional requirements. + +- Centralise and enable AWS Security services like: + - [AWS Security Hub](https://aws.amazon.com/security-hub/) + - [Amazon GuardDuty](https://aws.amazon.com/guardduty/) + - [Amazon Macie](https://aws.amazon.com/macie/) + - [AWS IAM Access Analyzer](https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html) + +- Enable [AWS Web Application Firewall](https://aws.amazon.com/waf/) for Web Applications. +- Enable additional [Control Tower Guardrails](https://docs.aws.amazon.com/controltower/latest/userguide/guardrails.html). +- Use [Amazon EC2 reserved instances](https://aws.amazon.com/ec2/pricing/reserved-instances/). +- [Optimize](https://docs.aws.amazon.com/whitepapers/latest/best-practices-for-deploying-amazon-appstream-2/cost-optimization.html) how you use AppStream. diff --git a/res/images/TREEHOOSE-architecture.png b/res/images/TREEHOOSE-architecture.png index d66f1661..b3bd1ba6 100644 Binary files a/res/images/TREEHOOSE-architecture.png and b/res/images/TREEHOOSE-architecture.png differ diff --git a/res/images/multi-account-setup.png b/res/images/multi-account-setup.png index a4dd539f..7063c24e 100644 Binary files a/res/images/multi-account-setup.png and b/res/images/multi-account-setup.png differ