Skip to content

Commit 5bcd255

Browse files
ea-mtenhoorea-mtenhoor
committed
Add updates for 7.x
1 parent c86f9cd commit 5bcd255

21 files changed

+1015
-294
lines changed

‎.github/workflows/linting.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: CFN Lint
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- '7.x'
7+
8+
permissions: read-all
9+
10+
jobs:
11+
security-scan:
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
python-version: ["3.12"]
16+
steps:
17+
- name: Checkout Repository
18+
uses: actions/checkout@v4
19+
with:
20+
ref: ${{ github.head_ref }}
21+
- name: Set up Python ${{ matrix.python-version }}
22+
uses: actions/setup-python@v3
23+
with:
24+
python-version: ${{ matrix.python-version }}
25+
- name: Install dependencies
26+
run: |
27+
python -m pip install --upgrade pip
28+
pip install cfn-lint
29+
pip install checkov
30+
- name: CFN Lint Templates
31+
run: |
32+
cfn-lint $(git ls-files './templates/*.yml') --include-checks I
33+
- name: Checkov
34+
run: |
35+
checkov --directory templates/ --quiet

‎.vscode/extensions.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"recommendations": [
3+
"kddejong.vscode-cfn-lint",
4+
"xargsuk.checkov-prismaless"
5+
]
6+
}

‎README.md‎

Lines changed: 51 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -2,41 +2,55 @@
22

33
Welcome to the StartingBlocks Open Source repository! The artifacts in this repository will allow users to deploy a cloud native Ed-Fi ODS/API implementation using AWS resources. StartingBlocks is additionally inclusive of a suite of management functions that allow users to manage resources to configure their Ed-Fi deployment appropraitely for a variety of use cases.
44

5-
[Deployment Steps](./docs/sbe-deployment-steps.md)
5+
>[!TIP]
6+
>Documentation is available in the [docs](./docs/) folder in this repo, and at [docs.startingblocks.org](https://docs.startingblocks.org).
7+
8+
# Getting Started
9+
10+
## Prerequisites
11+
1. You must have an AWS account
12+
2. You must have deployed a custom VPC, 2 public subnets, and 2 private subnets before deploying the StartingBlocks templates. <b>It is not advisable for users to use the default VPC initially deployed by AWS</b>.
13+
3. You must have created a Hosted Zone and NS record for the domain at which you intend to deploy your StartingBlocks environment(s).
14+
15+
If you would like more guidance and support with deploying StartingBlocks, [please reach out to our team using this form.](https://edanalytics.atlassian.net/helpcenter/products-and-services/portal/15/group/45/create/300)
16+
17+
## Deployment Steps
18+
Steps to deploy StartingBlocks templates.
19+
20+
1. In your AWS account, create an S3 bucket that will be used to hold the StartingBlocks templates.
21+
Example bucket name:
22+
- `{orgname}-{environment label}-{version}-cloudformation`
23+
- `EducationAnalytics-Prod2425-7.1-cloudformation`
24+
#####
25+
aws s3api create-bucket --bucket my-cfn-bucket
26+
2. Clone the StartingBlocks OSS repository to your local workstation and checkout the relevant version branch.
27+
- All Ed-Fi ODS/API 7.x versions can be deployed from the StartingBlocks `7.x` branch. Use this branch to deploy Ed-Fi API 7.1, 7.2, or 7.3.
28+
#####
29+
git clone https://github.com/edanalytics/startingblocks_oss.git
30+
cd startingblocks_oss
31+
git checkout 7.x
32+
3. Upload the contents of the repository to your S3 bucket location. You can do this via AWS CLI or dragging the folders from your file explorer to the S3 console for your bucket.
33+
#####
34+
aws s3 sync . s3://my-cfn-bucket/ --exclude ".git/*"
35+
4. Copy the S3 URL for the `templates > 1-StartingBlocks-Main-Template.yml` file.
36+
#####
37+
https://my-cfn-bucket.s3.REGION.amazonaws.com/templates/1-StartingBlocks-Main-Template.yml
38+
1. Navigate to the [CloudFormation console](https://us-east-2.console.aws.amazon.com/cloudformation/) in AWS. Create a new stack with new resources.
39+
2. Select `Template is ready > Amazon S3 URL` and paste your copied URL for the `1-StartingBlocks-Main-Template.yml` file into the field.
40+
3. Enter stack parameter values. [Please read the doc here for more information on parameter values.](./docs/sbe-parameter-values.md)
41+
4. Navigate through the various screens until you are able to start the deployment.
42+
43+
## Post Deploymnet Configuration
44+
45+
After a successful CloudFormation deployment of the StartingBlocks stack:
46+
1. Use the [TenantManagement](./docs/sbe-functions.md#variable-requirements) function to create Ed-Fi Tenants.
47+
2. Use the [ODSManagement](./docs/sbe-functions.md#variable-requirements-1) function to create ODSs within Tenants.
48+
3. Use the [EdOrgManagement](./docs/sbe-functions.md#variable-requirements-2) function to create preliminary EdOrg records within ODSs.
49+
- The EdOrg records created using this function are meant to be updated with full information via integrating systems. It is not meant to create comprehensive records, but rather populate enough information to create API credentials.
50+
4. Use the [TenantManagement](./docs/sbe-functions.md#keygen) KeyGen action to create AdminAPI credentials for a given Tenant.
51+
5. Use the AdminAPI to create application (Ed-Fi API) credentials for the Tenants.
52+
- [Please read the AdminAPI documentation published by the Alliance for detailed information on usage.](https://github.com/Ed-Fi-Alliance-OSS/AdminAPI-2.x)
53+
54+
>[!TIP]
55+
>Education Analytics offers StartingBlocks as a managed service. [Inquire here](https://www.edanalytics.org/products/starting-blocks)!
656
7-
[Management and Reporting Functions Usage](./docs/sbe-functions.md)
8-
9-
# Overall Architecture
10-
Below is a high level architecture diagram and inventory of <i>some</i> AWS resources deployed when using these templates. It's important to note, that once these resources are deployed, users are immediately responsible for any incurred costs, even if the environment is not being used. Please also note that there are some pre-requisite resources that are included in this diagram for informational purposes. Specifically, public subnets, VPC and private subnets must already exist prior to deploying StartingBlocks. For deployment steps [please click here](./docs/sbe-deployment-steps.md)
11-
12-
13-
![](./docs/imgs/StartingBlocks-OSS-diagram.svg)
14-
15-
---
16-
17-
**NOTE:**
18-
19-
Currently we do not support deployments of the Ed-Fi Admin App for StartingBlocks v7.1. There is no Admin App currently supported by the alliance that is compatible with Ed-Fi v7.x. When there is a release of the Admin App for v7.x, we are commited to supporting it in StartingBlocks.
20-
21-
---
22-
23-
# Lambda Functions
24-
The diagram highlights that there is a suite of AWS Lambda functions used in StartingBlocks. There are Ed-Fi environment management functions, but there are also general utility functions or functions created as custom CloudFormation resources. Below is a complete inventory of Lambda functions deployed with StartingBlocks. There is also more detail specifically on the environment management functions [in our docs folder here.](./docs/sbe-functions.md) All Lambda functions deployed by CloudFormation are prefixed by the `EnvLabel` parameter value to make them easy to find in the Lambda console.
25-
26-
## Utility Functions
27-
- <b>DbRestore - </b> Restores template databases on initial StartingBlocks deployments.
28-
- <b>EdFiBeanstalkSNSToSlack - </b> Forwards SNS messages sent from Beanstalk Env and RDS instance to Slack.
29-
- <b>East1Alarm - </b> Creates a Route53 Healthcheck Alarm in us-east-1.
30-
- <b>API-Publisher-getmaxchangeversion - </b> Optionally deployed if publisher is also deployed. Lambda Function to replace the getmaxchangeversion ODS function.
31-
## Custom CloudFormation Resource Functions
32-
- <b>EncryptionKeyGenerator - </b> CloudFormation Custom Resource provider. Creates and stores a base64 encoded 256-bit key.
33-
- <b>SetCloudWatchRetention - </b> CloudFormation Custom Resource provider. Sets retention on CloudWatch Log Groups.
34-
- <b>ODSDerivatives - </b> CloudFormation Custom Resource provider. Adds/removes ODS instance derivative in admin db when ODS instance derivative is created/deleted.
35-
## Management and Reporting Functions
36-
- <b>TenantManagement - </b> Used to manage Tenants in Ed-Fi 7.x environments.
37-
- <b>ODSManagement - </b> Used to manage ODSs in Ed-Fi 7.x environments.
38-
- <b>EdOrgManagement - </b> Used to managed Education Organizations in Ed-Fi 7.x environments.
39-
- <b>SbeMetadata - </b> Optionally Deployed if SBAA admin interface is chosen. Provides ARNs for all Lambda Ed-Fi resource management functions.
40-
- <b>TenantResourceTree - </b> Provides tree structure of resources in a tenant.
41-
- <b>DataFreshnessJson - </b> Provides a JSON output for resource counts and dates per table within a given ODS and Tenant.
42-
- <b>ODSUserPermissions - </b> Grants permissions to users in the ODS managed by database groups.

‎database/README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Artifact Storage Guide
2+
3+
Artifacts are stored in Amazon S3, and the **`ArtifactsS3SourceBucket`** parameter determines whether to use **default artifacts** from the central S3 bucket or **custom artifacts** stored in the same directory as this README.md file.
4+
5+
## Choosing the Artifact Source
6+
7+
- If the **`ArtifactsS3SourceBucket`** parameter is set to **Use artifacts from EA's central artifact store**, artifacts will be retrieved from the **central S3 bucket**, which will be used in StartingBlocks CloudFromation stack
8+
9+
- If the **`ArtifactsS3SourceBucket`** parameter is set to **Use my own custom artifacts in the CloudFormation bucket**, make sure to have the artifacts saved in the same directory as this README file to be used by the StartingBlocks CloudFormation stack.
10+
11+

‎docker/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Artifact Storage Guide
2+
3+
Artifacts are stored in Amazon S3, and the **`ArtifactsS3SourceBucket`** parameter determines whether to use **default artifacts** from the central S3 bucket or **custom artifacts** stored in the same directory as this README.md file.
4+
5+
## Choosing the Artifact Source
6+
7+
- If the **`ArtifactsS3SourceBucket`** parameter is set to **Use artifacts from EA's central artifact store**, artifacts will be retrieved from the **central S3 bucket**, which will be used in StartingBlocks CloudFromation stack
8+
9+
- If the **`ArtifactsS3SourceBucket`** parameter is set to **Use my own custom artifacts in the CloudFormation bucket**, make sure to have the artifacts saved in the same directory as this README file to be used by the StartingBlocks CloudFormation stack.
10+

‎docs/sbe-architecture.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Overall Architecture
2+
Below is a high level architecture diagram and inventory of <i>some</i> AWS resources deployed when using these templates. It's important to note that once these resources are deployed, users are immediately responsible for any incurred costs, even if the environment is not being used. Please also note that there are some pre-requisite resources that are included in this diagram for informational purposes. Specifically, public subnets, VPC and private subnets must already exist prior to deploying StartingBlocks. For deployment steps [please click here](../README.md)
3+
4+
5+
![](imgs/StartingBlocks-OSS-diagram.svg)
6+
7+
>[!NOTE]
8+
Currently we do not support deployments of the Ed-Fi Admin App for StartingBlocks v7.1. There is no Admin App currently supported by the Alliance that is compatible with Ed-Fi v7.x. When there is a release of the Admin App for v7.x, we are commited to supporting it in StartingBlocks.
9+
10+
11+
# Lambda Functions
12+
The diagram highlights that there is a suite of AWS Lambda functions used in StartingBlocks. There are Ed-Fi environment management functions, but there are also general utility functions or functions created as custom CloudFormation resources. Below is a complete inventory of Lambda functions deployed with StartingBlocks. There is also more detail specifically on the environment management functions [in our docs folder here.](sbe-functions.md) All Lambda functions deployed by CloudFormation are prefixed by the `EnvLabel` parameter value to make them easy to find in the Lambda console.
13+
14+
## Utility Functions
15+
- <b>DbRestore - </b> Restores template databases on initial StartingBlocks deployments.
16+
- <b>EdFiBeanstalkSNSToSlack - </b> Forwards SNS messages sent from Beanstalk Env and RDS instance to Slack.
17+
- <b>East1Alarm - </b> Creates a Route53 Healthcheck Alarm in us-east-1.
18+
- <b>API-Publisher-getmaxchangeversion - </b> Optionally deployed if publisher is also deployed. Lambda Function to replace the getmaxchangeversion ODS function.
19+
## Custom CloudFormation Resource Functions
20+
- <b>EncryptionKeyGenerator - </b> CloudFormation Custom Resource provider. Creates and stores a base64 encoded 256-bit key.
21+
- <b>SetCloudWatchRetention - </b> CloudFormation Custom Resource provider. Sets retention on CloudWatch Log Groups.
22+
- <b>ODSDerivatives - </b> CloudFormation Custom Resource provider. Adds/removes ODS instance derivative in admin db when ODS instance derivative is created/deleted.
23+
## Management and Reporting Functions
24+
- <b>TenantManagement - </b> Used to manage Tenants in Ed-Fi 7.x environments.
25+
- <b>ODSManagement - </b> Used to manage ODSs in Ed-Fi 7.x environments.
26+
- <b>EdOrgManagement - </b> Used to managed Education Organizations in Ed-Fi 7.x environments.
27+
- <b>SbeMetadata - </b> Optionally Deployed if SBAA admin interface is chosen. Provides ARNs for all Lambda Ed-Fi resource management functions.
28+
- <b>TenantResourceTree - </b> Provides tree structure of resources in a tenant.
29+
- <b>DataFreshnessJson - </b> Provides a JSON output for resource counts and dates per table within a given ODS and Tenant.
30+
- <b>ODSUserPermissions - </b> Grants permissions to users in the ODS managed by database groups.

‎docs/sbe-contribution-guidelines.md‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,10 @@ Any feature requests or questions should be raised using [GitHub Issues on the O
1010
5. Community Feature Addition Pull Requests
1111
6. StartingBlocks Product Team Feature Addition Pull Requests
1212

13-
---
13+
>[!NOTE]
14+
>Security releases, documentation updates, and updates to the Ed-Fi application bundles will be released in special priority as deemed appropriate.
1415
15-
**NOTE:**
16-
Security releases, documentation updates, and updates to the Ed-Fi application bundles will be released in special priority as deemed appropriate.
17-
18-
---
19-
20-
If you'd like to recieve up to date notifications for updates you can change your `watch` settings on the repository.
16+
If you'd like to receive up to date notifications for updates you can change your `watch` settings on the repository.
2117
![](./imgs/watch_screenshot.png).
2218

23-
For most recent release please [visit the release page.](https://github.com/edanalytics/startingblocks_oss/releases/)
19+
For the most recent release please [visit the release page.](https://github.com/edanalytics/startingblocks_oss/releases/)

‎docs/sbe-deployment-steps.md‎

Lines changed: 0 additions & 35 deletions
This file was deleted.

0 commit comments

Comments
 (0)