[RND-581] Deploy Meadowlark with Azure CLI (#259)

Ed-Fi-Exchange-OSS · Jun 22, 2023 · 8c9fb7d · 8c9fb7d
1 parent ed13d94
commit 8c9fb7d
Show file tree

Hide file tree

Showing 3 changed files with 90 additions and 62 deletions.
diff --git a/eng/deploy/azure/.env.example b/eng/deploy/azure/.env.example
@@ -1,2 +1,7 @@
 # The OAUTH_SIGNING_KEY may need quotation marks around the value, unlike other keys.
-OAUTH_SIGNING_KEY="<run `openssl rand -base64 256` to create a key>"
+OAUTH_SIGNING_KEY="<run `openssl rand -base64 256` to create a key>"
+
+# The following settings are values required for the URL, which is formed by {ED_FI_DOMAIN_NAME}.{AZURE_REGION}.azurecontainer.io
+# ED_FI_DOMAIN_NAME must be unique per Azure subscription
+ED_FI_DOMAIN_NAME="meadowlark-ABC"
+AZURE_REGION="southcentralus"
diff --git a/eng/deploy/azure/README.md b/eng/deploy/azure/README.md
@@ -1,21 +1,80 @@
 # Azure Deployment
 
-To deploy to Azure, this can be done through Azure Container Instances (ACI),
-with the [Docker Azure
-Integration](https://docs.docker.com/cloud/aci-integration/), or with [Azure
-CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).
+To deploy to Azure, this can be done thorough Azure Container Instances (ACI) deploying with the [Azure
+CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) or with the [Docker Azure
+Integration](https://docs.docker.com/cloud/aci-integration/).
+
+## Deploy with Azure CLI
+
+For Azure CLI, it's necessary to specify all environment variables in the command line since it is not possible to read a
+.env file. Additionally, it is not possible to add all containers into the same container group, it must be one container per
+group.
+
+```pwsh
+# Login to Azure
+az login
+
+$resourceGroup={resource group name}
+
+# The combination of DNS labels and azure regions must be globally unique.
+$meadowlarkDnsLabel={meadowlark dns}
+$mongoDnsLabel={mongo dns}
+$openSearchDnsLabel={opensearch dns}
+
+# Create the mongo container
+az container create --resource-group $resourceGroup -n ml-mongo `
+    --image edfialliance/meadowlark-mongo:latest `
+    --ports 27017 --dns-name-label $mongoDnsLabel `
+    --command-line "mongod --bind_ip_all --replSet rs0"
+
+# Initialize mongodb replica set
+az container exec --resource-group $resourceGroup -n ml-mongo `
+    --container-name ml-mongo --exec-command 'mongo --eval rs.initiate()'
+
+# Create OpenSearch container
+az container create --resource-group $resourceGroup -n ml-opensearch `
+    --image edfialliance/meadowlark-opensearch:latest `
+    --ports 9200 --dns-name-label $openSearchDnsLabel
+
+# Define variables
+# Replace with signing key
+$signingKey="<run `openssl rand -base64 256` to create a key>"
+$mongoUri='"mongodb://'+$mongoDnsLabel+'.southcentralus.azurecontainer.io:27017/?replicaSet=rs0&directConnection=true"'
+$openSearchUrl="http://${openSearchDnsLabel}.southcentralus.azurecontainer.io:9200"
+$documentStore="@edfi/meadowlark-mongodb-backend"
+$queryHandler="@edfi/meadowlark-opensearch-backend"
+$listenerPlugin="@edfi/meadowlark-opensearch-backend"
+$authorizationPlugin="@edfi/meadowlark-mongodb-backend"
+
+# Create meadowlark container
+az container create --resource-group $resourceGroup -n ml-api `
+    --image edfialliance/meadowlark-ed-fi-api:pre --ports 3000 `
+    --dns-name-label $meadowlarkDnsLabel `
+    --environment-variables OAUTH_SIGNING_KEY=$signingKey `OAUTH_HARD_CODED_CREDENTIALS_ENABLED=true `
+    OWN_OAUTH_CLIENT_ID_FOR_CLIENT_AUTH=meadowlark_verify-only_key_1 `
+    OWN_OAUTH_CLIENT_SECRET_FOR_CLIENT_AUTH=meadowlark_verify-only_secret_1 `
+    OAUTH_SERVER_ENDPOINT_FOR_OWN_TOKEN_REQUEST=http://${meadowlarkDnsLabel}.southcentralus.azurecontainer.io:3000/stg/oauth/token `
+    OAUTH_SERVER_ENDPOINT_FOR_TOKEN_VERIFICATION=http://${meadowlarkDnsLabel}.southcentralus.azurecontainer.io:3000/stg/oauth/verify `
+    OPENSEARCH_USERNAME=admin OPENSEARCH_PASSWORD=admin OPENSEARCH_ENDPOINT=$openSearchUrl OPENSEARCH_REQUEST_TIMEOUT=10000 `
+    DOCUMENT_STORE_PLUGIN=$documentStore QUERY_HANDLER_PLUGIN=$queryHandler LISTENER1_PLUGIN=$listenerPlugin `
+    FASTIFY_RATE_LIMIT=false FASTIFY_PORT=3000 FASTIFY_NUM_THREADS=10 MEADOWLARK_STAGE=stg `
+    LOG_LEVEL=info IS_LOCAL=false AUTHORIZATION_STORE_PLUGIN=$authorizationPlugin `
+    BEGIN_ALLOWED_SCHOOL_YEAR=2022 END_ALLOWED_SCHOOL_YEAR=2034 ALLOW_TYPE_COERCION=true `
+    ALLOW__EXT_PROPERTY=true MONGO_URI=$mongoUri
+```
 
 ## Deploy with Docker Azure Integration
 
-- [Log into Azure from
-  Docker](https://docs.docker.com/cloud/aci-integration/#log-into-azure).
+> **Warning** The Docker Azure Integration will be retired in November 2023.
 
-- [Create an ACI Docker
-  context](https://docs.docker.com/cloud/aci-integration/#create-an-aci-context)
+- [Log into Azure from Docker](https://docs.docker.com/cloud/aci-integration/#log-into-azure).
+
+- [Create an ACI Docker context](https://docs.docker.com/cloud/aci-integration/#create-an-aci-context)
 
 - Browse to `../eng/deploy/azure`
 
-- Create a .env file. Update URLs to match your correct Azure region.
+- Create a .env file. Set the OAUTH_SIGNING_KEY, AZURE_REGION and ED_FI_DOMAIN_NAME. The combination of domain name an azure
+  region must be globally unique.
 
 - Execute the following script:
 
@@ -32,54 +91,18 @@ az container exec --resource-group {resource group name} -n meadowlark `
 
 ```
 
-> **Note** Not all functionality available in a Docker Compose file is available
-> when deploying to ACI. To review the available features, check [the
-> documentation](https://docs.docker.com/cloud/aci-compose-features/) .
+> **Note** Not all functionality available in a Docker Compose file is available when deploying to ACI. To review the
+> available features, check [the documentation](https://docs.docker.com/cloud/aci-compose-features/) .
 
 ### Removing the containers
 
-Given that `docker compose down` is not available. To remove all the containers
-in the group, execute:
+Given that `docker compose down` is not available. To remove all the containers in the group, execute:
 
 ```Shell
 az container delete --resource-group {resource group name} -n meadowlark
 ```
 
-## Deploy with Azure CLI
-
-For Azure CLI, it's necessary to specify all environment variables in the
-command line since it is not possible to read a .env file. Additionally, it is
-not possible to add all containers into the same container group, it must be one
-container per group.
-
-```Shell
-# Login to Azure
-az login
-
-# Create the mongo container
-az container create --resource-group {resource group name} -n ml-mongo `
-    --image edfialliance/meadowlark-mongo:latest `
-    --ports 27017 --dns-name-label mlmongo1 `
-    --command-line "mongod --replSet rs0"
-
-# Initialize mongodb replica set
-az container exec --resource-group {resource group name} -n ml-mongo `
-    --container-name ml-mongo --exec-command 'mongo --eval rs.initiate()'
-
-# Create OpenSearch container
-az container create --resource-group {resource group name} -n ml-opensearch `
-    --image edfialliance/meadowlark-opensearch:latest `
-    --ports 9200 --dns-name-label ml-opensearch
-
-# Create meadowlark container
-az container create --resource-group {resource group name} -n ml-api `
-    --image edfialliance/meadowlark-ed-fi-api:pre --ports 80 `
-    --environment-variables {specify all env variables required}
-```
-
-> **Warning** Not ready for production usage. This example is using a single
-> mongo node with a simulated replica set and bypassing security with a direct
-> connection, also, it's using the OAUTH hardcoded credentials. The current
-> configuration is initializing the mongo replica manually, and this is not
-> saved. Therefore, if the container instance is stopped, it's necessary to
-> reinitialize the replica set.
+> **Warning** Not ready for production usage. This example is using a single mongo node with a simulated replica set and
+> bypassing security with a direct connection, also, it's using the OAUTH hardcoded credentials. The current configuration is
+> initializing the mongo replica manually, and this is not saved. Therefore, if the container instance is stopped, it's
+> necessary to reinitialize the replica set.
diff --git a/eng/deploy/azure/azure-docker-compose.yml b/eng/deploy/azure/azure-docker-compose.yml
@@ -5,7 +5,7 @@ services:
   meadowlark-ed-fi-api:
     image: edfialliance/meadowlark-ed-fi-api:pre
     container_name: ml-api
-    domainname: meadowlark
+    domainname: ${ED_FI_DOMAIN_NAME}
     ports:
       - 3000:3000
     environment:
@@ -14,16 +14,16 @@ services:
       OAUTH_HARD_CODED_CREDENTIALS_ENABLED: ${OAUTH_HARD_CODED_CREDENTIALS_ENABLED:-true}
       OWN_OAUTH_CLIENT_ID_FOR_CLIENT_AUTH: ${OWN_OAUTH_CLIENT_ID_FOR_CLIENT_AUTH:-meadowlark_verify-only_key_1}
       OWN_OAUTH_CLIENT_SECRET_FOR_CLIENT_AUTH: ${OWN_OAUTH_CLIENT_SECRET_FOR_CLIENT_AUTH:-meadowlark_verify-only_secret_1}
-      OAUTH_SERVER_ENDPOINT_FOR_OWN_TOKEN_REQUEST: ${OAUTH_SERVER_ENDPOINT_FOR_OWN_TOKEN_REQUEST:-http://meadowlark.southcentralus.azurecontainer.io:3000/local/oauth/token}
-      OAUTH_SERVER_ENDPOINT_FOR_TOKEN_VERIFICATION: ${OAUTH_SERVER_ENDPOINT_FOR_TOKEN_VERIFICATION:-http://meadowlark.southcentralus.azurecontainer.io:3000/local/oauth/verify}
+      OAUTH_SERVER_ENDPOINT_FOR_OWN_TOKEN_REQUEST: http://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:3000/local/oauth/token
+      OAUTH_SERVER_ENDPOINT_FOR_TOKEN_VERIFICATION: http://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:3000/local/oauth/verify
       OPENSEARCH_USERNAME: ${OPENSEARCH_USERNAME:-admin}
       OPENSEARCH_PASSWORD: ${OPENSEARCH_PASS:-admin}
-      OPENSEARCH_ENDPOINT: ${OPENSEARCH_ENDPOINT:-http://meadowlark.southcentralus.azurecontainer.io:9200}
+      OPENSEARCH_ENDPOINT: http://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:9200
       OPENSEARCH_REQUEST_TIMEOUT: '10000'
       DOCUMENT_STORE_PLUGIN: "${DOCUMENT_STORE_PLUGIN:-@edfi/meadowlark-mongodb-backend}"
       QUERY_HANDLER_PLUGIN: "${QUERY_HANDLER_PLUGIN:-@edfi/meadowlark-opensearch-backend}"
       LISTENER1_PLUGIN: "${LISTENER1_PLUGIN:-@edfi/meadowlark-opensearch-backend}"
-      MONGO_URI: ${MONGODB_URI:-mongodb://meadowlark.southcentralus.azurecontainer.io:27017/?replicaSet=rs0&directConnection=true}
+      MONGO_URI: mongodb://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:27017/?replicaSet=rs0&directConnection=true
       FASTIFY_RATE_LIMIT: ${FASTIFY_RATE_LIMIT:-false}
       FASTIFY_PORT: ${FASTIFY_PORT:-3000}
       FASTIFY_NUM_THREADS: ${FASTIFY_NUM_THREADS:-10}
@@ -37,15 +37,15 @@ services:
       ALLOW__EXT_PROPERTY: ${ALLOW__EXT_PROPERTY:-true}
     restart: unless-stopped
     healthcheck:
-      test: curl -s http://meadowlark.southcentralus.azurecontainer.io:3000/local >/dev/null || exit 1
+      test: curl -s http://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:3000/local >/dev/null || exit 1
       interval: 30s
       timeout: 10s
       retries: 50
 
   mongo1:
     image: edfialliance/meadowlark-mongo:latest
     container_name: ml-mongo1
-    domainname: meadowlark
+    domainname: ${ED_FI_DOMAIN_NAME}
     hostname: ml-mongo1
     ports:
       - 27017:27017
@@ -66,7 +66,7 @@ services:
   opensearch:
     image: edfialliance/meadowlark-opensearch:latest
     container_name: ml-opensearch1
-    domainname: meadowlark
+    domainname: ${ED_FI_DOMAIN_NAME}
     hostname: ml-opensearch1
     environment:
       OPENSEARCH_JAVA_OPTS: -Xms512m -Xmx512m # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
@@ -84,7 +84,7 @@ services:
       - 9200:9200
     restart: unless-stopped
     healthcheck:
-      test: curl -s http://meadowlark.southcentralus.azurecontainer.io:9200/_cat/health >/dev/null || exit 1
+      test: curl -s http://${ED_FI_DOMAIN_NAME}.${AZURE_REGION}.azurecontainer.io:9200/_cat/health >/dev/null || exit 1
       interval: 30s
       timeout: 10s
       retries: 50