diff --git a/PREREQS.md b/PREREQS.md
index 5eea2f8..bfa8e64 100644
--- a/PREREQS.md
+++ b/PREREQS.md
@@ -1,46 +1,18 @@
## Prerequisites
-### AMI Acceptance
+### Create an external Route53 zone that is resolvable
-Before you start, make sure the AMI that is listed in the `backendless_chef.yaml` for your region has been accepted/approved for use in the AWS Marketplace. We are using native Amazon Linux AMIs, the most up to date list is [always here](https://aws.amazon.com/amazon-linux-ami/).
+In order to use one of the "full stack" templates such as `main.yaml` or `marketplace.yaml`, you must have a domain that is hosted on Route53. If you're not ready to take the plunge for your entire domain, you can create a subdomain and route all traffic to that (for example chef.mycompany.com). AWS provides [instructions](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html) and [a video](https://aws.amazon.com/premiumsupport/knowledge-center/create-subdomain-route-53/) on how to do that.
-### For SSL
-1. You must have a DNS Record Name and Zone Name planned for this cluster to match the SSL certificate.
-1. You must have already created and uploaded an SSL cert to AWS. Once you've uploaded the cert, you will need to follow these steps to update the `backendless_chef.yaml` template with the correct values:
- * Update the `SSLCertificateARN` parameter with the `ARN` for the SSL Cert. _For info on how to get the ARN, please follow the [AWS Certificate Manger](https://aws.amazon.com/certificate-manager/) docs._
- * **If you're using AWS Route 53**, fill in the `Route53RecordName` and `Route53HostedZone` parameters for the template
- * **If you're not using AWS Route 53**, create the DNS record in your own nameservers to be a CNAME record that points at the stack's Application Load Balancer (e.g. my-chef-stack-1942464223.us-west-2.elb.amazonaws.com)
-
-### Using with Automate
-
-At this time, this template does not setup Automate as part of it's deployment. However, you can configure it to point to an Automate server that you will setup afterwards, or that is already setup.
-1. You should have a token already generated, follow the [instructions here](https://automate.chef.io/docs/data-collection/) on how to generate an Automate token.
-1. The Chef Automate URL, for more information on how this URL is formatted, [read this](https://automate.chef.io/docs/data-collection/).
+In the parameters, fill in the `Route53HostedZone` parameter to match your Route53 zone, as well as the `AutomateDnsRecordName`, `ChefServerDnsRecordName` and `SupermarketDnsRecordName` values.
-### Package Versions
-It's recommended to have the packages downloaded and hosted locally before proceeding, an S3 bucket, Artifactory/Nexus or YUM/APT repository cache works well for this purpose. Once you've downloaded the correct EL7 packages and have them hosted, adjust the following variables accordingly to point to the proper URLs.
-
-* `ChefServerPackage`, `ChefManagePackage`, `PushJobsPackage`
-
-_Here's an example of setting up an S3 cache:_
+### For SSL
-1. Install `mixlib-install` Ruby Gem from the [mixlib-install repo](https://github.com/chef/mixlib-install) on an EC2 instance that has access to the S3 bucket being used.
-1. Install the `aws` cli tool on the same instance from [aws cli](https://aws.amazon.com/cli/).
-1. Run the following commands:
- ```
- mixlib-install download chef-server --platform el --platform-version 7.5 --architecture x86_64
- # Starting download https://packages.chef.io/files/stable/chef-server/12.17.33/el/7/chef-server-core-12.17.33-1.el7.x86_64.rpm
- # Download saved to /Users/myname/chef-server-core-12.17.33-1.el7.x86_64.rpm
+You must create or upload at least one SSL certficate to AWS Certificate Manager (ACM). If you wish, ACM can provide free SSL certificates for you and automatically manages renewals of those certificates. You may create 3 separate certificates, or a single wildcard certificate (ex: `*.chef.mycompany.com`) that is used in all 3 places. Since your Route53 zone is now working, use the `DNS Validation` option as it is far faster and more convenient.
- aws s3 cp /Users/myname/chef-server-core-12.17.33-1.el7.x86_64.rpm s3://mybucket/package-cache/ --acl public-read
- # upload: ./chef-server-core-12.17.33-1.el7.x86_64.rpm to s3://mybucket/package-cache/chef-server-core-12.17.33-1.el7.x86_64.rpm
+In the parameters, fill in the `ChefSSLCertificateARN`, `AutomateSSLCertificateARN` and `SupermarketSSLCertificateARN` values with the ARNs for the certificates. The ARNs are always viewable in the ACM console, an example ACM ARN looks like: `arn:aws:acm:us-west-2:446539779517:certificate/82d30a13-b420-4f43-80de-9e7872f70b96`
- aws s3 presign s3://mybucket/package-cache/chef-server-core-12.17.33-1.el7.x86_64.rpm | cut -d '?' -f 1
- # https://mybucket.s3.amazonaws.com/package-cache/chef-server-core-12.17.33-1.el7.x86_64.rpm
- ```
-1. Set the last output (e.g. `https://mybucket.s3.amazonaws.com/package-cache/chef-server-core-12.17.33-1.el7.x86_64.rpm` as shown above) as your `ChefServerPackage` value in `backendless_chef.yaml`.
-
## Network
@@ -48,14 +20,15 @@ You must already have a VPC setup properly before continuing setting up the stac
* Have enough IP's available to assign to nodes
* Be split up into 3 subnets, each in different Availability Zones (AZ's)
+* Provide the VPC ID and associated subnets to the `VPC` and `ServerSubnets` parameters
## Security
-* You should already have created/uploaded an SSH key to AWS and have the ARN available.
-* You should already have an Admin SG created for inbound SSH connections. The Security Group ID should be provided for the `InboundAdminSecurityGroupId` parameter in `backendless_chef.yaml`, otherwise you won't have any ssh connectivity to your cluster.
+* You should already have created/uploaded an SSH key to AWS. Provide the keypair name to the `KeyName` parameter
+* If you're using the `main.yaml` stack, you must also create a security group in the referenced VPC to define your administrative access. Provide sg ID to the `InboundAdminSecurityGroupId` parameter
-## Amazon ElasticSearch and Service Linked Role (SLR)
+## Amazon Elasticsearch and Service Linked Role (SLR)
-Amazon ElasticSearch requires a specific SLR to be created prior to running this CloudFormation template, specifically one called `AWSServiceRoleForAmazonElasticsearchService`. This role cannot be created programmatically as it is created automatically when setting up a VPC access domain in the AWS console. For more information on this [please see this doc from AWS](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/slr-es.html), at this time, even though the doc points to a way to create this manually via the CLI, it still only works via console setup, however AWS adds new features all the time, and by the time you do this, it may work programmatically, please follow their documentation. Once you've created the VPC access domain for AWS ElasticSearch, you can then delete this domain, the role will still be there and you should be able to continue.
+Amazon Elasticsearch requires a specific SLR to be created prior to running this CloudFormation template, specifically one called `AWSServiceRoleForAmazonElasticsearchService`. This role cannot be created programmatically as it is created automatically when setting up a VPC access domain in the AWS console. For more information on this [please see this doc from AWS](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/slr-es.html), at this time, even though the doc points to a way to create this manually via the CLI, it still only works via console setup, however AWS adds new features all the time, and by the time you do this, it may work programmatically, please follow their documentation. Once you've created the VPC access domain for AWS ElasticSearch, you can then delete this domain, the role will still be there and you should be able to continue.
_Note: You will need to do this for each region you plan on setting up Chef in_
diff --git a/README.md b/README.md
index 697cc8d..5f553e1 100644
--- a/README.md
+++ b/README.md
@@ -33,26 +33,35 @@ Before you fire it up, there are a few things you should make sure you have prep
* Please review the [Prerequisites doc before proceeding](./PREREQS.md)
-## Fire up the Chef Server stack
+## Browser launch instructions (quickstart)
You can launch this stack with the push of a button:
-<p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2Fmain.yaml&stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p>
-However, the most repeatable and least error-prone way to launch this stack is to use the `aws` command-line. First copy file `stack_parameters.json.example` to `stack_parameters.json`, make the necessary changes, then run this command:
+| Edition | Version | Est Monthly cost | Launch button |
+| ------- | ------- | ---------------- | ------------- |
+| High Performance | 5.0.2 | [$1500*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-E9251374-D80C-45B7-BED7-C9C9778B1D6C) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.2%2Fmain.yaml&stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
+| Lowest Cost | 5.0.2 | [$320*](https://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=files/calc-165638db8e331664846c0d6654d743377bd3eac6&v=ver20190604sQ) | <p><a href="https://console.aws.amazon.com/cloudformation/home#/stacks/new?templateURL=https:%2F%2Fs3.amazonaws.com%2Faws-native-chef-server%2F5.0.2%2Fmarketplace.yaml&stackName=my-chef-stack" target="_blank"><img src="https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png" alt="Launch Stack" /></a></p> |
+
+_*NOTE: Estimated costs are just that, your mileage will vary. Use reserved instances to greatly reduce costs for long-running clusters._
+
+## CLI Launch instructions
+However, the most repeatable and least error-prone way to launch this stack is to use the `aws` command-line. First copy file `stack_parameters_main.json.example` to `stack_parameters_main.json`, make the necessary changes, then run this command:
```bash
MYBUCKET=aws-native-chef-server
MYID=mycompany
+VERSION=5.0.2
+EDITION=main # main or marketplace
# Configure the automate_stack_parameters.json and then launch the cloudformation stack:
# If you're using your own bucket, uncomment the next line:
-# aws s3 sync . s3://$MYBUCKET/ --exclude "*" --include "*.yaml" --include "files/*" && \
-aws cloudformation validate-template --template-url https://s3.amazonaws.com/$MYBUCKET/main.yaml && \
+# aws s3 sync . s3://$MYBUCKET/$VERSION/ --exclude "*" --include "*.yaml" --include "files/*" && \
+aws cloudformation validate-template --template-url https://s3.amazonaws.com/$MYBUCKET/$VERSION/$EDITION.yaml && \
aws cloudformation create-stack \
--stack-name ${MYID}-chef-stack \
- --template-url https://s3.amazonaws.com/$MYBUCKET/main.yaml \
+ --template-url https://s3.amazonaws.com/$MYBUCKET/$VERSION/$EDITION.yaml \
--capabilities CAPABILITY_IAM \
--on-failure DO_NOTHING \
- --parameters file://stack_parameters.json
+ --parameters file://stack_parameters_$EDITION.json
```
## Updating the stack
@@ -62,14 +71,16 @@ If you've made changes to the template content or parameters and you wish to upd
```bash
MYBUCKET=aws-native-chef-server
MYID=mycompany
+VERSION=5.0.2
+EDITION=main # main or marketplace
# If you're using your own bucket, uncomment the next line:
-# aws s3 sync . s3://$MYBUCKET/ --exclude "*" --include "*.yaml" --include "files/*" && \
-aws cloudformation validate-template --template-url https://s3.amazonaws.com/$MYBUCKET/main.yaml && \
+# aws s3 sync . s3://$MYBUCKET/$VERSION/ --exclude "*" --include "*.yaml" --include "files/*" && \
+aws cloudformation validate-template --template-url https://s3.amazonaws.com/$MYBUCKET/$VERSION/$EDITION.yaml && \
aws cloudformation update-stack \
--stack-name ${MYID}-chef-stack \
- --template-url https://s3.amazonaws.com/$MYBUCKET/main.yaml \
+ --template-url https://s3.amazonaws.com/$MYBUCKET/$VERSION/$EDITION.yaml \
--capabilities CAPABILITY_IAM \
- --parameters file://stack_parameters.json
+ --parameters file://stack_parameters_$EDITION.json
```
Note: For production instances it is recommended to use the CloudFormation console so that you can get a report of all changes before executing them. Particularly pay attention to any resources that are being replaced.
@@ -79,10 +90,8 @@ Note: For production instances it is recommended to use the CloudFormation conso
If you're using a bastion host and need to SSH from the outside:
```bash
-ssh -o ProxyCommand="ssh -W %h:%p -q user@bastion" -l user <chef server private ip>
+ssh -o ProxyCommand="ssh -W %h:%p -q centos@bastion" -l centos <chef server private ip>
```
-Where "user" is `ec2-user` on the RHEL AMI, and `centos` on the CentOS AMI
-
otherwise just SSH directly to the public IPs of the chef servers
## Upgrading the Chef Server
diff --git a/automate.yaml b/automate.yaml
index c924202..de7cd43 100644
--- a/automate.yaml
+++ b/automate.yaml
@@ -1,12 +1,8 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Automate 2 (auto-recovering standalone) v4.1.0
+Description: AWS Native Automate 2 (auto-recovering standalone) v5.0.2
Parameters:
# Required Parameters
- AutomationBucket:
- Default: 'aws-native-chef-server'
- Type: String
- Description: AWS S3 bucket name that contains all of the cloudformation templates and scripts for this stack
VPC:
Description: Choose VPC to use
Type: AWS::EC2::VPC::Id
@@ -52,10 +48,12 @@ Parameters:
Default: 1000
###############################################################################
# Security Settings
+ AutomateAdminPassword:
+ Description: Provide a password for the Chef Automate admin user
+ Type: String
AutomateDataCollectorToken:
Description: The token used for ingesting Data Collector (Chef) events
Type: String
- Default: 93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506
LoadBalancerScheme:
Description: Network Scheme for the ELB
Type: String
@@ -77,18 +75,11 @@ Parameters:
Route53HostedZone:
Type: String
Default: ''
- Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com.) for auto-creating a DNS record. Must end in a dot. (Leave blank to disable)
+ Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com) for auto-creating a DNS record. Must NOT end in a dot. (Leave blank to disable)
Route53RecordName:
Type: String
Default: automate
Description: Supply a DNS record name that will be prepended to the Route 53 Hosted Zone
- AMIFlavor:
- Description: Choose the AMI Flavor, the High-Perf CentOS image or the stock and paid official RHEL image
- Type: String
- Default: 'centos'
- AllowedValues:
- - 'centos'
- - 'rhel'
SystemLogsGroup:
Description: The logical name of the Cloudwatch Logs log group for systemn logs
Type: String
@@ -105,8 +96,8 @@ Metadata:
Conditions:
CreateRoute53Record:
!Not [!Equals [ !Ref Route53HostedZone, '' ] ]
- RHELAmiFlavor:
- !Equals [ !Ref AMIFlavor, 'rhel' ]
+ UsePublicIp:
+ !Equals [ !Ref LoadBalancerScheme, 'internet-facing']
Resources:
AutomateInstanceProfile:
@@ -175,7 +166,7 @@ Resources:
BlockDeviceMappings:
- DeviceName: /dev/sda1
Ebs:
- VolumeSize: 100
+ VolumeSize: 20
VolumeType: gp2
DeleteOnTermination: true
- DeviceName: /dev/xvdb
@@ -203,40 +194,19 @@ Resources:
"Fn::Base64": !Sub |
#!/bin/bash -ex
trap '/opt/aws/bin/cfn-signal -e 1 --region ${AWS::Region} --stack ${AWS::StackName} --resource AutomateServer' ERR
- # Update the system first
- yum upgrade -y
- # Run before script, make sure to install cfn-init if not installed
- # attempt to fetch the script first as a signed request (private bucket), then unsigned, then fallback
- aws s3 cp s3://${AutomationBucket}/files/before.sh /root/before.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/before.sh -o /root/before.sh || \
- curl -s https://aws-native-chef-server.s3.amazonaws.com/files/before.sh -o /root/before.sh
- bash -ex /root/before.sh
# Execute AWS::CloudFormation::Init
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource AutomateServer --region ${AWS::Region}
/opt/aws/bin/cfn-signal -e 0 --region ${AWS::Region} --stack ${AWS::StackName} --resource AutomateServer
CreationPolicy:
ResourceSignal:
Count: 1
- Timeout: PT10M
+ Timeout: PT20M
Metadata:
AWS::CloudFormation::Init:
configSets:
- default: [awslogs, automate_setup, nightly_snapshots]
+ default: [awslogs, automate_setup, nightly_snapshots, final_stage]
automate_setup:
- packages:
- yum:
- lvm2: []
- xfsprogs: []
- perl: []
- perl-DateTime: []
- perl-Sys-Syslog: []
- perl-LWP-Protocol-https: []
- perl-Digest-SHA: []
- zip: []
- unzip: []
files:
- /root/chef-automate.zip:
- source: https://packages.chef.io/files/current/automate/latest/chef-automate_linux_amd64.zip
/etc/automate-config.toml:
content: !Sub
- |
@@ -330,62 +300,33 @@ Resources:
[auth_n.v1.sys.service]
a1_data_collector_token = "${AutomateDataCollectorToken}"
- { FQDN: !If [CreateRoute53Record, !Ref AutomateDNSRecord, !GetAtt ChefALB.DNSName] }
- /etc/sysctl.d/00-chef.conf:
- content: |
- vm.swappiness=10
- vm.max_map_count=262144
- vm.dirty_ratio=20
- vm.dirty_background_ratio=30
- vm.dirty_expire_centisecs=30000
- /etc/security/limits.d/20-nproc.conf:
- content: |
- * soft nproc 65536
- * hard nproc 65536
- * soft nofile 1048576
- * hard nofile 1048576
commands:
02_before_script:
command: |
- sysctl -p /etc/sysctl.d/00-chef.conf
mkdir -p /hab
pvs | grep nvme1n1 || pvcreate /dev/nvme1n1
vgs | grep data-vg || vgcreate data-vg /dev/nvme1n1
lvs | grep data-lv || (lvcreate -n data-lv -l 95%VG data-vg ; mkfs.xfs /dev/data-vg/data-lv)
grep hab /proc/mounts || mount /dev/data-vg/data-lv /hab
+ grep 'data--vg-data--lv' /etc/fstab || echo '/dev/mapper/data--vg-data--lv /hab xfs defaults 0 0' >> /etc/fstab
03_configure_automate:
- command: |
- cd /root
- unzip chef-automate.zip -d /usr/local/sbin
- /usr/local/sbin/chef-automate deploy /etc/automate-config.toml --accept-terms-and-mlsa --skip-preflight
- 04_after_script:
command: !Sub |
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/after.sh /root/after.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/after.sh -o /root/after.sh
- bash -ex /root/after.sh
+ cd /root
+ /var/cache/marketplace/chef-automate deploy /etc/automate-config.toml --accept-terms-and-mlsa --skip-preflight --airgap-bundle /var/cache/marketplace/chef-automate.bundle
+ sleep 10
+ /bin/chef-automate iam admin-access restore '${AutomateAdminPassword}'
05_configure_cloudwatch_monitoring:
command: |
- mkdir /opt/cloudwatch_monitoring
- cd /opt/cloudwatch_monitoring
- curl -s -O http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip
- unzip CloudWatchMonitoringScripts-1.2.1.zip
- rm -f CloudWatchMonitoringScripts-1.2.1.zip
crontab -l | { cat; echo "*/5 * * * * /opt/cloudwatch_monitoring/aws-scripts-mon/mon-put-instance-data.pl --mem-util --disk-space-util --disk-path=/ --from-cron"; } | crontab -
+
crontab -l | { cat; echo "*/5 * * * * /opt/cloudwatch_monitoring/aws-scripts-mon/mon-put-instance-data.pl --disk-space-util --disk-path=/hab --from-cron"; } | crontab -
nightly_snapshots:
- files:
- /usr/local/bin/ebs-snapshot.sh:
- source: https://raw.githubusercontent.com/CaseyLabs/aws-ec2-ebs-automatic-snapshot-bash/master/ebs-snapshot.sh
- mode: '000755'
commands:
create_nightly_backup_cronjob:
command: |
crontab -l | { cat; echo "55 23 * * * root /usr/local/bin/ebs-snapshot.sh"; } | crontab -
awslogs:
files:
- /root/awslogs-agent-setup.py:
- source: 'https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py'
- mode: '000755'
/etc/awslogs/awscli.conf:
content: !Sub |
[default]
@@ -440,7 +381,16 @@ Resources:
01_setup:
command: !Sub |
mkdir -p /var/lib/awslogs
- python /root/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
+ python /var/cache/marketplace/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
+ final_stage:
+ commands:
+ 09_cleanup:
+ command: |
+ # clean up the chef package cache, to maximize available disk space after install
+ if [ -d /var/cache/marketplace ]; then
+ rm -f /var/cache/marketplace/*.rpm
+ rm -f /var/cache/marketplace/*.aib
+ fi
# EC2 System AutoRecovery
# https://aws.amazon.com/blogs/aws/new-auto-recovery-for-amazon-ec2/
@@ -489,9 +439,9 @@ Resources:
Type: AWS::Route53::RecordSet
Condition: CreateRoute53Record
Properties:
- HostedZoneName: !Ref Route53HostedZone
+ HostedZoneName: !Sub ${Route53HostedZone}.
+ Name: !Sub ${Route53RecordName}.${Route53HostedZone}.
Comment: !Sub Created by Cloudformation ${AWS::StackName}
- Name: !Sub ${Route53RecordName}.${Route53HostedZone}
Type: CNAME
TTL: '900'
ResourceRecords:
@@ -536,7 +486,7 @@ Resources:
- Name: MountPath
Value: '/'
- Name: Filesystem
- Value: !If [RHELAmiFlavor, "/dev/nvme0n1p2", "/dev/nvme0n1p1"]
+ Value: "/dev/nvme0n1p1"
DiskSpaceUtilizationDataDisk:
Type: AWS::CloudWatch::Alarm
@@ -603,4 +553,4 @@ Outputs:
Name: !Sub "${AWS::StackName}-DNSName"
ServerIP:
Description: The IP Address of the Automate Server
- Value: !GetAtt AutomateServer.PublicIp
+ Value: !If [UsePublicIp, !GetAtt AutomateServer.PublicIp, !GetAtt AutomateServer.PrivateIp]
diff --git a/chef_elasticsearch.yaml b/chef_elasticsearch.yaml
index 1ac6a3e..9b7be64 100644
--- a/chef_elasticsearch.yaml
+++ b/chef_elasticsearch.yaml
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: Chef Server Elasticsearch v4.1.0
+Description: Chef Server Elasticsearch 5.0.2
Parameters:
# Required Parameters
@@ -27,15 +27,16 @@ Parameters:
'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
ElasticSearchVersion:
- Description: Version of ElasticSearch to use. Chef 12.16 supports v2.3 or 5.x. (5.5 recommended for new installs)
+ Description: Version of ElasticSearch to use
Type: String
- Default: '5.5'
+ Default: '5.6'
AllowedValues:
- '2.3'
- '5.3'
- '5.5'
+ - '5.6'
ElasticSearchShardCount:
- Description: Number of ElasticSearch hosts to provision at launch (3 recommended)
+ Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
Default: 3
Type: Number
###############################################################################
@@ -52,6 +53,10 @@ Parameters:
Description: SNS topic
Type: String
+Conditions:
+ 2ZoneES:
+ !Equals [!Ref ElasticSearchShardCount, 2]
+
Resources:
# ElasticSearch
#########################################################################################
@@ -72,7 +77,7 @@ Resources:
ElasticsearchVersion: !Ref ElasticSearchVersion
ElasticsearchClusterConfig:
InstanceCount: !Sub ${ElasticSearchShardCount}
- ZoneAwarenessEnabled: false
+ ZoneAwarenessEnabled: !If [2ZoneES, true, false]
InstanceType: !Ref ElasticSearchInstanceType
DedicatedMasterEnabled: false
SnapshotOptions:
@@ -86,8 +91,11 @@ Resources:
Action: "es:*"
Resource: !Sub "arn:aws:es:${AWS::Region}:${AWS::AccountId}:*"
VPCOptions:
- SubnetIds:
- - !Select [ 0, !Ref ChefServerSubnets ]
+ SubnetIds: !If [
+ 2ZoneES,
+ [ !Select [ 0, !Ref ChefServerSubnets ], !Select [ 1, !Ref ChefServerSubnets ]],
+ [ !Select [ 0, !Ref ChefServerSubnets ] ]
+ ]
SecurityGroupIds:
- !Ref ESSecurityGroup
AdvancedOptions:
@@ -155,7 +163,7 @@ Resources:
Period: 60
# Reports when any node in the cluster falls below the threshold.
Statistic: Minimum
- Threshold: 5000
+ Threshold: 2000
Dimensions:
- Name: DomainName
Value: !Ref ElasticsearchDomain
diff --git a/chef_rds.yaml b/chef_rds.yaml
index 9e1990c..ec774ba 100644
--- a/chef_rds.yaml
+++ b/chef_rds.yaml
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: Chef Server PostgreSQL v4.1.0
+Description: Chef Server PostgreSQL v5.0.2
Parameters:
# Required Parameters
@@ -22,12 +22,18 @@ Parameters:
###############################################################################
# Performance Settings
DBInstanceClass:
- Description: EC2 Instance type for RDS DBs (EBS Optimized instances recommended)
- Default: 'db.m4.large'
+ Description: EC2 Instance type for RDS DBs (M5 class recommended for performance)
+ Default: 'db.m5.large'
Type: String
- AllowedValues: [db.t2.medium, db.t2.large, db.t2.xlarge, db.t2.2xlarge,
+ AllowedValues: [db.t3.small, db.t3.medium, db.t3.large, db.t3.xlarge, db.t3.2xlarge,
db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge, db.m4.16xlarge,
- db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge]
+ db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge,
+ db.m5.large, db.m5.xlarge, db.m5.2xlarge, db.m5.4xlarge, db.m5.12xlarge, db.m5.24xlarge]
+ DBStorageType:
+ Description: Select gp2 or io1 DB storage. io1 is much higher performing, must select minimum 100GB storage and an IOPS value
+ Default: 'gp2'
+ Type: String
+ AllowedValues: [gp2, io1]
DBAllocatedStorage:
Description: Storage size allocated for the database
Default: '100'
@@ -42,11 +48,23 @@ Parameters:
Description: Supply a security group for your chef frontends
Type: AWS::EC2::SecurityGroup::Id
###############################################################################
+ # Backup settings
+ BackupDaystoKeep:
+ Description: Number of days of automatic backups to keep
+ Default: 7
+ Type: Number
+ MinValue: 0
+ MaxValue: 35
+ ###############################################################################
# Input from Parent Stack
AlertNotificationTopic:
Description: SNS topic
Type: String
+Conditions:
+ UsePiopsStorage:
+ !Equals [ !Ref DBStorageType, 'io1' ]
+
Resources:
DBSecurityGroup:
Type: AWS::EC2::SecurityGroup
@@ -71,17 +89,17 @@ Resources:
Properties:
DBName: chef
AllocatedStorage: !Ref DBAllocatedStorage
- Iops: !Ref DBIops
+ Iops: !If [UsePiopsStorage, !Ref DBIops, 0]
MasterUsername: chefadmin
MasterUserPassword: !Ref DBPassword
DBInstanceClass: !Ref DBInstanceClass
- StorageType: io1
+ StorageType: !Ref DBStorageType
MultiAZ: true
Engine: postgres
- EngineVersion: 9.6.8
+ EngineVersion: 9.6.12
AllowMajorVersionUpgrade: true
AutoMinorVersionUpgrade: true
- BackupRetentionPeriod: '35'
+ BackupRetentionPeriod: !Ref BackupDaystoKeep
DBSubnetGroupName: !Ref DBSubnetGroup
VPCSecurityGroups:
- !Ref DBSecurityGroup
diff --git a/chef_server_ha.yaml b/chef_server_ha.yaml
index 5af2892..705e021 100644
--- a/chef_server_ha.yaml
+++ b/chef_server_ha.yaml
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Chef Server (auto-scaling HA) v4.1.0
+Description: AWS Native Chef Server (auto-scaling HA) v5.0.2
Parameters:
# Required Parameters
@@ -60,12 +60,18 @@ Parameters:
Description: Enter how many licenses you have purchased
Type: String
DBInstanceClass:
- Description: EC2 Instance type for RDS DBs (EBS Optimized instances recommended)
- Default: 'db.m4.large'
+ Description: EC2 Instance type for RDS DBs (M5 class recommended for performance)
+ Default: 'db.m5.large'
Type: String
- AllowedValues: [db.t2.medium, db.t2.large, db.t2.xlarge, db.t2.2xlarge,
+ AllowedValues: [db.t3.small, db.t3.medium, db.t3.large, db.t3.xlarge, db.t3.2xlarge,
db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge, db.m4.16xlarge,
- db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge]
+ db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge,
+ db.m5.large, db.m5.xlarge, db.m5.2xlarge, db.m5.4xlarge, db.m5.12xlarge, db.m5.24xlarge]
+ DBStorageType:
+ Description: Select gp2 or io1 DB storage. io1 is much higher performing, must select minimum 100GB storage and an IOPS value
+ Default: 'gp2'
+ Type: String
+ AllowedValues: [gp2, io1]
DBAllocatedStorage:
Description: Storage size allocated for the database
Default: '100'
@@ -74,6 +80,12 @@ Parameters:
Description: IOPS allocated to the storage (storage size * 10)
Default: '1000'
Type: String
+ DBBackupDaystoKeep:
+ Description: Number of days of automatic backups to keep
+ Default: 7
+ Type: Number
+ MinValue: 0
+ MaxValue: 35
ElasticSearchInstanceType:
Description: The Instance type to use for ElasticSearch instances (Note, must have ephemeral storage, the instance type affects the total amount of elasticsearch storage. i3 strongly recommended)
Type: String
@@ -84,36 +96,23 @@ Parameters:
'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
ElasticSearchVersion:
- Description: Version of ElasticSearch to use. Chef 12.16 supports v2.3 or 5.x. (5.5 recommended for new installs)
+ Description: Version of ElasticSearch to use
Type: String
- Default: '5.5'
+ Default: '5.6'
AllowedValues:
- '2.3'
- '5.3'
- '5.5'
+ - '5.6'
ElasticSearchShardCount:
Description: Number of ElasticSearch hosts to provision at launch (3 recommended)
Default: 3
Type: Number
ElasticSearchReplicaCount:
- Description: Replication factor for ElasticSearch sha vrds (how many extra copies to keep)
+ Description: Replication factor for ElasticSearch shards (how many extra copies to keep)
Default: 2
Type: Number
###############################################################################
- # Package Versions & Locations
- ChefServerPackage:
- Description: The URL to the chef server EL7 (chef-server-core) package which will be downloaded
- Type: String
- Default: 'https://packages.chef.io/files/stable/chef-server/12.17.33/el/7/chef-server-core-12.17.33-1.el7.x86_64.rpm'
- ChefManagePackage:
- Description: The URL to the chef-manage EL7 package which will be downloaded
- Type: String
- Default: 'https://packages.chef.io/files/stable/chef-manage/2.5.16/el/7/chef-manage-2.5.16-1.el7.x86_64.rpm'
- PushJobsPackage:
- Description: The URL to the push jobs server package which will be downloaded
- Type: String
- Default: 'https://packages.chef.io/files/stable/opscode-push-jobs-server/2.2.8/el/7/opscode-push-jobs-server-2.2.8-1.el7.x86_64.rpm'
- ###############################################################################
# Security Settings
LoadBalancerScheme:
Description: Network Scheme for the ELB
@@ -166,27 +165,27 @@ Parameters:
Default: ''
Type: String
Description: The S3 location of a custom chef-server configuration snippet that will be injected into the chef-server.rb (optional)
+ SupermarketUrl:
+ Description: The URL of the Private Supermarket Server, for oc-id authentication. Leave blank to disable.
+ Type: String
+ Default: ''
Route53HostedZone:
Type: String
Default: ''
- Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com.) for auto-creating a DNS record. Must end in a dot. (Leave blank to disable)
+ Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com) for auto-creating a DNS record. Must NOT end in a dot. (Leave blank to disable)
Route53RecordName:
Type: String
Default: 'chef'
Description: Supply a DNS record name that will be prepended to the Route 53 Hosted Zone
- AMIFlavor:
- Description: Choose the AMI Flavor, the High-Perf CentOS image or the stock and paid official RHEL image
- Type: String
- Default: 'centos'
- AllowedValues:
- - 'centos'
- - 'rhel'
SystemLogsGroup:
Description: The logical name of the Cloudwatch Logs log group for systemn logs
Type: String
AppLogsGroup:
Description: The logical name of the Cloudwatch Logs log group for systemn logs
Type: String
+ TemplateVersion:
+ Type: String
+ Default: "5.0.2"
Metadata:
cfn-lint:
@@ -197,8 +196,6 @@ Metadata:
Conditions:
CreateRoute53Record:
!Not [!Equals [ !Ref Route53HostedZone, '' ] ]
- RHELAmiFlavor:
- !Equals [ !Ref AMIFlavor, 'rhel' ]
Resources:
# Frontend Autoscale Groups
@@ -426,19 +423,8 @@ Resources:
UserData:
"Fn::Base64": !Sub |
#!/bin/bash -ex
- trap "{
- /opt/aws/bin/cfn-signal -e 1 --region ${AWS::Region} --stack ${AWS::StackName} --resource ServerLaunchConfig
- # Uncomment this next line before going to production
- # aws autoscaling set-instance-health --health-status Unhealthy --region ${AWS::Region} --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id)
- }" ERR
- # Update the system first
- yum upgrade -y
- # Run before script, make sure to install cfn-init if not installed
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/before.sh /root/before.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/before.sh -o /root/before.sh || \
- curl -s https://aws-native-chef-server.s3.amazonaws.com/files/before.sh -o /root/before.sh
- bash -ex /root/before.sh
+ # Uncomment this next line before going to production, note: It can cause instances to cycle rapidly, watch Cloudwatch Logs for errors
+ # trap "aws autoscaling set-instance-health --health-status Unhealthy --region ${AWS::Region} --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id)" ERR
# Execute AWS::CloudFormation::Init
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource ServerLaunchConfig --region ${AWS::Region}
# All is well so signal success and let CF know wait function is complete
@@ -446,12 +432,9 @@ Resources:
Metadata:
AWS::CloudFormation::Init:
configSets:
- default: [awslogs, es_proxy_setup, chef_server_setup, filebeat]
+ default: [awslogs, es_proxy_setup, chef_server_setup, filebeat, final_stage]
es_proxy_setup:
files:
- /usr/local/bin/aws-signing-proxy:
- source: https://github.com/chef-customers/aws-signing-proxy/releases/download/v0.5.0/aws-signing-proxy
- mode: '000755'
/etc/systemd/system/aws-signing-proxy.service:
content: |
[Unit]
@@ -474,21 +457,9 @@ Resources:
stdout-log: true
commands:
01_start_es_proxy:
- command: systemctl daemon-reload && systemctl start aws-signing-proxy
+ command: |
+ systemctl daemon-reload && systemctl start aws-signing-proxy
chef_server_setup:
- packages:
- rpm:
- chef-server-core: !Sub ${ChefServerPackage}
- chef-manage: !Sub ${ChefManagePackage}
- opscode-push-jobs-server: !Sub ${PushJobsPackage}
- yum:
- perl: []
- perl-DateTime: []
- perl-Sys-Syslog: []
- perl-LWP-Protocol-https: []
- perl-Digest-SHA: []
- zip: []
- unzip: []
files:
/etc/opscode-push-jobs-server/opscode-push-jobs-server.rb:
content: !Sub |
@@ -538,34 +509,27 @@ Resources:
rabbitmq['queue_length_monitor_enabled'] = false
opscode_expander['enable'] = false
dark_launch['actions'] = false
- oc_id['applications'] ||= {}
- oc_id['applications']['supermarket'] = {
- 'redirect_uri' => 'https://supermarket.${Route53HostedZone}/auth/chef_oauth2/callback'
- }
+ unless '${SupermarketUrl}'.empty?
+ oc_id['applications'] ||= {}
+ oc_id['applications']['supermarket'] = {
+ 'redirect_uri' => '${SupermarketUrl}/auth/chef_oauth2/callback'
+ }
+ end
default_orgname '${ChefDefaultOrgName}'.empty? ? null : '${ChefDefaultOrgName}'
# special handling for Chef Automate URL and Token which might be empty strings
unless '${ChefAutomateServerUrl}'.empty? || '${ChefAutomateToken}'.empty?
- data_collector['root_url'] = '${ChefAutomateServerUrl}'
+ data_collector['root_url'] = '${ChefAutomateServerUrl}/data-collector/v0/'
data_collector['token'] = '${ChefAutomateToken}'
+ profiles['root_url'] = '${ChefAutomateServerUrl}'
end
- { DNSNAME: !If [CreateRoute53Record, !Sub "${Route53RecordName}.${Route53HostedZone}", !GetAtt ChefALB.DNSName] }
- /etc/sysctl.d/00-chef.conf:
- content: |
- vm.swappiness=10
- vm.max_map_count=262144
- vm.dirty_ratio=20
- vm.dirty_background_ratio=30
- vm.dirty_expire_centisecs=30000
- /etc/security/limits.d/20-nproc.conf:
- content: |
- * soft nproc 65536
- * hard nproc 65536
- * soft nofile 1048576
- * hard nofile 1048576
commands:
02_before_script:
command: !Sub |
- sysctl -p /etc/sysctl.d/00-chef.conf
+ # If we're using a marketplace, install locally cached packages
+ pushd /var/cache/marketplace
+ rpm -ivh chef-server-core.rpm chef-manage.rpm push-jobs-server.rpm
+ popd
if [ -n '${ChefServerCustomConfig}' ]; then
aws s3 cp ${ChefServerCustomConfig} /etc/opscode/chef-server-custom.rb
echo "# Appended from chef-server-custom.rb" >> /etc/opscode/chef-server.rb
@@ -577,29 +541,13 @@ Resources:
export BUCKET=${ChefSecretsBucket}
export AWS_REGION=${AWS::Region}
export WAITHANDLE="${WaitHandle}"
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/main.sh /root/main.sh || \
- aws s3 cp --no-sign-request s3://${AutomationBucket}/files/main.sh /root/main.sh
- bash -x /root/main.sh
- 04_after_script:
- command: !Sub |
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/after.sh /root/after.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/after.sh -o /root/after.sh
- bash -ex /root/after.sh
+
+ bash -x /var/cache/marketplace/main.sh
05_configure_cloudwatch_monitoring:
command: |
- mkdir /opt/cloudwatch_monitoring
- cd /opt/cloudwatch_monitoring
- curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O
- unzip CloudWatchMonitoringScripts-1.2.1.zip
- rm -f CloudWatchMonitoringScripts-1.2.1.zip
crontab -l | { cat; echo "*/5 * * * * /opt/cloudwatch_monitoring/aws-scripts-mon/mon-put-instance-data.pl --auto-scaling --mem-util --disk-space-util --disk-path=/ --from-cron"; } | crontab -
awslogs:
files:
- /root/awslogs-agent-setup.py:
- source: 'https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py'
- mode: '000755'
/etc/awslogs/awscli.conf:
content: !Sub |
[default]
@@ -706,11 +654,8 @@ Resources:
01_setup:
command: !Sub |
mkdir -p /var/lib/awslogs
- python /root/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
+ python /var/cache/marketplace/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
filebeat:
- packages:
- rpm:
- filebeat: https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.6.6-x86_64.rpm
files:
/etc/filebeat/filebeat.yml:
content: |
@@ -732,7 +677,18 @@ Resources:
group: root
commands:
01_systemd:
- command: systemctl daemon-reload && systemctl start filebeat.service
+ command: |
+ rpm -ivh /var/cache/marketplace/filebeat-5.6.6-x86_64.rpm
+ systemctl daemon-reload && systemctl start filebeat.service
+ final_stage:
+ commands:
+ 09_cleanup:
+ command: |
+ # clean up the chef package cache, to maximize available disk space after install
+ if [ -d /var/cache/marketplace ]; then
+ rm -f /var/cache/marketplace/*.rpm
+ rm -f /var/cache/marketplace/*.aib
+ fi
WaitCondition:
Type: AWS::CloudFormation::WaitCondition
@@ -749,7 +705,7 @@ Resources:
PostgreSQLStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/chef_rds.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/chef_rds.yaml
Parameters:
VPC: !Sub ${VPC}
ChefServerSubnets: !Join [ ",", !Ref ChefServerSubnets ]
@@ -757,8 +713,10 @@ Resources:
ContactEmail: !Ref ContactEmail
ContactDept: !Ref ContactDept
DBInstanceClass: !Ref DBInstanceClass
+ DBStorageType: !Ref DBStorageType
DBAllocatedStorage: !Ref DBAllocatedStorage
DBIops: !Ref DBIops
+ BackupDaystoKeep: !Ref DBBackupDaystoKeep
FrontendSecurityGroupId: !Ref FrontendSecurityGroupId
AlertNotificationTopic: !Ref AlertNotificationTopic
@@ -767,7 +725,7 @@ Resources:
ElasticsearchStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/chef_elasticsearch.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/chef_elasticsearch.yaml
Parameters:
VPC: !Sub ${VPC}
ChefServerSubnets: !Join [ ",", !Ref ChefServerSubnets ]
@@ -786,9 +744,9 @@ Resources:
Type: AWS::Route53::RecordSet
Condition: CreateRoute53Record
Properties:
- HostedZoneName: !Ref Route53HostedZone
+ HostedZoneName: !Sub ${Route53HostedZone}.
+ Name: !Sub ${Route53RecordName}.${Route53HostedZone}.
Comment: !Sub Created by Cloudformation ${AWS::StackName}
- Name: !Sub ${Route53RecordName}.${Route53HostedZone}
Type: CNAME
TTL: '900'
ResourceRecords:
@@ -800,7 +758,7 @@ Resources:
OpsDashboardStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/chef_server_ops_dashboard.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/chef_server_ops_dashboard.yaml
Parameters:
BootstrapAutoScaleGroup: !Ref BootstrapAutoScaleGroup
FrontendAutoScaleGroup: !Ref FrontendAutoScaleGroup
@@ -831,7 +789,7 @@ Resources:
- Name: MountPath
Value: '/'
- Name: Filesystem
- Value: !If [RHELAmiFlavor, "/dev/nvme0n1p2", "/dev/nvme0n1p1"]
+ Value: '/dev/nvme0n1p1'
BootstrapDiskSpaceUtilizationNvme:
Type: AWS::CloudWatch::Alarm
@@ -853,7 +811,7 @@ Resources:
- Name: MountPath
Value: '/'
- Name: Filesystem
- Value: !If [RHELAmiFlavor, "/dev/nvme0n1p2", "/dev/nvme0n1p1"]
+ Value: '/dev/nvme0n1p1'
MemoryUtilization:
Type: AWS::CloudWatch::Alarm
diff --git a/chef_server_ops_dashboard.yaml b/chef_server_ops_dashboard.yaml
index 84731c1..35dc58d 100644
--- a/chef_server_ops_dashboard.yaml
+++ b/chef_server_ops_dashboard.yaml
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native OpsDashboard for Chef Server v4.1.0
+Description: AWS Native OpsDashboard for Chef Server v5.0.2
Parameters:
# Required Parameters
diff --git a/files/after.sh b/files/after.sh
deleted file mode 100644
index 60c817c..0000000
--- a/files/after.sh
+++ /dev/null
@@ -1,5 +0,0 @@
-#!/bin/bash
-
-# place your customizations here, for things that run after main.sh
-
-exit 0
diff --git a/generate_os_mappings.sh b/generate_os_mappings.sh
index 37b5a48..4e24a71 100755
--- a/generate_os_mappings.sh
+++ b/generate_os_mappings.sh
@@ -1,23 +1,17 @@
#!/bin/bash
-# get the latest Name value from:
-# RHEL: aws ec2 describe-images --owners 309956199498 --filters "Name=name,Values=RHEL-7.6*" --query "Images[*].Name" --output text
-# CentOS highperf: `aws ec2 describe-images --owners 446539779517 --filters "Name=name,Values=chef-highperf-centos7*" --query "Images[*].Name" | sort`
-
-RHEL_RELEASE='RHEL-7.6_HVM_GA-20190128-x86_64-0-Hourly2-GP2'
-CENTOS_RELEASE='chef-highperf-centos7-201902131539'
+CENTOS_RELEASE='aws-native-chef-server-5.0.*'
+IMAGE_OWNERID="406084061336"
printf "Mappings:\n AMI:\n"
regions=$(aws ec2 describe-regions --query "Regions[].RegionName" --output text)
for region in $regions; do
- rhel_ami=$(aws --region $region ec2 describe-images \
- --filters "Name=name,Values=${RHEL_RELEASE}" \
- --query "Images[0].ImageId" --output "text")
-
centos_ami=$(aws --region $region ec2 describe-images \
+ --owners $IMAGE_OWNERID \
--filters "Name=name,Values=${CENTOS_RELEASE}" \
- --query "Images[0].ImageId" --output "text")
+ --query "Images[0].ImageId" \
+ --output "text")
- printf " $region:\n rhel: $rhel_ami\n centos: $centos_ami\n"
+ printf " $region:\n centos: $centos_ami\n"
done
diff --git a/main.yaml b/main.yaml
index 3b55e46..44457bd 100644
--- a/main.yaml
+++ b/main.yaml
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Chef Stack v4.1.0
+Description: AWS Native Chef Stack v5.0.2
Parameters:
# Required Parameters
@@ -27,6 +27,21 @@ Parameters:
Type: String
###############################################################################
# Automate Stack Settings
+ ChefAutomateAdminPassword:
+ Description: Provide a password for the Chef Automate admin user
+ NoEcho: true
+ Type: String
+ MinLength: 8
+ MaxLength: 64
+ ConstraintDescription: Must be at least 8 characters
+ ChefAutomateToken:
+ Description: The secret token used to communicate with the Chef Automate Server, for forwarding Visibility data. Leave blank to disable.
+ Type: String
+ Default: 93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506
+ AllowedPattern: ^[a-z0-9]*$
+ MinLength: 64
+ MaxLength: 64
+ ConstraintDescription: Must be 64 alphanumeric characters
AutomateSSLCertificateARN:
Description: SSL Certficate ARN for SSL Certficate (must begin with arn:aws:iam::)
Type: String
@@ -45,6 +60,8 @@ Parameters:
Description: Amount of storage space to allocate on a dedicated data volume (must be between 500 and 16000)
Type: Number
Default: 1000
+ MinValue: 500
+ MaxValue: 16000
###############################################################################
# Chef Server Stack Settings
ChefSSLCertificateARN:
@@ -55,33 +72,47 @@ Parameters:
Type: String
Default: 'chef'
ChefInstanceType:
- Description: EC2 Instance type for Chef Server Frontends (high-CPU recommended)
+ Description: EC2 Instance type for Chef Server Frontends (C5 recommended for performance)
Default: c5.large
Type: String
AllowedValues: [t3.small, t3.medium, t3.large, t3.xlarge, t3.2xlarge,
m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.12xlarge, m5.24xlarge,
c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.18xlarge]
ChefDBInstanceClass:
- Description: EC2 Instance type for RDS DBs (EBS Optimized instances recommended)
- Default: 'db.m4.large'
+ Description: EC2 Instance type for RDS DBs (M5 class recommended for performance)
+ Default: 'db.m5.large'
Type: String
- AllowedValues: [db.t2.medium, db.t2.large, db.t2.xlarge, db.t2.2xlarge,
+ AllowedValues: [db.t3.medium, db.t3.large, db.t3.xlarge, db.t3.2xlarge,
db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge, db.m4.16xlarge,
- db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge]
+ db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge,
+ db.m5.large, db.m5.xlarge, db.m5.2xlarge, db.m5.4xlarge, db.m5.12xlarge, db.m5.24xlarge]
+ ChefDBStorageType:
+ Description: Select gp2 or io1 DB storage. io1 is much higher performing, must select minimum 100GB storage and an IOPS value
+ Default: 'io1'
+ Type: String
+ AllowedValues: [gp2, io1]
ChefDBAllocatedStorage:
- Description: Storage size allocated for the database
+ Description: Storage size allocated for the database (gp2 minimum 20, maximum 16000) (io1 minimum 100, maximum 32768)
Default: '100'
Type: String
ChefDBIops:
- Description: IOPS allocated to the storage (storage size * 10)
+ Description: IOPS reserved for DB storage, only applicable if ChefDBStorageType is io1 (ChefDBAllocatedStorage * 10)
Default: '1000'
Type: String
+ ChefDBBackupDaystoKeep:
+ Description: Number of days of automatic backups to keep
+ Default: 7
+ Type: Number
+ MinValue: 0
+ MaxValue: 35
ChefDBPassword:
- Description: Enter DB Password
+ Description: Create a strong RDS admin password
NoEcho: true
Type: String
+ MinLength: 8
+ MaxLength: 64
ChefElasticSearchInstanceType:
- Description: The Instance type to use for ElasticSearch instances (Note, must have ephemeral storage, the instance type affects the total amount of elasticsearch storage. i3 strongly recommended)
+ Description: The Instance type to use for ElasticSearch instances (i3 recommended for performance)
Type: String
Default: 'i3.large.elasticsearch'
AllowedValues: [
@@ -90,21 +121,19 @@ Parameters:
'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
ChefElasticSearchVersion:
- Description: Version of ElasticSearch to use. Chef 12.16 supports v2.3 or 5.x. (5.5 recommended for new installs)
+ Description: Version of ElasticSearch to use. (5.6 recommended for new installs)
Type: String
- Default: '5.5'
+ Default: '5.6'
AllowedValues:
- - '2.3'
- '5.3'
- '5.5'
+ - '5.6'
ChefElasticSearchShardCount:
- Description: Number of ElasticSearch hosts to provision at launch (3 recommended)
+ Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
Default: 3
Type: Number
- ChefElasticSearchReplicaCount:
- Description: Replication factor for ElasticSearch sha vrds (how many extra copies to keep)
- Default: 2
- Type: Number
+ MinValue: 2
+ MaxValue: 3
###############################################################################
# Automate Stack Settings
SupermarketSSLCertificateARN:
@@ -124,7 +153,7 @@ Parameters:
SupermarketDataVolumeSize:
Description: Amount of storage space to allocate for the Supermarket single OS volume
Type: Number
- Default: 100
+ Default: 20
###############################################################################
# Security Settings (these apply to all stacks)
LoadBalancerScheme:
@@ -159,21 +188,14 @@ Parameters:
Default: ''
###############################################################################
# Other Settings
- ChefAutomateToken:
- Description: The secret token used to communicate with the Chef Automate Server, for forwarding Visibility data. Leave blank to disable.
+ TemplateVersion:
+ Description: The version of this template to use (do not change this unless directed by support)
Type: String
- Default: 93a49a4f2482c64126f7b6015e6b0f30284287ee4054ff8807fb63d9cbd1c506
+ Default: "5.0.2"
Route53HostedZone:
Type: String
Default: ''
- Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com.) for auto-creating a DNS record. Must end in a dot. (Leave blank to disable)
- AMIFlavor:
- Description: Choose the AMI Flavor, the High-Perf CentOS image or the stock and paid official RHEL image
- Type: String
- Default: 'centos'
- AllowedValues:
- - 'centos'
- - 'rhel'
+ Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com) for auto-creating a DNS record. Do not append a dot at the end. (Leave blank to disable)
LogsRetentionInDays:
Description: Specifies the number of days you want to retain cloudwatch log events.
Type: Number
@@ -185,51 +207,54 @@ Metadata:
config:
ignore_checks:
- W2507
- # AWS::CloudFormation::Interface:
- # ParameterGroups:
- # - Label:
- # default: "Required Parameters"
- # Parameters:
- # - VPC
- # - ChefServerSubnets
- # - SSLCertificateARN
- # - InboundAdminSecurityGroupId
- # - KeyName
- # - DBPassword
- # - ContactEmail
- # - ContactDept
- # - Label:
- # default: "Performance Settings"
- # Parameters:
- # - InstanceType
- # - MaxFrontendInstances
- # - MinFrontendInstances
- # - LicenseCount
- # - DBInstanceClass
- # - DBAllocatedStorage
- # - DBIops
- # - ElasticSearchInstanceType
- # - ElasticSearchVersion
- # - ElasticSearchShardCount
- # - ElasticSearchReplicaCount
- # - Label:
- # default: "Package Versions & Locations"
- # Parameters:
- # - ChefServerPackage
- # - ChefManagePackage
- # - PushJobsPackage
- # - BeforeScriptLocation
- # - AfterScriptLocation
- # - Label:
- # default: "Security Settings"
- # Parameters:
- # - LoadBalancerScheme
- # - LoadBalancerSecurityGroupId
- # - FrontendSecurityGroupId
- # - ChefServerAssociatePublicIpAddress
- # - DisableSignup
- # - ChefServerIamRole
- # - ChefSecretsBucket
+ AWS::CloudFormation::Interface:
+ ParameterGroups:
+ - Label:
+ default: "Required Parameters"
+ Parameters:
+ - VPC
+ - ServerSubnets
+ - KeyName
+ - InboundAdminSecurityGroupId
+ - ContactEmail
+ - ContactDept
+ - Route53HostedZone
+ - ChefAutomateAdminPassword
+ - AutomateSSLCertificateARN
+ - ChefSSLCertificateARN
+ - SupermarketSSLCertificateARN
+ - ChefDBPassword
+ - Label:
+ default: "Chef Automate 2 Settings"
+ Parameters:
+ - ChefAutomateToken
+ - AutomateDnsRecordName
+ - AutomateInstanceType
+ - AutomateDataVolumeSize
+ - Label:
+ default: "Chef Server Settings"
+ Parameters:
+ - ChefServerDnsRecordName
+ - ChefInstanceType
+ - ChefDBInstanceClass
+ - ChefDBStorageType
+ - ChefDBAllocatedStorage
+ - ChefDBIops
+ - ChefDBBackupDaystoKeep
+ - ChefElasticSearchInstanceType
+ - ChefElasticSearchVersion
+ - ChefElasticSearchShardCount
+ - Label:
+ default: "Supermarket Settings"
+ Parameters:
+ - SupermarketDnsRecordName
+ - SupermarketInstanceType
+ - SupermarketDataVolumeSize
+ - Label:
+ default: "Security Settings"
+ Parameters:
+ - LoadBalancerScheme
+ - AssociatePublicIpAddress
Conditions:
CreateChefIamRole:
@@ -240,62 +265,16 @@ Conditions:
!Equals [ !Ref LoadBalancerSecurityGroupId, '' ]
CreateFrontendSecurityGroup:
!Equals [ !Ref FrontendSecurityGroupId, '' ]
- CreateRoute53Record:
- !Not [!Equals [ !Ref Route53HostedZone, '' ] ]
- RHELAmiFlavor:
- !Equals [ !Ref AMIFlavor, 'rhel' ]
-# Periodically regenerate this list using the 'generate_os_mappings.sh' script
+# STOP: Do not replace the AMI list unless you 100% know what you're doing
+# and are fully prepared to lose all support. This template now depends
+# on AMIs that are pre-baked with all needed packages and scripts.
Mappings:
AMI:
- eu-north-1:
- rhel: ami-66f67f18
- centos: ami-00d82e36d4cf13239
- ap-south-1:
- rhel: ami-0963937a03c01ecd4
- centos: ami-0d121b70d4f2a7c07
- eu-west-3:
- rhel: ami-0c4224e392ec4e440
- centos: ami-0cb3a832f7fa08045
- eu-west-2:
- rhel: ami-0188c0c5eddd2d032
- centos: ami-0dd3bbb85610696bb
- eu-west-1:
- rhel: ami-0202869bdd0fc8c75
- centos: ami-0355255a4ed05d5af
- ap-northeast-2:
- rhel: ami-041b16ca28f036753
- centos: ami-0c3e242e39ef33638
- ap-northeast-1:
- rhel: ami-00b95502a4d51a07e
- centos: ami-02c58f1040d612a45
- sa-east-1:
- rhel: ami-05c1c16cac05a7c0b
- centos: ami-0c7108774ddaedbaf
- ca-central-1:
- rhel: ami-06ca3c0058d0275b3
- centos: ami-0dfd14de667c4e8c9
- ap-southeast-1:
- rhel: ami-055c55112e25b1f1f
- centos: ami-0d4af852545da36f7
- ap-southeast-2:
- rhel: ami-036b423b657376f5b
- centos: ami-0fa6aa803dc7f6191
- eu-central-1:
- rhel: ami-09de4a4c670389e4b
- centos: ami-0ba5a0a4a8e5d70c1
us-east-1:
- rhel: ami-000db10762d0c4c05
- centos: ami-062a84bcfcb87a1ab
- us-east-2:
- rhel: ami-094720ddca649952f
- centos: ami-05e444eb3b5bb499f
- us-west-1:
- rhel: ami-04642fc8fca1e8e67
- centos: ami-0719ffe132cd3bc6c
+ centos: ami-08d306e6de29d5cc1
us-west-2:
- rhel: ami-0a7e1ebfee7a4570e
- centos: ami-05178c5d62c39cb82
+ centos: ami-06fbb1e0ba7c99cfd
Resources:
ChefRole:
@@ -451,18 +430,15 @@ Resources:
AutomateStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/automate.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/automate.yaml
Parameters:
- AutomationBucket: !Ref AutomationBucket
VPC: !Ref VPC
LoadBalancerSubnets: !Join [ ",", !Ref ServerSubnets ]
AutomateSubnet: !Select [ "0", !Ref ServerSubnets ]
SSLCertificateARN: !Ref AutomateSSLCertificateARN
+ AutomateAdminPassword: !Ref ChefAutomateAdminPassword
InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
- ImageId: !If
- - RHELAmiFlavor
- - !FindInMap [AMI, !Ref "AWS::Region", rhel]
- - !FindInMap [AMI, !Ref "AWS::Region", centos]
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
KeyName: !Ref KeyName
ContactEmail: !Ref ContactEmail
ContactDept: !Ref ContactDept
@@ -474,9 +450,8 @@ Resources:
LoadBalancerSecurityGroupId: !If [CreateLoadBalancerSecurityGroup, !Ref LoadBalancerSecurityGroup, !Ref LoadBalancerSecurityGroupId]
FrontendSecurityGroupId: !If [CreateFrontendSecurityGroup, !Ref FrontendSecurityGroup, !Ref FrontendSecurityGroupId]
AutomateIamRole: !If [CreateChefIamRole, !Ref ChefRole, !Ref ExistingIamRole]
- Route53HostedZone: !If [CreateRoute53Record, !Ref Route53HostedZone, '']
+ Route53HostedZone: !Ref Route53HostedZone
Route53RecordName: !Ref AutomateDnsRecordName
- AMIFlavor: !Ref AMIFlavor
SystemLogsGroup: !Ref SystemLogs
AppLogsGroup: !Ref AutomateAppLogs
@@ -485,17 +460,14 @@ Resources:
ChefServerStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/chef_server_ha.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/chef_server_ha.yaml
Parameters:
AutomationBucket: !Ref AutomationBucket
VPC: !Ref VPC
ChefServerSubnets: !Join [ ",", !Ref ServerSubnets ]
SSLCertificateARN: !Ref ChefSSLCertificateARN
InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
- ImageId: !If
- - RHELAmiFlavor
- - !FindInMap [AMI, !Ref "AWS::Region", rhel]
- - !FindInMap [AMI, !Ref "AWS::Region", centos]
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
KeyName: !Ref KeyName
DBPassword: !Ref ChefDBPassword
ContactEmail: !Ref ContactEmail
@@ -503,12 +475,13 @@ Resources:
AlertNotificationTopic: !Ref AlertNotificationTopic
InstanceType: !Ref ChefInstanceType
DBInstanceClass: !Ref ChefDBInstanceClass
+ DBStorageType: !Ref ChefDBStorageType
DBAllocatedStorage: !Ref ChefDBAllocatedStorage
DBIops: !Ref ChefDBIops
+ DBBackupDaystoKeep: !Ref ChefDBBackupDaystoKeep
ElasticSearchInstanceType: !Ref ChefElasticSearchInstanceType
ElasticSearchVersion: !Ref ChefElasticSearchVersion
ElasticSearchShardCount: !Ref ChefElasticSearchShardCount
- ElasticSearchReplicaCount: !Ref ChefElasticSearchReplicaCount
LoadBalancerScheme: !Ref LoadBalancerScheme
LoadBalancerSecurityGroupId: !If [CreateLoadBalancerSecurityGroup, !Ref LoadBalancerSecurityGroup, !Ref LoadBalancerSecurityGroupId]
FrontendSecurityGroupId: !If [CreateFrontendSecurityGroup, !Ref FrontendSecurityGroup, !Ref FrontendSecurityGroupId]
@@ -517,29 +490,26 @@ Resources:
ChefSecretsBucket: !If [CreateChefSecretsBucket, !Ref ChefBucket, !Ref ExistingSecretsBucket]
ChefAutomateServerUrl: !Sub "https://${AutomateStack.Outputs.DNSName}"
ChefAutomateToken: !Ref ChefAutomateToken
- Route53HostedZone: !If [CreateRoute53Record, !Ref Route53HostedZone, '']
+ SupermarketUrl: !Sub "https://${SupermarketDnsRecordName}.${Route53HostedZone}"
+ Route53HostedZone: !Ref Route53HostedZone
Route53RecordName: !Ref ChefServerDnsRecordName
- AMIFlavor: !Ref AMIFlavor
SystemLogsGroup: !Ref SystemLogs
AppLogsGroup: !Ref ChefAppLogs
+ TemplateVersion: !Ref TemplateVersion
# Supermarket Stack
#########################################################################################
SupermarketStack:
Type: AWS::CloudFormation::Stack
Properties:
- TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/supermarket.yaml
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/supermarket.yaml
Parameters:
- AutomationBucket: !Ref AutomationBucket
VPC: !Ref VPC
LoadBalancerSubnets: !Join [ ",", !Ref ServerSubnets ]
SupermarketSubnet: !Select [ "0", !Ref ServerSubnets ]
SSLCertificateARN: !Ref SupermarketSSLCertificateARN
InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
- ImageId: !If
- - RHELAmiFlavor
- - !FindInMap [AMI, !Ref "AWS::Region", rhel]
- - !FindInMap [AMI, !Ref "AWS::Region", centos]
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
KeyName: !Ref KeyName
ContactEmail: !Ref ContactEmail
ContactDept: !Ref ContactDept
@@ -553,9 +523,8 @@ Resources:
SupermarketIamRole: !If [CreateChefIamRole, !Ref ChefRole, !Ref ExistingIamRole]
ChefSecretsBucket: !If [CreateChefSecretsBucket, !Ref ChefBucket, !Ref ExistingSecretsBucket]
ChefStackName: !GetAtt ChefServerStack.Outputs.StackName
- Route53HostedZone: !If [CreateRoute53Record, !Ref Route53HostedZone, '']
+ Route53HostedZone: !Ref Route53HostedZone
Route53RecordName: !Ref SupermarketDnsRecordName
- AMIFlavor: !Ref AMIFlavor
SystemLogsGroup: !Ref SystemLogs
AppLogsGroup: !Ref SupermarketAppLogs
@@ -575,9 +544,3 @@ Outputs:
Value: !Sub "https://${SupermarketStack.Outputs.DNSName}"
Export:
Name: !Sub "${AWS::StackName}-SupermarketURL"
- AutomateAdminPassword:
- Description: The password for the Chef Automate admin user
- Value: !If
- - RHELAmiFlavor
- - !Sub "To retrieve, run: ssh ec2-user@${AutomateStack.Outputs.ServerIP} sudo cat /root/automate-credentials.toml"
- - !Sub "To retrieve, run: ssh centos@${AutomateStack.Outputs.ServerIP} sudo cat /root/automate-credentials.toml"
diff --git a/marketplace.yaml b/marketplace.yaml
new file mode 100644
index 0000000..a584edf
--- /dev/null
+++ b/marketplace.yaml
@@ -0,0 +1,543 @@
+AWSTemplateFormatVersion: '2010-09-09'
+Description: AWS Native Chef Stack for Marketplace v5.0.2
+
+Parameters:
+ # Required Parameters
+ AutomationBucket:
+ Default: 'aws-native-chef-server'
+ Type: String
+ Description: AWS S3 bucket name that contains all of the cloudformation templates and scripts for this stack
+ VPC:
+ Description: Choose VPC to use
+ Type: AWS::EC2::VPC::Id
+ ServerSubnets:
+ Description: Provide a list of Subnet IDs for the Chef Servers (must be within the specified VPC)
+ Type: List<AWS::EC2::Subnet::Id>
+ KeyName:
+ Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
+ Type: AWS::EC2::KeyPair::KeyName
+ InboundClientCidr:
+ Description: Your internal network range in CIDR notation, from where client traffic will originate
+ AllowedPattern: '((\d{1,3})\.){3}\d{1,3}/\d{1,2}'
+ ConstraintDescription: Must be in CIDR notation (e.g. 10.0.0.0/8 or 0.0.0.0/0)
+ Type: String
+ InboundAdminCidr:
+ Description: Your internal administrative network source range in CIDR notation
+ AllowedPattern: '((\d{1,3})\.){3}\d{1,3}/\d{1,2}'
+ ConstraintDescription: Must be in CIDR notation (e.g. 10.0.0.0/8 or 0.0.0.0/0)
+ Type: String
+ ContactEmail:
+ Description: Contact email for Cloudwatch notifications and instance tagging
+ Type: String
+ ContactDept:
+ Description: Contact department for resource tagging purposes
+ Type: String
+ Route53HostedZone:
+ Type: String
+ Default: ''
+ Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com) for auto-creating a DNS record. Do not append a dot at the end. (Leave blank to disable)
+ ###############################################################################
+ # Automate Stack Settings
+ AutomateSSLCertificateARN:
+ Description: ARN for SSL Certficate, pre-create this in AWS Certificate Manager
+ Type: String
+ AllowedPattern: "arn:aws:acm:.*"
+ ChefAutomateAdminPassword:
+ Description: Provide a password for the Chef Automate admin user
+ NoEcho: true
+ Type: String
+ MinLength: 8
+ MaxLength: 64
+ ConstraintDescription: Must be at least 8 characters
+ ChefAutomateToken:
+ Description: The secret token used to communicate with the Chef Automate Server, for forwarding Visibility data.
+ Type: String
+ AllowedPattern: ^[a-z0-9]*$
+ MinLength: 64
+ MaxLength: 64
+ ConstraintDescription: Must be 64 alphanumeric characters
+ AutomateDnsRecordName:
+ Description: The DNS A-record name to automatically create in the Route53 zone (if enabled). Do not include the domain name.
+ Type: String
+ Default: 'automate'
+ AutomateInstanceType:
+ Description: EC2 Instance type for Automate server (M5 class recommended)
+ Default: t3.medium
+ Type: String
+ AllowedValues: [t3.medium, t3.large, t3.xlarge, t3.2xlarge,
+ m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.12xlarge, m5.24xlarge,
+ c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.18xlarge]
+ AutomateDataVolumeSize:
+ Description: Amount of storage space in GB to allocate on a dedicated data volume (must be between 500 and 16000)
+ Type: Number
+ Default: 500
+ MinValue: 500
+ MaxValue: 16000
+ ###############################################################################
+ # Chef Server Stack Settings
+ ChefSSLCertificateARN:
+ Description: ARN for SSL Certficate, pre-create this in AWS Certificate Manager
+ Type: String
+ AllowedPattern: "arn:aws:acm:.*"
+ ChefServerDnsRecordName:
+ Description: The DNS A-record name to automatically create in the Route53 zone (if enabled). Do not include the domain name.
+ Type: String
+ Default: 'chef'
+ ChefInstanceType:
+ Description: EC2 Instance type for Chef Server Frontends (C5 recommended for performance)
+ Default: t3.small
+ Type: String
+ AllowedValues: [t3.small, t3.medium, t3.large, t3.xlarge, t3.2xlarge,
+ m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.12xlarge, m5.24xlarge,
+ c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.18xlarge]
+ ChefDBInstanceClass:
+ Description: EC2 Instance type for RDS DBs (M5 class recommended for performance)
+ Default: 'db.t3.small'
+ Type: String
+ AllowedValues: [db.t3.small, db.t3.medium, db.t3.large, db.t3.xlarge, db.t3.2xlarge,
+ db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge, db.m4.16xlarge,
+ db.r4.large, db.r4.xlarge, db.r4.2xlarge, db.r4.4xlarge, db.r4.8xlarge, db.r4.16xlarge,
+ db.m5.large, db.m5.xlarge, db.m5.2xlarge, db.m5.4xlarge, db.m5.12xlarge, db.m5.24xlarge]
+ ChefDBStorageType:
+ Description: Select gp2 or io1 DB storage. io1 is much higher performing, must select minimum 100GB storage and an IOPS value
+ Default: 'gp2'
+ Type: String
+ AllowedValues: [gp2, io1]
+ ChefDBAllocatedStorage:
+ Description: Storage size allocated for the database (gp2 minimum 20, maximum 16000) (io1 minimum 100, maximum 32768)
+ Default: '20'
+ Type: String
+ ChefDBIops:
+ Description: IOPS reserved for DB storage, only applicable if ChefDBStorageType is io1 (ChefDBAllocatedStorage * 10)
+ Default: '1000'
+ Type: String
+ ChefDBBackupDaystoKeep:
+ Description: Number of days of automatic DB backups to keep
+ Default: 5
+ Type: Number
+ MinValue: 0
+ MaxValue: 35
+ ChefDBPassword:
+ Description: Create a strong RDS admin password
+ NoEcho: true
+ Type: String
+ MinLength: 8
+ MaxLength: 64
+ ChefElasticSearchInstanceType:
+ Description: The Instance type to use for ElasticSearch instances (i3 recommended for performance)
+ Type: String
+ Default: 'm3.medium.elasticsearch'
+ AllowedValues: [
+ 'i3.large.elasticsearch', 'i3.xlarge.elasticsearch', 'i3.2xlarge.elasticsearch', 'i3.4xlarge.elasticsearch', 'i3.8xlarge.elasticsearch', 'i3.16xlarge.elasticsearch',
+ 'i2.xlarge.elasticsearch', 'i2.2xlarge.elasticsearch',
+ 'm3.medium.elasticsearch', 'm3.large.elasticsearch', 'm3.xlarge.elasticsearch', 'm3.medium.elasticsearch',
+ 'r3.large.elasticsearch', 'r3.xlarge.elasticsearch', 'r3.2xlarge.elasticsearch', 'r3.4xlarge.elasticsearch', 'r3.8xlarge.elasticsearch' ]
+ ChefElasticSearchVersion:
+ Description: Version of ElasticSearch to use. (5.6 recommended for new installs)
+ Type: String
+ Default: '5.6'
+ AllowedValues:
+ - '5.6'
+ ChefElasticSearchShardCount:
+ Description: Number of ElasticSearch hosts to provision at launch (3 recommended, 2 provides HA)
+ Default: 2
+ Type: Number
+ MinValue: 2
+ MaxValue: 3
+ ###############################################################################
+ # Automate Stack Settings
+ SupermarketSSLCertificateARN:
+ Description: ARN for SSL Certficate, pre-create this in AWS Certificate Manager
+ Type: String
+ AllowedPattern: "arn:aws:acm:.*"
+ SupermarketDnsRecordName:
+ Description: The DNS A-record name to automatically create in the Route53 zone (if enabled). Do not include the domain name.
+ Type: String
+ Default: 'supermarket'
+ SupermarketInstanceType:
+ Description: EC2 Instance type for Supermarket server (M5 class recommended)
+ Default: t3.small
+ Type: String
+ AllowedValues: [t3.small, t3.medium, t3.large, t3.xlarge, t3.2xlarge,
+ m5.large, m5.xlarge, m5.2xlarge, m5.4xlarge, m5.12xlarge, m5.24xlarge,
+ c5.large, c5.xlarge, c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.18xlarge]
+ SupermarketDataVolumeSize:
+ Description: Amount of storage space in GB to allocate for the Supermarket OS+data volume
+ Type: Number
+ Default: 20
+###############################################################################
+# Security Settings (these apply to all stacks)
+ LoadBalancerScheme:
+ Description: Network Scheme for the ELB
+ Type: String
+ Default: internet-facing
+ AllowedValues:
+ - 'internet-facing'
+ - 'internal'
+ AssociatePublicIpAddress:
+ Description: Assign public IP addresses to the Chef Servers or not
+ Type: String
+ Default: true
+ AllowedValues:
+ - true
+ - false
+ ###############################################################################
+ # Other Settings
+ TemplateVersion:
+ Description: The version of this template to use (do not change this unless directed by support)
+ Type: String
+ Default: "5.0.2"
+ LogsRetentionInDays:
+ Description: Specifies the number of days you want to retain cloudwatch log events.
+ Type: Number
+ Default: 90
+ AllowedValues: [1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]
+
+Conditions:
+ 2ZoneES:
+ !Equals [!Ref ChefElasticSearchShardCount, 2]
+
+Metadata:
+ cfn-lint:
+ config:
+ ignore_checks:
+ - W2507
+ AWS::CloudFormation::Interface:
+ ParameterGroups:
+ - Label:
+ default: "Required Parameters"
+ Parameters:
+ - VPC
+ - ServerSubnets
+ - KeyName
+ - InboundClientCidr
+ - InboundAdminCidr
+ - ContactEmail
+ - ContactDept
+ - Route53HostedZone
+ - ChefAutomateAdminPassword
+ - AutomateSSLCertificateARN
+ - ChefSSLCertificateARN
+ - SupermarketSSLCertificateARN
+ - ChefDBPassword
+ - Label:
+ default: "Chef Automate 2 Settings"
+ Parameters:
+ - ChefAutomateToken
+ - AutomateDnsRecordName
+ - AutomateInstanceType
+ - AutomateDataVolumeSize
+ - Label:
+ default: "Chef Server Settings"
+ Parameters:
+ - ChefServerDnsRecordName
+ - ChefInstanceType
+ - ChefDBInstanceClass
+ - ChefDBStorageType
+ - ChefDBAllocatedStorage
+ - ChefDBIops
+ - ChefDBBackupDaystoKeep
+ - ChefElasticSearchInstanceType
+ - ChefElasticSearchVersion
+ - ChefElasticSearchShardCount
+ - Label:
+ default: "Supermarket Settings"
+ Parameters:
+ - SupermarketDnsRecordName
+ - SupermarketInstanceType
+ - SupermarketDataVolumeSize
+ - Label:
+ default: "Security Settings"
+ Parameters:
+ - LoadBalancerScheme
+ - AssociatePublicIpAddress
+
+# STOP: Do not replace the AMI list unless you 100% know what you're doing
+# and are fully prepared to lose all support. This template now depends
+# on AMIs that are pre-baked with all needed packages and scripts.
+Mappings:
+ AMI:
+ us-east-1:
+ centos: ami-08d306e6de29d5cc1
+ us-west-2:
+ centos: ami-06fbb1e0ba7c99cfd
+
+Resources:
+ ChefRole:
+ Type: AWS::IAM::Role
+ Properties:
+ AssumeRolePolicyDocument:
+ Statement:
+ - Action:
+ - sts:AssumeRole
+ Effect: Allow
+ Principal:
+ Service:
+ - ec2.amazonaws.com
+ Version: '2012-10-17'
+ Path: /
+
+ RolePolicies:
+ Type: AWS::IAM::Policy
+ Properties:
+ PolicyName: !Sub ${AWS::StackName}-ChefStack-Policy
+ PolicyDocument:
+ Version: '2012-10-17'
+ Statement:
+ # Allow all actions to one bucket (the supplied one, or the one you provided)
+ - Action: s3:*
+ Effect: Allow
+ Resource:
+ - !Join ['', [ 'arn:aws:s3:::', !Ref ChefBucket]]
+ - !Join ['', [ 'arn:aws:s3:::', !Ref ChefBucket, '/*' ]]
+ # Allow ability to list all buckets
+ - Action: s3:List*
+ Effect: Allow
+ Resource: arn:aws:s3:::*
+ # Allow instances to set themselves as unhealthy if one of the scripts fail
+ - Action: autoscaling:*
+ Effect: Allow
+ Resource: "*"
+ # Allow instance to create a nightly snapshot
+ - Action: ["ec2:CreateSnapshot", "ec2:CreateTags", "ec2:DeleteSnapshot", "ec2:DescribeSnapshots", "ec2:DescribeVolumes"]
+ Effect: Allow
+ Resource: "*"
+ # Allow instances to read their own tags (needed for setup script below)
+ - Action: ec2:DescribeTags
+ Effect: Allow
+ Resource: "*"
+ - Action: cloudwatch:PutMetricData
+ Effect: Allow
+ Resource: "*"
+ - Action: cloudwatch:GetMetricStatistics
+ Effect: Allow
+ Resource: "*"
+ - Action: cloudwatch:ListMetrics
+ Effect: Allow
+ Resource: "*"
+ # Allow instances to write to cloudwatch logs
+ - Action: ["logs:PutLogEvents", "logs:CreateLogStream", "logs:CreateLogGroup"]
+ Effect: Allow
+ Resource: !Sub arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:*
+ Roles:
+ - !Ref ChefRole
+
+ ChefBucket:
+ Type: AWS::S3::Bucket
+ DeletionPolicy: Retain
+ Properties:
+ AccessControl: Private
+ VersioningConfiguration:
+ Status: Enabled
+
+ InboundAdminSecurityGroupId:
+ Type: AWS::EC2::SecurityGroup
+ Properties:
+ GroupDescription: Trusted IP addresses, ranges or security groups
+ SecurityGroupIngress:
+ - CidrIp: !Ref InboundAdminCidr
+ IpProtocol: tcp
+ FromPort: 22
+ ToPort: 22
+ - CidrIp: !Ref InboundAdminCidr
+ IpProtocol: tcp
+ FromPort: 443
+ ToPort: 443
+ Tags:
+ - Key: Name
+ Value: !Sub ${AWS::StackName}-Trusted-SG
+ VpcId: !Ref VPC
+
+ FrontendSecurityGroup:
+ Type: AWS::EC2::SecurityGroup
+ Properties:
+ GroupDescription: Setup Ingress/Egress for Automate Server
+ SecurityGroupEgress:
+ - CidrIp: 0.0.0.0/0
+ FromPort: 0
+ IpProtocol: tcp
+ ToPort: 65535
+ SecurityGroupIngress:
+ - FromPort: 80
+ IpProtocol: tcp
+ SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
+ ToPort: 80
+ - FromPort: 443
+ IpProtocol: tcp
+ SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup
+ ToPort: 443
+ Tags:
+ - Key: Name
+ Value: !Sub ${AWS::StackName}-FE-SG
+ VpcId: !Ref VPC
+
+ LoadBalancerSecurityGroup:
+ Type: AWS::EC2::SecurityGroup
+ Properties:
+ GroupDescription: Setup Ingress/Egress for Automate Load Balancer
+ SecurityGroupEgress:
+ - CidrIp: 0.0.0.0/0
+ FromPort: 0
+ IpProtocol: tcp
+ ToPort: 65535
+ SecurityGroupIngress:
+ - CidrIp: !Ref InboundClientCidr
+ FromPort: 80
+ IpProtocol: tcp
+ ToPort: 80
+ - CidrIp: !Ref InboundClientCidr
+ FromPort: 443
+ IpProtocol: tcp
+ ToPort: 443
+ Tags:
+ - Key: Name
+ Value: !Sub ${AWS::StackName}-ELB-SG
+ VpcId: !Ref VPC
+
+ AlertNotificationTopic:
+ Type: AWS::SNS::Topic
+ Properties:
+ Subscription:
+ - Endpoint:
+ !Ref ContactEmail
+ Protocol: email
+
+# Logging Groups (Cloudwatch Logs)
+#########################################################################################
+ SystemLogs:
+ Type: AWS::Logs::LogGroup
+ Properties:
+ RetentionInDays: !Ref LogsRetentionInDays
+ ChefAppLogs:
+ Type: AWS::Logs::LogGroup
+ Properties:
+ RetentionInDays: !Ref LogsRetentionInDays
+ SupermarketAppLogs:
+ Type: AWS::Logs::LogGroup
+ Properties:
+ RetentionInDays: !Ref LogsRetentionInDays
+ AutomateAppLogs:
+ Type: AWS::Logs::LogGroup
+ Properties:
+ RetentionInDays: !Ref LogsRetentionInDays
+
+# Automate Stack
+#########################################################################################
+ AutomateStack:
+ Type: AWS::CloudFormation::Stack
+ Properties:
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/automate.yaml
+ Parameters:
+ VPC: !Ref VPC
+ LoadBalancerSubnets: !Join [ ",", !Ref ServerSubnets ]
+ AutomateSubnet: !Select [ "0", !Ref ServerSubnets ]
+ SSLCertificateARN: !Ref AutomateSSLCertificateARN
+ AutomateAdminPassword: !Ref ChefAutomateAdminPassword
+ InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
+ KeyName: !Ref KeyName
+ ContactEmail: !Ref ContactEmail
+ ContactDept: !Ref ContactDept
+ AlertNotificationTopic: !Ref AlertNotificationTopic
+ InstanceType: !Ref AutomateInstanceType
+ DataVolumeSize: !Ref AutomateDataVolumeSize
+ AutomateDataCollectorToken: !Ref ChefAutomateToken
+ LoadBalancerScheme: !Ref LoadBalancerScheme
+ LoadBalancerSecurityGroupId: !Ref LoadBalancerSecurityGroup
+ FrontendSecurityGroupId: !Ref FrontendSecurityGroup
+ AutomateIamRole: !Ref ChefRole
+ Route53HostedZone: !Ref Route53HostedZone
+ Route53RecordName: !Ref AutomateDnsRecordName
+ SystemLogsGroup: !Ref SystemLogs
+ AppLogsGroup: !Ref AutomateAppLogs
+
+# Chef Server Stack
+#########################################################################################
+ ChefServerStack:
+ Type: AWS::CloudFormation::Stack
+ Properties:
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/chef_server_ha.yaml
+ Parameters:
+ AutomationBucket: !Ref AutomationBucket
+ VPC: !Ref VPC
+ ChefServerSubnets: !Join [ ",", !Ref ServerSubnets ]
+ SSLCertificateARN: !Ref ChefSSLCertificateARN
+ InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
+ KeyName: !Ref KeyName
+ DBPassword: !Ref ChefDBPassword
+ ContactEmail: !Ref ContactEmail
+ ContactDept: !Ref ContactDept
+ AlertNotificationTopic: !Ref AlertNotificationTopic
+ InstanceType: !Ref ChefInstanceType
+ DBInstanceClass: !Ref ChefDBInstanceClass
+ DBStorageType: !Ref ChefDBStorageType
+ DBAllocatedStorage: !Ref ChefDBAllocatedStorage
+ DBIops: !Ref ChefDBIops
+ DBBackupDaystoKeep: !Ref ChefDBBackupDaystoKeep
+ ElasticSearchInstanceType: !Ref ChefElasticSearchInstanceType
+ ElasticSearchVersion: !Ref ChefElasticSearchVersion
+ ElasticSearchShardCount: !Ref ChefElasticSearchShardCount
+ ElasticSearchReplicaCount: !If [2ZoneES, 1, 2]
+ LoadBalancerScheme: !Ref LoadBalancerScheme
+ LoadBalancerSecurityGroupId: !Ref LoadBalancerSecurityGroup
+ FrontendSecurityGroupId: !Ref FrontendSecurityGroup
+ ChefServerAssociatePublicIpAddress: !Ref AssociatePublicIpAddress
+ ChefServerIamRole: !Ref ChefRole
+ ChefSecretsBucket: !Ref ChefBucket
+ ChefAutomateServerUrl: !Sub "https://${AutomateStack.Outputs.DNSName}"
+ ChefAutomateToken: !Ref ChefAutomateToken
+ SupermarketUrl: !Sub "https://${SupermarketDnsRecordName}.${Route53HostedZone}"
+ Route53HostedZone: !Ref Route53HostedZone
+ Route53RecordName: !Ref ChefServerDnsRecordName
+ SystemLogsGroup: !Ref SystemLogs
+ AppLogsGroup: !Ref ChefAppLogs
+ TemplateVersion: !Ref TemplateVersion
+
+# Supermarket Stack
+#########################################################################################
+ SupermarketStack:
+ Type: AWS::CloudFormation::Stack
+ Properties:
+ TemplateURL: !Sub https://s3.amazonaws.com/${AutomationBucket}/${TemplateVersion}/supermarket.yaml
+ Parameters:
+ VPC: !Ref VPC
+ LoadBalancerSubnets: !Join [ ",", !Ref ServerSubnets ]
+ SupermarketSubnet: !Select [ "0", !Ref ServerSubnets ]
+ SSLCertificateARN: !Ref SupermarketSSLCertificateARN
+ InboundAdminSecurityGroupId: !Ref InboundAdminSecurityGroupId
+ ImageId: !FindInMap [AMI, !Ref "AWS::Region", centos]
+ KeyName: !Ref KeyName
+ ContactEmail: !Ref ContactEmail
+ ContactDept: !Ref ContactDept
+ AlertNotificationTopic: !Ref AlertNotificationTopic
+ ChefUrl: !Sub "https://${ChefServerStack.Outputs.DNSName}"
+ InstanceType: !Ref SupermarketInstanceType
+ DataVolumeSize: !Ref SupermarketDataVolumeSize
+ LoadBalancerScheme: !Ref LoadBalancerScheme
+ LoadBalancerSecurityGroupId: !Ref LoadBalancerSecurityGroup
+ FrontendSecurityGroupId: !Ref FrontendSecurityGroup
+ SupermarketIamRole: !Ref ChefRole
+ ChefSecretsBucket: !Ref ChefBucket
+ ChefStackName: !GetAtt ChefServerStack.Outputs.StackName
+ Route53HostedZone: !Ref Route53HostedZone
+ Route53RecordName: !Ref SupermarketDnsRecordName
+ SystemLogsGroup: !Ref SystemLogs
+ AppLogsGroup: !Ref SupermarketAppLogs
+
+Outputs:
+ AutomateUrl:
+ Description: The URL of the Chef Automate Server
+ Value: !Sub "https://${AutomateStack.Outputs.DNSName}"
+ Export:
+ Name: !Sub "${AWS::StackName}-AutomateURL"
+ ChefServerUrl:
+ Description: The URL of the Chef Automate Server
+ Value: !Sub "https://${ChefServerStack.Outputs.DNSName}"
+ Export:
+ Name: !Sub "${AWS::StackName}-ChefServerURL"
+ SupermarketUrl:
+ Description: The URL of the Chef Automate Server
+ Value: !Sub "https://${SupermarketStack.Outputs.DNSName}"
+ Export:
+ Name: !Sub "${AWS::StackName}-SupermarketURL"
diff --git a/stack_parameters_main.json.example b/stack_parameters_main.json.example
new file mode 100644
index 0000000..89c525f
--- /dev/null
+++ b/stack_parameters_main.json.example
@@ -0,0 +1,50 @@
+[
+ {
+ "ParameterKey": "VPC",
+ "ParameterValue": "vpc-41d45124"
+ },
+ {
+ "ParameterKey": "ServerSubnets",
+ "ParameterValue": "subnet-7424b611,subnet-7fd06308,subnet-4dc71114"
+ },
+ {
+ "ParameterKey": "InboundAdminSecurityGroupId",
+ "ParameterValue": "sg-c9beb2ac"
+ },
+ {
+ "ParameterKey": "KeyName",
+ "ParameterValue": "id_rsa"
+ },
+ {
+ "ParameterKey": "ContactEmail",
+ "ParameterValue": "me@mycompany.com"
+ },
+ {
+ "ParameterKey": "ContactDept",
+ "ParameterValue": "Cool Automation Team"
+ },
+ {
+ "ParameterKey": "ChefAutomateAdminPassword",
+ "ParameterValue": "SuperSecurePassword"
+ },
+ {
+ "ParameterKey": "ChefSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "AutomateSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "SupermarketSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "ChefDBPassword",
+ "ParameterValue": "SuperSecurePassword"
+ },
+ {
+ "ParameterKey": "Route53HostedZone",
+ "ParameterValue": "chef.mycompany.com"
+ }
+]
diff --git a/stack_parameters_marketplace.json.example b/stack_parameters_marketplace.json.example
new file mode 100644
index 0000000..d24edd8
--- /dev/null
+++ b/stack_parameters_marketplace.json.example
@@ -0,0 +1,54 @@
+[
+ {
+ "ParameterKey": "VPC",
+ "ParameterValue": "vpc-41d45124"
+ },
+ {
+ "ParameterKey": "ServerSubnets",
+ "ParameterValue": "subnet-7424b611,subnet-7fd06308,subnet-4dc71114"
+ },
+ {
+ "ParameterKey": "InboundClientCidr",
+ "ParameterValue": "0.0.0.0/0"
+ },
+ {
+ "ParameterKey": "InboundAdminCidr",
+ "ParameterValue": "0.0.0.0/0"
+ },
+ {
+ "ParameterKey": "KeyName",
+ "ParameterValue": "id_rsa"
+ },
+ {
+ "ParameterKey": "ContactEmail",
+ "ParameterValue": "me@mycompany.com"
+ },
+ {
+ "ParameterKey": "ContactDept",
+ "ParameterValue": "Cool Automation Team"
+ },
+ {
+ "ParameterKey": "ChefAutomateAdminPassword",
+ "ParameterValue": "SuperSecurePassword"
+ },
+ {
+ "ParameterKey": "ChefSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "AutomateSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "SupermarketSSLCertificateARN",
+ "ParameterValue": "arn:aws:acm:us-west-2:446539779517:certificate/60f573b3-f8ed-48d9-a6d1-e89f79da2e8f"
+ },
+ {
+ "ParameterKey": "ChefDBPassword",
+ "ParameterValue": "SuperSecurePassword"
+ },
+ {
+ "ParameterKey": "Route53HostedZone",
+ "ParameterValue": "chef.mycompany.com"
+ }
+]
diff --git a/supermarket.yaml b/supermarket.yaml
index 84da691..e465730 100644
--- a/supermarket.yaml
+++ b/supermarket.yaml
@@ -1,12 +1,8 @@
AWSTemplateFormatVersion: '2010-09-09'
-Description: AWS Native Supermarket (auto-recovering standalone) v4.1.0
+Description: AWS Native Supermarket (auto-recovering standalone) v5.0.2
Parameters:
# Required Parameters
- AutomationBucket:
- Default: 'aws-native-chef-server'
- Type: String
- Description: AWS S3 bucket name that contains all of the cloudformation templates and scripts for this stack
VPC:
Description: Choose VPC to use
Type: AWS::EC2::VPC::Id
@@ -62,12 +58,6 @@ Parameters:
Type: Number
Default: 100
###############################################################################
- # Package Versions & Locations
- SupermarketPackage:
- Description: The URL to the chef server EL7 (chef-server-core) package which will be downloaded
- Type: String
- Default: 'https://packages.chef.io/files/stable/supermarket/3.1.63/el/7/supermarket-3.1.63-1.el7.x86_64.rpm'
- ###############################################################################
# Security Settings
LoadBalancerScheme:
Description: Network Scheme for the ELB
@@ -96,18 +86,11 @@ Parameters:
Route53HostedZone:
Type: String
Default: ''
- Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com.) for auto-creating a DNS record. Must end in a dot. (Leave blank to disable)
+ Description: Supply a Route 53 Hosted Zone name (eg. mydomain.com) for auto-creating a DNS record. Must NOT end in a dot. (Leave blank to disable)
Route53RecordName:
Type: String
Default: 'supermarket'
Description: Supply a DNS record name that will be prepended to the Route 53 Hosted Zone
- AMIFlavor:
- Description: Choose the AMI Flavor, the High-Perf CentOS image or the stock and paid official RHEL image
- Type: String
- Default: 'centos'
- AllowedValues:
- - 'centos'
- - 'rhel'
SystemLogsGroup:
Description: The logical name of the Cloudwatch Logs log group for systemn logs
Type: String
@@ -124,8 +107,6 @@ Metadata:
Conditions:
CreateRoute53Record:
!Not [!Equals [ !Ref Route53HostedZone, '' ] ]
- RHELAmiFlavor:
- !Equals [ !Ref AMIFlavor, 'rhel' ]
Resources:
ChefInstanceProfile:
@@ -215,14 +196,6 @@ Resources:
"Fn::Base64": !Sub |
#!/bin/bash -ex
trap '/opt/aws/bin/cfn-signal -e 1 --region ${AWS::Region} --stack ${AWS::StackName} --resource SupermarketServer' ERR
- # Update the system first
- yum upgrade -y
- # Run before script, make sure to install cfn-init if not installed
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/before.sh /root/before.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/before.sh -o /root/before.sh || \
- curl -s https://aws-native-chef-server.s3.amazonaws.com/files/before.sh -o /root/before.sh
- bash -ex /root/before.sh
# Execute AWS::CloudFormation::Init
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource SupermarketServer --region ${AWS::Region}
/opt/aws/bin/cfn-signal -e 0 --region ${AWS::Region} --stack ${AWS::StackName} --resource SupermarketServer
@@ -233,27 +206,12 @@ Resources:
Metadata:
AWS::CloudFormation::Init:
configSets:
- default: [awslogs, supermarket_setup, nightly_snapshots]
+ default: [awslogs, supermarket_setup, nightly_snapshots, final_stage]
supermarket_setup:
- packages:
- rpm:
- supermarket: !Sub ${SupermarketPackage}
- yum:
- perl: []
- perl-DateTime: []
- perl-Sys-Syslog: []
- perl-LWP-Protocol-https: []
- perl-Digest-SHA: []
- zip: []
- unzip: []
commands:
02_write_supermarket_json:
command: !Sub
- |
- yum install -y jq || \
- rpm -ivh \
- https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/o/oniguruma-5.9.5-3.el7.x86_64.rpm \
- https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/j/jq-1.5-1.el7.x86_64.rpm
aws s3 cp s3://${ChefSecretsBucket}/${ChefStackName}/etc_opscode/oc-id-applications/supermarket.json /tmp/supermarket.json
OAUTH2_APP_ID=$(cat /tmp/supermarket.json | jq -r ".uid")
OAUTH2_SECRET=$(cat /tmp/supermarket.json | jq -r ".secret")
@@ -267,35 +225,18 @@ Resources:
- { FQDN: !If [CreateRoute53Record, !Ref SupermarketDNSRecord, !GetAtt ChefALB.DNSName] }
03_configure_supermarket:
command: |
+ rpm -ivh /var/cache/marketplace/supermarket.rpm
supermarket-ctl reconfigure
- 04_after_script:
- command: !Sub |
- # attempt to fetch the script first as a signed request (private bucket), then unsigned (public bucket)
- aws s3 cp s3://${AutomationBucket}/files/after.sh /root/after.sh || \
- curl -s https://${AutomationBucket}.s3.amazonaws.com/files/after.sh -o /root/after.sh
- bash -ex /root/after.sh
05_configure_cloudwatch_monitoring:
command: |
- mkdir /opt/cloudwatch_monitoring
- cd /opt/cloudwatch_monitoring
- curl http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip -O
- unzip CloudWatchMonitoringScripts-1.2.1.zip
- rm -f CloudWatchMonitoringScripts-1.2.1.zip
crontab -l | { cat; echo "*/5 * * * * /opt/cloudwatch_monitoring/aws-scripts-mon/mon-put-instance-data.pl --mem-util --disk-space-util --disk-path=/ --from-cron"; } | crontab -
nightly_snapshots:
- files:
- /usr/local/bin/ebs-snapshot.sh:
- source: https://raw.githubusercontent.com/CaseyLabs/aws-ec2-ebs-automatic-snapshot-bash/master/ebs-snapshot.sh
- mode: '000755'
commands:
create_nightly_backup_cronjob:
command: |
crontab -l | { cat; echo "55 23 * * * root /usr/local/bin/ebs-snapshot.sh"; } | crontab -
awslogs:
files:
- /root/awslogs-agent-setup.py:
- source: 'https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py'
- mode: '000755'
/etc/awslogs/awscli.conf:
content: !Sub |
[default]
@@ -374,7 +315,16 @@ Resources:
01_setup:
command: !Sub |
mkdir -p /var/lib/awslogs
- python /root/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
+ python /var/cache/marketplace/awslogs-agent-setup.py --region ${AWS::Region} --non-interactive -c /etc/awslogs/awslogs.conf
+ final_stage:
+ commands:
+ 09_cleanup:
+ command: |
+ # clean up the chef package cache, to maximize available disk space after install
+ if [ -d /var/cache/marketplace ]; then
+ rm -f /var/cache/marketplace/*.rpm
+ rm -f /var/cache/marketplace/*.aib
+ fi
# EC2 System AutoRecovery
# https://aws.amazon.com/blogs/aws/new-auto-recovery-for-amazon-ec2/
@@ -423,9 +373,9 @@ Resources:
Type: AWS::Route53::RecordSet
Condition: CreateRoute53Record
Properties:
- HostedZoneName: !Ref Route53HostedZone
+ HostedZoneName: !Sub ${Route53HostedZone}.
+ Name: !Sub ${Route53RecordName}.${Route53HostedZone}.
Comment: !Sub Created by Cloudformation ${AWS::StackName}
- Name: !Sub ${Route53RecordName}.${Route53HostedZone}
Type: CNAME
TTL: '900'
ResourceRecords:
@@ -470,7 +420,7 @@ Resources:
- Name: MountPath
Value: '/'
- Name: Filesystem
- Value: !If [RHELAmiFlavor, "/dev/nvme0n1p2", "/dev/nvme0n1p1"]
+ Value: '/dev/nvme0n1p1'
MemoryUtilization:
Type: AWS::CloudWatch::Alarm