Skip to content

Commit 351ed8e

Browse files
authored
Merge pull request #172 from alan-turing-institute/docs/network-policy
Update docs to deploy cluster that obeys network policies
2 parents e8f7e76 + 69ae54e commit 351ed8e

8 files changed

+327
-49
lines changed

docs/_posts/2010-01-01-azure-keyvault.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ Test if the resource group exists by running the following: ```az group exists -
6868

6969
#### 4. Create the key vault
7070

71-
Key vault names must be lower case and/or numerical and may only include hyphens (`-`), no underscores (`_`) or other non-alphanumeric characters. They must also be unique.
71+
**NOTE:** Key vault names must be lower case and/or numerical and may only include hyphens (`-`), no underscores (`_`) or other non-alphanumeric characters. They must also be unique.
7272
{: .notice--info}
7373

7474
```bash

docs/_posts/2010-01-02-deploy-standard-k8s-cluster.md

Lines changed: 73 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ We assume you have the following CLIs installed:
1515

1616
- [Setup](#setup)
1717
- [Download the required secrets](#download-the-required-secrets)
18+
- [Enable Network Policies](#enable-network-policies)
1819
- [Create the Kubernetes cluster](#create-the-kubernetes-cluster)
1920

2021
---
@@ -61,7 +62,8 @@ az group create --name Hub23 --location westeurope --output table
6162
- `--location` sets the [region of data centres](https://azure.microsoft.com/en-gb/global-infrastructure/locations/) that will host the resources.
6263
- `--output table` prints the info in a human-readable format.
6364

64-
**N.B.:** If you have already followed the docs on creating a key vault, then this resource group should already exist and this step can be skipped.
65+
**NOTE:** If you have already followed the docs on creating a key vault, then this resource group should already exist and this step can be skipped.
66+
{: .notice--info}
6567

6668
## Download the required secrets
6769

@@ -82,7 +84,6 @@ We will require the following secrets:
8284

8385
- Service Principal app ID and key
8486
- public SSH key
85-
- API and secret tokens
8687

8788
They should be downloaded to files in the `.secret` folder so that they are git-ignored.
8889

@@ -111,20 +112,67 @@ az keyvault secret download \
111112
--file .secret/ssh-key-hub23cluster.pub
112113
```
113114

114-
Download the API and secret tokens:
115+
## Enable Network Policies
116+
117+
The BinderHub helm chart contains network policies designed to restrict access to pods and the JupyterHub.
118+
However, the Kubernetes cluster will not be automatically configured to obey these network policies.
119+
Therefore, we need create a virtual network (vnet) and sub network (subnet) with network policies enabled so that these pod traffic restrictions are obeyed.
120+
121+
See the following documentation: <https://docs.microsoft.com/en-us/azure/aks/use-network-policies#create-an-aks-cluster-and-enable-network-policy>
122+
123+
#### 1. Create a VNET
115124

116125
```bash
117-
az keyvault secret download \
118-
--vault-name hub23-keyvault \
119-
--name apiToken \
120-
--file .secret/apiToken.txt
126+
az network vnet create \
127+
--resource-group Hub23 \
128+
--name hub23-vnet \
129+
--address-prefixes 10.0.0.0/8 \
130+
--subnet-name hub23-subnet \
131+
--subnet-prefix 10.240.0.0/16
121132
```
122133

134+
- `--address-prefixes`: IP address prefixes for the VNet;
135+
- `--subnet-prefix`: IP address prefixes in CIDR format for the new subnet.
136+
137+
#### 2. Retrieve the VNet ID
138+
139+
This saves the VNet ID into a bash variable.
140+
123141
```bash
124-
az keyvault secret download \
125-
--vault-name hub23-keyvault \
126-
--name secretToken \
127-
--file .secret/secretToken.txt
142+
VNET_ID=$(
143+
az network vnet show \
144+
--resource-group Hub23 \
145+
--name hub23-vnet \
146+
--query id \
147+
--output tsv
148+
)
149+
```
150+
151+
#### 3. Assign the Contributor role to the Service Principal for accessing the VNet
152+
153+
```bash
154+
az role assignment create \
155+
--assignee $(cat .secret/appID.txt) \
156+
--scope $VNET_ID \
157+
--role Contributor
158+
```
159+
160+
**WARNING:** You must have Owner permissions on the subscription for this step to work.
161+
{: .notice--warning}
162+
163+
#### 4. Retrieve the subnet ID
164+
165+
This will save the subnet ID to a bash variable.
166+
167+
```bash
168+
SUBNET_ID=$(
169+
az network vnet subnet show \
170+
--resource-group Hub23 \
171+
--vnet-name hub23-vnet \
172+
--name hub23-subnet \
173+
--query id \
174+
--output tsv
175+
)
128176
```
129177

130178
## Create the Kubernetes cluster
@@ -137,16 +185,30 @@ This command has been known to take between 7 and 30 minutes to execute dependin
137185
```bash
138186
az aks create --name hub23cluster \
139187
--resource-group Hub23 \
188+
--kubernetes-version 1.14.8 \
140189
--ssh-key-value .secret/ssh-key-hub23cluster.pub \
141190
--node-count 3
142191
--node-vm-size Standard_D2s_v3 \
143192
--service-principal $(cat .secret/appID.txt) \
144193
--client-secret $(cat .secret/key.txt) \
194+
--dns-service-ip 10.0.0.10 \
195+
--docker-bridge-address 172.17.0.1/16 \
196+
--network-plugin azure \
197+
--network-policy azure \
198+
--service-cidr 10.0.0.0/16 \
199+
--vnet-subnet-id $SUBNET_ID \
145200
--output table
146201
```
147202

203+
- `--kubernetes-version`: It's recommended to use the most up-to-date version of Kubernetes.
148204
- `--node-count` is the number of nodes to be deployed. 3 is recommended for a stable, scalable cluster.
149205
- `--node-vm-size` denotes the type of virtual machine to be deployed. A list of Linux types can be found [here](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/).
206+
- `--dns-service-ip`: An IP address assigned to the Kubernetes DNS service.
207+
- `--docker-bridge-address`: A specific IP address and netmask for the Docker bridge, using standard CIDR notation.
208+
- `--network-plugin`: The Kubernetes network plugin to use.
209+
- `--network-policy`: The Kubernetes network policy to use.
210+
- `--service-cidr`: A CIDR notation IP range from which to assign service cluster IPs.
211+
- `--vnet-subnet-id`: The ID of a subnet in an existing VNet into which to deploy the cluster.
150212

151213
#### Delete local copies of the secret files
152214

docs/_posts/2010-01-03-deploy-autoscaling-k8s-cluster.md

Lines changed: 74 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ We assume you have the following CLIs installed:
1515

1616
- [Setup](#setup)
1717
- [Download the required secrets](#download-the-required-secrets)
18+
- [Enable Network Policies](#enable-network-policies)
1819
- [Set up for Autoscaling](#set-up-for-autoscaling)
1920
- [Create the Kubernetes cluster](#create-the-kubernetes-cluster)
2021
- [Enabling Autoscaling](#enabling-autoscaling)
@@ -47,7 +48,7 @@ If not, request access by opening a TopDesk ticket.
4748
To activate the subscription, run the following:
4849

4950
```bash
50-
az account set --subcription Turing-BinderHub
51+
az account set --subscription Turing-BinderHub
5152
```
5253

5354
#### 3. Create a Resource Group
@@ -64,7 +65,8 @@ az group create --name Hub23 --location westeurope --output table
6465
- `--location` sets the [data centre](https://azure.microsoft.com/en-gb/global-infrastructure/locations/) that will host the resources.
6566
- `--output table` prints the info in a human-readable format.
6667

67-
**N.B.:** If you have already followed the docs on creating a key vault, then this resource group should already exist and this step can be skipped.
68+
**NOTE:** If you have already followed the docs on creating a key vault, then this resource group should already exist and this step can be skipped.
69+
{: .notice--info}
6870

6971
## Download the required secrets
7072

@@ -85,7 +87,6 @@ We will require the following secrets:
8587

8688
- Service Principal app ID and key
8789
- public SSH key
88-
- API and secret tokens
8990

9091
They should be downloaded to files in the `.secret` folder so that they are git-ignored.
9192
Download the Service Principal:
@@ -113,20 +114,67 @@ az keyvault secret download \
113114
--file .secret/ssh-key-hub23cluster.pub
114115
```
115116

116-
Download the API and secret tokens:
117+
## Enable Network Policies
118+
119+
The BinderHub helm chart contains network policies designed to restrict access to pods and the JupyterHub.
120+
However, the Kubernetes cluster will not be automatically configured to obey these network policies.
121+
Therefore, we need create a virtual network (vnet) and sub network (subnet) with network policies enabled so that these pod traffic restrictions are obeyed.
122+
123+
See the following documentation: <https://docs.microsoft.com/en-us/azure/aks/use-network-policies#create-an-aks-cluster-and-enable-network-policy>
124+
125+
#### 1. Create a VNET
117126

118127
```bash
119-
az keyvault secret download \
120-
--vault-name hub23-keyvault \
121-
--name apiToken \
122-
--file .secret/apiToken.txt
128+
az network vnet create \
129+
--resource-group Hub23 \
130+
--name hub23-vnet \
131+
--address-prefixes 10.0.0.0/8 \
132+
--subnet-name hub23-subnet \
133+
--subnet-prefix 10.240.0.0/16
123134
```
124135

136+
- `--address-prefixes`: IP address prefixes for the VNet;
137+
- `--subnet-prefix`: IP address prefixes in CIDR format for the new subnet.
138+
139+
#### 2. Retrieve the VNet ID
140+
141+
This saves the VNet ID into a bash variable.
142+
125143
```bash
126-
az keyvault secret download \
127-
--vault-name hub23-keyvault \
128-
--name secretToken \
129-
--file .secret/secretToken.txt
144+
VNET_ID=$(
145+
az network vnet show \
146+
--resource-group Hub23 \
147+
--name hub23-vnet \
148+
--query id \
149+
--output tsv
150+
)
151+
```
152+
153+
#### 3. Assign the Contributor role to the Service Principal for accessing the VNet
154+
155+
```bash
156+
az role assignment create \
157+
--assignee $(cat .secret/appID.txt) \
158+
--scope $VNET_ID \
159+
--role Contributor
160+
```
161+
162+
**WARNING:** You must have Owner permissions on the subscription for this step to work.
163+
{: .notice--warning}
164+
165+
#### 4. Retrieve the subnet ID
166+
167+
This will save the subnet ID to a bash variable.
168+
169+
```bash
170+
SUBNET_ID=$(
171+
az network vnet subnet show \
172+
--resource-group Hub23 \
173+
--vnet-name hub23-vnet \
174+
--name hub23-subnet \
175+
--query id \
176+
--output tsv
177+
)
130178
```
131179

132180
## Set up for Autoscaling
@@ -175,7 +223,7 @@ This command has been known to take between 7 and 30 minutes to execute dependin
175223
az aks create \
176224
--resource-group Hub23 \
177225
--name hub23cluster \
178-
--kubernetes-version 1.14.6 \ # Or whichever is newest
226+
--kubernetes-version 1.14.8 \
179227
--node-count 3 \
180228
--enable-vmss \
181229
--enable-cluster-autoscaler \
@@ -184,13 +232,26 @@ az aks create \
184232
--ssh-key-value .secret/ssh-key-hub23cluster.pub \
185233
--service-principal $(cat .secret/appID.txt) \
186234
--client-secret $(cat .secret/key.txt) \
235+
--dns-service-ip 10.0.0.10 \
236+
--docker-bridge-address 172.17.0.1/16 \
237+
--network-plugin azure \
238+
--network-policy azure \
239+
--service-cidr 10.0.0.0/16 \
240+
--vnet-subnet-id $SUBNET_ID \
187241
--output table
188242
```
189243

190244
- `--node-count` is the number of nodes to be deployed. 3 is recommended for a stable, scalable cluster.
191245
- `--kubernetes-version` needs to be 1.12.4 or greater to be compatible with the cluster autoscaler.
246+
It's recommended to use the most up-to-date version of Kubernetes.
192247
- `--enable-vmss` enables the Virtual Machine Scale Set of scalable VMs.
193248
- `--min-count`/`--max-count` defines the minimum and maximum number of nodes to be spun up/down.
249+
- `--dns-service-ip`: An IP address assigned to the Kubernetes DNS service.
250+
- `--docker-bridge-address`: A specific IP address and netmask for the Docker bridge, using standard CIDR notation.
251+
- `--network-plugin`: The Kubernetes network plugin to use.
252+
- `--network-policy`: The Kubernetes network policy to use.
253+
- `--service-cidr`: A CIDR notation IP range from which to assign service cluster IPs.
254+
- `--vnet-subnet-id`: The ID of a subnet in an existing VNet into which to deploy the cluster.
194255

195256
#### Delete local copies of the secret files
196257

0 commit comments

Comments
 (0)