Skip to content

Commit 1832011

Browse files
kamszlinki
kamsz
authored and
linki
committed
Setting up ExternalDNS using same domain for public/private Route53 zones (kubernetes-sigs#472)
* setting up externaldns using the same domain for public and private route53 zones * update nginx-ingress-controller image to 0.11.0
1 parent 06a8da0 commit 1832011

File tree

1 file changed

+386
-0
lines changed

1 file changed

+386
-0
lines changed

‎docs/tutorials/public-private-route53.md

Lines changed: 386 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,386 @@
1+
# Setting up ExternalDNS using the same domain for public and private Route53 zones
2+
3+
This tutorial describes how to setup ExternalDNS using the same domain for public and private Route53 zones and [nginx-ingress-controller](https://github.com/kubernetes/ingress-nginx). It also outlines how to use [cert-manager](https://github.com/jetstack/cert-manager) to automatically issue SSL certificates from [Let's Encrypt](https://letsencrypt.org/) for both public and private records.
4+
5+
## Deploy public nginx-ingress-controller
6+
7+
Consult [External DNS nginx ingress docs](nginx-ingress.md) for installation guidelines.
8+
9+
Specify `ingress-class` in nginx-ingress-controller container args:
10+
11+
```yaml
12+
apiVersion: extensions/v1beta1
13+
kind: Deployment
14+
metadata:
15+
labels:
16+
app: external-ingress
17+
name: external-ingress-controller
18+
spec:
19+
replicas: 1
20+
selector:
21+
matchLabels:
22+
app: external-ingress
23+
template:
24+
metadata:
25+
labels:
26+
app: external-ingress
27+
spec:
28+
containers:
29+
- args:
30+
- /nginx-ingress-controller
31+
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
32+
- --configmap=$(POD_NAMESPACE)/external-ingress-configuration
33+
- --tcp-services-configmap=$(POD_NAMESPACE)/external-tcp-services
34+
- --udp-services-configmap=$(POD_NAMESPACE)/external-udp-services
35+
- --annotations-prefix=nginx.ingress.kubernetes.io
36+
- --ingress-class=external-ingress
37+
- --publish-service=$(POD_NAMESPACE)/external-ingress
38+
env:
39+
- name: POD_NAME
40+
valueFrom:
41+
fieldRef:
42+
apiVersion: v1
43+
fieldPath: metadata.name
44+
- name: POD_NAMESPACE
45+
valueFrom:
46+
fieldRef:
47+
apiVersion: v1
48+
fieldPath: metadata.namespace
49+
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0
50+
livenessProbe:
51+
failureThreshold: 3
52+
httpGet:
53+
path: /healthz
54+
port: 10254
55+
scheme: HTTP
56+
initialDelaySeconds: 10
57+
periodSeconds: 10
58+
successThreshold: 1
59+
timeoutSeconds: 1
60+
name: external-ingress-controller
61+
ports:
62+
- containerPort: 80
63+
name: http
64+
protocol: TCP
65+
- containerPort: 443
66+
name: https
67+
protocol: TCP
68+
readinessProbe:
69+
failureThreshold: 3
70+
httpGet:
71+
path: /healthz
72+
port: 10254
73+
scheme: HTTP
74+
periodSeconds: 10
75+
successThreshold: 1
76+
timeoutSeconds: 1
77+
```
78+
79+
Set `type: LoadBalancer` in your public nginx-ingress-controller Service definition.
80+
81+
```yaml
82+
apiVersion: v1
83+
kind: Service
84+
metadata:
85+
annotations:
86+
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
87+
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
88+
labels:
89+
app: external-ingress
90+
name: external-ingress
91+
spec:
92+
externalTrafficPolicy: Cluster
93+
ports:
94+
- name: http
95+
port: 80
96+
protocol: TCP
97+
targetPort: http
98+
- name: https
99+
port: 443
100+
protocol: TCP
101+
targetPort: https
102+
selector:
103+
app: external-ingress
104+
sessionAffinity: None
105+
type: LoadBalancer
106+
```
107+
108+
## Deploy private nginx-ingress-controller
109+
110+
Consult [External DNS nginx ingress docs](nginx-ingress.md) for installation guidelines.
111+
112+
Make sure to specify `ingress-class` in nginx-ingress-controller container args:
113+
114+
```yaml
115+
apiVersion: extensions/v1beta1
116+
kind: Deployment
117+
metadata:
118+
labels:
119+
app: internal-ingress
120+
name: internal-ingress-controller
121+
spec:
122+
replicas: 1
123+
selector:
124+
matchLabels:
125+
app: internal-ingress
126+
template:
127+
metadata:
128+
labels:
129+
app: internal-ingress
130+
spec:
131+
containers:
132+
- args:
133+
- /nginx-ingress-controller
134+
- --default-backend-service=$(POD_NAMESPACE)/default-http-backend
135+
- --configmap=$(POD_NAMESPACE)/internal-ingress-configuration
136+
- --tcp-services-configmap=$(POD_NAMESPACE)/internal-tcp-services
137+
- --udp-services-configmap=$(POD_NAMESPACE)/internal-udp-services
138+
- --annotations-prefix=nginx.ingress.kubernetes.io
139+
- --ingress-class=internal-ingress
140+
- --publish-service=$(POD_NAMESPACE)/internal-ingress
141+
env:
142+
- name: POD_NAME
143+
valueFrom:
144+
fieldRef:
145+
apiVersion: v1
146+
fieldPath: metadata.name
147+
- name: POD_NAMESPACE
148+
valueFrom:
149+
fieldRef:
150+
apiVersion: v1
151+
fieldPath: metadata.namespace
152+
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0
153+
livenessProbe:
154+
failureThreshold: 3
155+
httpGet:
156+
path: /healthz
157+
port: 10254
158+
scheme: HTTP
159+
initialDelaySeconds: 10
160+
periodSeconds: 10
161+
successThreshold: 1
162+
timeoutSeconds: 1
163+
name: internal-ingress-controller
164+
ports:
165+
- containerPort: 80
166+
name: http
167+
protocol: TCP
168+
- containerPort: 443
169+
name: https
170+
protocol: TCP
171+
readinessProbe:
172+
failureThreshold: 3
173+
httpGet:
174+
path: /healthz
175+
port: 10254
176+
scheme: HTTP
177+
periodSeconds: 10
178+
successThreshold: 1
179+
timeoutSeconds: 1
180+
```
181+
182+
Set additional annotations in your private nginx-ingress-controller Service definition to create an internal load balancer.
183+
184+
```yaml
185+
apiVersion: v1
186+
kind: Service
187+
metadata:
188+
annotations:
189+
service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
190+
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
191+
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
192+
labels:
193+
app: internal-ingress
194+
name: internal-ingress
195+
spec:
196+
externalTrafficPolicy: Cluster
197+
ports:
198+
- name: http
199+
port: 80
200+
protocol: TCP
201+
targetPort: http
202+
- name: https
203+
port: 443
204+
protocol: TCP
205+
targetPort: https
206+
selector:
207+
app: internal-ingress
208+
sessionAffinity: None
209+
type: LoadBalancer
210+
```
211+
212+
## Deploy the public zone ExternalDNS
213+
214+
Consult [AWS ExternalDNS setup docs](aws.md) for installation guidelines.
215+
216+
In ExternalDNS containers args, make sure to specify `annotation-filter` and `aws-zone-type`:
217+
218+
```yaml
219+
apiVersion: apps/v1beta2
220+
kind: Deployment
221+
metadata:
222+
labels:
223+
app: external-dns-public
224+
name: external-dns-public
225+
namespace: kube-system
226+
spec:
227+
replicas: 1
228+
selector:
229+
matchLabels:
230+
app: external-dns-public
231+
strategy:
232+
type: Recreate
233+
template:
234+
metadata:
235+
labels:
236+
app: external-dns-public
237+
spec:
238+
containers:
239+
- args:
240+
- --source=ingress
241+
- --provider=aws
242+
- --registry=txt
243+
- --txt-owner-id=external-dns
244+
- --annotation-filter=kubernetes.io/ingress.class=external-ingress
245+
- --aws-zone-type=public
246+
image: registry.opensource.zalan.do/teapot/external-dns:0.4.8
247+
name: external-dns-public
248+
```
249+
250+
## Deploy the private zone ExternalDNS
251+
252+
Consult [AWS ExternalDNS setup docs](aws.md) for installation guidelines.
253+
254+
In ExternalDNS containers args, make sure to specify `annotation-filter` and `aws-zone-type`:
255+
256+
```yaml
257+
apiVersion: apps/v1beta2
258+
kind: Deployment
259+
metadata:
260+
labels:
261+
app: external-dns-private
262+
name: external-dns-private
263+
namespace: kube-system
264+
spec:
265+
replicas: 1
266+
selector:
267+
matchLabels:
268+
app: external-dns-private
269+
strategy:
270+
type: Recreate
271+
template:
272+
metadata:
273+
labels:
274+
app: external-dns-private
275+
spec:
276+
containers:
277+
- args:
278+
- --source=ingress
279+
- --provider=aws
280+
- --registry=txt
281+
- --txt-owner-id=dev.k8s.nexus
282+
- --annotation-filter=kubernetes.io/ingress.class=internal-ingress
283+
- --aws-zone-type=private
284+
image: registry.opensource.zalan.do/teapot/external-dns:0.4.8
285+
name: external-dns-private
286+
```
287+
288+
## Create application Service definitions
289+
290+
For this setup to work, you've to create two Service definitions for your application.
291+
292+
At first, create public Service definition:
293+
294+
```yaml
295+
apiVersion: extensions/v1beta1
296+
kind: Ingress
297+
metadata:
298+
annotations:
299+
kubernetes.io/ingress.class: "external-ingress"
300+
labels:
301+
app: app
302+
name: app-public
303+
spec:
304+
rules:
305+
- host: app.domain.com
306+
http:
307+
paths:
308+
- backend:
309+
serviceName: app
310+
servicePort: 80
311+
```
312+
313+
Then create private Service definition:
314+
315+
```yaml
316+
apiVersion: extensions/v1beta1
317+
kind: Ingress
318+
metadata:
319+
annotations:
320+
kubernetes.io/ingress.class: "internal-ingress"
321+
labels:
322+
app: app
323+
name: app-private
324+
spec:
325+
rules:
326+
- host: app.domain.com
327+
http:
328+
paths:
329+
- backend:
330+
serviceName: app
331+
servicePort: 80
332+
```
333+
334+
Additionally, you may leverage [cert-manager](https://github.com/jetstack/cert-manager) to automatically issue SSL certificates from [Let's Encrypt](https://letsencrypt.org/). To do that, request a certificate in public service definition:
335+
336+
```yaml
337+
apiVersion: extensions/v1beta1
338+
kind: Ingress
339+
metadata:
340+
annotations:
341+
certmanager.k8s.io/acme-challenge-type: "dns01"
342+
certmanager.k8s.io/acme-dns01-provider: "route53"
343+
certmanager.k8s.io/cluster-issuer: "letsencrypt-production"
344+
kubernetes.io/ingress.class: "external-ingress"
345+
kubernetes.io/tls-acme: "true"
346+
labels:
347+
app: app
348+
name: app-public
349+
spec:
350+
rules:
351+
- host: app.domain.com
352+
http:
353+
paths:
354+
- backend:
355+
serviceName: app
356+
servicePort: 80
357+
tls:
358+
- hosts:
359+
- app.domain.com
360+
secretName: app-tls
361+
```
362+
363+
And reuse the requested certificate in private Service definition:
364+
365+
```yaml
366+
apiVersion: extensions/v1beta1
367+
kind: Ingress
368+
metadata:
369+
annotations:
370+
kubernetes.io/ingress.class: "internal-ingress"
371+
labels:
372+
app: app
373+
name: app-private
374+
spec:
375+
rules:
376+
- host: app.domain.com
377+
http:
378+
paths:
379+
- backend:
380+
serviceName: app
381+
servicePort: 80
382+
tls:
383+
- hosts:
384+
- app.domain.com
385+
secretName: app-tls
386+
```

0 commit comments

Comments
 (0)