Skip to content

Commit

Permalink
Stateful master nodes (pires#211)
Browse files Browse the repository at this point in the history
  • Loading branch information
skokovic authored and pires committed Aug 21, 2018
1 parent 3053cdd commit 6b8050f
Show file tree
Hide file tree
Showing 3 changed files with 84 additions and 5 deletions.
8 changes: 4 additions & 4 deletions stateful/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# Elasticsearch StatefulSet Data Pod

This directory contains Kubernetes configurations which run elasticsearch data pods as a [`StatefulSet`](https://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/), using storage provisioned using a [`StorageClass`](http://blog.kubernetes.io/2016/10/dynamic-provisioning-and-storage-in-kubernetes.html). Be sure to read and understand the documentation in the root directory, which deploys the data pods as a `Deployment` using an `emptyDir` for storage.
This directory contains Kubernetes configurations which run elasticsearch data and master pods as a [`StatefulSet`](https://kubernetes.io/docs/concepts/abstractions/controllers/statefulsets/), using storage provisioned using a [`StorageClass`](http://blog.kubernetes.io/2016/10/dynamic-provisioning-and-storage-in-kubernetes.html). Be sure to read and understand the documentation in the root directory, which deploys the data pods as a `Deployment` using an `emptyDir` for storage.

## Storage

The [`es-data-stateful.yaml`](es-data-stateful.yaml) file contains a `volumeClaimTemplates` section which requests a 12 GB disk. This is plenty of space for a demonstration cluster, but will fill up quickly under moderate to heavy load. Consider modifying the disk size to your needs.
The [`es-data-stateful.yaml`](es-data-stateful.yaml) and [`es-master-stateful.yaml`](es-master-stateful.yaml) files contain `volumeClaimTemplates` sections which request 2GB volume for each master node, and 12GB volume for each data node. This is plenty of space for a demonstration cluster, but will fill up quickly under moderate to heavy load. Consider modifying the disk size to your needs.

## Deploy
The root directory contains instructions for deploying elasticsearch using a `Deployment` with transient storage for data pods. These brief instructions show a deployment using the `StatefulSet` and `StorageClass`.

```
kubectl create -f ../es-discovery-svc.yaml
kubectl create -f ../es-svc.yaml
kubectl create -f ../es-master.yaml
kubectl rollout status -f ../es-master.yaml
kubectl create -f es-master-stateful.yaml
kubectl rollout status -f es-master-stateful.yaml
kubectl create -f ../es-ingest-svc.yaml
kubectl create -f ../es-ingest.yaml
Expand Down
2 changes: 1 addition & 1 deletion stateful/es-data-stateful.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
apiVersion: apps/v1beta1
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-data
Expand Down
79 changes: 79 additions & 0 deletions stateful/es-master-stateful.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-master
labels:
component: elasticsearch
role: master
spec:
replicas: 3
template:
metadata:
labels:
component: elasticsearch
role: master
spec:
initContainers:
- name: init-sysctl
image: busybox:1.27.2
command:
- sysctl
- -w
- vm.max_map_count=262144
securityContext:
privileged: true
containers:
- name: es-master
image: quay.io/pires/docker-elasticsearch-kubernetes:6.3.0
env:
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: CLUSTER_NAME
value: myesdb
- name: NUMBER_OF_MASTERS
value: "2"
- name: NODE_MASTER
value: "true"
- name: NODE_INGEST
value: "false"
- name: NODE_DATA
value: "false"
- name: HTTP_ENABLE
value: "false"
- name: ES_JAVA_OPTS
value: -Xms256m -Xmx256m
- name: PROCESSORS
valueFrom:
resourceFieldRef:
resource: limits.cpu
resources:
requests:
cpu: 0.25
limits:
cpu: 1
ports:
- containerPort: 9300
name: transport
livenessProbe:
tcpSocket:
port: transport
initialDelaySeconds: 20
periodSeconds: 10
volumeMounts:
- name: storage
mountPath: /data
volumeClaimTemplates:
- metadata:
name: storage
spec:
storageClassName: standard
accessModes: [ ReadWriteOnce ]
resources:
requests:
storage: 2Gi

0 comments on commit 6b8050f

Please sign in to comment.