Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[community-operator Hem chart] Add very useful values for pod scheduling #329

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[community-operator Hem chart] Add very useful values for pod scheduling #329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
971c30f
Add pod affinity, nodeSelector, schedulerName, tolerations & topology…
MaesterZ Jun 27, 2024
505cd9a
Spacing
MaesterZ Jun 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/community-operator/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: v2
name: community-operator
description: MongoDB Kubernetes Community Operator
version: 0.10.0
version: 0.11.0
type: application
appVersion: 0.10.0
kubeVersion: '>=1.16-0'
Expand Down
64 changes: 34 additions & 30 deletions charts/community-operator/templates/operator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ metadata:
name: {{ .Values.operator.name }}
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.operator.replicas }}
replicas: {{ int .Values.operator.replicas }}
selector:
matchLabels:
name: {{ .Values.operator.name }}
Expand All @@ -22,35 +22,24 @@ spec:
labels:
name: {{ .Values.operator.name }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- with .Values.operator.affinity }}
affinity: {{- toYaml . | nindent 8 }}
{{- end }}
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: name
operator: In
values:
- {{ .Values.operator.name }}
topologyKey: kubernetes.io/hostname
containers:
- command:
- /usr/local/bin/entrypoint
env:
{{- if .Values.operator.extraEnvs }}
{{ toYaml .Values.operator.extraEnvs | nindent 12 }}
{{- end }}
{{- with .Values.operator.extraEnvs }}
{{ toYaml . | nindent 12 }}
{{- end }}
- name: WATCH_NAMESPACE
{{- if .Values.operator.watchNamespace}}
{{- if .Values.operator.watchNamespace}}
value: "{{ .Values.operator.watchNamespace }}"
{{- else }}
{{- else }}
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- end }}
{{- end }}
- name: POD_NAME
valueFrom:
fieldRef:
Expand All @@ -70,17 +59,32 @@ spec:
image: {{ .Values.registry.operator }}/{{ .Values.operator.operatorImageName }}:{{ .Values.operator.version }}
imagePullPolicy: {{ .Values.registry.pullPolicy}}
name: {{ .Values.operator.deploymentName }}
resources:
{{- toYaml .Values.operator.resources | nindent 12 }}
{{- if .Values.operator.securityContext }}
securityContext:
{{- toYaml .Values.operator.securityContext | nindent 12 }}
{{- with .Values.operator.resources }}
resources: {{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.operator.priorityClassName }}
priorityClassName: {{ .Values.operator.priorityClassName }}
{{- with .Values.operator.securityContext }}
securityContext: {{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.nodeSelector }}
nodeSelector: {{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.priorityClassName }}
priorityClassName: {{ . | quote }}
{{- end }}
{{- if .Values.operator.podSecurityContext }}
securityContext:
{{- toYaml .Values.operator.podSecurityContext | nindent 8 }}
{{- with .Values.operator.schedulerName }}
schedulerName: {{ . | quote }}
{{- end }}
{{- with .Values.operator.podSecurityContext }}
securityContext: {{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ .Values.operator.name }}
{{- with .Values.operator.tolerations }}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be implementing tolerations for the operator, which is only a subset of the chart, right? I tried to add tolerations myself and I'm not sure how to add tolerations to the custom resource definition

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure to understand: this is to schedule the operator pod itself and not the MongoDB pods with CRDs?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I thought the intention was to add tolerations also to the mongoDB pods.

I've found a problem when deploying mongoDB to an ARM based environment in GCP. GCP taints their new ARM based nodes with a specific arm64 tag/label, and the only way I found to deploy pods into them was adding this toleration:

  tolerations:
    - key: kubernetes.io/arch
      operator: Equal
      value: arm64
      effect: NoSchedule

Unfortunately, I haven't been able to deploy mongodb to an GCP arm64 environment because of this problem. I forked this repo and added the ability to set tolerations for the operator, and that didn't fix my issue. I tried to figure how to also add tolerations to CRDs but I didn't know how.

Should I open a different issue? Am I doing something very wrong here?

Copy link
Author

@MaesterZ MaesterZ Oct 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to make some time to double check the helm chart is fully functional, I didn't try on different architectures but should not be a problem.

tolerations: {{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.topologySpreadConstraints }}
topologySpreadConstraints: {{- toYaml . | nindent 8 }}
{{- end }}
20 changes: 20 additions & 0 deletions charts/community-operator/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
imagePullSecrets: []
# - name: "image-pull-secret"

## Operator
operator:
# Name that will be assigned to most of internal Kubernetes objects like
Expand Down Expand Up @@ -48,6 +49,25 @@ operator:

securityContext: {}

affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: name
operator: In
values:
- mongodb-kubernetes-operator
topologyKey: kubernetes.io/hostname

nodeSelector: {}

schedulerName: ""

tolerations: []

topologySpreadConstraints: []

## Operator's database
database:
name: mongodb-database
Expand Down