Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[community-operator Hem chart] Add very useful values for pod scheduling #329

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/community-operator/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: v2
name: community-operator
description: MongoDB Kubernetes Community Operator
version: 0.10.0
version: 0.11.0
type: application
appVersion: 0.10.0
kubeVersion: '>=1.16-0'
Expand Down
64 changes: 34 additions & 30 deletions charts/community-operator/templates/operator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ metadata:
name: {{ .Values.operator.name }}
namespace: {{ .Release.Namespace }}
spec:
replicas: {{ .Values.operator.replicas }}
replicas: {{ int .Values.operator.replicas }}
selector:
matchLabels:
name: {{ .Values.operator.name }}
Expand All @@ -22,35 +22,24 @@ spec:
labels:
name: {{ .Values.operator.name }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- with .Values.operator.affinity }}
affinity: {{- toYaml . | nindent 8 }}
{{- end }}
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: name
operator: In
values:
- {{ .Values.operator.name }}
topologyKey: kubernetes.io/hostname
containers:
- command:
- /usr/local/bin/entrypoint
env:
{{- if .Values.operator.extraEnvs }}
{{ toYaml .Values.operator.extraEnvs | nindent 12 }}
{{- end }}
{{- with .Values.operator.extraEnvs }}
{{ toYaml . | nindent 12 }}
{{- end }}
- name: WATCH_NAMESPACE
{{- if .Values.operator.watchNamespace}}
{{- if .Values.operator.watchNamespace}}
value: "{{ .Values.operator.watchNamespace }}"
{{- else }}
{{- else }}
valueFrom:
fieldRef:
fieldPath: metadata.namespace
{{- end }}
{{- end }}
- name: POD_NAME
valueFrom:
fieldRef:
Expand All @@ -70,17 +59,32 @@ spec:
image: {{ .Values.registry.operator }}/{{ .Values.operator.operatorImageName }}:{{ .Values.operator.version }}
imagePullPolicy: {{ .Values.registry.pullPolicy}}
name: {{ .Values.operator.deploymentName }}
resources:
{{- toYaml .Values.operator.resources | nindent 12 }}
{{- if .Values.operator.securityContext }}
securityContext:
{{- toYaml .Values.operator.securityContext | nindent 12 }}
{{- with .Values.operator.resources }}
resources: {{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.operator.priorityClassName }}
priorityClassName: {{ .Values.operator.priorityClassName }}
{{- with .Values.operator.securityContext }}
securityContext: {{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.nodeSelector }}
nodeSelector: {{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.priorityClassName }}
priorityClassName: {{ . | quote }}
{{- end }}
{{- if .Values.operator.podSecurityContext }}
securityContext:
{{- toYaml .Values.operator.podSecurityContext | nindent 8 }}
{{- with .Values.operator.schedulerName }}
schedulerName: {{ . | quote }}
{{- end }}
{{- with .Values.operator.podSecurityContext }}
securityContext: {{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ .Values.operator.name }}
{{- with .Values.operator.tolerations }}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be implementing tolerations for the operator, which is only a subset of the chart, right? I tried to add tolerations myself and I'm not sure how to add tolerations to the custom resource definition

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure to understand: this is to schedule the operator pod itself and not the MongoDB pods with CRDs?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I thought the intention was to add tolerations also to the mongoDB pods.

I've found a problem when deploying mongoDB to an ARM based environment in GCP. GCP taints their new ARM based nodes with a specific arm64 tag/label, and the only way I found to deploy pods into them was adding this toleration:

  tolerations:
    - key: kubernetes.io/arch
      operator: Equal
      value: arm64
      effect: NoSchedule

Unfortunately, I haven't been able to deploy mongodb to an GCP arm64 environment because of this problem. I forked this repo and added the ability to set tolerations for the operator, and that didn't fix my issue. I tried to figure how to also add tolerations to CRDs but I didn't know how.

Should I open a different issue? Am I doing something very wrong here?

Copy link
Author

@MaesterZ MaesterZ Oct 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to make some time to double check the helm chart is fully functional, I didn't try on different architectures but should not be a problem.

tolerations: {{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.operator.topologySpreadConstraints }}
topologySpreadConstraints: {{- toYaml . | nindent 8 }}
{{- end }}
20 changes: 20 additions & 0 deletions charts/community-operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
imagePullSecrets: []
# - name: "image-pull-secret"

## Operator
operator:
# Name that will be assigned to most of internal Kubernetes objects like
Expand Down Expand Up @@ -48,6 +49,25 @@ operator:

securityContext: {}

affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: name
operator: In
values:
- mongodb-kubernetes-operator
topologyKey: kubernetes.io/hostname

nodeSelector: {}

schedulerName: ""

tolerations: []

topologySpreadConstraints: []

## Operator's database
database:
name: mongodb-database
Expand Down