apply-k3s-agent pending on nodes with taints

[clouedoc]

I get these pods which are pending, on my server groups with taints:

Am I doing something wrong?

[vitobotta]

I converted this to a discussion since it's not an issue with the tool. Please use discussions for generic Kubernetes questions.

You need to configure deployments to tolerate taints otherwise the pods cannot be scheduled on tainted nodes.

[clouedoc]

PS: I'll make sure to open a discussion next time, if I'm not 100% sure that it's a problem with hetzner-k3s 🙂

[vitobotta]

Not sure why you think it's a problem with the tool. If you adapt the configuration to taint some nodes with custom taints, these are taints that e.g. the system upgrade controller is not aware of, and therefore the pods will remain unscheduled if there are no nodes that can satisfy the requirements of those pods. The tool installs the system upgrade controller, the CSI driver and the CCM as they are provided by the respective authors, it doesn't change them. Same thing with the autoscaler, apart from a little configuration to configure the node pools. Therefore if you use taints that are not recognized by e.g. the system upgrade controller, it's expected that its pods will remain unscheduled.

[clouedoc]

Here's why: I expected hetzner-k3s to automatically configure the built-in tools to tolerate user taints because I believe that they are critical to the maintenance of the system -- it seemed important to me that these pods get scheduled even on nodes with taints. I was concurrently having random nodes disconnecting from the cluster, which made me suspect these pending pods (turns out that it was just a resource usage issue)

However, I understand your point; I should've expected that using taints might prevent potentially critical workloads to be scheduled on my nodes, even if I did it through hetzner-k3s itself.

To sum up: in this specific issue that I encountered; does the responsibility lie in me (not having read enough about k8s) or within the tool (is it reasonable to expect the tolerations to be created by hetzner-k3s when configuring taints through it's configuration file?)

PS: apologies for the super short original issue message... I now realise that it was not complete enough to convey my perspective

[vitobotta]

Like I mentioned the tool installs system components as they are without any modification otherwise it would be more difficult to manage them. If you want to use custom taints just keep a static pool with a single node, even small one, untainted so those components can be scheduled. Another option is to download the manifests for those components and add tolerations for your tains, and specify the paths of those manifests in the configuration file. v1.1.3 allows you to configure these paths so you can do that if you prefer.

[clouedoc]

The issue with these pods is that they cannot schedule onto other nodes; they require to run on node X, Y, Z
So using a single untainted node to run them is not an option