What's the difference between node-token, token, agent-token?

[arches-richard]

What's the difference between the 3 tokens in /var/lib/rancher/k3s/server/? I'm trying to figure out what command I need to use to install a new K3s server (but for an existing cluster), it doesn't seem to be detailed in the quick start page.

[brandond]

https://docs.k3s.io/cli/token#token-format

The agent token is a less privileged token that can only be used to join agents, not servers.

If you don't specify a different --agent-token when starting the server, they are all the same thing:

root@k3s-server-1:/# ls -la /var/lib/rancher/k3s/server/*token
lrwxrwxrwx 1 root root  33 Mar 14 17:48 /var/lib/rancher/k3s/server/agent-token -> /var/lib/rancher/k3s/server/token
lrwxrwxrwx 1 root root  33 Mar 14 17:48 /var/lib/rancher/k3s/server/node-token -> /var/lib/rancher/k3s/server/token
-rw------- 1 root root 109 Mar 14 17:48 /var/lib/rancher/k3s/server/token

[brandond]

I'm not sure what you mean by "the installer fails". What specific sequence of events are you asking about?

[arches-richard]

Sorry, here's the journalctl logs. I believe the first failure was from forgetting to add a search domain. After I fixed that I uninstalled, reboot, and reinstall for a couple other tries. Somewhere down the line the rest of the cluster apart from the machine I was trying to add went down, kubectl just gives ServiceUnavailable, so the failed installs after that were probably just from before I noticed it. I think I should have removed the node via kubectl before running the k3s uninstaller so that might be part of the problem.

k3s.log

[brandond]

If you have a two-server cluster using embedded etcd, then your etcd cluster has no fault tolerance, and stopping either node will cause the other node to fail as well. It sounds kinda like that's what you caused when you uninstalled the second node?

[arches-richard]

Does K3S support installation inside of LXC by any chance?

[brandond]

It's not something we test, so I would say that it's not supported but could work?