You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks in advance for any responses to my discussion.
What am I trying to do?
Spin up a k3d single node cluster (master only) and connect external k3s worker nodes to this cluster.
Why am I trying to do this?
I have found having a raspberry pi as a K3S master over time the CPU gets overworked and becomes un-realiable. So I am attempting to get some stability into my cluster without breaking the bank and use what I currently have.
I have an M1 Mac Mini that has plenty of grunt so my desire is to run the master there.
Some issues I had with other ideas:
you cannot run k3s master natively on m1 atm via install script
free hypervisor virtualization options on m1 are not so great and virtualbox which I have used in the past is a no go for M1.
I have used Parallels Desktop in the past which is good but I have found it interferes with my Line 6 Helix FX pedal (USB ports) and the helix editor causing drop outs so would rather avoid that as its annoy and not tenable.
Hence why K3D seemed like a great option as I have use it for a couple of years now for testing and it does run on Mac M1. Plus it is isolated as it runs in docker.
What have I tried?
1. Spin up a k3d master node using the following config file
Command used to spin up the cluster k3d cluster create demo --config k3d-config.yaml
apiVersion: k3d.io/v1alpha4kind: Simpleservers: 1# same as `--servers 1`kubeAPI: # same as `--api-port myhost.my.domain:6445` (where the name would resolve to 127.0.0.1)host: "192.168.0.3"# important for the `server` setting in the kubeconfighostIP: "0.0.0.0"# where the Kubernetes API will be listening onhostPort: "6445"# where the Kubernetes API listening port will be mapped to on your host systemimage: rancher/k3s:v1.23.6-k3s1token: superSecretTokenports:
- port: 8081:80# same as `--port '8080:80@loadbalancer'`nodeFilters:
- loadbalanceroptions:
k3s: # options passed on to K3s itselfextraArgs: # additional arguments passed to the `k3s server|agent` command; same as `--k3s-arg`
- arg: --tls-san=192.168.0.3nodeFilters:
- server:*
2. Attempted to connect a K3S Client (no love)
export K3S_TOKEN="superSecretToken"export K3S_URL="https://192.168.0.3:6445"export INSTALL_K3S_CHANNEL='stable'
curl -sfL https://get.k3s.io | sh -
From the K3S node I can curl the K3D master.
curl -ks https://192.168.0.3:6445/ping
pong
However if we look at the agent service it appears a CA cert issue from this output.
It says to use the "full" token but I am not sure how to go about that using K3D (details in logs below)
So this is where I get stuck.
journalctl -u k3s-agent.service --since "10 minutes ago" - extract to show what that the systemctl does not, setup of load balancer
Jul 18 22:23:25 pi6-k3sn systemd[1]: Started Lightweight Kubernetes.
Jul 18 22:23:25 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:25+10:00" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Jul 18 22:23:25 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:25+10:00" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/0519414e2a8313c0184273224578271d46c25d19a6299c8bc1780f4ec16b9a02"
Jul 18 22:23:33 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:33+10:00" level=info msg="Starting k3s agent v1.23.8+k3s2 (fe3cecc2)"
Jul 18 22:23:33 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:33+10:00" level=info msg="Running load balancer k3s-agent-load-balancer 127.0.0.1:6444 -> [192.168.0.3:6445]"
Jul 18 22:23:33 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:33+10:00" level=warning msg="Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token from the server's node-token file to enable Cluster CA validation."
Jul 18 22:23:45 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:45+10:00" level=info msg="Module overlay was already loaded"
Jul 18 22:23:45 pi6-k3sn k3s[2010182]: time="2022-07-18T22:23:45+10:00" level=info msg="Module nf_conntrack was already loaded"
systemctl status k3s-agent.service
k3s-agent.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2022-07-18 22:23:25 AEST; 13min ago
Docs: https://k3s.io
Process: 2010178 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
Process: 2010180 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 2010181 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 2010182 (k3s-agent)
Tasks: 21
Memory: 245.3M
CPU: 16.390s
CGroup: /system.slice/k3s-agent.service
├─2010182 /usr/local/bin/k3s agent
└─2010271 containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd
Jul 18 22:35:47 pi6-k3sn k3s[2010182]: time="2022-07-18T22:35:47+10:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: CA cert validation failed: Get \"https://127.0.0.1:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 22:36:07 pi6-k3sn k3s[2010182]: time="2022-07-18T22:36:07+10:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: CA cert validation failed: Get \"https://127.0.0.1:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 22:36:26 pi6-k3sn k3s[2010182]: W0718 22:36:26.353587 2010182 reflector.go:324] k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": net/http: TLS handshake timeout
Jul 18 22:36:26 pi6-k3sn k3s[2010182]: I0718 22:36:26.353992 2010182 trace.go:205] Trace[768658237]: "Reflector ListAndWatch" name:k8s.io/[email protected]/tools/cache/reflector.go:167 (18-Jul-2022 22:36:16.351) (total time: 10002ms):
Jul 18 22:36:26 pi6-k3sn k3s[2010182]: Trace[768658237]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": net/http: TLS handshake timeout 10002ms (22:36:26.353)
Jul 18 22:36:26 pi6-k3sn k3s[2010182]: Trace[768658237]: [10.002710004s] [10.002710004s] END
Jul 18 22:36:26 pi6-k3sn k3s[2010182]: E0718 22:36:26.354080 2010182 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": net/http: TLS handshake timeout
Jul 18 22:36:27 pi6-k3sn k3s[2010182]: time="2022-07-18T22:36:27+10:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: CA cert validation failed: Get \"https://127.0.0.1:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 22:36:47 pi6-k3sn k3s[2010182]: time="2022-07-18T22:36:47+10:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: CA cert validation failed: Get \"https://127.0.0.1:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 18 22:37:07 pi6-k3sn k3s[2010182]: time="2022-07-18T22:37:07+10:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: CA cert validation failed: Get \"https://127.0.0.1:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi all,
Thanks in advance for any responses to my discussion.
What am I trying to do?
Spin up a k3d single node cluster (master only) and connect external k3s worker nodes to this cluster.
Why am I trying to do this?
I have found having a raspberry pi as a K3S master over time the CPU gets overworked and becomes un-realiable. So I am attempting to get some stability into my cluster without breaking the bank and use what I currently have.
I have an M1 Mac Mini that has plenty of grunt so my desire is to run the master there.
Some issues I had with other ideas:
Hence why K3D seemed like a great option as I have use it for a couple of years now for testing and it does run on Mac M1. Plus it is isolated as it runs in docker.
What have I tried?
1. Spin up a k3d master node using the following config file
Command used to spin up the cluster
k3d cluster create demo --config k3d-config.yaml
2. Attempted to connect a K3S Client (no love)
From the K3S node I can curl the K3D master.
However if we look at the agent service it appears a CA cert issue from this output.
It says to use the "full" token but I am not sure how to go about that using K3D (details in logs below)
So this is where I get stuck.
journalctl -u k3s-agent.service --since "10 minutes ago"
- extract to show what that the systemctl does not, setup of load balancersystemctl status k3s-agent.service
Beta Was this translation helpful? Give feedback.
All reactions