Skip to content

VPN node IP not being added to TLS cert for kubelet logs endpoint #13659

@cdanis

Description

@cdanis

Environmental Info:
K3s Version:
k3s version v1.34.4+k3s1 (c601791)
go version go1.24.12

Node(s) CPU architecture, OS, and Version:
Linux fuzzydunlop 6.12.69+deb13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.69-1 (2026-02-08) x86_64 GNU/Linux

Cluster Configuration:
5 bare-metal servers (and 0 agents), all running Debian trixie. Also using Tailscale.

Describe the bug:

Container logs cannot be viewed, as the endpoint that provides container logs does not have the node VPN IPs as part of its TLS SANs:

💙cdanis@kima ~ 🕑☕ kubectl -n gadget logs gadget-q698x                 
Error from server: Get "https://100.101.207.31:10250/containerLogs/gadget/gadget-q698x/gadget": tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, ::1, 192.168.1.20, not 100.101.207.31

Steps To Reproduce:

  • Installed K3s: using the usual get.k3s.io script.
    Possibly-relevant configuration options:
--vpn-auth=name=tailscale,joinKey=tskey-redacted-.... \
--embedded-registry \
--disable=servicelb \
--tls-san=k8sapi \
--tls-san=k8sapi.bobcat-beta.ts.net \
--tls-san=192.168.1.41

One other thing possibly of note -- this is a long-lived cluster, running system-update-controller against the stable release. The automatic upgrade to v1.34.4+k3s1, which incorporates #13457, did not fix this issue.

Expected behavior:
kubectl logs should work and return container logs.

Actual behavior:
TLS errors about missing VPN IP SANs are returned when fetching logs from any container running on any node.

Additional context / logs:
I do see that the VPN node IP is being provided to the kubelet, but perhaps this code path is not triggered before certificate generation?

Feb 16 15:29:09 fuzzydunlop k3s[2019274]: time="2026-02-16T15:29:09-05:00" level=info msg="Node-ip changed to [100.101.207.31 fd7a:115c:a1e0:ab12:4843:cd96:6265:cf1f] due to VPN"
Feb 16 15:29:17 fuzzydunlop k3s[2019274]: time="2026-02-16T15:29:17-05:00" level=info msg="Running kubelet --cloud-provider=external --config-dir=/var/lib/rancher/k3s/agent/etc/kubelet.conf.d --containerd=/run/k3s/containerd/containerd.sock --hostname-override=fuzzydunlop --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=100.101.207.31,fd7a:115c:a1e0:ab12:4843:cd96:6265:cf1f --node-labels= --read-only-port=0"

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

Status

Accepted

Relationships

None yet

Development

No branches or pull requests

Issue actions