Containerd, kubelet is failing on Talos 1.7.2

[Rammurthy5]

I have talos 1.7.2 running and one of the worker nodes this morning went to NotReady state. Dmesg shows the below. Could i ask how to get this up & running again please?

x.x.xx: user: warning: [2024-06-18T09:45:03.062320046Z]: [talos] service[cri](Running): Process Process(["/bin/containerd" "--address" "/run/containerd/containerd.sock" "--config" "/etc/cri/containerd.toml"]) started with PID 193001 
x.x.xx: user: warning: [2024-06-18T09:45:03.231232046Z]: [talos] service[cri](Waiting): Error running Process(["/bin/containerd" "--address" "/run/containerd/containerd.sock" "--config" "/etc/cri/containerd.toml"]), going to restart forever: exit status 1
x.x.xx: user: warning: [2024-06-18T09:45:06.482470046Z]: [talos] controller failed {"component": "controller-runtime", "controller": "k8s.EndpointController", "error": "stat /etc/kubernetes/kubeconfig-kubelet: input/output error"}
x.x.xx: user: warning: [2024-06-18T09:45:07.740285046Z]: [talos] service[kubelet](Waiting): Error running Containerd(kubelet), going to restart forever: failed to create task: "kubelet": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: connection refused": unavailable
x.x.xx: user: warning: [2024-06-18T09:45:08.234730046Z]: [talos] service[cri](Running): Process Process(["/bin/containerd" "--address" "/run/containerd/containerd.sock" "--config" "/etc/cri/containerd.toml"]) started with PID 193020
x.x.xx: user: warning: [2024-06-18T09:45:08.383217046Z]: [talos] service[cri](Waiting): Error running Process(["/bin/containerd" "--address" "/run/containerd/containerd.sock" "--config" "/etc/cri/containerd.toml"]), going to restart forever: exit status 1

containerd logs says nothing suspicious, last few lines from the logs.

"level":"info","msg":"loading plugin \"io.containerd.grpc.v1.transfer\"...","time":"2024-06-07T10:23:24.784630141Z","type":"io.containerd.grpc.v1"}
 {"level":"info","msg":"loading plugin \"io.containerd.grpc.v1.version\"...","time":"2024-06-07T10:23:24.784640897Z","type":"io.containerd.grpc.v1"}
 {"level":"info","msg":"loading plugin \"io.containerd.internal.v1.restart\"...","time":"2024-06-07T10:23:24.784659780Z","type":"io.containerd.internal.v1"}
{"level":"info","msg":"loading plugin \"io.containerd.grpc.v1.healthcheck\"...","time":"2024-06-07T10:23:24.784706794Z","type":"io.containerd.grpc.v1"}
{"address":"/system/run/containerd/containerd.sock.ttrpc","level":"info","msg":"serving...","time":"2024-06-07T10:23:24.784907960Z"}
{"address":"/system/run/containerd/containerd.sock","level":"info","msg":"serving...","time":"2024-06-07T10:23:24.784953567Z"}
{"level":"info","msg":"containerd successfully booted in 0.034933s","time":"2024-06-07T10:23:24.784979123Z"}
 time="2024-06-07T10:29:29.591691354Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-06-07T10:29:29.591808503Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
 time="2024-06-07T10:29:29.591828496Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-06-07T10:29:29.592510124Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1

kubelet logs says some errors

{"ts":1718687682596.5786,"caller":"machine/info.go:104","msg":"Failed to get disk map: open /sys/block/nvme1c1n1/dev: no such file or directory\n"}
{"ts":1718687742670.2568,"caller":"iptables/iptables.go:575","msg":"Could not set up iptables canary","table":"mangle","chain":"KUBE-KUBELET-CANARY","err":"error creating chain \"KUBE-KUBELET-CANARY\": executable file not found in $PATH: "}