Using helm controller in k3s in docker, helm-install-xxx-c5czq pod CrashLoopBackOff

[zzzhangqi]

Environmental Info:
K3s Version: v1.30.4+k3s1

Node(s) CPU architecture, OS, and Version:

[root@localhost ~]# cat /etc/os-release
NAME="Kylin Linux Advanced Server"
VERSION="V10 (Tercel)"
ID="kylin"
VERSION_ID="V10"
PRETTY_NAME="Kylin Linux Advanced Server V10 (Tercel)"
ANSI_COLOR="0;31"

uname -a
Linux localhost.localdomain 4.19.90-23.8.v2101.ky10.x86_64 #1 SMP Mon May 17 17:08:34 CST 2021 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: Single node

Describe the bug:

kubectl get pod -A
NAMESPACE     NAME                                  READY   STATUS             RESTARTS         AGE
kube-system   coredns-576bfc4dc7-fhs6t              1/1     Running            1 (85m ago)      91m
kube-system   metrics-server-557ff575fb-njp72       1/1     Running            1 (85m ago)      91m
rbd-system    helm-install-rainbond-cluster-c5czq   0/1     CrashLoopBackOff   27 (2m13s ago)   91m

kubectl describe pod -n rbd-system helm-install-rainbond-cluster-c5czq
Name:             helm-install-rainbond-cluster-c5czq
Namespace:        rbd-system
Priority:         0
Service Account:  helm-rainbond-cluster
Node:             node/172.17.0.2
Start Time:       Tue, 11 Feb 2025 12:02:46 +0000
Labels:           batch.kubernetes.io/controller-uid=b0b2081d-962d-49fc-b701-94da26c1c49c
                  batch.kubernetes.io/job-name=helm-install-rainbond-cluster
                  controller-uid=b0b2081d-962d-49fc-b701-94da26c1c49c
                  helmcharts.helm.cattle.io/chart=rainbond-cluster
                  job-name=helm-install-rainbond-cluster
Annotations:      helmcharts.helm.cattle.io/configHash: SHA256=57CD1CACB4A09C82C9BF42EB1E7F0DD21CF508B3D8F47A85B98F7B72E772F328
Status:           Running
SeccompProfile:   RuntimeDefault
IP:               10.42.0.6
IPs:
  IP:           10.42.0.6
Controlled By:  Job/helm-install-rainbond-cluster
Containers:
  helm:
    Container ID:  containerd://d165730eae5bdfbb57877656634e83398bd5290418894f205c55f101bdba3f93
    Image:         rancher/klipper-helm:v0.8.4-build20240523
    Image ID:      sha256:0c1204a22f1327aaf550fa4c9ba2f9bfd8935d60aeaa3b29dfe3a41833b2c0a0
    Port:          <none>
    Host Port:     <none>
    Args:
      install
      --namespace
      rbd-system
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       StartError
      Message:      failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "entry": executable file not found in $PATH: unknown
      Exit Code:    128
      Started:      Thu, 01 Jan 1970 00:00:00 +0000
      Finished:     Tue, 11 Feb 2025 13:31:44 +0000
    Ready:          False
    Restart Count:  27
    Environment:
      NAME:                   rainbond-cluster
      VERSION:
      REPO:
      HELM_DRIVER:            secret
      CHART_NAMESPACE:        rbd-system
      CHART:                  https://%{KUBERNETES_API}%/static/rainbond-cluster.tgz
      HELM_VERSION:
      TARGET_NAMESPACE:       rbd-system
      AUTH_PASS_CREDENTIALS:  false
      NO_PROXY:               .svc,.cluster.local,10.42.0.0/16,10.43.0.0/16
      FAILURE_POLICY:         reinstall
    Mounts:
      /chart from content (rw)
      /config from values (rw)
      /home/klipper-helm/.cache from klipper-cache (rw)
      /home/klipper-helm/.config from klipper-config (rw)
      /home/klipper-helm/.helm from klipper-helm (rw)
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bb72r (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 True
  Ready                       False
  ContainersReady             False
  PodScheduled                True
Volumes:
  klipper-helm:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  klipper-cache:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  klipper-config:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  values:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  chart-values-rainbond-cluster
    Optional:    false
  content:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      chart-content-rainbond-cluster
    Optional:  false
  kube-api-access-bb72r:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason   Age                 From     Message
  ----     ------   ----                ----     -------
  Warning  BackOff  7s (x397 over 85m)  kubelet  Back-off restarting failed container helm in pod helm-install-rainbond-cluster-c5czq_rbd-system(708b47ce-2b41-4473-8efe-b838ad7d7462)

[zzzhangqi]

$ docker version
Client:
 Version:           18.09.0
 EulerVersion:      18.09.0.100
 API version:       1.39
 Go version:        go1.15.7
 Git commit:        62eb848
 Built:             Thu Dec 26 02:12:43 2024
 OS/Arch:           linux/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.0
  EulerVersion:     18.09.0.100
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.15.7
  Git commit:       62eb848
  Built:            Thu Dec 26 02:12:01 2024
  OS/Arch:          linux/amd64
  Experimental:     false

All images are automatically imported via /var/lib/rancher/k3s/agent/images/xxx.tar

[brandond]

failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "entry": executable file not found in $PATH: unknown

$ docker version
Client:
Version: 18.09.0

Why do you have Docker installed on this node? K3s does not use docker, you should remove it from this node. Also that is a terribly old version to have around in the first place.

If you still have problems without Docker present on the node, confirm that your airgap image tarball has not been corrupted and is for the correct architecture. The error message indicates that the entrypoint is missing from the image, which indicates a problem with the image or containerd image store.