Skip to content

Conversation

nirs
Copy link
Contributor

@nirs nirs commented Aug 15, 2025

krunkit-1.0.0[1] allows disabling offloading for faster networking. Keep the Offloading option in case offloading is improved in future versions or we want to add a flag to use int for specific workload.

Testing shows 6.7 times faster network performance, and 9p mount is 3 times faster.

Starting cluster:

% minikube start --driver krunkit --container-runtime containerd
😄  minikube v1.36.0 on Darwin 15.6 (arm64)
✨  Using the krunkit (experimental) driver based on user configuration
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🔥  Creating krunkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
📦  Preparing Kubernetes v1.33.2 on containerd 1.7.23 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Testing iperf3:

% kubectl apply -f iper3-server.yaml
deployment.apps/iperf3 created
service/iperf3 created

% kubectl get deploy iperf3
NAME     READY   UP-TO-DATE   AVAILABLE   AGE
iperf3   1/1     1            1           9s

% kubectl get service iperf3
NAME     TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
iperf3   NodePort   10.105.127.180   <none>        5201:30201/TCP   17s

% iperf3 -c $(minikube ip) -p 30201
Connecting to host 192.168.105.10, port 30201
[  5] local 192.168.105.1 port 50630 connected to 192.168.105.10 port 30201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.05 GBytes  9.03 Gbits/sec
[  5]   1.00-2.01   sec  1.09 GBytes  9.29 Gbits/sec
[  5]   2.01-3.01   sec  1.06 GBytes  9.09 Gbits/sec
[  5]   3.01-4.00   sec  1.08 GBytes  9.33 Gbits/sec
[  5]   4.00-5.00   sec  1.08 GBytes  9.31 Gbits/sec
[  5]   5.00-6.00   sec  1.07 GBytes  9.23 Gbits/sec
[  5]   6.00-7.00   sec  1.09 GBytes  9.35 Gbits/sec
[  5]   7.00-8.01   sec  1.08 GBytes  9.20 Gbits/sec
[  5]   8.01-9.00   sec  1.07 GBytes  9.18 Gbits/sec
[  5]   9.00-10.01  sec  1.08 GBytes  9.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  sender
[  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  receiver

Testing 9p mount:

% minikube mount ~/models:/mnt/models
📁  Mounting host path /Users/nir/models into VM as /mnt/models ...
    ▪ Mount type:   9p
    ▪ User ID:      docker
    ▪ Group ID:     docker
    ▪ Version:      9p2000.L
    ▪ Message Size: 262144
    ▪ Options:      map[]
    ▪ Bind Address: 192.168.105.1:50614
🚀  Userspace file server:
ufs starting
✅  Successfully mounted /Users/nir/models to /mnt/models

📌  NOTE: This process must stay alive for the mount to be accessible ...

$ time cat /mnt/models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf >/dev/null

real    0m10.219s
user    0m0.007s
sys     0m0.263s

[1] https://github.com/containers/krunkit/releases/tag/v1.0.0

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Aug 15, 2025
@k8s-ci-robot k8s-ci-robot requested review from medyagh and prezha August 15, 2025 17:00
@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 15, 2025
@nirs nirs force-pushed the krunkit-no-offloading branch 2 times, most recently from dfb2dfd to 390e6a1 Compare August 24, 2025 14:23
@nirs nirs force-pushed the krunkit-no-offloading branch 2 times, most recently from 089b576 to 301051f Compare September 2, 2025 13:37
krunkit-1.0.0[1] allows disabling offloading for faster networking. Keep
the Offloading option in case offloading is improved in future versions
or we want to add a flag to use int for specific workload.

Testing shows 6.7 times faster network performance, and 9p mount
is 3 times faster.

Starting cluster:

    % minikube start --driver krunkit --container-runtime containerd
    😄  minikube v1.36.0 on Darwin 15.6 (arm64)
    ✨  Using the krunkit (experimental) driver based on user configuration
    👍  Starting "minikube" primary control-plane node in "minikube" cluster
    🔥  Creating krunkit VM (CPUs=2, Memory=6144MB, Disk=20000MB) ...
    📦  Preparing Kubernetes v1.33.2 on containerd 1.7.23 ...
    🔗  Configuring bridge CNI (Container Networking Interface) ...
    🔎  Verifying Kubernetes components...
        ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
    🌟  Enabled addons: default-storageclass, storage-provisioner
    🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

Testing iperf3:

    % kubectl apply -f iper3-server.yaml
    deployment.apps/iperf3 created
    service/iperf3 created

    % kubectl get deploy iperf3
    NAME     READY   UP-TO-DATE   AVAILABLE   AGE
    iperf3   1/1     1            1           9s

    % kubectl get service iperf3
    NAME     TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
    iperf3   NodePort   10.105.127.180   <none>        5201:30201/TCP   17s

    % iperf3 -c $(minikube ip) -p 30201
    Connecting to host 192.168.105.10, port 30201
    [  5] local 192.168.105.1 port 50630 connected to 192.168.105.10 port 30201
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-1.00   sec  1.05 GBytes  9.03 Gbits/sec
    [  5]   1.00-2.01   sec  1.09 GBytes  9.29 Gbits/sec
    [  5]   2.01-3.01   sec  1.06 GBytes  9.09 Gbits/sec
    [  5]   3.01-4.00   sec  1.08 GBytes  9.33 Gbits/sec
    [  5]   4.00-5.00   sec  1.08 GBytes  9.31 Gbits/sec
    [  5]   5.00-6.00   sec  1.07 GBytes  9.23 Gbits/sec
    [  5]   6.00-7.00   sec  1.09 GBytes  9.35 Gbits/sec
    [  5]   7.00-8.01   sec  1.08 GBytes  9.20 Gbits/sec
    [  5]   8.01-9.00   sec  1.07 GBytes  9.18 Gbits/sec
    [  5]   9.00-10.01  sec  1.08 GBytes  9.28 Gbits/sec
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bitrate
    [  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  sender
    [  5]   0.00-10.01  sec  10.7 GBytes  9.23 Gbits/sec                  receiver

Testing 9p mount:

    % minikube mount ~/models:/mnt/models
    📁  Mounting host path /Users/nir/models into VM as /mnt/models ...
        ▪ Mount type:   9p
        ▪ User ID:      docker
        ▪ Group ID:     docker
        ▪ Version:      9p2000.L
        ▪ Message Size: 262144
        ▪ Options:      map[]
        ▪ Bind Address: 192.168.105.1:50614
    🚀  Userspace file server:
    ufs starting
    ✅  Successfully mounted /Users/nir/models to /mnt/models

    📌  NOTE: This process must stay alive for the mount to be accessible ...

    $ time cat /mnt/models/DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf >/dev/null

    real    0m10.219s
    user    0m0.007s
    sys     0m0.263s

[1] https://github.com/containers/krunkit/releases/tag/v1.0.0
@nirs nirs force-pushed the krunkit-no-offloading branch from 301051f to 890003c Compare September 3, 2025 15:15
@nirs nirs marked this pull request as ready for review September 3, 2025 15:17
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 3, 2025
@nirs nirs requested a review from medyagh September 3, 2025 15:17
@nirs
Copy link
Contributor Author

nirs commented Sep 3, 2025

@medyagh this is ready for merge

@medyagh
Copy link
Member

medyagh commented Sep 3, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Sep 3, 2025
Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can merge after full tests are back :)

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, nirs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 3, 2025
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21341 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 49.1s    │ 48.9s                  │
│ enable ingress │ 15.4s    │ 15.6s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube start: 51.2s 46.7s 47.5s 48.5s 51.5s
Times for minikube (PR 21341) start: 49.8s 47.9s 49.9s 49.6s 47.5s

Times for minikube ingress: 14.8s 15.8s 15.8s 15.8s 14.8s
Times for minikube (PR 21341) ingress: 15.8s 15.8s 15.8s 14.8s 15.8s

docker driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21341 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 24.3s    │ 23.5s                  │
│ enable ingress │ 14.4s    │ 15.3s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube ingress: 13.6s 12.7s 13.6s 18.6s 13.6s
Times for minikube (PR 21341) ingress: 13.6s 13.6s 18.7s 17.6s 13.1s

Times for minikube start: 22.1s 22.8s 27.3s 22.9s 26.2s
Times for minikube (PR 21341) start: 22.2s 23.8s 25.9s 23.2s 22.5s

docker driver with containerd runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21341 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 22.1s    │ 22.8s                  │
│ enable ingress │ 30.5s    │ 24.7s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube start: 21.9s 23.1s 22.2s 22.6s 20.8s
Times for minikube (PR 21341) start: 24.0s 22.8s 21.1s 21.7s 24.7s

Times for minikube ingress: 24.2s 40.1s 40.1s 24.1s 24.1s
Times for minikube (PR 21341) ingress: 24.1s 24.1s 23.6s 24.1s 27.6s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants