Forwarding fails on the first time

[nort3x]

i think i'm not able to run this,
when i try to connect for the first time i get this:

kubevpn connect -n playground
Starting connect
Getting network CIDR from cluster info...
Getting network CIDR from CNI...
Getting network CIDR from services...
Labeling Namespace playground
Creating ServiceAccount kubevpn-traffic-manager
Creating Roles kubevpn-traffic-manager
Creating RoleBinding kubevpn-traffic-manager
Creating Service kubevpn-traffic-manager
Creating MutatingWebhookConfiguration kubevpn-traffic-manager
Creating Deployment kubevpn-traffic-manager

Pod kubevpn-traffic-manager-75dc49c46f-62pn9 is Pending...
Container Reason Message

Pod kubevpn-traffic-manager-75dc49c46f-62pn9 is Running...
Container     Reason           Message
control-plane ContainerRunning 
vpn           ContainerRunning 
webhook       ContainerRunning 

Forwarding port...
Create tun device error: ip route add 10.42.2.5/24 dev utun1: invalid argument
Failed to create tun listener: ip route add 10.42.2.5/24 dev utun1: invalid argument
Start local tun service failed: ip route add 10.42.2.5/24 dev utun1: invalid argument
Failed to connect: ip route add 10.42.2.5/24 dev utun1: invalid argument
Performing cleanup operations
No proxy resources found
Error: rpc error: code = Unknown desc = ip route add 10.42.2.5/24 dev utun1: invalid argument

second time it connects, but doesn't resolve anything (and i loose the ability to resolve any domain name)

$ kubevpn status
ID    Mode   Cluster   Kubeconfig                 Namespace    Status      Netif
0     full   default   /home/human/.kube/config   playground   Connected   utun2

$ kubectl get pods -n playground -o wide
NAME                                       READY   STATUS    RESTARTS   AGE     IP           NODE                 NOMINATED NODE   READINESS GATES
kubevpn-traffic-manager-75dc49c46f-62pn9   3/3     Running   0          4m6s    10.42.0.55   simin-master-1       <none>           <none>
pg-7746f46bcc-jg57k                        1/1     Running   0          3h19m   10.42.2.5    simin-worker-set-1   <none>           <none>
pg-admin-565659c79b-nt5nk                  1/1     Running   0          3h19m   10.42.2.6    simin-worker-set-1   <none>           <none>

$ ping 10.42.2.5
PING 10.42.2.5 (10.42.2.5) 56(84) bytes of data.
^C
--- 10.42.2.5 ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8185ms

[wencaiwulue]

@nort3x Sorry to hear that.

1. I am trying to find the reason Forwarding fails on the first time. Invalid argument normally means tun device gone while add route.
2. On the second time, can you exec into pod kubevpn-traffic-manager and ping other PodIP ? or just curl _PodIP:port to verify connection?

[nort3x]

@wencaiwulue thanks for your fast response <3

It's hard to reproduce and it sometimes connects flawlessly on the first time,
I will come back to this issue if i could pin point what the issue might be...
But to just add some context for anyone who might read this,

I see couple of tun devices being add to my interfaces (one for each failed attempt) currently i'm connecting with tun2 meaning i experienced two failures (tun0 and tun1)

when failure happens the resolve system (which mine would be systemd) doesn't function at all - doesn't resolve in cluster domain names nor external domain names, i can ping external IP addresses but i can't connect to in-cluster IP addresses (indicating the vpn is not connected but system thinks it is)

[wencaiwulue]

@wencaiwulue thanks for your fast response <3

It's hard to reproduce and it sometimes connects flawlessly on the first time, I will come back to this issue if i could pin point what the issue might be... But to just add some context for anyone who might read this,

I see couple of tun devices being add to my interfaces (one for each failed attempt) currently i'm connecting with tun2 meaning i experienced two failures (tun0 and tun1)

when failure happens the resolve system (which mine would be systemd) doesn't function at all - doesn't resolve in cluster domain names nor external domain names, i can ping external IP addresses but i can't connect to in-cluster IP addresses (indicating the vpn is not connected but system thinks it is)

Ok. i got it

1. How about connect with options netstack gvisor?
2. Which Linux system are you using？Ubuntu？
3. Which version of kubevpn are you using？

[wencaiwulue]

@wencaiwulue thanks for your fast response <3

It's hard to reproduce and it sometimes connects flawlessly on the first time, I will come back to this issue if i could pin point what the issue might be... But to just add some context for anyone who might read this,

I see couple of tun devices being add to my interfaces (one for each failed attempt) currently i'm connecting with tun2 meaning i experienced two failures (tun0 and tun1)

when failure happens the resolve system (which mine would be systemd) doesn't function at all - doesn't resolve in cluster domain names nor external domain names, i can ping external IP addresses but i can't connect to in-cluster IP addresses (indicating the vpn is not connected but system thinks it is)

if no failure happens. Does all function works fine?

[nort3x]

I will check netstack the next time it happens,

versions:

• kubevpn: 2.3.13
• server and client: Ubuntu 24.04.1
• k8s cluster single master running k3s v1.31.5+k3s1
• networking is flannel: 0.26.4
• container runtime: docker v26.1.3

when no failure happens everything work as expected.

[wencaiwulue]

I will check netstack the next time it happens,

versions:

• kubevpn: 2.3.13
• server and client: Ubuntu 24.04.1
• k8s cluster single master running k3s v1.31.5+k3s1
• networking is flannel: 0.26.4
• container runtime: docker v26.1.3

when no failure happens everything work as expected.

Got it, can you provide log file ~/.kubevpn/daemon/daemon.log? help to dignose the issue? Thanks

[nort3x]

ofcourse,
plus i'm willing to help, if you give me some clue and direction i will happily collaborate
thank you

kubevpn.zip

[wencaiwulue]

ofcourse, plus i'm willing to help, if you give me some clue and direction i will happily collaborate thank you

kubevpn.zip

Cool~, we are seek more contributor, Welcome. 🎉

1. You can startup debug mode with your favourite IDE. because kubevpn is client/server mode. we can debug server. one GRPC server is user daemon. another is sudo daemon. debug sudo daemon is ok. ref: https://github.com/kubenetworks/kubevpn?tab=readme-ov-file#contributions. the try to find why tun device is gone(maybe golang context done?/why context done).

2. We can try to set env KUBECTL_REMOTE_COMMAND_WEBSOCKETS to use websocket protocol to port-forward. use command: export KUBECTL_REMOTE_COMMAND_WEBSOCKETS=true && kubevpn quit && kubevpn connect -n playgroundref: 8658521 by this commit, i changed it to spdy, maybe should not to change it from websocket to spdy?

3. We can try to use kubectl to port-forward to check k8s cluster port-forward function is health or not?

[wencaiwulue]

@nort3x you can try new version 2.4.2, it will print more log, should help to diagnose the issue.