Why does a VXLAN tunnel need to be established through the vx-submariner?

[zcz-1020]

In a cluster, the network between nodes is connected. Why do VXLAN tunnels need to be established between nodes in the cluster through the vx-submariner?

[sridhargaddam]

Submariner creates vx-submariner tunnels to the local Gateway nodes from all the other nodes in the cluster mainly to preserve the source-ip address of the Pod for inter-cluster communication. For this, it uses UDP/4800 port as captured in the pre-requisites.

[zcz-1020]

Thank you for your answer, I'm still a little confused. In my opinion, CNI can ensure that pod-to-pod or node-to-pod communication within a cluster does not need to use NAT. In this way, the source-ip address of the packet that reaches the gateway node is not changed, so the source-ip address of the pod can be preserved. Is it possible to omit the vx-submariner to establish a tunnel? After all, VXLAN encapsulation and decapsulation cause performance loss.

[sridhargaddam]

Usually the CNI will preserve the sourceIP of the traffic if the destination pod address belong to the local cluster, but when the destination address/CIDR is a external address (i.e., address of a remote cluster connected via Submariner), most of the CNIs will perform SNAT with the nodeIP when the traffic leaves the worker node. To avoid this, we send the traffic over the vx-submariner tunnel to the local gateway node.

[zcz-1020]

Thank you， I understand. I also have a question, even if the source-ip address of the pod is changed by snat, does this not affect cross-cluster access between pods? Is it understood that vx-submariner establishes a tunnel only to preserve the source IP address of the pod ，but it is not necessary?

[sridhargaddam]

I also have a question, even if the source-ip address of the pod is changed by snat, does this not affect cross-cluster access between pods?

I hope you meant, "....does this affect ....". If so, yes it would because Submariner only advertises the local cluster CIDRs and not the host CIDRs to remote cluster. So if the traffic reaches the remote cluster using the NodeIP (usually with which the SNAT is performed) the return traffic will not make its way through.

Is it understood that vx-submariner establishes a tunnel only to preserve the source IP address of the pod ，but it is not necessary?

Well, in Submariner design we want to identify the source of the traffic when it reaches the remote cluster. The sourceIP of the pod will help us identifying from where the traffic is coming from. The idea is to use this information when we support multi-cluster network policies.

[zcz-1020]

Thank you. I see, thank you very much.