Skip to content

Conversation

@mkmks
Copy link
Contributor

@mkmks mkmks commented Oct 20, 2025

Description of changes

Currently, we use chains of proxies to enable network access in the enclave. Thus, it takes kms-server two hops (TCP→vsock→TCP) to communicate with an external API (e.g., AWS) and three hops (TCP→vsock→TCP→vsock→TCP) to communicate with another MPC peer.

We began investigating the move to direct IP connectivity as a solution to the performance issues observed in enclave deployments (https://github.com/zama-ai/kms-internal/issues/2633) and while the root cause wasn't the network, we still might want to ditch TCP-vsock proxies in favor of proxying IP.

The advantages are the following:

  1. potentially increased stability (a single TCP handshake per connection is better than three)
  2. potentially decreased latency (avoiding unpacking and repacking TCP payload twice)
  3. better enclave development experience due to fewer differences with non-enclave deployments

This PR makes outgoing vsock proxies unneeded by adding a TUN interface in the enclave and on the parent instance joined through vsock and enabling a masquerading NAT. Making it work in a real k8s cluster would probably require creating the parent TUN interface in a privileged container.

Supporting incoming connections would require adding DNAT rules, however, additional work would be probably needed to make them play along with k8s Service declarations.

Issue ticket number and link

Closes https://github.com/zama-ai/kms-internal/issues/2778

PR Checklist

  • Title follows conventional commits (e.g. chore: ...).
  • Tests added for every new pub item and test coverage has not decreased.
  • Public APIs and non-obvious logic documented; unfinished work marked as TODO(#issue).
  • unwrap/expect/panic only in tests or for invariant bugs (documented if present).
  • No dependency version changes OR (if changed) only minimal required fixes.
  • No architectural protocol changes OR linked spec PR/issue provided.
  • No breaking deployment config changes OR devops label + infra notified + infra-team reviewer assigned.
  • No breaking gRPC / serialized data changes OR commit marked with ! and affected teams notified.
  • No modifications to existing versionized structs OR backward compatibility tests updated.
  • No critical business logic / crypto changes OR ≥2 reviewers assigned.
  • No new sensitive data fields added OR Zeroize + ZeroizeOnDrop implemented.
  • No new public storage data OR data is verifiable (signature / digest).
  • No unsafe; if unavoidable: minimal, justified, documented, and test/fuzz covered.
  • Strongly typed boundaries: typed inputs validated at the edge; no untyped values or errors cross modules.
  • Self-review completed.

@mkmks mkmks added the devops Label to notify infrastructure of a change with the deployment or configuration in a PR label Oct 20, 2025
@cla-bot cla-bot bot added the cla-signed The CLA has been signed. label Oct 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The CLA has been signed. devops Label to notify infrastructure of a change with the deployment or configuration in a PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant