Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server-side connection management #355

Open
lucaspin opened this issue Mar 25, 2024 · 4 comments
Open

Server-side connection management #355

lucaspin opened this issue Mar 25, 2024 · 4 comments

Comments

@lucaspin
Copy link

Is your feature request related to a problem? Please describe.

I'm running into an issue regarding load balancing fairness. The situation is:

  • On the server side of the equation, I have multiple replicas of a GRPC.Server, running behind a Kubernetes headless service.
  • On the client side of the equation, I have a gRPC gateway translating HTTP calls into gRPC calls that go into the gRPC server above. The gRPC gateway is configured to use the round_robin load balancing policy, so each gRPC request is sent to one of the known IPs exposed by the headless Kubernetes service.

Everything works well and load is distributed fairly until the number of replicas for the gRPC server is scaled up. When that happens, the new pods receive no traffic unless I restart the gRPC clients, forcing them to re-establish the connections again.

Describe the solution you'd like

One of the solutions to this problem is described in the server-side connection management gRPC proposal. For this particular issue with load balancing fairness, having the ability to specify the MAX_CONNECTION_AGE and MAX_CONNECTION_AGE_GRACE options for the gRPC server could help force the clients to re-create connections after a certain period of time.

Describe alternatives you've considered

Another alternative would be to handle the re-creation of connections on the client side. The drawback with that is there's no way to handle that easily unless you take full control of the connection management on the client side, which is not always possible, especially when using libs like grpc-gateway.

Additional context

For gRPC servers implemented in Go, you can specify the maximum connection age with these parameters.

@sleipnir
Copy link
Collaborator

@lucaspin A possible workaround if you have the appropriate infrastructure in your kubernetes cluster is to use layer 7 routing instead of the native Kubernetes routing which is layer 4. Istio and Cilium are examples of products that add this capability to your kubernetes cluster

@lucaspin
Copy link
Author

@sleipnir thanks for the suggestion. That's certainly something I'd like to use, but right now it wouldn't be possible to add a layer 7 load balancer in that mix, at least not without complicating things a bit.

@sleipnir
Copy link
Collaborator

Is your feature request related to a problem? Please describe.

I'm running into an issue regarding load balancing fairness. The situation is:

  • On the server side of the equation, I have multiple replicas of a GRPC.Server, running behind a Kubernetes headless service.
  • On the client side of the equation, I have a gRPC gateway translating HTTP calls into gRPC calls that go into the gRPC server above. The gRPC gateway is configured to use the round_robin load balancing policy, so each gRPC request is sent to one of the known IPs exposed by the headless Kubernetes service.

Just a comment unrelated to the load balancer issue. We now support HTTP transcoding, so perhaps this would eliminate the need for the gRPC gateway in your infrastructure. Documentation here

@lucaspin
Copy link
Author

@sleipnir nice, thanks for the pointer. I'll take a look at it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants