Expose custom metadata from EDS response to custom LB policies

[hpathak01]

Please see the FAQ in our main README.md before submitting your issue.

Use case(s) - what problem will this feature solve?

Presently, when we create a custom load balancer, it cannot access the endpoint or locality weights because they are keyed by an internal struct, so we cannot read these from Attributes. Similarly, the "metadata" returned from EDS response is also not exposed. As a result, the custom load balancer cannot make decision based on weights or metadata

Proposed Solution

Expose at least the weights (locality weight and endpoint weight) so they can be read by custom load balancer; there is a precedence in ringash.go - clients can use ringhash.HashKey(endpoint) to get this info. We can have something similar for weights.

Alternatives Considered

There are no clean or viable alternatives for this missing data.

Additional Context

NA

[arjan-bal]

Hi @hpathak01, could you describe your specific use case? I'm trying to understand if it can be satisfied with an existing LB policy.

Also, could the request be scoped down to only the attributes you actually need? Exposing a generic Attribute through our public API means gRPC can't change it without breaking users. We need to maintain this flexibility since we frequently make internal changes to xDS-related functionality. For example, exposing the weight attribute might be easier since it's already decoupled from xDS:

grpc-go/internal/balancer/weight/weight.go

Lines 60 to 66 in 98573cb

	// FromEndpoint returns the EndpointInfo stored in the Attributes field of an
	// endpoint. It returns an empty EndpointInfo if attribute is not found.
	func FromEndpoint(endpoint resolver.Endpoint) EndpointInfo {
	v := endpoint.Attributes.Value(attributeKey{})
	ei, _ := v.(EndpointInfo)
	return ei
	}

[hpathak01]

hi @arjan-bal we are writing a custom gRPC load balancer (based on p2c deterministic aperture as described in https://blog.x.com/engineering/en_us/topics/infrastructure/2019/daperture-load-balancer) that needs the endpoint and locality weights returned from xDS (specifically EDS) response. However, we are unable to access the weight because the weight package is internal, so we cannot access weight.FromEndpoint or weight.attributeKey.

[easwars]

@hpathak01

We have an existing issue to perform exactly what you want: #8757

[arjan-bal]

@easwars, I believe the weights are already present in an attribute and are get/set using functions in this internal package:

grpc-go/internal/balancer/weight/weight.go

Lines 60 to 66 in 98573cb

	// FromEndpoint returns the EndpointInfo stored in the Attributes field of an
	// endpoint. It returns an empty EndpointInfo if attribute is not found.
	func FromEndpoint(endpoint resolver.Endpoint) EndpointInfo {
	v := endpoint.Attributes.Value(attributeKey{})
	ei, _ := v.(EndpointInfo)
	return ei
	}

For this feature request, we may only need to expose this internal/balancer/weight package to custom balancers.

[easwars]

I've started a discussion with some core gRPC maintainers about making this information available through a public API.

we are writing a custom gRPC load balancer (based on p2c deterministic aperture as described in blog.x.com/engineering/en_us/topics/infrastructure/2019/daperture-load-balancer)

Please note that this was considered in the past. See #6370.

And the LB policy that came out of it is https://github.com/grpc/proposal/blob/master/A68-random-subsetting.md, which is already implemented in grpc-go. We would be extremely happy if you want to try that out and give us feedback.

[easwars]

@hpathak01 Can you please let us know if the random_subsetting policy is something that meets your needs? We still haven't implemented the xDS plumbing for it. But that should be a minor change and we can either make it happen or we will be happy to review a PR if you are interested in contributing.