Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(eval): flag match #2979

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

anthonpetrow
Copy link

if the index does not match the length of the distribution, then we will get a flag mismatch, although this should not have affected the flag match

@anthonpetrow anthonpetrow requested a review from a team as a code owner April 12, 2024 06:46
@anthonpetrow anthonpetrow changed the title fix flag match fix(eval): flag match Apr 12, 2024
@markphelps
Copy link
Collaborator

Thanks @anthonpetrow ! could you add a little more info about how this situation could occur? like what your setup/evaluation request is?

@anthonpetrow
Copy link
Author

@markphelps
I can't reproduce such a case through a query, but the presence of the possibility to influence the distribution in the code on the segmentation of the flag doesn't sound very convincing

@GeorgeMac
Copy link
Contributor

GeorgeMac commented Apr 16, 2024

This is a bit of a fiddly one. Definitely shines a light on a subtle bit of the evaluator that needs some thought.
I don't think anything is necessarily broken here, it is perhaps just different expectations around the contract.

What you're saying @anthonpetrow with this change makes sense on paper. The constraints / rules has matched the context to a segment. It is just that we cannot identify an assocated variant on the flag for the matched segment, since none of them fall into any identified distribution.

The thing we have to decide is whether changing this creates an unexpected / undesirable break in behaviour for folks. Currently, (without this change) there is an implicit assumption you can make, which is that matched == true implies a variant key is present in .Value.

Can we safely break that assumption @markphelps ? I worry we might not.

Putting on the retrospective glasses, perhaps the concept of a default variant would've helped here. Or perhaps some validation that the entire 100% of buckets point to a valid variant in a distribution. However, this kind of change is hard to retrofit.

Or perhaps the existing behaviour is still reasonable and that perhaps it just needs to be clearer in the documentation.
That being that: adding distributions to a rule implies a further condition that could cause the match to fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants