Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Feedback Objects Serializable #133

Open
acbart opened this issue Nov 14, 2024 · 0 comments
Open

Make Feedback Objects Serializable #133

acbart opened this issue Nov 14, 2024 · 0 comments

Comments

@acbart
Copy link
Collaborator

acbart commented Nov 14, 2024

A design goal that I never really wrote down or achieved is that Feedback Objects should be JSON serializable. This was to be important for data analysis, since we could dump a list of the considered feedback and process it later. But currently, non-serializable python objects are getting dumped too, and sometimes recursive data structures end up in there, and generally it just isn't reliable right now.

This is a problem for the newly proposed alchemy resolver, which might be planning to send all of the Report to a backend in order to hand it off to GPT or some other LLM. We'll need to be able to reliably (and compactly?) send the entire Report, or at least all the good bits.

The first step will be to review all of the existing feedback functions and inventory them to see who is offending and who is not. I assume there will not be a simple solution, but I expect that we can implement a kind of "data allowlist" for the fields of FFs. Basically, we mark what fields ARE safe to send and use that list to do the serialization instead of just assuming we can bundle up everything. It's possible that this can be done safely automatically, but I suspect in practice we need to be intentional in some places.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant