Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug Fixes in sampler.py to correct sequence score mismatch #14

Open
carlossolugen opened this issue Oct 29, 2024 · 0 comments
Open

Bug Fixes in sampler.py to correct sequence score mismatch #14

carlossolugen opened this issue Oct 29, 2024 · 0 comments

Comments

@carlossolugen
Copy link

I identified and fixed several bugs in the sampler.py file which resulted in:

  1. Accepting proposed sequences even when accepted is False.
  2. Lost alignment between the sequences in the chains and their scores.
    In combination, these issues resulted in a mismatch between the scores returned by the sampler, and those obtained by rescoring the same sequences with the model used for sampling.

The fixes make sure that only proposed sequences for which accepted is True are accepted, and that the sampler state is correctly updated to reflect this, ensuring ongoing alignment between the sequences and their scores throughout the optimization.

To reproduce the issue, it suffices to run EvoProtGrad with any expert, e.g., ESM2, to produce all the sequences and scores found, and then rescoring the sequences with the same expert. You will find that a decent percent of sequences end up with non-matching scores.

I will create a pull request shortly with my bugfixes to sampler.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant