Incorporate new PheWeb manhattan plot filtering

[abought]

Our Manhattan plots draw from the PheWeb codebase, with modifications.

PheWeb recently gained the ability to filter manhattan plots by variant type and annotation. We should investigate incorporating that feature into the upload service, as well as abstracting manhattan plots in a way that better supports code sharing.

[pjvandehaar]

The python to get the top 100k variants is nice: https://github.com/statgen/pheweb/blob/master/pheweb/load/best_of_pheno.py#L35 .

The js is not great. It's almost-but-not-quite the same as the normal manhattan code, and I didn't do the work of factoring out common sections. And even before the duplication it wasn't elegant code. Somehow I never learned how to make d3 readable. https://github.com/statgen/pheweb/blob/master/pheweb/serve/static/pheno-filter.js .

[abought]

Thanks! If I do incorporate this, I'll try to modularize my JS code to be useful across both services. I've been trying to make LZ's d3 usages more readable in the upcoming release as well.

Two examples of potential code sharing:

• Provide "save as svg/png" buttons for the manhattan plot. It's a frequent request in the my.locuszoom.org service and I imagine it would be useful to pheweb too. As of 0.12, we've made the feature a bit faster and cleaner.
• Upgrade the manhattan and qq plots to d3 v5 (if pheweb isn't there already), when I upgrade pheweb to LZ 0.12-0.13. Changes here: https://github.com/statgen/locuszoom-hosted/blob/develop/assets/js/util/pheweb_plots.js

Last question: Is the "top 100k variants" code a replacement for the old manhattan binner, or a supplement used for filtering?

[pjvandehaar]

https://github.com/statgen/pheweb/blob/master/pheweb/load/best_of_pheno.py#L35 just makes a normal tsv file with only 100k variants in it. https://github.com/statgen/pheweb/blob/8f43ca0075cc14c4b58673c6c3e2dcaac0a9951d/pheweb/serve/server.py#L124 reads that file, filters variants, feeds them into the manhattan plot binner, and then sends the result to the client. During pre-processing, the same manhattan plot binner runs on the full input file to make the normal manhattan plots.