Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[superissue] Improve data handling #304

Open
3 tasks
ivan-aksamentov opened this issue Apr 29, 2022 · 0 comments
Open
3 tasks

[superissue] Improve data handling #304

ivan-aksamentov opened this issue Apr 29, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ivan-aksamentov
Copy link
Member

ivan-aksamentov commented Apr 29, 2022

This superissue tracks issues and PRs related to the improvement of data handling in CoVariants:

Goals:

  • Improve user experience by making web UI more responsive to user actions. Currently, after navigation to pages with plots, there is a few seconds where nothing happens.
  • Speedup rendering of plots. If data is split into pieces, one piece for every plot, we can make some plots rendering earlier, providing quick feedback to the user, while other plots finish rendering later.
  • Reduce amount of downloaded data. Web app can download only the required chunks.
  • Parallelize downloads. Array items can be downloaded concurrently, so that earlier chunks fully arrive and are ready to be rendered, without waiting the whole data array.
  • Reduce web app bundle size. Currently all data is bundled into JS files and they are fully downloaded on first render or on page navigation. The rendering is blocked until the whole thing arrives.
  • Cache app code and data independently. Data is changing more often than the app's code. If they are decoupled, the unmodified code can stay in user's browser's cache longer, saving bandwidth and making things faster.
  • Cache data chunks independently. If some of the elements of the data collection are unchanged during release, then there is no need to redownload it.
  • Manage data chunks separately within the app state. Each element of the data collection can be kept in a separate Recoil atom, and then this Recoil atom can be used only in the component that needs it (alternatively it can be a separate react-query query). This may allow for concurrent rendering of the components that depend only on individual data chunks.
  • Reduce size of the git repository
  • Reduce load and failure rates on CI
  • Improve developer experience
  • Improve or at least preserve scientist experience
  • Preserve correctness and feature-completeness
@ivan-aksamentov ivan-aksamentov added the enhancement New feature or request label Apr 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant