Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamline the implementation of collect and caching in readers #262

Open
hunterhector opened this issue Sep 29, 2020 · 0 comments
Open
Labels
enhancement New feature or request priority: medium topic: data Issue about data loader modules and data processing related topic: interface The design and implementation of the Forte interface

Comments

@hunterhector
Copy link
Member

Is your feature request related to a problem? Please describe.
The design of collect and cache_key_function is to help develop a caching function during data reading. This design expects the users to cache the results of collect. and use the cache_key to retrieve it. But this creates the following problem:

  1. If one does not want to develop caching, this design is confusing and cumbersome to them. Even worse, they will arbitrarily implement these functions, which are not effective for caching, providing bad examples for future adopters.
  2. The test coverage for these is not enough.

Describe the solution you'd like

  1. We probably need to provide two ways to implement readers: a simple reader, or a reader with some caching ability.
  2. as requested several times, some base class for reader tests should be implemented, that automatically test these function branches (e.g. caching).
  3. And we should provide standard implementations for readers.

Describe alternatives you've considered

Additional context
Some issues are related to this, such as this one: #239

@hunterhector hunterhector added enhancement New feature or request priority: medium topic: data Issue about data loader modules and data processing related topic: interface The design and implementation of the Forte interface labels Sep 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority: medium topic: data Issue about data loader modules and data processing related topic: interface The design and implementation of the Forte interface
Projects
None yet
Development

No branches or pull requests

1 participant