Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve description for custom collector/provider #74

Open
raphaelrpl opened this issue Apr 4, 2023 · 0 comments
Open

Improve description for custom collector/provider #74

raphaelrpl opened this issue Apr 4, 2023 · 0 comments
Assignees
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@raphaelrpl
Copy link
Contributor

raphaelrpl commented Apr 4, 2023

We should improve the documentation in how to create custom providers and attach it into collectors. It should contain the following entries:

  • Add example for module initialization and class for this
  • brief entrypoint management and how it is loaded by bdc-collectors

For example, consider an extension of current provider SciHub named MyProvider which uses different api location. You may have few ways to use this

  • Change runtime parameter api_url:
from bdc_collectors.scihub import SciHub

catalog = SciHub(username="user", password="pass", api_url="https://my-url-provider-hub")
catalog.search(...)

# or
# from bdc_collectors.ext import CollectorExtension
# ext = CollectorExtension(app)  # flask_app
# catalog = ext.get_provider("SciHub")(username="user", password="pass", api_url="https://my-url-provider-hub")
  • Custom Provider named MyProvider (in bdc-collectors bdc_collectors/my_provider/__init__.py or in other github repo):
from typing import Type

from ..base import BaseCollection
from ..scihub import SciHub

def init_provider():
    return dict(MyProvider=MyProviderAPI)


class MyProviderAPI(SciHub):
    def __init__(self, *args, **kwargs):
        kwargs.setdefault("api_url", "https://my-url-provider-hub")
        super(MyProviderAPI, self).__init__(*args, **kwargs)
    def get_collector(self, collection: str) -> Type[BaseCollection]:
        # ... logic for custom data collection
        return super().get_collector(collection) # optional

Register in setup.py the entrypoint for loading this provider:

entry_points={
        ...,
        'bdc_collectors.providers': [
            ...  # other providers
            'my_provider = bdc_collectors.my_provider'
        ]
}

and use it with:

from bdc_collectors.ext import CollectorExtension
ext = CollectorExtension(app)  # flask_app
catalog = ext.get_provider("MyProvider")(username="user", password="pass")
@raphaelrpl raphaelrpl added the documentation Improvements or additions to documentation label Apr 4, 2023
@raphaelrpl raphaelrpl added this to the Version 1.0 milestone Apr 4, 2023
@raphaelrpl raphaelrpl self-assigned this Apr 4, 2023
@raphaelrpl raphaelrpl modified the milestones: Version 1.0, Version 1.1 Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

1 participant