Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler to fetch data from Git repositories #645

Closed
sduenas opened this issue Apr 1, 2024 · 1 comment
Closed

Scheduler to fetch data from Git repositories #645

sduenas opened this issue Apr 1, 2024 · 1 comment

Comments

@sduenas
Copy link
Member

sduenas commented Apr 1, 2024

The current version of the platform needs of several instances for analyzing more than 5000 data sources. For example, for a project with around 3500 high-activity repositories that retrieves data from GitHub (commits, issues and pull requests), the platform needs 3 days to start analyzing new data.

The goal is to start working on a new scheduler that allows to scale the platform according to what's defined on the current version of the roadmap.

The first iteration of this scheduler will be to add support for fetching data from Git repositories.

@sduenas
Copy link
Member Author

sduenas commented Feb 21, 2025

Our first version of the new scheduler supports the retrieval of git repositories, transforming the commits into events.

@sduenas sduenas closed this as completed Feb 21, 2025
@sduenas sduenas moved this from Near Term to Delivered in GrimoireLab Roadmap Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Delivered
Development

No branches or pull requests

1 participant