Subpath Crawler

[wpreimes]

In order to automate data updates, some of our data packages (gldas, ecmwf_models, ...) should contain functions to go through existing data structures and determine what data (start date, end date etc.) is already stored and what data is missing. E.g. in the gldas package there are functions for that https://github.com/TUW-GEO/gldas/blob/75ca48f620c1b64d7c6246f081aaa6924834b7ff/gldas/download.py#L43 and https://github.com/TUW-GEO/gldas/blob/75ca48f620c1b64d7c6246f081aaa6924834b7ff/gldas/download.py#L119

Without looking too much into this package now, is this something that could fit here? It would be nice if I don't have to add functions as the ones above to all our packages because that would mean a lot of duplicate code.

[sebhahn]

I think this is the right place