Skip to content

Download Function in Package Could be Multi-Threaded #312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jjacobson95 opened this issue Jan 27, 2025 · 2 comments
Open

Download Function in Package Could be Multi-Threaded #312

jjacobson95 opened this issue Jan 27, 2025 · 2 comments
Assignees
@ymahlich
Labels
enhancement New feature or request package

Comments

@jjacobson95
Copy link
Collaborator

Currently the download function is quite slow. This has become more of an issue now that there are ~100 data files to download and it takes a significant amount of time to download them all. It would be nice to multi-thread this function and download the files in parallel.
@jjacobson95 jjacobson95 added enhancement New feature or request package labels Jan 27, 2025
@sgosline
Copy link
Member

Need to look into how this would happen. Are there packages available to facilitate this? How about error checking/file fidelity?

@ymahlich
Copy link
Collaborator

We should also consider to do checksum verification of the downloaded files (see also here: https://help.figshare.com/article/how-to-use-the-figshare-api#basic-coding)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ymahlich ymahlich

Labels
enhancement New feature or request package
Projects
CoderData
Status: No status
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sgosline @ymahlich @jjacobson95