Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-threading preprocess #76

Open
RoyJames opened this issue May 17, 2018 · 0 comments
Open

Multi-threading preprocess #76

RoyJames opened this issue May 17, 2018 · 0 comments

Comments

@RoyJames
Copy link

The preprocess (as well as converter) basically iterates and extracts features for each file independently. This can be easily multi-threaded with python on a multi-core machine or bigger clusters and get linear speedup. This can be beneficial especially when you want to experiment with preprocess hyperparameters. I can possibly do that (not sure when) if nobody has such plan yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant