We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The Scheduler methods 'queue' and 'start_job' are blocking functions.
'start_new_jobs' is run in a ThreadPoolExecutor inside JobManager, but 'queue' is never run in a separate thread, and will block the event loop.
One fix would be to use 'asyncio.run_in_thread' everywhere these methods are called. An alternative is to make these methods async.
The first fix is easier to do, but also easier to footgun (forgetting to 'run_in_thread' impacts performance). Second fix would require more changes.
The text was updated successfully, but these errors were encountered:
The reason I call start_new_jobs in a ThreadPoolExecutor is because when profiling, this part took a lot of time. queue seemed to be pretty fast.
start_new_jobs
ThreadPoolExecutor
queue
Open to all improvements though!
Sorry, something went wrong.
No branches or pull requests
The Scheduler methods 'queue' and 'start_job' are blocking functions.
'start_new_jobs' is run in a ThreadPoolExecutor inside JobManager, but 'queue' is never run in a separate thread, and will block the event loop.
One fix would be to use 'asyncio.run_in_thread' everywhere these methods are called. An alternative is to make these methods async.
The first fix is easier to do, but also easier to footgun (forgetting to 'run_in_thread' impacts performance). Second fix would require more changes.
The text was updated successfully, but these errors were encountered: