Skip to content

Ensure to Pre-Load imports in main thread #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

stefanhahmann
Copy link
Contributor

@stefanhahmann stefanhahmann commented May 8, 2025

This draft PR is a suggestion to fix apposed/appose#13

This issue is a bit difficult to debug. As already found and communiated, it only can be reproduced in Windows environments. It could not be reproduced in Linux. Situation on Mac is unclear (not yet tested on Mac), cf: apposed/appose#13 (comment)

I tried to reproduce on Windows from the command line using the env from this code example (apposed/appose#13 (comment)) and the following command

micromamba run -p C:\Users\xxxx\.local\share\appose\stardist C:\Users\xxxx\.local\share\appose\stardist\python.exe -c "import appose.python_worker; appose.python_worker.main()"

and then interactively inputing this task to stdin

{"task":"c388f270-d6be-48df-9e02-e42b48cab98b","requestType":"EXECUTE","inputs":{},"script":"import numpy\n"}

This results in a perfectly working process:

[SERVICE-0] {"task":"c388f270-d6be-48df-9e02-e42b48cab98b","requestType":"EXECUTE","inputs":{},"script":"import numpy\n"}
[SERVICE-0] {"task":"c388f270-d6be-48df-9e02-e42b48cab98b","responseType":"LAUNCH"}
[SERVICE-0] {"task":"c388f270-d6be-48df-9e02-e42b48cab98b","responseType":"COMPLETION","outputs":{}}

However, when I try to run this from a Java process similar to the code example (apposed/appose#13 (comment)), I only get this response and then the whole process is stuck in a dead lock:

[SERVICE-0] {"task":"c388f270-d6be-48df-9e02-e42b48cab98b","requestType":"EXECUTE","inputs":{},"script":"import numpy\n"}
[SERVICE-0] {"task":"c388f270-d6be-48df-9e02-e42b48cab98b","responseType":"LAUNCH"}

The code added in this PR fixes this situation. It seems that loading imports like (e.g.) numpy can lead to deadlocks when being loaded in multiple threads.

I am not sure, if the proposed fix is a good one however. Since the problem occurs only on Windows, one could think about adding if sys.platform == 'win32': or something like this to only perform the extra code on Windows. Also, one could think about making this optional and controlable from outside the python_worker by perhaps only performing the new code, if some global variable is set.

@ctrueden curious to learn your opinion about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Import of a Python module leads to task not being finished in Windows
1 participant