Creating Event when in from_thread.run_sync() context

[tapetersen]

When using to_thread.run_sync and then calling back again to the event loop with from_thread.run_sync I can't create an anyio.Event since sniffio fails to detect the running loop (no current_task is set in asyncio which of course is kindof true).

Reproduced in a gist here https://gist.github.com/tapetersen/e85a808f0bba7dd396fa68eafb41012d

[tapetersen]

This is not critical for us and can be worked around by creating asyncio.Event objects directly (not ideal to tie to backend but ok in our case) or by making the function async which I would want to avoid since it doesn't really yield to the event loop or await anything.

from_thread here is really just used to ensure thread safety since the method modifies some other state that isn't explicitly locked but relies on only being modified by the main thread running the loop.

I don't know about triobut at least in asyncio it has been ok to create Event objects outside async contexts and also in different event loops as they don't actually bind to a loop until waited upon since 3.9

[agronholm]

Overall, using from_thread() for the purpose of ensuring thread safety already sounds a bit dubious. I could probably elaborate if you are willing to do the same regarding the actual code being run.

[tapetersen]

I can elaborate about the usecase below but what I was surpised by and I don't understand is why get_current_task is used instead of get_running_loop to detect asyncio since the latter should guarantee that asyncio is the backend as much as the former and will work in cases where a current task isn't actually needed as here. We're in an executing eventloop but in a "sync task" rather than an async one.

Basically it's a framework for running test-sequences that will run parallell in threads and the main thread runs an operation and resource scheduler that is implemented using anyIO. The sequences themself run synchronous in their threads but need to schedule operations that are handled by the scheduler.

The schedule operation doesn't need to yield since it only creates an operation handle and depending on if it's possible starts the operation with a call to start_soon on a task_group owned by the scheduler. In this case the operation handle has an event that is used to wait for some operation to be done and I would prefer to create that at the time the handle is to avoid having it Optional.

While making it async and changing run_sync to run would work fine here It doesn't feel like it should be needed as it will never yield and If nothing I find that as a good marker that it is atomik with respect to the schedulers data-structures.

[tapetersen]

Regarding using from_threadfor synchronization instead of locks is a question of keeping it simple without mixing synchronization primitives between async and threading. My intuition is that from thread is relatively expensive in latency from the threads perspective but that it would let the scheduler have good throughput without handling fine-grained locks which is the priority in our use case (sequences are mostly waiting on operations and handling the synchonization and messaging between the instrument processes is the expected bottleneck.

[agronholm]

Ok, so I think I initially misunderstood the issue a bit. You could state your case for better asyncio detection on the sniffio issue tracker. I wonder if AnyIO should propagate the context in run_sync(), as that would potentially solve this for you.

[tapetersen]

I opened an issue python-trio/sniffio#35 and submitted a PR python-trio/sniffio#36 to sniffIO if you want to take a look