Error on Moving from Sequential to LocalExecutor

[sheharbano96]

I just updated my airflow setup to use LocalExecutor but it started giving this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: 'QueuedLocalWorker' object has no attribute 'wrapper'

It works fine with Sequential Executor but keeps queuing tasks with this error on the scheduler.

[Shubhisnottaken]

have you resolved this?

[hshabi]

I'm having the same issue, with macOS.
I installed it on the virtual environments and tried Python 3.11.8 / 3.10.12, but it didn't work.
SequentialExecutor worked fine.
Before the upgrade to Airflow 2.10.2, I used Airflow 2.6.3 with Python 3.8.18, and the localexecutor worked fine without errors.
Any solution on how to use localexecutor?

[pbjc]

I think the problem is that the span decorator does not use functools.wraps, making the decorated methods unable to be serialized by pickle. Removing @span from LocalWorkerBase and its child classes fixed the issue for me (ex). I'm currently using a modified LocalExecutor with AIRFLOW__CORE__EXECUTOR to get around this issue locally.

[jhb115]

I am facing the exact same issue. I hope this will be fixed soon.

[potiuk]

Can someone here submit a fix for that please after testing it locally? That would definitely help in fixing it faster.

[lok419]

same issue here, looks like MacOS only

[worst-username-ever]

I had the same issue and tried commenting out some Darwin-specific logic in the multiprocessing module in cpython 3.12, as it seemed to be deprecated by now. Specifically in those files:
https://github.com/python/cpython/blob/3.12/Lib/multiprocessing/context.py
https://github.com/python/cpython/blob/3.12/Lib/multiprocessing/reduction.py

This seemed to fix the issue for me. However, please be aware that I have absolutely no idea what this code does let alone which bugs it was trying to fix. I have almost zero Python experience so I feel unsure about submitting a PR for a change which may cause more problems than it solves.