Async client uses blocking call when uploading file with v2

[tkdrob]

This project is used to support the Slack integration for the Home-Assistant project. Home-Assistant uses a single event loop and prevents any calls in the event loop that are blocking.

The Slack SDK version

3.27.1

Python runtime version

3.12.3

OS info

ProductName: macOS
ProductVersion: 14.4.1
BuildVersion: 23E224

Steps to reproduce:

Check out the pr below for context. We would still use upload v1 until the upload can be fully async.
home-assistant/core#116868

Expected result:

The files should ideally be uploaded in an asynchronous manner both for local files and remote urls using aiohttp with the ClientSession object give by AsyncBaseClient.session.

Actual result:

The call to putrequest would block the event loop and so it cannot move forward.

logs:

Traceback (most recent call last):
  File "/core/homeassistant/helpers/script.py", line 512, in _async_step
    await getattr(self, handler)()
  File "/core/homeassistant/helpers/script.py", line 747, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/helpers/script.py", line 710, in _async_run_long_action
    return await long_task
           ^^^^^^^^^^^^^^^
  File "/core/homeassistant/core.py", line 2725, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/core/homeassistant/core.py", line 2766, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/components/notify/legacy.py", line 275, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/core/homeassistant/components/slack/notify.py", line 320, in async_send_message
    return await self._async_send_remote_file_message(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/components/slack/notify.py", line 220, in _async_send_remote_file_message
    await asyncio.gather(
  File "/core/venv/lib/python3.12/site-packages/slack_sdk/web/async_client.py", line 3587, in files_upload_v2
    upload_result = _upload_file_via_v2_url(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/venv/lib/python3.12/site-packages/slack_sdk/web/internal_utils.py", line 385, in _upload_file_via_v2_url
    resp = urlopen(req, context=ssl, timeout=timeout)  # skipcq: BAN-B310
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 515, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 532, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 1392, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 1344, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1336, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1347, in _send_request
    self.putrequest(method, url, **skips)
  File "/core/homeassistant/util/loop.py", line 136, in protected_loop_func
    check_loop(
  File "/core/homeassistant/util/loop.py", line 82, in check_loop
    raise RuntimeError(  # noqa: TRY200
RuntimeError: Detected blocking call to putrequest inside the event loop in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py, line 1347: self.putrequest(method, url, **skips). Use `await hass.async_add_executor_job()`; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue
2024-05-12 14:38:59.670 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [5200108480] Error handling message: Unknown error (unknown_error) test from 127.0.0.1 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36)
Traceback (most recent call last):
  File "/core/homeassistant/util/loop.py", line 66, in check_loop
    integration_frame = get_integration_frame()
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/helpers/frame.py", line 110, in get_integration_frame
    raise MissingIntegrationFrame
homeassistant.helpers.frame.MissingIntegrationFrame

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/core/homeassistant/components/websocket_api/decorators.py", line 27, in _handle_async_response
    await func(hass, connection, msg)
  File "/core/homeassistant/components/websocket_api/commands.py", line 796, in handle_execute_script
    script_result = await script_obj.async_run(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/helpers/script.py", line 1738, in async_run
    return await asyncio.shield(create_eager_task(run.async_run()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/helpers/script.py", line 462, in async_run
    await self._async_step(log_exceptions=False)
  File "/core/homeassistant/helpers/script.py", line 514, in _async_step
    self._handle_exception(
  File "/core/homeassistant/helpers/script.py", line 544, in _handle_exception
    raise exception
  File "/core/homeassistant/helpers/script.py", line 512, in _async_step
    await getattr(self, handler)()
  File "/core/homeassistant/helpers/script.py", line 747, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/helpers/script.py", line 710, in _async_run_long_action
    return await long_task
           ^^^^^^^^^^^^^^^
  File "/core/homeassistant/core.py", line 2725, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/core/homeassistant/core.py", line 2766, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/components/notify/legacy.py", line 275, in _async_notify_message_service
    await self.async_send_message(**kwargs)
  File "/core/homeassistant/components/slack/notify.py", line 320, in async_send_message
    return await self._async_send_remote_file_message(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/homeassistant/components/slack/notify.py", line 220, in _async_send_remote_file_message
    await asyncio.gather(
  File "/core/venv/lib/python3.12/site-packages/slack_sdk/web/async_client.py", line 3587, in files_upload_v2
    upload_result = _upload_file_via_v2_url(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/core/venv/lib/python3.12/site-packages/slack_sdk/web/internal_utils.py", line 385, in _upload_file_via_v2_url
    resp = urlopen(req, context=ssl, timeout=timeout)  # skipcq: BAN-B310
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 215, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 515, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 532, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 1392, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 1344, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1336, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py", line 1347, in _send_request
    self.putrequest(method, url, **skips)
  File "/core/homeassistant/util/loop.py", line 136, in protected_loop_func
    check_loop(
  File "/core/homeassistant/util/loop.py", line 82, in check_loop
    raise RuntimeError(  # noqa: TRY200
RuntimeError: Detected blocking call to putrequest inside the event loop in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/http/client.py, line 1347: self.putrequest(method, url, **skips). Use `await hass.async_add_executor_job()`; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue

[seratch]

Hi @tkdrob, thanks for reporting this issue. You're right that this should be improved, but as you can see here, the source file is auto-generated. Therefore, modifying the code like you did at #1497 does not work. I will come up with a new pull request to make similar changes that are compatible with the codegen.