You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using aiohttp with an HTTP proxy, the following exception occurs in some cases:
After consulting the developer of the proxy tool Gost, it was identified that both the server-side and client-side are not adhering to RFC-9110, which causes this issue.
According to the RFC-9110:
A server MUST NOT send any Transfer-Encoding or Content-Length header fields in a 2xx (Successful) response to CONNECT.
A client MUST ignore any Content-Length or Transfer-Encoding header fields received in a successful response to CONNECT.
The error occurs because the Gost HTTP server returns a Content-Length header, and aiohttp does not ignore it, contrary to the RFC. This makes it difficult to identify the root cause of the issue (especially since the Requests library works well with the same proxy server setup).
Now, the Gost program has been fixed, and aiohttp works well with it. However, addressing this issue in aiohttp would help avoid similar problems for other users in the future. If aiohttp could ignore the related HTTP headers as specified in the RFC, it would greatly improve its robustness. Even if the server sends the Content-Length header, no error would occur (Just like third-libs lke requests and httpx ...etc).
To Reproduce
setup a plain http proxy, using tools like: gost, goproxy
testing the proxy server to ensure it works normally
using aiohttp make request behind that proxy server
It should return the partial text of the target web.
Logs/tracebacks
Traceback (most recent call last):
File "/home/max/remote_projects/test-pro/test_aiohttp.py", line 12, in <module>
asyncio.run(main())
File "/usr/lib/python3.11/asyncio/runners.py", line 190, in runreturn runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/runners.py", line 118, in runreturnself._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_completereturn future.result()
^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/test_aiohttp.py", line 8, in mainasyncwith cs.get("https://www.python.org", proxy=env) as r:
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 1197, in __aenter__self._resp =awaitself._coro
^^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 581, in _request
conn =awaitself._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect
proto =awaitself._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/connector.py", line 942, in _create_connection
_, proto =awaitself._create_proxy_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/connector.py", line 1379, in _create_proxy_connectionreturnawaitself._start_tls_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/max/remote_projects/test-pro/.venv/lib/python3.11/site-packages/aiohttp/connector.py", line 1172, in _start_tls_connectionraise client_error(req.connection_key, OSError(msg))
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host www.python.org:443 ssl:default [None]
Python Version
$ python --versionPython 3.12.4 on WindowsPython 3.11.2 on Debian Bookworm
aiohttp Version
$ python -m pip show aiohttpName: aiohttpVersion: 3.9.5Summary: Async http client/server framework (asyncio)Home-page: https://github.com/aio-libs/aiohttpAuthor:Author-email:License: Apache 2Location: C:\Python\312\Lib\site-packagesRequires: aiosignal, attrs, frozenlist, multidict, yarlRequired-by:same version both on Windows and Debian
multidict Version
$ python -m pip show multidictName: multidictVersion: 6.0.5Summary: multidict implementationHome-page: https://github.com/aio-libs/multidictAuthor: Andrew SvetlovAuthor-email: [email protected]License: Apache 2Location: C:\Python\312\Lib\site-packagesRequires:Required-by: aiohttp, yarlsame version both on Windows and Debian
yarl Version
$ python -m pip show yarlName: yarlVersion: 1.9.4Summary: Yet another URL libraryHome-page: https://github.com/aio-libs/yarlAuthor: Andrew SvetlovAuthor-email: [email protected]License: Apache-2.0Location: C:\Python\312\Lib\site-packagesRequires: idna, multidictRequired-by: aiohttpsame version both on Windows and Debian
OS
Windows 10 2021 LTSC
Debian bookworm
Related component
Client
Additional context
No response
Code of Conduct
I agree to follow the aio-libs Code of Conduct
The text was updated successfully, but these errors were encountered:
Describe the bug
When using aiohttp with an HTTP proxy, the following exception occurs in some cases:
After consulting the developer of the proxy tool Gost, it was identified that both the server-side and client-side are not adhering to RFC-9110, which causes this issue.
According to the RFC-9110:
The error occurs because the Gost HTTP server returns a Content-Length header, and aiohttp does not ignore it, contrary to the RFC. This makes it difficult to identify the root cause of the issue (especially since the Requests library works well with the same proxy server setup).
Now, the Gost program has been fixed, and aiohttp works well with it. However, addressing this issue in aiohttp would help avoid similar problems for other users in the future. If aiohttp could ignore the related HTTP headers as specified in the RFC, it would greatly improve its robustness. Even if the server sends the Content-Length header, no error would occur (Just like third-libs lke requests and httpx ...etc).
To Reproduce
Expected behavior
It should return the partial text of the target web.
Logs/tracebacks
Python Version
aiohttp Version
multidict Version
yarl Version
OS
Windows 10 2021 LTSC
Debian bookworm
Related component
Client
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: