[SCRAPER] - TrueNAS Scale Docker Error #4476
Labels
Comments
|
having ismus scraping the afforementioned recipe, could this be an issing with my internal dns or with the way truenas docker is set up? |
|
I can't replicate your issue, and i had never any issues scraping chefkoch.de. |
|
I'd definitely double check that other applications you're running are able to access things like the URL you've got here. A couple of times w/ TrueNAS Scale, I've had to essentially reboot the node to get some of the app networking to work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
First Check
I used the GitHub search to find a similar issue and didn't find it.
I have verified that this issue is not related to the underlying library
hhyrsev/recipe-scrapers by 1) checking
the debugger and data is returned, 2)
verifying that there are errors in the log related to application level code, or
3) verified that the site provides recipe data, or is otherwise supported by
hhyrsev/recipe-scrapers
This issue can be replicated on the demo site (https://demo.mealie.io/)
Please provide 1-5 example URLs that are having errors
https://www.chefkoch.de/rezepte/4228681685379518/One-Pan-Thunfisch-Pasta.html
Please provide your logs for the Mealie container
docker logs <container-id> > mealie.logs2024-10-30 20:09:20.120419+00:00ERROR 2024-10-30T21:09:20 - Exception in ASGI application
2024-10-30 20:09:20.120504+00:00Traceback (most recent call last):
2024-10-30 20:09:20.120537+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-10-30 20:09:20.120566+00:00result = await app( # type: ignore[func-returns-value]
2024-10-30 20:09:20.120685+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
2024-10-30 20:09:20.120717+00:00return await self.app(scope, receive, send)
2024-10-30 20:09:20.120746+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
2024-10-30 20:09:20.120804+00:00await super().call(scope, receive, send)
2024-10-30 20:09:20.120833+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in call
2024-10-30 20:09:20.120859+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.120916+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in call
2024-10-30 20:09:20.120946+00:00raise exc
2024-10-30 20:09:20.120973+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in call
2024-10-30 20:09:20.120999+00:00await self.app(scope, receive, _send)
2024-10-30 20:09:20.121248+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/sessions.py", line 85, in call
2024-10-30 20:09:20.121328+00:00await self.app(scope, receive, send_wrapper)
2024-10-30 20:09:20.121362+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 20, in call
2024-10-30 20:09:20.121434+00:00await responder(scope, receive, send)
2024-10-30 20:09:20.121463+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 39, in call
2024-10-30 20:09:20.121490+00:00await self.app(scope, receive, self.send_with_gzip)
2024-10-30 20:09:20.121517+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in call
2024-10-30 20:09:20.121671+00:00await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-30 20:09:20.121730+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.121761+00:00raise exc
2024-10-30 20:09:20.121911+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.121950+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.121980+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in call
2024-10-30 20:09:20.122009+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.122075+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
2024-10-30 20:09:20.122105+00:00await route.handle(scope, receive, send)
2024-10-30 20:09:20.122132+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
2024-10-30 20:09:20.122195+00:00await self.app(scope, receive, send)
2024-10-30 20:09:20.122226+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
2024-10-30 20:09:20.122253+00:00await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-30 20:09:20.122280+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.122341+00:00raise exc
2024-10-30 20:09:20.122371+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.122398+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.122425+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
2024-10-30 20:09:20.122486+00:00response = await f(request)
2024-10-30 20:09:20.122515+00:00File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler
2024-10-30 20:09:20.122545+00:00response = await original_route_handler(request)
2024-10-30 20:09:20.122605+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
2024-10-30 20:09:20.122640+00:00raw_response = await run_endpoint_function(
2024-10-30 20:09:20.122666+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2024-10-30 20:09:20.122694+00:00return await dependant.call(**values)
2024-10-30 20:09:20.122762+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 233, in parse_recipe_url
2024-10-30 20:09:20.122792+00:00return await self._create_recipe_from_web(req)
2024-10-30 20:09:20.122819+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 244, in _create_recipe_from_web
2024-10-30 20:09:20.122879+00:00recipe, extras = await create_from_html(url, self.translator, html)
2024-10-30 20:09:20.122909+00:00File "/app/mealie/services/scraper/scraper.py", line 45, in create_from_html
2024-10-30 20:09:20.122937+00:00new_recipe, extras = await scraper.scrape(url, html)
2024-10-30 20:09:20.122965+00:00File "/app/mealie/services/scraper/recipe_scraper.py", line 41, in scrape
2024-10-30 20:09:20.123035+00:00raw_html = html or await safe_scrape_html(url)
2024-10-30 20:09:20.123065+00:00File "/app/mealie/services/scraper/scraper_strategies.py", line 47, in safe_scrape_html
2024-10-30 20:09:20.123094+00:00async with client.stream(
2024-10-30 20:09:20.123120+00:00File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
2024-10-30 20:09:20.123377+00:00return await anext(self.gen)
2024-10-30 20:09:20.123658+00:00ERROR 2024-10-30T21:09:20 - Exception in ASGI application
2024-10-30 20:09:20.123713+00:00Traceback (most recent call last):
2024-10-30 20:09:20.123741+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi
2024-10-30 20:09:20.123824+00:00result = await app( # type: ignore[func-returns-value]
2024-10-30 20:09:20.123858+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in call
2024-10-30 20:09:20.123887+00:00return await self.app(scope, receive, send)
2024-10-30 20:09:20.123956+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
2024-10-30 20:09:20.123987+00:00await super().call(scope, receive, send)
2024-10-30 20:09:20.124014+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 113, in call
2024-10-30 20:09:20.124082+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.124114+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in call
2024-10-30 20:09:20.124142+00:00raise exc
2024-10-30 20:09:20.124168+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in call
2024-10-30 20:09:20.124236+00:00await self.app(scope, receive, _send)
2024-10-30 20:09:20.124266+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/sessions.py", line 85, in call
2024-10-30 20:09:20.124294+00:00await self.app(scope, receive, send_wrapper)
2024-10-30 20:09:20.124320+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 20, in call
2024-10-30 20:09:20.125356+00:00await responder(scope, receive, send)
2024-10-30 20:09:20.125413+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 39, in call
2024-10-30 20:09:20.123419+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1628, in stream
2024-10-30 20:09:20.125594+00:00response = await self.send(
2024-10-30 20:09:20.125651+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
2024-10-30 20:09:20.125680+00:00response = await self._send_handling_auth(
2024-10-30 20:09:20.125707+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
2024-10-30 20:09:20.126015+00:00response = await self._send_handling_redirects(
2024-10-30 20:09:20.126050+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
2024-10-30 20:09:20.126079+00:00response = await self._send_single_request(request)
2024-10-30 20:09:20.126182+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
2024-10-30 20:09:20.126218+00:00response = await transport.handle_async_request(request)
2024-10-30 20:09:20.126247+00:00File "/app/mealie/pkgs/safehttp/transport.py", line 64, in handle_async_request
2024-10-30 20:09:20.126463+00:00ip_str = socket.gethostbyname(netloc)
2024-10-30 20:09:20.125444+00:00await self.app(scope, receive, self.send_with_gzip)
2024-10-30 20:09:20.126696+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in call
2024-10-30 20:09:20.126727+00:00await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-30 20:09:20.126824+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.126858+00:00raise exc
2024-10-30 20:09:20.126887+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.127126+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.127166+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 715, in call
2024-10-30 20:09:20.127193+00:00await self.middleware_stack(scope, receive, send)
2024-10-30 20:09:20.127219+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
2024-10-30 20:09:20.127288+00:00await route.handle(scope, receive, send)
2024-10-30 20:09:20.127317+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
2024-10-30 20:09:20.127344+00:00await self.app(scope, receive, send)
2024-10-30 20:09:20.127371+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
2024-10-30 20:09:20.127431+00:00await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-30 20:09:20.127461+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2024-10-30 20:09:20.127489+00:00raise exc
2024-10-30 20:09:20.127687+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2024-10-30 20:09:20.127857+00:00await app(scope, receive, sender)
2024-10-30 20:09:20.127951+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
2024-10-30 20:09:20.127981+00:00response = await f(request)
2024-10-30 20:09:20.126974+00:00socket.gaierror: [Errno -3] Temporary failure in name resolution
2024-10-30 20:09:20.128298+00:00File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler
2024-10-30 20:09:20.128334+00:00response = await original_route_handler(request)
2024-10-30 20:09:20.128360+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
2024-10-30 20:09:20.128501+00:00raw_response = await run_endpoint_function(
2024-10-30 20:09:20.128539+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2024-10-30 20:09:20.128568+00:00return await dependant.call(**values)
2024-10-30 20:09:20.128633+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 233, in parse_recipe_url
2024-10-30 20:09:20.128664+00:00return await self._create_recipe_from_web(req)
2024-10-30 20:09:20.128691+00:00File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 244, in _create_recipe_from_web
2024-10-30 20:09:20.128717+00:00recipe, extras = await create_from_html(url, self.translator, html)
2024-10-30 20:09:20.128901+00:00File "/app/mealie/services/scraper/scraper.py", line 45, in create_from_html
2024-10-30 20:09:20.128940+00:00new_recipe, extras = await scraper.scrape(url, html)
2024-10-30 20:09:20.128993+00:00File "/app/mealie/services/scraper/recipe_scraper.py", line 41, in scrape
2024-10-30 20:09:20.129087+00:00raw_html = html or await safe_scrape_html(url)
2024-10-30 20:09:20.129119+00:00File "/app/mealie/services/scraper/scraper_strategies.py", line 47, in safe_scrape_html
2024-10-30 20:09:20.129174+00:00async with client.stream(
2024-10-30 20:09:20.129202+00:00File "/usr/local/lib/python3.10/contextlib.py", line 199, in aenter
2024-10-30 20:09:20.129289+00:00return await anext(self.gen)
2024-10-30 20:09:20.129346+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1628, in stream
2024-10-30 20:09:20.129375+00:00response = await self.send(
2024-10-30 20:09:20.129424+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
2024-10-30 20:09:20.129513+00:00response = await self._send_handling_auth(
2024-10-30 20:09:20.129546+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
2024-10-30 20:09:20.129597+00:00response = await self._send_handling_redirects(
2024-10-30 20:09:20.129625+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
2024-10-30 20:09:20.129811+00:00response = await self._send_single_request(request)
2024-10-30 20:09:20.129934+00:00File "/opt/pysetup/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
2024-10-30 20:09:20.129966+00:00response = await transport.handle_async_request(request)
2024-10-30 20:09:20.130080+00:00File "/app/mealie/pkgs/safehttp/transport.py", line 64, in handle_async_request
2024-10-30 20:09:20.130112+00:00ip_str = socket.gethostbyname(netloc)
2024-10-30 20:09:20.130140+00:00socket.gaierror: [Errno -3] Temporary failure in name resolution
Deployment
Docker (Linux)
The text was updated successfully, but these errors were encountered: