Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 stream X canceled by remote with error code 0 #1105

Open
warksit opened this issue Nov 9, 2023 · 28 comments
Open

🐛 stream X canceled by remote with error code 0 #1105

warksit opened this issue Nov 9, 2023 · 28 comments
Labels
Priority: Normal Minor issue impacting one or more users Type: Bug Something isn't working

Comments

@warksit
Copy link

warksit commented Nov 9, 2023 •
edited
Loading

Describe the bug
Cloudflare Tunnel disconnects after a few hours

To Reproduce
Steps to reproduce the behavior:

  1. Configure Cloudflared in HomeAssistant Add-on
  2. Start it and wait a few hours
  3. See error

If it's an issue with Cloudflare Tunnel:
4. Tunnel ID : a123c10a-8b4b-49ff-870b-5880edbd505e
5. cloudflared config: remote managed tunnel

Expected behavior
I expect tunnel to stay open. It disconnects and requires a restart of the cloudflared add-on to reconnect tunnel

Environment and versions

  • OS: HomeAssistant OS
  • Architecture: Intel i5 running Proxmox
  • 2023.11.1

Logs and errors
Will add when this tunnel fails (HomeAssistant doesn't t display logs after a restart of the add-on)
@warksit warksit added Priority: Normal Minor issue impacting one or more users Type: Bug Something isn't working labels Nov 9, 2023
@warksit
Copy link
Author

warksit commented Nov 13, 2023

Logs now tunnel has failed:


-----------------------------------------------------------
 Add-on: Cloudflared
 Use a Cloudflare Tunnel to remotely connect to Home Assistant without opening any ports
-----------------------------------------------------------
 Add-on version: 4.2.10
 You are running the latest version of this add-on.
 System: Home Assistant OS 11.1  (amd64 / qemux86-64)
 Home Assistant Core: 2023.11.1
 Home Assistant Supervisor: 2023.11.0
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[14:44:25] INFO: Using Cloudflare Remote Management Tunnel
[14:44:25] INFO: All add-on configuration options except tunnel_token will be ignored.
[14:44:25] INFO: Connecting Cloudflare Tunnel...
2023-11-09T14:44:25Z INF Starting tunnel tunnelID=a123c10a-8b4b-49ff-870b-5880edbd505e
2023-11-09T14:44:25Z INF Version 2023.10.0
2023-11-09T14:44:25Z INF GOOS: linux, GOVersion: go1.20.6, GoArch: amd64
2023-11-09T14:44:25Z INF Settings: map[loglevel:info metrics:0.0.0.0:36500 no-autoupdate:true token:*****]
2023-11-09T14:44:25Z INF Generated Connector ID: 08334dad-86e1-463d-b244-1af437bd0542
2023-11-09T14:44:25Z INF Initial protocol quic
2023-11-09T14:44:25Z INF ICMP proxy will use 172.30.33.3 as source for IPv4
2023-11-09T14:44:25Z INF ICMP proxy will use :: as source for IPv6
2023-11-09T14:44:25Z INF Starting metrics server on [::]:36500/metrics
2023-11-09T14:44:26Z INF Registered tunnel connection connIndex=0 connection=a29feb16-1807-4948-9945-08eb3f9bdd7c event=0 ip=198.41.192.47 location=lhr09 protocol=quic
2023-11-09T14:44:26Z INF Registered tunnel connection connIndex=1 connection=efda6390-9b65-4ced-94a5-2a7aab7450fe event=0 ip=198.41.200.43 location=man03 protocol=quic
2023-11-09T14:44:27Z INF Registered tunnel connection connIndex=2 connection=1ebf3386-5a05-4e2a-b982-2561eee3bd87 event=0 ip=198.41.200.113 location=man03 protocol=quic
2023-11-09T14:44:27Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"ha.warks.it\",\"service\":\"http://192.168.20.2:8123\"},{\"service\":\"http_status:404\"}],\"warp-routing\":{\"enabled\":false}}" version=1
2023-11-09T14:44:28Z INF Registered tunnel connection connIndex=3 connection=71951bd7-2d7f-437c-bc78-cc232f32d5da event=0 ip=198.41.192.57 location=lhr11 protocol=quic
2023-11-10T13:22:54Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.43
2023-11-10T13:22:54Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.43
2023-11-10T13:22:54Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.43
2023-11-10T13:22:54Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.43
2023-11-10T13:22:54Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.200.113
2023-11-10T13:22:54Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.113
2023-11-10T13:22:54Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.113
2023-11-10T13:22:54Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.200.113
2023-11-10T13:22:54Z INF Unregistered tunnel connection connIndex=0 event=0 ip=198.41.192.47
2023-11-10T13:22:54Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.192.47
2023-11-10T13:22:54Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.192.47
2023-11-10T13:22:54Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.192.47
2023-11-10T13:22:55Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.192.57
2023-11-10T13:22:55Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.192.57
2023-11-10T13:22:55Z ERR Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.192.57
2023-11-10T13:22:55Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.192.57
2023-11-10T13:22:55Z ERR Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-10T13:22:55Z ERR Connection terminated error="timeout: no recent network activity" connIndex=2
2023-11-10T13:22:56Z ERR Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-10T13:22:57Z INF Registered tunnel connection connIndex=0 connection=a4358d0b-6f31-4a8a-9dc2-26aeca0a0e1f event=0 ip=198.41.192.67 location=lhr11 protocol=quic
2023-11-10T13:23:08Z INF Registered tunnel connection connIndex=1 connection=966d5256-9609-49d2-ba6b-fc464179e498 event=0 ip=198.41.200.193 location=man01 protocol=quic
2023-11-10T13:23:08Z INF Registered tunnel connection connIndex=3 connection=29d9f907-f23b-4875-aeaf-a9a929b69e98 event=0 ip=198.41.192.227 location=lhr12 protocol=quic
2023-11-10T13:23:08Z INF Registered tunnel connection connIndex=2 connection=f3f74f26-3fc8-42ee-8de1-028f20d14ad4 event=0 ip=198.41.200.53 location=man03 protocol=quic
2023-11-10T14:17:20Z ERR  error="Incoming request ended abruptly: context canceled" cfRay=823eeb1cbaa93862-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-10T14:17:20Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=3 dest=https://ha.warks.it/api/webhook/39dfb309cc8987ef7f43cc09cb335131c92269ca23543de707784786da503126 event=0 ip=198.41.192.227 type=http
2023-11-11T14:54:16Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.200.53
2023-11-11T14:54:16Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.53
2023-11-11T14:54:16Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.53
2023-11-11T14:54:16Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.200.53
2023-11-11T14:54:17Z WRN Connection terminated error="timeout: no recent network activity" connIndex=2
2023-11-11T14:54:20Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.192.227
2023-11-11T14:54:20Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.192.227
2023-11-11T14:54:20Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.192.227
2023-11-11T14:54:20Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.192.227
2023-11-11T14:54:21Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-11T14:54:21Z INF Registered tunnel connection connIndex=2 connection=66845de4-7e67-472b-ae25-1062ee1c5a2e event=0 ip=198.41.200.113 location=man03 protocol=quic
2023-11-11T14:54:40Z INF Registered tunnel connection connIndex=3 connection=5a1cead6-a634-4853-bcc4-d3d4961f0f79 event=0 ip=198.41.192.107 location=lhr09 protocol=quic
2023-11-11T22:49:07Z ERR  error="stream 381 canceled by remote with error code 0" cfRay=824a14c42e4fdd64-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-11T22:49:07Z ERR Request failed error="stream 381 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/api/webhook/39dfb309cc8987ef7f43cc09cb335131c92269ca23543de707784786da503126 event=0 ip=198.41.192.67 type=http
2023-11-11T22:49:23Z ERR  error="stream 389 canceled by remote with error code 0" cfRay=824a1639bf92dd64-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-11T22:49:23Z ERR Request failed error="stream 389 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/api/webhook/39dfb309cc8987ef7f43cc09cb335131c92269ca23543de707784786da503126 event=0 ip=198.41.192.67 type=http
2023-11-12T00:06:50Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.200.113
2023-11-12T00:06:50Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.113
2023-11-12T00:06:50Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.200.113
2023-11-12T00:06:50Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.200.113
2023-11-12T00:06:52Z WRN Connection terminated error="timeout: no recent network activity" connIndex=2
2023-11-12T00:07:09Z INF Registered tunnel connection connIndex=2 connection=1bcf9648-26d2-451f-8b17-d88083b6a7a3 event=0 ip=198.41.200.33 location=dub01 protocol=quic
2023-11-12T10:30:41Z ERR  error="Incoming request ended abruptly: context canceled" cfRay=824e19df68f423b8-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-12T10:30:41Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=3 dest=https://ha.warks.it/api/webhook/39dfb309cc8987ef7f43cc09cb335131c92269ca23543de707784786da503126 event=0 ip=198.41.192.107 type=http
2023-11-12T10:47:37Z ERR  error="Incoming request ended abruptly: context canceled" cfRay=824e32adee35652b-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-12T10:47:37Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=3 dest=https://ha.warks.it/api/webhook/39dfb309cc8987ef7f43cc09cb335131c92269ca23543de707784786da503126 event=0 ip=198.41.192.107 type=http
2023-11-13T08:21:20Z ERR  error="stream 721 canceled by remote with error code 0" cfRay=825598e4c941dc25-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-13T08:21:20Z ERR Request failed error="stream 721 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.192.67 type=http

@warksit
Copy link
Author

warksit commented Nov 14, 2023

More logs

2023-11-14T14:18:48Z ERR  error="stream 345 canceled by remote with error code 0" cfRay=825fe054dcca63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:48Z ERR Request failed error="stream 345 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:49Z ERR  error="stream 349 canceled by remote with error code 0" cfRay=825fe05b7fe563d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:49Z ERR Request failed error="stream 349 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:50Z ERR  error="stream 353 canceled by remote with error code 0" cfRay=825fe061b8c663d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:50Z ERR Request failed error="stream 353 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:51Z ERR  error="stream 357 canceled by remote with error code 0" cfRay=825fe0682a2063d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:51Z ERR Request failed error="stream 357 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 365 canceled by remote with error code 0" cfRay=825fe0740b6163d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 365 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 373 canceled by remote with error code 0" cfRay=825fe0808e7163d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR  error="stream 381 canceled by remote with error code 0" cfRay=825fe08d18c763d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 381 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 385 canceled by remote with error code 0" cfRay=825fe093393063d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 385 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 409 canceled by remote with error code 0" cfRay=825fe0b93d1e63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 409 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 405 canceled by remote with error code 0" cfRay=825fe0b2bc6d63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 405 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 429 canceled by remote with error code 0" cfRay=825fe0f1f95d63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 429 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 441 canceled by remote with error code 0" cfRay=825fe0f7b9ff63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 441 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 417 canceled by remote with error code 0" cfRay=825fe0cc6e6263d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 417 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 421 canceled by remote with error code 0" cfRay=825fe0d1cd8763d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 421 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 433 canceled by remote with error code 0" cfRay=825fe0f2199e63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 433 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 397 canceled by remote with error code 0" cfRay=825fe0a8ff2f63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 397 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 425 canceled by remote with error code 0" cfRay=825fe0e5285663d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 425 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 493 canceled by remote with error code 0" cfRay=825fe2799d4563d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 493 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 437 canceled by remote with error code 0" cfRay=825fe0f29a4b63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 437 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 497 canceled by remote with error code 0" cfRay=825fe27a0de063d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 497 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/manifest.json event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 361 canceled by remote with error code 0" cfRay=825fe06dba0363d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 361 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 369 canceled by remote with error code 0" cfRay=825fe07a3c4c63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 369 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 377 canceled by remote with error code 0" cfRay=825fe086bfa763d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 377 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR Request failed error="stream 373 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 413 canceled by remote with error code 0" cfRay=825fe0c5fdc863d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR  error="stream 389 canceled by remote with error code 0" cfRay=825fe099ead363d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 413 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR Request failed error="stream 389 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 401 canceled by remote with error code 0" cfRay=825fe0ac3bc963d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 401 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:18:52Z ERR  error="stream 393 canceled by remote with error code 0" cfRay=825fe0a00b7b63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:18:52Z ERR Request failed error="stream 393 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:19:14Z ERR  error="stream 445 canceled by remote with error code 0" cfRay=825fe0f7da3c63d7-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:19:14Z ERR Request failed error="stream 445 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/hacsfiles/iconset.js event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 545 canceled by remote with error code 0" cfRay=825fe2de1eac7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR  error="stream 593 canceled by remote with error code 0" cfRay=825fe2de1eb27306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 545 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR Request failed error="stream 593 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/hacsfiles/iconset.js event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 557 canceled by remote with error code 0" cfRay=825fe2de1eaa7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 557 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 501 canceled by remote with error code 0" cfRay=825fe2de1e977306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 501 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 581 canceled by remote with error code 0" cfRay=825fe2de1e9e7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR  error="stream 561 canceled by remote with error code 0" cfRay=825fe2de1eb47306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 581 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR Request failed error="stream 561 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/manifest.json event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 537 canceled by remote with error code 0" cfRay=825fe2de1ea37306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 537 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 569 canceled by remote with error code 0" cfRay=825fe2de1eb37306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR  error="stream 509 canceled by remote with error code 0" cfRay=825fe2de1e9a7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 569 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR Request failed error="stream 509 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 521 canceled by remote with error code 0" cfRay=825fe2de1e9f7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 521 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 513 canceled by remote with error code 0" cfRay=825fe2de1ea77306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 513 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 529 canceled by remote with error code 0" cfRay=825fe2de1ea27306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 529 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 553 canceled by remote with error code 0" cfRay=825fe2de1eaf7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 553 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 517 canceled by remote with error code 0" cfRay=825fe2de1e9d7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR  error="stream 533 canceled by remote with error code 0" cfRay=825fe2de1ea87306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 533 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR Request failed error="stream 517 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 565 canceled by remote with error code 0" cfRay=825fe2de1eb17306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 565 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/manifest.json event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 541 canceled by remote with error code 0" cfRay=825fe2de1ea57306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR  error="stream 573 canceled by remote with error code 0" cfRay=825fe2de1ea17306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 541 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR Request failed error="stream 573 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 505 canceled by remote with error code 0" cfRay=825fe2de1e997306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 505 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 577 canceled by remote with error code 0" cfRay=825fe2de1ead7306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 577 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 525 canceled by remote with error code 0" cfRay=825fe2de1ea47306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 525 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http
2023-11-14T14:20:00Z ERR  error="stream 549 canceled by remote with error code 0" cfRay=825fe2de1ea97306-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-14T14:20:00Z ERR Request failed error="stream 549 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/dashboard-mushroom event=0 ip=198.41.200.23 type=http

@warksit
Copy link
Author

warksit commented Nov 17, 2023 •
edited
Loading

More logs


2023-11-15T00:20:09Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.192.57
2023-11-15T00:20:09Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.192.57
2023-11-15T00:20:09Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.192.57
2023-11-15T00:20:09Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.192.57
2023-11-15T00:20:09Z WRN Connection terminated error="timeout: no recent network activity" connIndex=2
2023-11-15T00:20:10Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.73
2023-11-15T00:20:10Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.73
2023-11-15T00:20:10Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.73
2023-11-15T00:20:10Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.73
2023-11-15T00:20:10Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-15T00:20:12Z INF Registered tunnel connection connIndex=1 connection=44c44173-45df-4416-bca2-affb39e3b806 event=0 ip=198.41.200.53 location=bru01 protocol=quic
2023-11-15T00:20:12Z INF Registered tunnel connection connIndex=2 connection=c26acb33-af67-41e7-bdfd-1d201b9c90db event=0 ip=198.41.192.167 location=lhr10 protocol=quic
2023-11-15T00:56:06Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.53
2023-11-15T00:56:06Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.53
2023-11-15T00:56:06Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.53
2023-11-15T00:56:06Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.53
2023-11-15T00:56:08Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-15T00:56:26Z INF Registered tunnel connection connIndex=1 connection=780aecda-d9d4-49a3-b6e0-f92c246c9c28 event=0 ip=198.41.200.73 location=dub01 protocol=quic
2023-11-15T09:21:27Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.73
2023-11-15T09:21:27Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.73
2023-11-15T09:21:27Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.73
2023-11-15T09:21:27Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.73
2023-11-15T09:21:27Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-15T09:21:40Z INF Registered tunnel connection connIndex=1 connection=d4a97f7d-ca69-4a0a-bd3a-6f2cfcc14412 event=0 ip=198.41.200.43 location=bru01 protocol=quic
2023-11-15T15:58:21Z ERR  error="Incoming request ended abruptly: context canceled" cfRay=8268b1fa79cfdc9b-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-15T15:58:21Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=0 dest=https://ha.warks.it/api/webhook/3d8228706261ff268b0981fef53ecefc822fa78870d0076590b6b53942bfe225 event=0 ip=198.41.192.37 type=http
2023-11-15T16:37:19Z ERR  error="Incoming request ended abruptly: context canceled" cfRay=8268eb09cceedc9b-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-15T16:37:19Z ERR Request failed error="Incoming request ended abruptly: context canceled" connIndex=0 dest=https://ha.warks.it/api/webhook/3d8228706261ff268b0981fef53ecefc822fa78870d0076590b6b53942bfe225 event=0 ip=198.41.192.37 type=http
2023-11-16T01:26:51Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.43
2023-11-16T01:26:51Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.43
2023-11-16T01:26:51Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.43
2023-11-16T01:26:51Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.43
2023-11-16T01:26:52Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.33
2023-11-16T01:26:52Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-16T01:26:52Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-16T01:26:52Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.33
2023-11-16T01:26:52Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-16T01:26:53Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-16T01:27:10Z INF Registered tunnel connection connIndex=1 connection=74e03f1d-64f0-40b3-9031-594e3abea5bd event=0 ip=198.41.200.63 location=man03 protocol=quic
2023-11-16T01:27:10Z INF Registered tunnel connection connIndex=3 connection=63c44183-45e3-4c64-9dcf-15f46824dac9 event=0 ip=198.41.200.233 location=dub01 protocol=quic
2023-11-16T01:56:26Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.63
2023-11-16T01:56:26Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.63
2023-11-16T01:56:26Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.63
2023-11-16T01:56:26Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.63
2023-11-16T01:56:27Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-16T01:56:30Z INF Registered tunnel connection connIndex=1 connection=4dcbfb40-e21f-465f-a3f9-e71915179264 event=0 ip=198.41.200.53 location=bru01 protocol=quic
2023-11-16T01:56:51Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.53
2023-11-16T01:56:51Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.53
2023-11-16T01:56:51Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.53
2023-11-16T01:56:51Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.53
2023-11-16T01:56:51Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-16T01:56:58Z INF Registered tunnel connection connIndex=1 connection=84d0da79-6bf8-4b6e-9400-0b8498e412e3 event=0 ip=198.41.200.33 location=man02 protocol=quic
2023-11-16T16:44:50Z ERR  error="stream 533 canceled by remote with error code 0" cfRay=827133621f1424e0-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:44:50Z ERR Request failed error="stream 533 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:44:52Z ERR  error="stream 537 canceled by remote with error code 0" cfRay=8271336edfa924e0-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:44:52Z ERR Request failed error="stream 537 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:44:55Z ERR  error="stream 545 canceled by remote with error code 0" cfRay=827133879a4224e0-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:44:55Z ERR Request failed error="stream 545 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:51:22Z ERR  error="stream 565 canceled by remote with error code 0" cfRay=82713cf3cd1a76cc-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:51:22Z ERR Request failed error="stream 565 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:51:24Z ERR  error="stream 569 canceled by remote with error code 0" cfRay=82713d005c3076cc-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:51:24Z ERR Request failed error="stream 569 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:51:26Z ERR  error="stream 573 canceled by remote with error code 0" cfRay=82713d0caa8a76cc-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:51:26Z ERR Request failed error="stream 573 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:51:28Z ERR  error="stream 577 canceled by remote with error code 0" cfRay=82713d18efc176cc-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:51:28Z ERR Request failed error="stream 577 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:51:29Z ERR  error="stream 581 canceled by remote with error code 0" cfRay=82713d25cf6d76cc-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:51:29Z ERR Request failed error="stream 581 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:57:50Z ERR  error="stream 593 canceled by remote with error code 0" cfRay=8271466cefd3654d-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:57:50Z ERR Request failed error="stream 593 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:57:52Z ERR  error="stream 597 canceled by remote with error code 0" cfRay=82714678dcbc654d-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:57:52Z ERR Request failed error="stream 597 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/dashboard-mushroom?external_auth=1 event=0 ip=198.41.200.33 type=http
2023-11-16T16:57:56Z ERR  error="stream 605 canceled by remote with error code 0" cfRay=827146879cc8654d-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T16:57:56Z ERR Request failed error="stream 605 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/static/translations/en-GB-066487eb4e9d4c6b63eb09a7cb3cb8fc.json event=0 ip=198.41.200.33 type=http
2023-11-16T17:06:58Z ERR  error="stream 625 canceled by remote with error code 0" cfRay=8271539d3f90889e-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:06:58Z ERR Request failed error="stream 625 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/static/translations/en-GB-066487eb4e9d4c6b63eb09a7cb3cb8fc.json event=0 ip=198.41.200.33 type=http
2023-11-16T17:07:01Z ERR  error="stream 613 canceled by remote with error code 0" cfRay=8271539d3f91889e-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:07:01Z ERR Request failed error="stream 613 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/frontend_latest/27714-fDFdfFIWm5k.js event=0 ip=198.41.200.33 type=http
2023-11-16T17:17:04Z ERR  error="stream 649 canceled by remote with error code 0" cfRay=827161a76fe923ab-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:17:04Z ERR Request failed error="stream 649 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/static/translations/en-GB-066487eb4e9d4c6b63eb09a7cb3cb8fc.json event=0 ip=198.41.200.33 type=http
2023-11-16T17:17:12Z ERR  error="stream 657 canceled by remote with error code 0" cfRay=827161a76fea23ab-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:17:12Z ERR Request failed error="stream 657 canceled by remote with error code 0" connIndex=1 dest=https://ha.warks.it/frontend_latest/27714-fDFdfFIWm5k.js event=0 ip=198.41.200.33 type=http
2023-11-16T17:18:47Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.200.33
2023-11-16T17:18:47Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.33
2023-11-16T17:18:47Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=1 event=0 ip=198.41.200.33
2023-11-16T17:18:47Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.200.33
2023-11-16T17:18:49Z WRN Connection terminated error="timeout: no recent network activity" connIndex=1
2023-11-16T17:19:02Z INF Registered tunnel connection connIndex=1 connection=c8ff9270-b31e-43c4-bc8e-95dd9f6dccbf event=0 ip=198.41.200.73 location=dub01 protocol=quic
2023-11-16T17:22:46Z ERR  error="stream 949 canceled by remote with error code 0" cfRay=82716ac2ec284889-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:22:46Z ERR Request failed error="stream 949 canceled by remote with error code 0" connIndex=2 dest=https://ha.warks.it/static/translations/en-786b126fa6d30d17b5e2062704f1e400.json event=0 ip=198.41.192.167 type=http
2023-11-16T17:30:38Z ERR  error="stream 1725 canceled by remote with error code 0" cfRay=82717647df4d60ed-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-16T17:30:38Z ERR Request failed error="stream 1725 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/static/translations/en-786b126fa6d30d17b5e2062704f1e400.json event=0 ip=198.41.192.37 type=http
2023-11-17T15:41:00Z ERR  error="unexpected EOF" cfRay=82791202c92c637d-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-17T15:41:00Z ERR Request failed error="unexpected EOF" connIndex=3 dest=https://ha.warks.it/api/camera_proxy_stream/camera.back_door?token=2b595d3ba035db0c39e0830e983c6c28b50764cf4f7aa6a8079957c08e66362d event=0 ip=198.41.200.233 type=http
2023-11-17T15:42:03Z ERR  error="stream 1789 canceled by remote with error code 0" cfRay=827911acaeda637d-LHR event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-17T15:42:03Z ERR Request failed error="stream 1789 canceled by remote with error code 0" connIndex=3 dest=https://ha.warks.it/api/camera_proxy_stream/camera.front_door_2?token=92292821ee95bfde63c81a48074777c5f401df2e0a03a9988eebef1d69ae0d99 event=0 ip=198.41.200.233 type=http
2023-11-17T17:20:04Z ERR  error="stream 1977 canceled by remote with error code 0" cfRay=8279a44a8d4b20fb-HKG event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-17T17:20:04Z ERR Request failed error="stream 1977 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/ event=0 ip=198.41.192.37 type=http
2023-11-17T17:20:15Z ERR  error="stream 1989 canceled by remote with error code 0" cfRay=8279a48ee86a8b5a-HKG event=1 ingressRule=0 originService=http://192.168.20.2:8123
2023-11-17T17:20:15Z ERR Request failed error="stream 1989 canceled by remote with error code 0" connIndex=0 dest=https://ha.warks.it/ event=0 ip=198.41.192.37 type=http

@warksit warksit changed the title 🐛 🐛 stream X canceled by remote with error code 0 Nov 17, 2023
@warksit
Copy link
Author

warksit commented Nov 18, 2023

Another failed tunnel. What else can I do to try and resolve this issue?


2023-11-17T22:18:29Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:18:29Z WRN Failed to serve quic connection error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:18:29Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:18:29Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:18:30Z WRN Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3
2023-11-17T22:18:38Z INF Registered tunnel connection connIndex=3 connection=9c37c412-77dd-4104-82d5-d8ad709efb7e event=0 ip=198.41.200.113 location=man03 protocol=quic
2023-11-17T22:19:00Z INF Unregistered tunnel connection connIndex=0 event=0 ip=198.41.200.43
2023-11-17T22:19:00Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.200.43
2023-11-17T22:19:00Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.200.43
2023-11-17T22:19:00Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.200.43
2023-11-17T22:19:00Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:19:00Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:19:00Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:19:00Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.113
2023-11-17T22:19:02Z INF Registered tunnel connection connIndex=0 connection=384c5161-ff4c-4877-92f5-5a70dd0f433b event=0 ip=198.41.200.33 location=mrs01 protocol=quic
2023-11-17T22:19:02Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T22:19:38Z INF Registered tunnel connection connIndex=3 connection=c3e8c451-b8ed-4d05-96ae-30d04ae29d59 event=0 ip=198.41.200.23 location=man01 protocol=quic
2023-11-17T22:19:46Z INF Unregistered tunnel connection connIndex=0 event=0 ip=198.41.200.33
2023-11-17T22:19:46Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.200.33
2023-11-17T22:19:46Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=0 event=0 ip=198.41.200.33
2023-11-17T22:19:46Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.200.33
2023-11-17T22:19:48Z INF Registered tunnel connection connIndex=0 connection=bb8c5bed-27e5-4c66-b6cf-13b3e4dfa295 event=0 ip=198.41.200.63 location=dub01 protocol=quic
2023-11-17T22:21:08Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.23
2023-11-17T22:21:08Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.23
2023-11-17T22:21:08Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.23
2023-11-17T22:21:08Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.23
2023-11-17T22:21:10Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T22:21:20Z INF Registered tunnel connection connIndex=3 connection=2c8a52b2-863d-49bc-b41a-3143e0ff8737 event=0 ip=198.41.200.53 location=mrs01 protocol=quic
2023-11-17T22:22:31Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:31Z WRN Failed to serve quic connection error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:31Z WRN Serve tunnel error error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:31Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:32Z WRN Connection terminated error="failed to accept QUIC stream: timeout: no recent network activity" connIndex=3
2023-11-17T22:22:40Z INF 

===================================================================================
You are hitting an error while using the experimental post-quantum tunnels feature.

Please check:

   https://pqtunnels.cloudflareresearch.com

for known problems.
===================================================================================


2023-11-17T22:22:40Z WRN Failed to create new quic connection error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:40Z INF Retrying connection in up to 4s connIndex=3 event=0 ip=198.41.200.53
2023-11-17T22:22:41Z WRN Connection terminated error="failed to dial to edge with quic: timeout: no recent network activity" connIndex=3
2023-11-17T22:23:13Z INF Registered tunnel connection connIndex=3 connection=7155820e-10ea-4463-a285-e0e6caa3592d event=0 ip=198.41.200.33 location=bru01 protocol=quic
2023-11-17T22:23:44Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.33
2023-11-17T22:23:44Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-17T22:23:44Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-17T22:23:44Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.33
2023-11-17T22:23:45Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T22:24:27Z INF Registered tunnel connection connIndex=3 connection=63309d36-92f2-43af-a47d-c9cb553177f2 event=0 ip=198.41.200.193 location=bru01 protocol=quic
2023-11-17T22:24:58Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.193
2023-11-17T22:24:58Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.193
2023-11-17T22:24:58Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.193
2023-11-17T22:24:58Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.193
2023-11-17T22:24:58Z ERR writing call: timeout: no recent network activity
2023-11-17T22:24:58Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T22:25:31Z INF Registered tunnel connection connIndex=3 connection=3c8095b8-5aeb-4c35-b37a-3e38428d5e2e event=0 ip=198.41.200.73 location=man01 protocol=quic
2023-11-17T22:26:25Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.73
2023-11-17T22:26:25Z ERR writing release: timeout: no recent network activity
2023-11-17T22:26:25Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.73
2023-11-17T22:26:25Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.73
2023-11-17T22:26:25Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.73
2023-11-17T22:26:25Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T22:31:37Z INF Registered tunnel connection connIndex=3 connection=3fb16340-3638-4059-b21e-b969792fc3c4 event=0 ip=198.41.200.33 location=man02 protocol=quic
2023-11-17T23:12:44Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.33
2023-11-17T23:12:44Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-17T23:12:44Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.33
2023-11-17T23:12:44Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.33
2023-11-17T23:12:44Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T23:12:56Z INF Registered tunnel connection connIndex=3 connection=5ae69f8e-344e-4d18-b22b-9274810368bd event=0 ip=198.41.200.233 location=man02 protocol=quic
2023-11-17T23:13:44Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.233
2023-11-17T23:13:44Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.233
2023-11-17T23:13:44Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.233
2023-11-17T23:13:44Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.233
2023-11-17T23:13:45Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T23:14:03Z INF Registered tunnel connection connIndex=3 connection=b1ff1b81-8560-433b-81f4-d8434bfc69d1 event=0 ip=198.41.200.53 location=bru01 protocol=quic
2023-11-17T23:37:11Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.53
2023-11-17T23:37:11Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.53
2023-11-17T23:37:11Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.53
2023-11-17T23:37:11Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.53
2023-11-17T23:37:13Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-17T23:37:33Z INF Registered tunnel connection connIndex=3 connection=2c0538a7-925b-48a4-aaad-217ecf00d861 event=0 ip=198.41.200.113 location=man03 protocol=quic
2023-11-18T02:34:16Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.200.113
2023-11-18T02:34:16Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-18T02:34:16Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=3 event=0 ip=198.41.200.113
2023-11-18T02:34:16Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.200.113
2023-11-18T02:34:16Z WRN Connection terminated error="timeout: no recent network activity" connIndex=3
2023-11-18T02:34:16Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.192.7
2023-11-18T02:34:16Z WRN Failed to serve quic connection error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.192.7
2023-11-18T02:34:16Z WRN Serve tunnel error error="timeout: no recent network activity" connIndex=2 event=0 ip=198.41.192.7
2023-11-18T02:34:16Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.192.7
2023-11-18T02:34:17Z WRN Connection terminated error="timeout: no recent network activity" connIndex=2
2023-11-18T02:34:27Z INF Registered tunnel connection connIndex=2 connection=71bf2184-0617-401d-bbef-b87172338d32 event=0 ip=198.41.192.57 location=lhr09 protocol=quic
2023-11-18T02:34:27Z INF Registered tunnel connection connIndex=3 connection=125ca80e-38e8-48a3-9ccc-f21900a32117 event=0 ip=198.41.200.23 location=man01 protocol=quic

@warksit
Copy link
Author

warksit commented Nov 18, 2023

A reboot of the Cloudflare add on in HomeAssistant resolves the issue temporarily

@mklaiber
Copy link

I have the exact same problem. Very annoying.

@warksit
Copy link
Author

warksit commented Dec 20, 2023

Hi @mklaiber

What I have done to deal with this is install https://github.com/mvdwetering/websitechecker and set that up to check my cloudflared tunnel url. Then an automation that restarts the Cloudflared add-on when the tunnel has failed.

This really is a workaround but until the tunnel failing issue is resolved it’s the best I can come up with.

@Scot-Survivor
Copy link

Can confirm this happened to me just now.
Is it false positives on Cloudflare's side causing these issues?

@morpig
Copy link

morpig commented Jan 21, 2024

This have something to do with cloudflared -> local service.

cloudflared received incoming requests but for some reason it is unable to forward it to the local service.

might want the CF team to check this out.

@amaciuc
Copy link

amaciuc commented Feb 16, 2024 •
edited
Loading

Same here, I tried to use Nextcloud as additional host in Cloudflare and I encountered so many errors with canceled by remote with error core 0 when I am trying to view a video, to upload / download a large size file.

@RyderCragie
Copy link

I have the same logs showing.

@Healzangels
Copy link

Wondering if there was any news on this/any steps to take to remedy. Thanks!

@Scot-Survivor
Copy link

Scot-Survivor commented Apr 14, 2024 •
edited
Loading

Hi,
I just had this reappear again in my services, a simple restart of the tunnel container made it work.
Unlike last time, this happened with many of my users and myself, rather than just the sole user it was myself.

Restarting the tunnel container isn't exactly a great solution, but it would be ideal to see a fix for this soon / or preventative measures that I can take,

Cheers,
Scot

@mrmelon54
Copy link

mrmelon54 commented Apr 23, 2024 •
edited
Loading

Looks like the error is caused by quic-go
https://github.com/quic-go/quic-go/blob/master/send_stream.go#L389
https://github.com/quic-go/quic-go/blob/master/receive_stream.go#L286

These parts of the library seems to handle read/write fails and return this specific error.

Potentially due to some server-side issue at cloudflare or network dropouts, though I have no way to test that theory.

@HerbCSO
Copy link

HerbCSO commented Jul 11, 2024 •
edited
Loading

I just ran into a similar problem (logs below). Also solved with a restart. Been running these tunnels for almost a year now and this is the first time I've encountered this. I have Uptime Kuma running and checking these connections, this started suddenly at 2024-07-11T09:09:23Z with these error messages, nothing I can find prior to that.

Strangely only 3 of my 5 active tunnels were affected. 2 of the tunnels connected to the same target host came down at the same time, 1 more (on a different target host) went down 10 minutes later. 2 of the connections came back by themselves, one after 2 minutes, the other one came back after 5 minutes. That last one went down again 13 minutes after that, then back up after 7 minutes, down again 13 minutes later, and back up 7 minutes after that, then stayed up until I restarted the tunnel. 🤷🏼‍♂️

2024-07-11T10:07:20Z ERR Request failed error="stream 38293 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:07:20Z ERR  error="stream 38281 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:07:20Z ERR Request failed error="stream 38281 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:07:20Z ERR  error="stream 38285 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:07:20Z ERR Request failed error="stream 38285 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:07:40Z ERR  error="stream 38305 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:07:40Z ERR Request failed error="stream 38305 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38313 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38313 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38317 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38317 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38325 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR  error="stream 38321 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38321 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38337 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38337 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38333 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38333 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR Request failed error="stream 38325 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38329 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38329 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38341 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38341 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38349 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38349 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38345 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38345 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:02Z ERR  error="stream 38357 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:08:02Z ERR Request failed error="stream 38357 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:08:19Z ERR  error="stream 38381 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=2 originService=<REDACTED>
2024-07-11T10:08:19Z ERR Request failed error="stream 38381 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:09:17Z ERR  error="stream 38401 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:09:17Z ERR Request failed error="stream 38401 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:10:17Z ERR  error="stream 38417 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=2 originService=<REDACTED>
2024-07-11T10:10:17Z ERR Request failed error="stream 38417 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:05Z ERR  error="stream 38493 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:05Z ERR Request failed error="stream 38493 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38413 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38413 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38453 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38453 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38433 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38433 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38449 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38449 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38445 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38445 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38465 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38465 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38461 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38461 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38469 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38469 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:46Z ERR  error="stream 38473 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:46Z ERR Request failed error="stream 38473 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:11:52Z ERR  error="stream 38485 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:11:52Z ERR Request failed error="stream 38485 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:12:05Z ERR  error="stream 38513 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=2 originService=<REDACTED>
2024-07-11T10:12:05Z ERR Request failed error="stream 38513 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38517 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38517 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38529 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38529 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38549 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38549 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38533 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38533 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38557 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38557 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:44Z ERR  error="stream 38553 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=5 originService=<REDACTED>
2024-07-11T10:13:44Z ERR Request failed error="stream 38553 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:13:53Z ERR  error="stream 38589 canceled by remote with error code 0" connIndex=3 event=1 ingressRule=2 originService=<REDACTED>
2024-07-11T10:13:53Z ERR Request failed error="stream 38589 canceled by remote with error code 0" connIndex=3 dest=<REDACTED> event=0 ip=<REDACTED> type=http
2024-07-11T10:14:08Z INF Initiating graceful shutdown due to signal terminated ...
2024-07-11T10:14:08Z INF Unregistered tunnel connection connIndex=1 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z INF Unregistered tunnel connection connIndex=0 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z ERR Failed to serve quic connection error="context canceled" connIndex=0 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z INF Unregistered tunnel connection connIndex=2 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z ERR Failed to serve quic connection error="context canceled" connIndex=2 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=<REDACTED>
2024/07/11 10:14:08 rpc: writing finish: Application error 0x0 (remote)
2024-07-11T10:14:08Z ERR Failed to serve quic connection error="Application error 0x0 (remote)" connIndex=1 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z ERR Serve tunnel error error="Application error 0x0 (remote)" connIndex=1 event=0 ip=<REDACTED>
2024-07-11T10:14:08Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=<REDACTED>
2024-07-11T10:14:28Z INF Starting tunnel tunnelID=9cf36c79-de77-4db2-b714-048fcb641f0d
2024-07-11T10:14:28Z INF Version 2024.6.1
2024-07-11T10:14:28Z INF GOOS: linux, GOVersion: go1.22.2-devel-cf, GoArch: amd64
2024-07-11T10:14:28Z INF Settings: map[metrics:cloudflare-tunnel:10082 no-autoupdate:true token:*****]
2024-07-11T10:14:28Z INF Generated Connector ID: 3c21ff02-8816-4356-a2fb-ed2baaa2e91f
2024-07-11T10:14:28Z INF Initial protocol quic
2024-07-11T10:14:28Z INF ICMP proxy will use <REDACTED> as source for IPv4
2024-07-11T10:14:28Z INF ICMP proxy will use ::1 in zone lo as source for IPv6
2024-07-11T10:14:28Z INF Starting metrics server on <REDACTED>:10082/metrics
2024-07-11T10:14:29Z INF Registered tunnel connection connIndex=0 connection=deffd49b-6b1e-42df-9d79-e18f84614822 event=0 ip=<REDACTED> location=ewr07 protocol=quic
2024-07-11T10:14:29Z INF Registered tunnel connection connIndex=1 connection=f89b2109-5159-47ba-ac02-9c8af79a2023 event=0 ip=<REDACTED> location=ewr13 protocol=quic
2024-07-11T10:14:30Z INF Registered tunnel connection connIndex=2 connection=a1c198ed-08a1-474b-bc99-f43e119ea184 event=0 ip=<REDACTED> location=ewr07 protocol=quic
2024-07-11T10:14:31Z INF Registered tunnel connection connIndex=3 connection=e84c69da-313e-4f46-a06b-a8e0364563c4 event=0 ip=<REDACTED> location=ewr13 protocol=quic
2024-07-11T10:14:32Z INF Updated to new configuration config="{\"ingress\":[{\"hostname\":\"<REDACTED>\",\"service\":\"<REDACTED>"},{\"hostname\":\"<REDACTED>\",\"originRequest\":{},\"service\":\"<REDACTED>"},{\"hostname\":\"<REDACTED>\",\"originRequest\":{},\"service\":\"<REDACTED>"},{\"hostname\":\"<REDACTED>\",\"originRequest\":{},\"service\":\"<REDACTED>"},{\"hostname\":\"<REDACTED>\",\"originRequest\":{\"http2Origin\":true},\"service\":\"<REDACTED>"},{\"hostname\":\"<REDACTED>\",\"originRequest\":{\"http2Origin\":false},\"service\":\"<REDACTED>"},{\"service\":\"http_status:404\"}],\"warp-routing\":{\"enabled\":false}}" version=63
2024/07/11 10:14:32 rpc: abort: rpc: aborted by remote: rpc: shutdown

@tarheelz
Copy link

Seeing the same thing. Restart of the Cloudflare add-on (a docker container) in Home Assistant fixes things for a few hours. I would post my logs but they are pretty much the same as everyone else's. Hate this. Cloudflare has been reliable for a very long time for me.

@Krakazybik
Copy link

Exactly the same issue occurs when connecting from a specific device and network (iPhone + cellular). When sharing the internet from the phone and connecting from a PC, there is no problem. When connecting from the phone via WiFi, there is no problem. Restarting cloudflared temporarily solves the issue.

@jhult
Copy link

jhult commented Jul 22, 2024 •
edited
Loading

We are also bumping into this issue. Restarting cloudflared fixes it (for a while).

@mshelto59
Copy link

Same Issue as above.
We had to discontinue usage as the tunnel was too unstable for production.
We've been using it for about a year, and this issue just appeared.

@zv3
Copy link

zv3 commented Jul 23, 2024

Have been using cloudflared for about a month without issues until last Sunday. Had to restart it and might need to add it to the crontab as I'm still seeing issues every other day since then.

@UsernameIHardlyKnowHerName
Copy link

What version of cloudflared is everyone running that is having this issue?

@irateadmin
Copy link

We also have this exact issue. We are running version 2024.6.1.

@14wkinnersley
Copy link

Same issue here. 2024.6.1 but the issue only started up within the past few days.

@irateadmin
Copy link

Our issue started on 7/19/2024 and has been randomly reoccurring since then. We have a ticket open with support and provided this thread as a reference. If I get a resolution I’ll make sure to share.

@dledfordcf
Copy link

Hey everyone, not a CF engineer here, just a support guy but to keep from blowing up this random person's post with comments I opened a bug report under this link #1300 lets move to leaving comments there as it will also give me a bit of visibility into people reporting this outside of our ticket system.

Definitely feel free to also open tickets for this as well, or your own Github bug reports, as mine just has generalized info.

@825i
Copy link

825i commented Jul 25, 2024

Our issue started on 7/19/2024 and has been randomly reoccurring since then. We have a ticket open with support and provided this thread as a reference. If I get a resolution I’ll make sure to share.

Sad to hear I am not the only one. This has started happening to me also in a recent update and none of my services resolve at all but load indefinitely and then time out with an HTTP 524 error.

Would be super grateful for a fix.

@irateadmin
Copy link

Wanted to respond for anyone who isn't on the other issue. It appears to be fixed now.

#1300 (comment)

@baptistejamin
Copy link

Hey there.

We are having the exact same issue, and it seems to be related to how Cloudflare handles peer discovery. Cloudflared stills picks proxy locations that can be monitored as "rerouted" from Cloudflare status page.

I updated our analysis here: #1360 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Priority: Normal Minor issue impacting one or more users Type: Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests