Caddy doesn't forward truncated responses

[garak95]

Issue Details

When an upstream responds with a non-zero Content-Length but then closes the connection before sending the content (or sends too small amount of content), Caddy will drop the connection and the client will not receive anything, instead the client connection is abruptly closed.

I'm assuming that Caddy is doing some buffering, realizes the mismatch, and drops everything. However I would expect caddy to be a little more transparent: as long as the response had fully formed headers caddy should pass it along.

To reproduce:
reverse_proxy 10.0.0.2:80 and cause the backend to issue HTTP 500 with Content-Length 123 but then close the connection right after the headers.

Curl sessions

Direct backend response (what I would expect caddy to pass along)

curl http://example.com/demo.mkv -v --resolve example.com:80:10.0.0.8
* Added example.com:80:10.0.0.8 to DNS cache
* Hostname example.com was found in DNS cache
*   Trying 10.0.0.8:80...
* Connected to example.com (10.0.0.8) port 80
* using HTTP/1.x
> GET /demo.mkv HTTP/1.1
> Host: example.com
> User-Agent: curl/8.14.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 500 Internal Server Error
< Server: nginx
< Date: Sun, 28 Jun 2026 16:13:57 GMT
< Content-Type: video/x-matroska
< Content-Length: 135708
< Connection: keep-alive
< Content-Disposition: inline; filename="demo.mkv"
< Accept-Ranges: bytes
<
* end of response with 135708 bytes missing
* closing connection #0
curl: (18) end of response with 135708 bytes missing

What caddy responds instead:

curl https://example.com/demo.mkv -v
* Host example.com:443 was resolved.
* IPv6: (none)
* IPv4: 1.2.3.4
*   Trying 1.2.3.4:443...
* ALPN: curl offers h2,http/1.1
[TLS STUFF...]
* Connected to example.com (1.2.3.4) port 443
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://example.com/demo.mkv
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: example.com]
* [HTTP/2] [1] [:path: /demo.mkv]
* [HTTP/2] [1] [user-agent: curl/8.14.1]
* [HTTP/2] [1] [accept: */*]
> GET /demo.mkv HTTP/2
> Host: example.com
> User-Agent: curl/8.14.1
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Request completely sent off
* HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
* Connection #0 to host example.com left intact
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

To make sure it wasn't HTTP2 weirdness, I forced caddy to respond in http 1.1 without tls:

curl http://example.com/demo.mkv -v
* Host example.com:80 was resolved.
* IPv6: (none)
* IPv4: 1.2.3.4
*   Trying 1.2.3.4:80...
* Connected to example.com (1.2.3.4) port 80
* using HTTP/1.x
> GET /demo.mkv HTTP/1.1
> Host: example.com
> User-Agent: curl/8.14.1
> Accept: */*
>
* Request completely sent off
* Empty reply from server
* shutting down connection #0
curl: (52) Empty reply from server

The stderr output

{"level":"warn","ts":1782663200.3506744,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"10.0.0.8:80","duration":0.000743837,"request":{"remote_ip":"10.0.0.1","remote_port":"35568","client_ip":"10.0.0.1","proto":"HTTP/1.1","method":"GET","host":"example.com","uri":"/demo.mkv","headers":{"X-Forwarded-Proto":["http"],"X-Forwarded-Host":["example.com"],"Via":["1.1 Caddy"],"Accept":["*/*"],"User-Agent":["curl/8.14.1"],"X-Forwarded-For":["10.0.0.1"]}},"error":"reading: unexpected EOF"}

Assistance Disclosure

AI not used

If AI was used, describe the extent to which it was used.

No response

[mohammed90]

However I would expect caddy to be a little more transparent: as long as the response had fully formed headers caddy should pass it along.

Why? Where is this specified? Can you cite the RFC?

[garak95]

Why? Where is this specified? Can you cite the RFC?

Sorry I didn't try to imply that Caddy was breaking any RFC. What I did mean is that Caddy should always give a response to the client, it shouldn't forcefully close the connection and let the user be faced with a weird SSL-is-broken (or connection reset if http) error.

It doesn't have to forward things from upstream if you feel like it's unwise, but it should then issue a 50X of its own, in my opinion.

[mohammed90]

I was just asking for supporting material for forwarding truncated data. I may have been too blunt. Sorry.

Replying with 50x feels more correct.

[WeidiDeng]

I created a simple backend and confirmed it's indeed a bug,

I'll show the result first before showing the codes needed to replicate this,

Test Case	Connection Type	HTTP Status Code	curl Output / Error	Behavior Summary
1. Header Only	Direct (:9999)	500 Internal Server Error	curl: (18) transfer closed with 500 bytes remaining to read	Server sent headers with Content-Length: 500 but dropped the connection before sending the body.
	Caddy (:10000)	None	curl: (52) Empty reply from server	Caddy swallowed the incomplete header response and dropped the client connection.
2. Partial Body	Direct (:9999)	500 Internal Server Error	curl: (18) ... 493 bytes remaining to read Body: partial	Server sent headers and a 7-byte body (partial), then prematurely closed the connection before reaching the promised 500 bytes.
	Caddy (:10000)	None	curl: (52) Empty reply from server	Caddy caught the premature closure from the upstream and dropped the client connection entirely without forwarding the partial data.
3. Full Body	Direct (:9999)	500 Internal Server Error	Connection left intact. Body: Full 500 bytes	Successful error response. Upstream sent headers and the full body matching Content-Length.
	Caddy (:10000)	500 Internal Server Error	Connection left intact. Body: Full 500 bytes	Successful proxying. Caddy successfully forwarded the complete headers and body, adding its own Via: 1.1 Caddy header.
4. No Response	Direct (:9999)	None	curl: (52) Empty reply from server	Upstream accepted the connection but closed it immediately without sending any HTTP data.
	Caddy (:10000)	502 Bad Gateway	Connection left intact. Content-Length: 0	Caddy handled the upstream's immediate drop gracefully by returning a standard 502 Bad Gateway error to the client.

Codes

package main

import (
	"io"
	"net/http"
)

func main() {
	http.HandleFunc("/header", func(writer http.ResponseWriter, request *http.Request) {
		writer.Header().Set("Content-Length", "500")
		writer.WriteHeader(500)
		_ = http.NewResponseController(writer).Flush()
		panic(http.ErrAbortHandler)
	})
	http.HandleFunc("/partial", func(writer http.ResponseWriter, request *http.Request) {
		writer.Header().Set("Content-Length", "500")
		writer.WriteHeader(500)
		_, _ = io.WriteString(writer, "partial")
		_ = http.NewResponseController(writer).Flush()
		panic(http.ErrAbortHandler)
	})
	http.HandleFunc("/full", func(writer http.ResponseWriter, request *http.Request) {
		writer.Header().Set("Content-Length", "500")
		writer.WriteHeader(500)
		for range 125 {
			_, _ = io.WriteString(writer, "full")
		}
		_ = http.NewResponseController(writer).Flush()
		panic(http.ErrAbortHandler)
	})
	http.HandleFunc("/empty", func(writer http.ResponseWriter, request *http.Request) {
		panic(http.ErrAbortHandler)
	})
	panic(http.ListenAndServe("127.0.0.1:9999", nil))
}

Caddyfile is

http://127.0.0.1:10000 {
        reverse_proxy 127.0.0.1:9999
}

Raw curl outputs when requesting the backend directly or through caddy.

1. header only

Direct

curl 127.0.0.1:9999/header -v
*   Trying 127.0.0.1:9999...
* Connected to 127.0.0.1 (127.0.0.1) port 9999
> GET /header HTTP/1.1
> Host: 127.0.0.1:9999
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Content-Length: 500
< Date: Tue, 30 Jun 2026 00:55:21 GMT
< 
* transfer closed with 500 bytes remaining to read
* Closing connection
curl: (18) transfer closed with 500 bytes remaining to read

Caddy

curl 127.0.0.1:10000/header -v
*   Trying 127.0.0.1:10000...
* Connected to 127.0.0.1 (127.0.0.1) port 10000
> GET /header HTTP/1.1
> Host: 127.0.0.1:10000
> User-Agent: curl/8.5.0
> Accept: */*
> 
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

2. partial body

Direct

curl 127.0.0.1:9999/partial -v
*   Trying 127.0.0.1:9999...
* Connected to 127.0.0.1 (127.0.0.1) port 9999
> GET /partial HTTP/1.1
> Host: 127.0.0.1:9999
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Content-Length: 500
< Date: Tue, 30 Jun 2026 01:01:01 GMT
< Content-Type: text/plain; charset=utf-8
< 
* transfer closed with 493 bytes remaining to read
* Closing connection
curl: (18) transfer closed with 493 bytes remaining to read
partial

Caddy

curl 127.0.0.1:10000/partial -v
*   Trying 127.0.0.1:10000...
* Connected to 127.0.0.1 (127.0.0.1) port 10000
> GET /partial HTTP/1.1
> Host: 127.0.0.1:10000
> User-Agent: curl/8.5.0
> Accept: */*
> 
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

3. full body

Direct

curl 127.0.0.1:9999/full -v
*   Trying 127.0.0.1:9999...
* Connected to 127.0.0.1 (127.0.0.1) port 9999
> GET /full HTTP/1.1
> Host: 127.0.0.1:9999
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Content-Length: 500
< Date: Tue, 30 Jun 2026 01:01:21 GMT
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host 127.0.0.1 left intact
fullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfull

Caddy

curl 127.0.0.1:10000/full -v
*   Trying 127.0.0.1:10000...
* Connected to 127.0.0.1 (127.0.0.1) port 10000
> GET /full HTTP/1.1
> Host: 127.0.0.1:10000
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Content-Length: 500
< Content-Type: text/plain; charset=utf-8
< Date: Tue, 30 Jun 2026 01:00:13 GMT
< Via: 1.1 Caddy
< 
* Connection #0 to host 127.0.0.1 left intact
fullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfullfull

4. no response

Direct

curl 127.0.0.1:9999/empty -v
*   Trying 127.0.0.1:9999...
* Connected to 127.0.0.1 (127.0.0.1) port 9999
> GET /empty HTTP/1.1
> Host: 127.0.0.1:9999
> User-Agent: curl/8.5.0
> Accept: */*
> 
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

Caddy

curl 127.0.0.1:10000/empty -v
*   Trying 127.0.0.1:10000...
* Connected to 127.0.0.1 (127.0.0.1) port 10000
> GET /empty HTTP/1.1
> Host: 127.0.0.1:10000
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 502 Bad Gateway
< Server: Caddy
< Date: Tue, 30 Jun 2026 01:00:34 GMT
< Content-Length: 0
< 
* Connection #0 to host 127.0.0.1 left intact

[kiwipaulrob]

I can confirm this bug across multiple versions. Here is a detailed reproduction:

Environment

• Caddy versions tested: v2.9.1, v2.11.3, v2.11.4 — all affected
• OS: Ubuntu 24.04.4 LTS on Proxmox LXC container
• Config: 24 reverse_proxy hostname blocks proxying to various backends (Docker containers, VMs, localhost services)

Symptoms

• HTTP 200 with Content-Length: 0 on approximately 30-50% of requests
• Intermittent — same hostname works on some requests, returns 0 bytes on others
• No errors in Caddy logs
• --no-keepalive on the client side does not help
• transport http { keepalive off } does not help
• max_conns_per_host 1 does not help
• Wiping persistent state (rm -rf /root/.config/caddy /root/.local/share/caddy) temporarily improves but does not fix

Backends

• Multiple backends affected: Docker containers on different IPs, VMs, localhost services
• Direct curl to backends works 100% of the time (10/10 requests)
• nginx with identical reverse_proxy config works 100% of the time (10/10 requests)

Workaround

Replaced Caddy with nginx. All 24 hostnames now return full responses consistently.

Happy to provide Caddyfile, debug logs, or run specific tests if helpful.

[mohammed90]

@kiwipaulrob, it's not clear what your issue is, honestly. The particular scenario of the OG issue shouldn't be consistent and recurrent often in a proper network or robust upstream/backend.