No recovery for persistent/pipelined HTTP requests after packet loss

[apekoolp]

First Observations

Zeek v6.0.1. PCAP with persistent HTTP sessions, some of them showing packet loss (almost exclusively client to server side). For sessions with packet loss, the fields from the HTTP requests are not shown in the http.log, but the response fields are present and filled correctly. Looking in the PCAP itself I see that the http requests show packet loss but plenty are also complete and could be shown in the http.log (but are not). Could it be that the HTTP request parsing is stopped completely for the full TCP session once packet loss is detected whereas the response parsing is continuing?

Analysis after discussion on Slack

It seems the intention was to stop parsing requests and replies if there are gaps in the client's request (when they are not fully within the body). There's no recovery/re-synchronization.

zeek/src/analyzer/protocol/http/HTTP.cc

Lines 1007 to 1021 in 8f56140

	if ( is_orig ) {
	// Stop parsing reply messages too, because whether a
	// reply contains a body may depend on knowing the
	// request method
	RequestMade(true, "message interrupted by a content gap");
	ReplyMade(true, "message interrupted by a content gap");
	content_line->SetSkipDeliveries(true);
	}
	else {
	ReplyMade(true, "message interrupted by a content gap");
	content_line->SetSkipDeliveries(true);
	}
	}

Resolution
Current behavior is reasonable for a single request response pair. It goes wrong for persistent or pipelined HTTP. Recovering for subsequent request responses would be helpful.

[0xxon]

Hi,

I don't think that this is something that we will try fixing - at least not in the current version of the analyzer. Binpac does not really lend itself to resynchronization.

Apart from that, one always has the issue that it can become hard correctly resynchronize - consider, e.g., someone downloading some text file that contains HTTP headers (like an RFC). This will probably end up with hillarious results. Which might be acceptable - if one marks them correctly in logs - but is definitely something to consider.