Implements route backend timeout

[AlexanderYastrebov]

• Adds backendTimeout filter to configure route backend timeout
• Proxy sets up request context with configured timeout and responds
  with 504 status on timeout (note: if response streaming has already
  started it will be terminated, client will receive backend status and
  truncated response body).

See #1041

[AlexanderYastrebov]

If response truncation is not acceptable I see three ways out:

• declare it a feature for limited usecases :)
• try to workaround/disable streaming termination
• attempt another approach - consider timeout as Transport.ResponseHeaderTimeout, keep a pool of transports per configured timeout (cleanup?) and wrap roundtripper around here

  skipper/proxy/proxy.go

  Line 951 in 8f32650

  response, err = p.roundTripper.RoundTrip(req)

[AlexanderYastrebov]

/cc @otrosien

[AlexanderYastrebov]

There is a general problem to notify client about backend that has stuck. At the moment copyStream stops writing on backend read error:

skipper/proxy/proxy.go

Lines 411 to 421 in 8f32650

	func copyStream(to flushedResponseWriter, from io.Reader, tracing *proxyTracing, span ot.Span) error {
	b := make([]byte, proxyBufferSize)
	for {
	l, rerr := from.Read(b)
	tracing.logStreamEvent(span, StreamBodyEvent, fmt.Sprintf("%d", l))
	if rerr != nil && rerr != io.EOF {
	return rerr
	}

which leads to truncated response.

httputil.ReverseProxy uses panic(http.ErrAbortHandler), that closes client connection, see golang/go#23643 (not sure what happens to multistream http/2 connection on this)

[aryszka]

If response truncation is not acceptable I see three ways out:
...
try to workaround/disable streaming termination

I believe this is possible if we implement our variant of context.Context and signal to it various phases like request headers sent, request stream done, response headers received, response stream done. But in my opinion, the current implementation is a good start, and covers a reasonable majority of the use cases.

[aryszka]

I would allow multiple filters where the last one would take effect (see my comment for the timeout.Request method. Apart of that, lgtm

[szuecs]

Code looks good to me.
I am pretty sure tests will hurt in the future (1ms/2ms sleeps to separate cases will break too often)

[AlexanderYastrebov]

This now covers client cancellation and backend request timeout

[szuecs]

nice!

[AlexanderYastrebov]

I have rebased and updated tests as suggested

[szuecs]

👍

[aryszka]

Thanks a lot!

[aryszka]

👍