proxy: fix flaky TestSlowService

[AlexanderYastrebov]

backendTimeout("1ms") in TestSlowService may cancel context during
dialing which results in i/o timeout error and 502 response status

$ go test ./proxy -run=TestSlowService -count=10000 | grep -F 'FAIL:' -C1
...
time="2023-08-18T20:50:57+02:00" level=error msg="error while proxying after 1.295026ms, route  with backend network http://127.0.0.1:38613/, status code 502: dialing failed true: failed to do backend roundtrip to 127.0.0.1:38613: dial tcp 127.0.0.1:38613: i/o timeout, remote host: 127.0.0.1, request: \"GET / HTTP/1.1\", host: 127.0.0.1:36801, user agent: \"Go-http-client/1.1\""
--- FAIL: TestSlowService (0.00s)
    backendtimeout_test.go:40: expected 504, got: &{502 Bad Gateway 502 HTTP/1.1 1 1 map[Content-Length:[12] Content-Type:[text/plain; charset=utf-8] Date:[Fri, 18 Aug 2023 18:50:57 GMT] Server:[Skipper] X-Content-Type-Options:[nosniff]] 0xc00068e100 12 [] false false map[] 0xc000255600 <nil>}

Increase TestSlowService filter parameter to 10ms.
Also make requests using test proxy client.

[AlexanderYastrebov]

#1566 (comment) 🔮

[szuecs]

Make it 1ms->100ms
We have the time 😉

Think about the computer that runs tests as the cheapest computer you can built. It runs as a shared machine that will be on average at >80% load or more.

[AlexanderYastrebov]

--- FAIL: TestTeeEndToEndBody2TeeRoutesAndClosing (0.00s)
    tee_test.go:268: shadow handler got a request, but should not
panic: close of closed channel [recovered]
	panic: close of closed channel

goroutine 49 [running]:
testing.tRunner.func1.2({0x11cb720, 0x141f9e0})
	/usr/local/go/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x397
panic({0x11cb720?, 0x141f9e0?})
	/usr/local/go/src/runtime/panic.go:914 +0x21f
runtime.Goexit()
	/usr/local/go/src/runtime/panic.go:523 +0x145
testing.(*common).FailNow(0xc000103ba0)
	/usr/local/go/src/testing/testing.go:999 +0x4a
testing.(*common).Fatal(0xc000103ba0, {0xc0001f7b70?, 0x0?, 0x0?})
	/usr/local/go/src/testing/testing.go:1076 +0x54
github.com/zalando/skipper/filters/tee.TestTeeEndToEndBody2TeeRoutesAndClosing(0xc000103ba0)
	/workspace/filters/tee/tee_test.go:268 +0xee5
testing.tRunner(0xc000103ba0, 0x134b030)
	/usr/local/go/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1648 +0x3ad

check-race:

--- FAIL: TestResponseHeaderTimeout (0.00s)
    proxy_test.go:1574: expected 504, got: &{200 OK 200 HTTP/1.1 1 1 map[Content-Length:[0] Date:[Fri, 18 Aug 2023 19:43:17 GMT] Server:[Skipper]] {} 0 [] false false map[] 0xc000175800 <nil>}

--- FAIL: TestConnectionRefused (0.98s)
    --- FAIL: TestConnectionRefused/Groups_with_only_failing_members_should__fail_on_multiple_calls_d (0.35s)
        lb_failingbackend_test.go:185: loadbalanced request 51 to /d should have statuscode != 502: 404

[AlexanderYastrebov]

Make it 1ms->100ms

Other tests use x*10ms timeouts and I only saw this test flaking so far. I propose we merge this fix and maybe rethink the whole file if the problem re-appears.

[AlexanderYastrebov]

👍

[MustafaSaber]

👍