WIP: puppet dry run

[kragniz]

Fixes #224.

Allow performing dry-run of puppet modules.

[jetstack-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: simonswine

If they are not already assigned, you can assign the PR to them by writing /assign @simonswine in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[simonswine]

OK I am running things locally now like that:

tmux has-session -t wing || tmux new-session -d -s wing /bin/zsh
tmux new-window -t wing:1 -n wing-server "./wing_linux_amd64 server --secure-port 8443  --etcd-servers http://127.0.0.1:2379"
tmux new-window -t wing:2 -n etcd "etcd"

How can I reproduce those time outs, which operations cause them?

[simonswine]

/assign @kragniz

[kragniz]

curl -k https://localhost:4433/apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs

[simonswine]

curl -vk https://localhost:8443/apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=localhost@1531923564
*  start date: Jul 18 14:19:24 2018 GMT
*  expire date: Jul 18 14:19:24 2019 GMT
*  issuer: CN=localhost@1531923564
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562e8173af80)
> GET /apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs HTTP/2
> Host: localhost:8443
> User-Agent: curl/7.61.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< content-type: application/json
< content-length: 218
< date: Wed, 18 Jul 2018 14:46:52 GMT
<
{
  "kind": "WingJobList",
  "apiVersion": "wing.tarmak.io/v1alpha1",
  "metadata": {
    "selfLink": "/apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs",
    "resourceVersion": "2"
  },
  "items": []
* Connection #0 to host localhost left intact
}

[simonswine]

etcd --version
etcd Version: 3.1.7

[kragniz]

 g/s/g/j/tarmak {puppet-dry-run} $ etcd                             1  
2018-07-18 15:56:26.478484 I | etcdmain: etcd Version: 3.2.8
2018-07-18 15:56:26.478547 I | etcdmain: Git SHA: e211fb6
2018-07-18 15:56:26.478553 I | etcdmain: Go Version: go1.8.3
2018-07-18 15:56:26.478558 I | etcdmain: Go OS/Arch: linux/amd64
2018-07-18 15:56:26.478563 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2018-07-18 15:56:26.478572 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
2018-07-18 15:56:26.478829 I | embed: listening for peers on http://localhost:2380

g/s/g/j/tarmak {puppet-dry-run} $ ./wing_linux_amd64 server --etcd-servers localhost --secure-port 4433 -v 1000                       1m 5s 405ms 
W0718 15:56:32.296740   10475 authorization.go:34] Authorization is disabled
W0718 15:56:32.296808   10475 authentication.go:56] Authentication is disabled
I0718 15:56:32.297473   10475 storage_factory.go:48] Storage caching is enabled for *wing.Instance with capacity 100
I0718 15:56:32.298163   10475 compact.go:54] compactor already exists for endpoints [localhost]
I0718 15:56:32.298317   10475 storage_factory.go:48] Storage caching is enabled for *wing.WingJob with capacity 100
I0718 15:56:32.298236   10475 reflector.go:240] Listing and watching *wing.Instance from storage/cacher.go:/wing.tarmak.io/instances
I0718 15:56:32.299188   10475 reflector.go:240] Listing and watching *wing.WingJob from storage/cacher.go:/wing.tarmak.io/wingjobs
I0718 15:56:32.300851   10475 healthz.go:74] Installing healthz checkers:"ping", "etcd"
I0718 15:56:32.301053   10475 serve.go:89] Serving securely on [::]:4433
I0718 15:56:36.473862   10475 handler.go:150] wing: GET "/apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs" satisfied by gorestful with webservice /apis/wing.tarmak.io/v1alpha1
I0718 15:57:36.474126   10475 wrap.go:42] GET /apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs: (1m0.000452935s) 504
goroutine 98 [running]:
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc420960c40, 0x1f8)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc420960c40, 0x1f8)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).timeout(0xc4203d4140, 0xc4200226c0)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:201 +0xb3
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP(0xc420195060, 0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:101 +0x26c
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters.WithWaitGroup.func1(0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters/waitgroup.go:51 +0xeb
net/http.HandlerFunc.ServeHTTP(0xc42041cc80, 0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithRequestInfo.func1(0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/endpoints/filters/requestinfo.go:45 +0x1f9
net/http.HandlerFunc.ServeHTTP(0xc42041ccc0, 0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/endpoints/request.WithRequestContext.func1(0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/endpoints/request/requestcontext.go:110 +0xcb
net/http.HandlerFunc.ServeHTTP(0xc4201950c0, 0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters.WithPanicRecovery.func1(0x1c114c0, 0xc420960c40, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/filters/wrap.go:41 +0x108
net/http.HandlerFunc.ServeHTTP(0xc420195100, 0x1c12040, 0xc42020e0c8, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server.(*APIServerHandler).ServeHTTP(0xc420412cf0, 0x1c12040, 0xc42020e0c8, 0xc42040ae00)
	/home/louis/go/src/github.com/jetstack/tarmak/vendor/k8s.io/apiserver/pkg/server/handler.go:198 +0x51
net/http.serverHandler.ServeHTTP(0xc420387450, 0x1c12040, 0xc42020e0c8, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:2694 +0xbc
net/http.initNPNRequest.ServeHTTP(0xc420238380, 0xc420387450, 0x1c12040, 0xc42020e0c8, 0xc42040ae00)
	/usr/local/go/src/net/http/server.go:3260 +0x9a
net/http.(Handler).ServeHTTP-fm(0x1c12040, 0xc42020e0c8, 0xc42040ae00)
	/usr/local/go/src/net/http/h2_bundle.go:5475 +0x4d
net/http.(*http2serverConn).runHandler(0xc420218000, 0xc42020e0c8, 0xc42040ae00, 0xc4203b6520)
	/usr/local/go/src/net/http/h2_bundle.go:5760 +0x89
created by net/http.(*http2serverConn).processHeaders
	/usr/local/go/src/net/http/h2_bundle.go:5494 +0x46b

logging error output: "{\"metadata\":{},\"status\":\"Failure\",\"message\":\"Timeout: request did not complete within 1m0s\",\"reason\":\"Timeout\",\"details\":{},\"code\":504}\n"
 [[curl/7.59.0] 127.0.0.1:44262]

 g/s/g/j/tarmak {puppet-dry-run} $ 
curl -k https://localhost:4433/apis/wing.tarmak.io/v1alpha1/namespaces/myenv-mycluster/wingjobs
{"metadata":{},"status":"Failure","message":"Timeout: request did not complete within 1m0s","reason":"Timeout","details":{},"code":504}

[jetstack-bot]

@kragniz: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[simonswine]

This depends on #663 which is taken out of 0.6

[jetstack-bot]

@kragniz: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
tarmak-quick-verify	98d321f	link	/test verify quick
tarmak-puppet-module-tarmak-acceptance-1-14-centos	98d321f	link	/test puppet-tarmak-acceptance-centos v1.14

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.