Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WbaPA access CPE testing with "401 Unauthorized" message #498

Open
VicLin66 opened this issue Mar 22, 2024 · 14 comments
Open

WbaPA access CPE testing with "401 Unauthorized" message #498

VicLin66 opened this issue Mar 22, 2024 · 14 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@VicLin66
Copy link

VicLin66 commented Mar 22, 2024

Hi,
Followed with scytale issue xmidt-org/scytale#350, the default talaria port is 6200, and I modify it to 6400 (for meet talaria.yaml), and restart again.
My CPE parodus client can accessed with talaria server (:6400),

talaria periodic log,
{"key":"debug","ts":"2024-03-22T15:29:49+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"192.168.1.1:53080"}
{"key":"debug","ts":"2024-03-22T15:29:52+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"192.168.1.1:52770"}

with below WebPA server terminal curl command result,
does log "401 Unauthorized" means curl command access with CPE success or fail?

LOG:

$ curl -i -H 'Authorization: Basic dXNlcjpwYXNz' http://localhost:6100/api/v3/device/mac:0050f1a1d74e/stat
HTTP/1.1 401 Unauthorized
X-Midt-Server: tr1d1um
X-Midt-Version: development
X-Scytale-Build: 0.1.4
X-Scytale-Flavor: mint
X-Scytale-Region: east
X-Scytale-Server: scytale
X-Scytale-Start-Time: 21 Mar 24 05:43 UTC
X-Talaria-Build: 0.1.4
X-Talaria-Flavor: mint
X-Talaria-Region: east
X-Talaria-Server: talaria
X-Talaria-Start-Time: 21 Mar 24 05:43 UTC
X-Webpa-Transaction-Id: kqvqOOypGOYErzThyw7n6w
X-Xmidt-Span: "http://localhost:6400/api/v3/device/mac:0050f1a1d74e/stat","2024-03-21T05:50:03Z","930.253µs"
Date: Thu, 21 Mar 2024 05:50:03 GMT
Content-Length: 0

# tr1d1um

{"key":"debug","ts":"2024-03-21T13:50:03+08:00","message":"authentication accepted by enforcer","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v3/device/mac:0050f1a1d74e/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v3/device/mac:0050f1a1d74e/stat","request.query":"","request.tid":"FqRJU-a6MfUM5V8L0gLofg"}
{"key":"error","ts":"2024-03-21T13:50:03+08:00","message":"Request arrival not capture for logger","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v3/device/mac:0050f1a1d74e/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v3/device/mac:0050f1a1d74e/stat","request.query":"","request.tid":"FqRJU-a6MfUM5V8L0gLofg","satClientID":"user","deviceid":"mac:0050f1a1d74e","tid":"kqvqOOypGOYErzThyw7n6w"}
{"key":"info","ts":"2024-03-21T13:50:03+08:00","message":"response","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v3/device/mac:0050f1a1d74e/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v3/device/mac:0050f1a1d74e/stat","request.query":"","request.tid":"FqRJU-a6MfUM5V8L0gLofg","satClientID":"user","deviceid":"mac:0050f1a1d74e","response":{"code":401,"headers":{"X-Midt-Server":["tr1d1um"],"X-Midt-Version":["development"],"X-Scytale-Build":["0.1.4"],"X-Scytale-Flavor":["mint"],"X-Scytale-Region":["east"],"X-Scytale-Server":["scytale"],"X-Scytale-Start-Time":["21 Mar 24 05:43 UTC"],"X-Talaria-Build":["0.1.4"],"X-Talaria-Flavor":["mint"],"X-Talaria-Region":["east"],"X-Talaria-Server":["talaria"],"X-Talaria-Start-Time":["21 Mar 24 05:43 UTC"],"X-Webpa-Transaction-Id":["kqvqOOypGOYErzThyw7n6w"],"X-Xmidt-Span":["\"http://localhost:6400/api/v3/device/mac:0050f1a1d74e/stat\",\"2024-03-21T05:50:03Z\",\"930.253µs\""]}}}
# scytale

{"key":"debug","ts":"2024-03-21T13:50:03+08:00","message":"accepted connection","serverName":"scytale","bindAddress":":6300","listenNetwork":"tcp","listenAddress":"[::]:6300","remoteAddress":"[::1]:35710"}
{"key":"debug","ts":"2024-03-21T13:50:03+08:00","caller":"/home/runner/go/pkg/mod/github.com/xmidt-org/[email protected]/basculehttp/enforcer.go:102","message":"authentication accepted by enforcer","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:0050f1a1d74e/stat","method":"GET","ts":"2024-03-21T05:50:03Z"}
{"key":"debug","ts":"2024-03-21T13:50:03+08:00","message":"fanout request complete","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:0050f1a1d74e/stat","method":"GET","statusCode":401,"url":"http://localhost:6400/api/v3/device/mac:0050f1a1d74e/stat"}
{"key":"error","ts":"2024-03-21T13:50:03+08:00","message":"all fanout requests failed","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:0050f1a1d74e/stat","method":"GET","statusCode":401,"url":"/api/v3/device/mac:0050f1a1d74e/stat"}
{"key":"debug","ts":"2024-03-21T13:50:03+08:00","message":"wrote fanout response","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:0050f1a1d74e/stat","method":"GET","statusCode":401}
# talaria

{"key":"debug","ts":"2024-03-21T13:50:02+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"192.168.1.1:33516"}
{"key":"debug","ts":"2024-03-21T13:50:03+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"[::1]:42484"}
{"key":"error","ts":"2024-03-21T13:50:03+08:00","caller":"/home/runner/go/pkg/mod/github.com/xmidt-org/[email protected]/basculehttp/constructor.go:117","message":"key not supported: [Basic]","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"],"X-Webpa-Device-Name":["mac:0050f1a1d74e"]},"requestURL":"/api/v3/device/mac:0050f1a1d74e/stat","method":"GET","requestURI":"/api/v3/device/mac:0050f1a1d74e/stat","remoteAddr":"[::1]:42484","ts":"2024-03-21T05:50:03Z","auth":"Basic YXV0aEhlYWRlcg=="}

I also tried different command, still got "401" error,

$ curl -H 'Authorization:Basic dXNlcjpwYXNz' -i http://localhost:6100/api/v2/device/mac:0050f120d74e/config?names=Device.DeviceInfo.ModelName
HTTP/1.1 401 Unauthorized
X-Midt-Server: tr1d1um
X-Midt-Version: development
X-Scytale-Build: 0.1.4
X-Scytale-Flavor: mint
X-Scytale-Region: east
X-Scytale-Server: scytale
X-Scytale-Start-Time: 21 Mar 24 05:43 UTC
X-Talaria-Build: 0.1.4
X-Talaria-Flavor: mint
X-Talaria-Region: east
X-Talaria-Server: talaria
X-Talaria-Start-Time: 21 Mar 24 05:43 UTC
X-Webpa-Transaction-Id: E5t_Ti84LgOfuCkxmRHhlg
X-Xmidt-Span: "http://localhost:6400/api/v3/device/send","2024-03-22T07:29:59Z","849.033µs"
Date: Fri, 22 Mar 2024 07:29:59 GMT
Content-Length: 0

@VicLin66
Copy link
Author

Hi, is there other way to check tr1d1um/scytale/talaria all worked well?
ex, curl -vvv http://localhost:6100 (tr1d1um)
But currently is seems can't use this way.

$ curl -vvv http://localhost:6100
* About to connect() to localhost port 6100 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 6100 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:6100
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< X-Midt-Server: tr1d1um
< X-Midt-Version: development
< Date: Tue, 26 Mar 2024 10:30:02 GMT
< Content-Length: 19
< 
404 page not found
* Connection #0 to host localhost left intact

@VicLin66
Copy link
Author

Update more, hope can be clarified the issue here.
Is there got something wrong with talaria log "message":"key not supported:?
image
Here are also 401 error message in tr1d1um and scytale log with my test command "curl -i -H 'Authorization: Basic dXNlcjpwYXNz' http://localhost:6100/api/v3/device/mac:0050f1a1d74e/stat"

# HTTP 401 error messages
image

@denopink
Copy link
Contributor

Hi, is there other way to check tr1d1um/scytale/talaria all worked well? ex, curl -vvv http://localhost:6100 (tr1d1um) But currently is seems can't use this way.

$ curl -vvv http://localhost:6100
* About to connect() to localhost port 6100 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 6100 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:6100
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< X-Midt-Server: tr1d1um
< X-Midt-Version: development
< Date: Tue, 26 Mar 2024 10:30:02 GMT
< Content-Length: 19
< 
404 page not found
* Connection #0 to host localhost left intact

you can use any of tr1d1um's api to test that flow (tr1d1um<->scytale<->talaria)

if you want to test tr1d1um<->scytale<->talaria<->cpe, use https://github.com/xmidt-org/tr1d1um?tab=readme-ov-file#crud-operations---config-endpoints

@denopink
Copy link
Contributor

denopink commented Mar 29, 2024

Update more, hope can be clarified the issue here. Is there got something wrong with talaria log "message":"key not supported:? image Here are also 401 error message in tr1d1um and scytale log with my test command "curl -i -H 'Authorization: Basic dXNlcjpwYXNz' http://localhost:6100/api/v3/device/mac:0050f1a1d74e/stat"

# HTTP 401 error messages image

nothing is wrong with talaria but your config talaria is likely missing the section (that's what the error log "message":"key not supported: [Basic]" is telling us):

inbound:
  # authKey is the basic auth token incoming api requests like /stat, and /devices
  # WARNING: DO NOT use this in production.
  authKey: YXV0aEhlYWRlcg==

again, do not use this exact config section in production

@denopink
Copy link
Contributor

Hi, is there other way to check tr1d1um/scytale/talaria all worked well? ex, curl -vvv http://localhost:6100 (tr1d1um) But currently is seems can't use this way.

$ curl -vvv http://localhost:6100
* About to connect() to localhost port 6100 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 6100 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:6100
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< X-Midt-Server: tr1d1um
< X-Midt-Version: development
< Date: Tue, 26 Mar 2024 10:30:02 GMT
< Content-Length: 19
< 
404 page not found
* Connection #0 to host localhost left intact

you can also curl their /health api to check if they're running

@VicLin66
Copy link
Author

VicLin66 commented Apr 1, 2024

Update more, hope can be clarified the issue here. Is there got something wrong with talaria log "message":"key not supported:? image Here are also 401 error message in tr1d1um and scytale log with my test command "curl -i -H 'Authorization: Basic dXNlcjpwYXNz' http://localhost:6100/api/v3/device/mac:0050f1a1d74e/stat"
# HTTP 401 error messages image

nothing is wrong with talaria but your config talaria is likely missing the section (that's what the error log "message":"key not supported: [Basic]" is telling us):

inbound:
  # authKey is the basic auth token incoming api requests like /stat, and /devices
  # WARNING: DO NOT use this in production.
  authKey: YXV0aEhlYWRlcg==

again, do not use this exact config section in production

@denopink I did not modify it only except port number (from 6200 to 6400, for meet scytale config) So how do I fix talaria config for solve this auth issue?
Maybe comment out authKey? ("# authKey: YXV0aEhlYWRlcg==" in talaria .yaml )
But seems same testing result.

@VicLin66
Copy link
Author

VicLin66 commented Apr 1, 2024

Hi, is there other way to check tr1d1um/scytale/talaria all worked well? ex, curl -vvv http://localhost:6100 (tr1d1um) But currently is seems can't use this way.

$ curl -vvv http://localhost:6100
* About to connect() to localhost port 6100 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 6100 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:6100
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< X-Midt-Server: tr1d1um
< X-Midt-Version: development
< Date: Tue, 26 Mar 2024 10:30:02 GMT
< Content-Length: 19
< 
404 page not found
* Connection #0 to host localhost left intact

you can also curl their /health api to check if they're running

@denopink I checked status with tr1d1um/scytale/talaria,
scytale/talaria showed "Connection: close", does it matter?

# tr1d1um
$ curl localhost:6102/health -i
HTTP/1.1 200 OK
X-Midt-Server: tr1d1um
X-Midt-Version: development
Date: Mon, 01 Apr 2024 06:01:04 GMT
Content-Length: 0

# scytale
$ curl localhost:6301/health -i
HTTP/1.1 200 OK
Content-Type: application/json
X-Scytale-Build: 0.1.4
X-Scytale-Flavor: mint
X-Scytale-Region: east
X-Scytale-Server: scytale
X-Scytale-Start-Time: 01 Apr 24 05:34 UTC
Date: Mon, 01 Apr 2024 06:01:08 GMT
Content-Length: 329
Connection: close

{"CurrentMemoryUtilizationActive":3529170944,"CurrentMemoryUtilizationAlloc":2864536,"CurrentMemoryUtilizationHeapSys":7733248,"MaxMemoryUtilizationActive":3574611968,"MaxMemoryUtilizationAlloc":4844816,"MaxMemoryUtilizationHeapSys":7766016,"TotalRequestsDenied":0,"TotalRequestsReceived":0,"TotalRequestsSuccessfullyServiced":0}

# talaria
$ curl localhost:6201/health -i
HTTP/1.1 200 OK
Content-Type: application/json
X-Talaria-Build: 0.1.4
X-Talaria-Flavor: mint
X-Talaria-Region: east
X-Talaria-Server: talaria
X-Talaria-Start-Time: 01 Apr 24 05:31 UTC
Date: Mon, 01 Apr 2024 06:01:36 GMT
Content-Length: 497
Connection: close

{"CurrentMemoryUtilizationActive":3529449472,"CurrentMemoryUtilizationAlloc":1078688,"CurrentMemoryUtilizationHeapSys":7634944,"DeviceCount":0,"MaxMemoryUtilizationActive":3633061888,"MaxMemoryUtilizationAlloc":3045688,"MaxMemoryUtilizationHeapSys":7667712,"TotalConnectionEvents":0,"TotalDisconnectionEvents":0,"TotalPingMessagesReceived":0,"TotalPongMessagesReceived":0,"TotalRequestsDenied":0,"TotalRequestsReceived":0,"TotalRequestsSuccessfullyServiced":0,"TotalWRPRequestResponseProcessed":0}

@VicLin66
Copy link
Author

VicLin66 commented Apr 1, 2024

ant to test tr1d1um<->scytale<->talaria<->cpe, us

Hi, is there other way to check tr1d1um/scytale/talaria all worked well? ex, curl -vvv http://localhost:6100 (tr1d1um) But currently is seems can't use this way.

$ curl -vvv http://localhost:6100
* About to connect() to localhost port 6100 (#0)
*   Trying ::1...
* Connected to localhost (::1) port 6100 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:6100
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< X-Midt-Server: tr1d1um
< X-Midt-Version: development
< Date: Tue, 26 Mar 2024 10:30:02 GMT
< Content-Length: 19
< 
404 page not found
* Connection #0 to host localhost left intact

you can use any of tr1d1um's api to test that flow (tr1d1um<->scytale<->talaria)

if you want to test tr1d1um<->scytale<->talaria<->cpe, use https://github.com/xmidt-org/tr1d1um?tab=readme-ov-file#crud-operations---config-endpoints

I just want to try "Deploy" section (./tr1d1um) of https://github.com/xmidt-org/tr1d1um?tab=readme-ov-file#crud-operations---config-endpoints
I start up with talaria, scytale and tr1d1um (/usr/bin/talaria, /usr/bin/scytale, /usr/bin/tr1d1um)
Then, CPE's parodus connected with talaria, and got success message.

{"key":"debug","ts":"2024-04-01T13:39:53+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"192.168.1.1:54476"}
{"key":"debug","ts":"2024-04-01T13:44:04+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"[::1]:41202"}

Then, remaining testing is, query config from tr1d1um to CPE.

@denopink
Copy link
Contributor

denopink commented Apr 1, 2024

Update more, hope can be clarified the issue here. Is there got something wrong with talaria log "message":"key not supported:? image Here are also 401 error message in tr1d1um and scytale log with my test command "curl -i -H 'Authorization: Basic dXNlcjpwYXNz' http://localhost:6100/api/v3/device/mac:0050f1a1d74e/stat"
# HTTP 401 error messages image

nothing is wrong with talaria but your config talaria is likely missing the section (that's what the error log "message":"key not supported: [Basic]" is telling us):

inbound:
  # authKey is the basic auth token incoming api requests like /stat, and /devices
  # WARNING: DO NOT use this in production.
  authKey: YXV0aEhlYWRlcg==

again, do not use this exact config section in production

@denopink I did not modify it only except port number (from 6200 to 6400, for meet scytale config) So how do I fix talaria config for solve this auth issue? Maybe comment out authKey? ("# authKey: YXV0aEhlYWRlcg==" in talaria .yaml ) But seems same testing result.

You could comment it out, then you'll have no auth.

Looks like the author who wrote the example config https://github.com/xmidt-org/talaria/blob/main/talaria.yaml used a bad auth key value:

echo YXV0aEhlYWRlcg== | base64 --decode
authHeader

Talaria expects the value for authKey: SOME_BASE64_STRING to have the format USER_NAME_VALUE:PASSWORD_VALUE
e.g.: the key Zm9vOmJhcgo= should work, make sure to update the yaml value for both talaria and scytale

echo foo:bar | base64
Zm9vOmJhcgo=
# TALARIA YAML
inbound:
  # WARNING: This is an example auth token. DO NOT use this in production.
  authKey: Zm9vOmJhcgo=
# SCYTALE YAML
fanout:
  # Authorization is the Basic Auth token to use for each request.
  # WARNING: This is an example auth token. DO NOT use this in production.
  authorization: Zm9vOmJhcgo=

I tested this solution locally and it works (talaria enables basic auth), this should fix your talaria error {"key":"error","ts":"2024-03-21T13:50:03+08:00","message":"key not supported: [Basic]","auth":"Basic YXV0aEhlYWRlcg=="} (causing 401 regardless of the auth header value)

again, do not use this example auth key in production

From here you should be able to send queries with basic auth resolving this ticket, so I'm closing this ticket

If you still get 401 and the talaria error {"key":"error","ts":"2024-03-21T13:50:03+08:00","message":"key not supported: [Basic]","auth":"Basic YXV0aEhlYWRlcg=="}, then feel free to reopen the ticket.

Otherwise, for any new issues open a new ticket in the appropriate repo

@VicLin66
Copy link
Author

VicLin66 commented Apr 2, 2024

@denopink Referred with your last suggestion about auth setting, I still got same 401 fail, would you have a review again with my setting and steps?

  1. Due to tr1d1um's config (tr1d1um.yaml) "auth" is "dXNlcjpwYXNz" by default, I filled this string into scytale and talaria configs (scytale.yaml, talaria.yaml). And this string seems followed the rule USER_NAME_VALUE:PASSWORD_VALUE. (text is user:pass)
$ echo dXNlcjpwYXNz | base64 --decode
user:pass
  1. When I issued same curl command, curl -H 'Authorization:Basic dXNlcjpwYXNz' -i http://localhost:6100/api/v2/device/mac:10e8a7a1c0d3/stat, here is the log result with curl command and tr1d1um/scytale/talaria:
### curl command
$ curl -H 'Authorization:Basic dXNlcjpwYXNz' -i http://localhost:6100/api/v2/device/mac:10e8a7a1c0d3/stat
HTTP/1.1 401 Unauthorized
X-Midt-Server: tr1d1um
X-Midt-Version: development
X-Scytale-Build: 0.1.4
X-Scytale-Flavor: mint
X-Scytale-Region: east
X-Scytale-Server: scytale
X-Scytale-Start-Time: 02 Apr 24 08:02 UTC
X-Talaria-Build: 0.1.4
X-Talaria-Flavor: mint
X-Talaria-Region: east
X-Talaria-Server: talaria
X-Talaria-Start-Time: 02 Apr 24 08:03 UTC
X-Webpa-Transaction-Id: glvll7WCScWwksp0iC-iSg
X-Xmidt-Span: "http://localhost:6400/api/v3/device/mac:10e8a7a1c0d3/stat","2024-04-02T08:03:05Z","1.745568ms"
Date: Tue, 02 Apr 2024 08:03:05 GMT
Content-Length: 0

---------------------------------------------------------
# tr1d1um
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","message":"authentication accepted by enforcer","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.query":"","request.tid":"-Mi2RF_U-DNqy7LvmY1GsQ"}
{"key":"error","ts":"2024-04-02T16:03:05+08:00","message":"Request arrival not capture for logger","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.query":"","request.tid":"-Mi2RF_U-DNqy7LvmY1GsQ","satClientID":"user","deviceid":"mac:10e8a7a1c0d3","tid":"glvll7WCScWwksp0iC-iSg"}
{"key":"info","ts":"2024-04-02T16:03:05+08:00","message":"response","request.Headers":{"Accept":["*/*"],"Authorization-Type":["Basic"],"User-Agent":["curl/7.29.0"]},"request.URL":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.Method":"GET","request.address":"::1","request.path":"/api/v2/device/mac:10e8a7a1c0d3/stat","request.query":"","request.tid":"-Mi2RF_U-DNqy7LvmY1GsQ","satClientID":"user","deviceid":"mac:10e8a7a1c0d3","response":{"code":401,"headers":{"X-Midt-Server":["tr1d1um"],"X-Midt-Version":["development"],"X-Scytale-Build":["0.1.4"],"X-Scytale-Flavor":["mint"],"X-Scytale-Region":["east"],"X-Scytale-Server":["scytale"],"X-Scytale-Start-Time":["02 Apr 24 08:02 UTC"],"X-Talaria-Build":["0.1.4"],"X-Talaria-Flavor":["mint"],"X-Talaria-Region":["east"],"X-Talaria-Server":["talaria"],"X-Talaria-Start-Time":["02 Apr 24 08:03 UTC"],"X-Webpa-Transaction-Id":["glvll7WCScWwksp0iC-iSg"],"X-Xmidt-Span":["\"http://localhost:6400/api/v3/device/mac:10e8a7a1c0d3/stat\",\"2024-04-02T08:03:05Z\",\"1.745568ms\""]}}}


---------------------------------------------------------
# scytale
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","message":"accepted connection","serverName":"scytale","bindAddress":":6300","listenNetwork":"tcp","listenAddress":"[::]:6300","remoteAddress":"[::1]:54814"}
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","caller":"/home/runner/go/pkg/mod/github.com/xmidt-org/[email protected]/basculehttp/enforcer.go:102","message":"authentication accepted by enforcer","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:10e8a7a1c0d3/stat","method":"GET","ts":"2024-04-02T08:03:05Z"}
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","message":"fanout request complete","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:10e8a7a1c0d3/stat","method":"GET","statusCode":401,"url":"http://localhost:6400/api/v3/device/mac:10e8a7a1c0d3/stat"}
{"key":"error","ts":"2024-04-02T16:03:05+08:00","message":"all fanout requests failed","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:10e8a7a1c0d3/stat","method":"GET","statusCode":401,"url":"/api/v3/device/mac:10e8a7a1c0d3/stat"}
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","message":"wrote fanout response","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"]},"requestURL":"/api/v3/device/mac:10e8a7a1c0d3/stat","method":"GET","statusCode":401}


---------------------------------------------------------
# trlaria
{"key":"debug","ts":"2024-04-02T16:03:05+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"[::1]:33356"}

{"key":"error","ts":"2024-04-02T16:03:05+08:00","caller":"/home/runner/go/pkg/mod/github.com/xmidt-org/[email protected]/basculehttp/constructor.go:117","message":"failed to parse and validate token: could not decode string: illegal base64 data at input byte 12","requestHeaders":{"Accept-Encoding":["gzip"],"Authorization-Type":["Basic"],"User-Agent":["Go-http-client/1.1"],"X-Webpa-Device-Name":["mac:10e8a7a1c0d3"]},"requestURL":"/api/v3/device/mac:10e8a7a1c0d3/stat","method":"GET","requestURI":"/api/v3/device/mac:10e8a7a1c0d3/stat","remoteAddr":"[::1]:33356","ts":"2024-04-02T08:03:05Z","auth":"Basic dXNlcjpwYXNzs"}

{"key":"debug","ts":"2024-04-02T16:03:06+08:00","message":"accepted connection","serverName":"talaria","bindAddress":":6400","listenNetwork":"tcp","listenAddress":"[::]:6400","remoteAddress":"192.168.1.1:55556"}

You can see that there is no "key not supported: [Basic]" in talaria logs now, but still got 401 error from curl result.

I also uplaod my current talaria/scytale/tr1d1um config files, maybe you will be interested with this.
webpa_configs.zip

@denopink
Copy link
Contributor

denopink commented Apr 2, 2024

I rather not download random files, please post your server configs here as raw text (using backticks if you like https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#fenced-code-blocks)

@denopink denopink reopened this Apr 2, 2024
@VicLin66
Copy link
Author

VicLin66 commented Apr 3, 2024

Hi, here is my talaria/scytale/tr1d1um config

#talaria.yaml

---
# SPDX-FileCopyrightText: 2017 Comcast Cable Communications Management, LLC
# SPDX-License-Identifier: Apache-2.0

# The unique fully-qualified-domain-name of the server.  It is provided to
# the X-Talaria-Server header for showing what server fulfilled the request
# sent.
# (Optional)
server: "talaria"

########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# Provides this build number to the X-Talaria-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
build: "0.1.4"

# Provides the region information to the X-Talaria-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
region: "east"

# Provides the flavor information to the X-Talaria-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
flavor: "mint"

##############################################################################
# WebPA Service configuration
##############################################################################

# For a complete view of the service config structure,
# checkout https://godoc.org/github.com/xmidt-org/webpa-common/server#WebPA

########################################
#   primary endpoint Configuration
########################################

# primary defines the details needed for the primary endpoint.  The
# primary endpoint accepts the events from talaria (typically).
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Basic
primary:
  # address provides the port number for the endpoint to bind to.
  # ":443" is ideal, but may require some special handling due to it being
  # a reserved (by the kernel) port.
  address: ":6400"
  # HTTPS/TLS
  #
  # certificateFile provides the public key and CA chain in PEM format if
  # TLS is used.  Note: the certificate needs to match the fqdn for clients
  # to accept without issue.
  #
  # keyFile provides the private key that matches the certificateFile
  # (Optional)
  # certificateFile: "/etc/talaria/public.pem"
  # keyFile: "/etc/talaria/private.pem"
########################################
#   health endpoint Configuration
########################################

# health defines the details needed for the health check endpoint.  The
# health check endpoint is generally used by services (like AWS Route53
# or consul) to determine if this particular machine is healthy or not.
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Health
health:
  # address provides the port number for the endpoint to bind to.
  # ":80" is ideal, but may require some special handling due to it being
  # a reserved (by the kernel) port.
  address: ":6201"

  # logInterval appears to be present from before we had formal metrics
  # (Deprecated)
  # logInterval: "60s"
  # options appears to be present from before we had formal metrics
  # (Deprecated)
  # options:
  #  - "PayloadsOverZero"
  #  - "PayloadsOverHundred"
  #  - "PayloadsOverThousand"
  #  - "PayloadsOverTenThousand"

########################################
#   Debugging/pprof Configuration
########################################

# pprof defines the details needed for the pprof debug endpoint.
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Basic
# (Optional)
pprof:
  # address provides the port number for the endpoint to bind to.
  address: ":6202"

########################################
#   Control Configuration
########################################

# control configures the details needed for the control server.
# defined https://godoc.org/github.com/xmidt-org/webpa-common/xhttp#ServerOptions
control:
  # address provides the port number for the endpoint to bind to.
  # defaults to the internal net/http default
  address: ":6203"

########################################
#   Metrics Configuration
########################################

# metric defines the details needed for the prometheus metrics endpoint
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Metric
# (Optional)
metric:
  # address provides the port number for the endpoint to bind to.  Port 6204
  # was chosen because it does not conflict with any of the other prometheus
  # metrics or other machines in the xmidt cluster.  You may use any port you
  # wish.
  address: ":6204"

  # metricsOptions provides the details needed to configure the prometheus
  # metric data.  Metrics generally have the form:
  #
  # {namespace}_{subsystem}_{metric}
  #
  # so if you use the suggested value below, your metrics are prefixed like
  # this:
  #
  # xmidt_talaria_{metric}
  #
  # (Optional)
  metricsOptions:
    # namespace is the namespace of the metrics provided
    # (Optional)
    namespace: "xmidt"
    # subsystem is the subsystem of the metrics provided
    # (Optional)
    subsystem: "talaria"

########################################
#   Logging Related Configuration
########################################

# log configures the logging subsystem details
log:
  # file is the name of the most recent log file.  If set to "stdout" this
  # will log to os.Stdout.
  # (Optional) defaults to os.TempDir()
  # file: "/var/log/talaria/talaria.log"
  file: "stdout"

  # level is the logging level to use - INFO, DEBUG, WARN, ERROR
  # (Optional) defaults to ERROR
  level: "DEBUG"

  # maxsize is the maximum file size in MB
  # (Optional) defaults to max 100MB
  maxsize: 50

  # maxage is the maximum number of days to retain old log files
  # (Optional) defaults to ignore age limit (0)
  maxage: 30

  # maxbackups is the maximum number of old log files to retain
  # (Optional) defaults to retain all (0)
  maxbackups: 10

  # json is a flag indicating whether JSON logging output should be used.
  # (Optional) defaults to false
  json: true

########################################
#   Device  Related Configuration
########################################

# device configures the generic talaria configuration.
# It has four parts: manager, rehasher, outbound, and inbound.
# The manager section handles the actual device connection configuration.
# The rehasher section configures the services for whose events the
# rehasher (a service discovery listener) should respond to.
# The outbound section configures the outbound requests to caduceus or some other receiver of messages.
# The inbound section configures the api inbound requests.
device:
  # manager handles the device manager related configuration.
  # defined by https://godoc.org/github.com/xmidt-org/webpa-common/device#Options
  manager:
    # wrpSourceCheck configures behavior around source validation of WRP messages
    # originating from the device. It's intended to prevent connected devices from
    # spoofing the source of the messages they send.
    wrpSourceCheck:
      # When type=enforce, all messages with a source which doesn't match the device
      # ID of the websocket connection (1) will be ignored, (2) logged as a security event and
      # (3) the wrp_source_check counter will be updated accordingly. When the type=monitor,
      # only the last two take place.
      type: monitor
    # upgrader is the gorilla mux websocket configuration, which upgrades an
    # incoming device connection from http to websocket.
    # defined by https://godoc.org/github.com/gorilla/websocket#Upgrader
    upgrader:
      # handshakeTimeout is the time to wait before rejecting an incoming/new websocket connection.
      # (Optional) defaults to (0), aka do not timeout
      handshakeTimeout: "10s"

    # maxDevices is the maximum number of devices allowed to connect to the talaria.
    # (Optional) defaults to math.MaxUint32
    maxDevices: 100

    # deviceMessageQueueSize is the capacity of the channel which stores messages waiting
    # to be transmitted to a device.
    # (Optional) defaults to 100
    deviceMessageQueueSize: 1000

    # pingPeriod is the time between pings sent to each device to ensure they
    # are still reachable.
    # (Optional) defaults to 45s
    pingPeriod: "1m"

    # writeTimeout is the length of time a device connection is allowed to be idle,
    # with no traffic coming from talaria. Must be longer than pingPeriod to maintain
    # a persistent connection when idle.
    # (Optional) defaults to 60s
    writeTimeout: "2m"

    # idlePeriod is the length of time a device connection is allowed to be idle,
    # with no traffic coming from the device. Must be longer than pingPeriod to
    # maintain a persistent connection when idle.
    # (Optional) defaults to 135s
    idlePeriod: "2m"


  # # rehasher defines the services for which a monitor listener will rehash
  # # and disconnect devices in response to service discovery events.
  # # https://pkg.go.dev/github.com/xmidt-org/[email protected]/device/rehasher
  # # (Optional) rehasher.services defaults to ["talaria"]
  # rehasher:
  #   services:
  #     - talaria

  # outbound handles api request to push messages to a receiver (usually caduceus).
  # defined by https://github.com/xmidt-org/talaria/blob/main/outbounder.go
  # TODO: link godoc instead
  outbound:
    # method is the http method to use against the receiving server.
    # (Optional) defaults to POST
    method: "POST"

    # retries is the number attempts to sent the message to the receiver.
    # (Optional) defaults to 1
    retries: 3

    # eventEndpoints is a map defining where to send the events to,
    # where the key is the device event type (https://godoc.org/github.com/xmidt-org/webpa-common/device#EventType)
    # and the value is the url.
    eventEndpoints:
      default: http://caduceus:6000/api/v4/notify

    # enableConsulRoundRobin will overwrite the eventEndpoints with using consul to discover the caduceus in the datacenter.
    # NOTE: eventEndpoints still must be set, and in the service section of this config caduceus must be added to the list
    # of services to watch.
    # if no services are found, talaria will fail back to the defined endpoint.
    # (Optional) defaults to false
    enableConsulRoundRobin: true

    # requestTimeout is how long an event will be held on to starting from when it is
    # received till completing the http request.
    # So if the event was in the queue for 124s and using the default value
    # the http request only has 1s to complete before moving on.
    # (Optional) defaults to 125s
    requestTimeout: "2m"

    # TODO:// double check if this is correct
    # defaultScheme is the default scheme for the http request.
    # (Optional) defaults to https
    defaultScheme: "https"

    # allowedSchemes is a list of schemes, used in conjunction with the defaultScheme
    # for the URLFilter.
    # (Optional) defaults to []string{"https"}
    allowedSchemes:
      - "http"
      - "https"

    # outboundQueueSize is the size of the buffer to queue messages for each
    # receiver.
    # (Optional) defaults to 1000
    outboundQueueSize: 2000

    # workerPoolSize configures how many active go threads send messages to the receivers.
    # (Optional) defaults to 100
    workerPoolSize: 50

    # transport is a way to overwrite the default golang http.Transport configuration.
    # defined  https://golang.org/pkg/net/http/#Transport
    # (Optional) defaults described below
    transport:
      # (Optional) defaults to 0, aka do not limit it
      maxIdleConns: 0
      # (Optional) defaults to 100
      maxIdleConnsPerHost: 100
      # (Optional) defaults to 0s, aka do not timeout while in idle
      idleConnTimeout: "120s"

    # clientTimeout specifies a time limit for requests made by this
    # Client. The timeout includes connection time, any
    # redirects, and reading the response body. The timer remains
    # running after Get, Head, Post, or Do return and will
    # interrupt reading of the Response.Body.
    # defined  https://golang.org/pkg/net/http/#Client
    # (Optional) defaults to 160s
    clientTimeout: "2m"

    # authKey is the basic auth token used for sending messages to the receiver.
    # (Optional) defaults to no auth token
    # WARNING: This is an example auth token. DO NOT use this in production.
    authKey: dXNlcjpwYXNzs

# inbound configures the api inbound requests.
# (Optional) defaults described below
inbound:
  # authKey is the basic auth token incoming api requests like /stat, and /devices
  # (Optional) defaults to no auth token
  # WARNING: This is an example auth token. DO NOT use this in production.
  authKey: dXNlcjpwYXNzs

  # The timeout for all inbound HTTP requests.
  # (Optional) defaults to 120s
  timeout: "120s"

########################################
#   Authorization Related Configuration
########################################

jwtValidator:
  Config:
    Resolve:
      # Template is a URI template used to fetch keys.  This template may
	    # use a single parameter named keyID, e.g. http://keys.com/{keyID}.
      # This field is required and has no default.
      Template: "http://localhost/{key_name}"

# Any combination of these configurations may be used for authorization.
# If ANY match, the request goes onwards.  If none are provided, no requests
# will be accepted.

# # jwtValidator provides the details about where to get the keys for JWT
# # kid values and their associated information (expiration, etc) for JWTs
# # used as authorization.
# # If configured, it is used to authenticate devices during registration.
# # (Optional)
# jwtValidator:
#   keys:
#     factory:
#       uri: "https://jwt.example.com/keys/{keyId}"
#     purpose: 0
#     updateInterval: 604800000000000

# # deviceAccessCheck configures the strategy to ensure WRP messages only reach those devices they
# # are authorized to (essential to secure multi-tenant clouds). The type can be "monitor" or "enforce".
# # If restrictions must be applied, select the "enforce" type, otherwise use "monitor" to view the
# # unauthorized events without explicit rejections. For either type, transaction
# # metrics are collected. If no valid type is provided, no checks are provided.
# # (Optional)
# deviceAccessCheck:
#   type: "enforce"
#   # sep is the delimeter used for the credential paths below
#   # (Optional) Defaults to "."
#   sep: ","

#   checks:
#     -
#       # name should be a concise, helpful description of the check
#       name: "PartnerID"

#       # deviceCredentialPath is the path to the credential within the device's metadata map representation.
#       # deviceCredentialPath: partner-ids

#       # wrpCredentialPath is the path to the credential within the WRP Message map representation.
#       # (Optional if inputValue is provided. If both provided, inputValue will be used during check).
#       wrpCredentialPath: PartnerIDs

#       # operation is the name of the operation that's is meant to be performed.
#       # Supported operations include: "intersects", "contains", "eq" (equal) and "gt" (greater than).
#       # Note: By default operation is applied from deviceCredential to wrpCredential
#       operation: intersect

#       # inversed should be set to true if operation must be applied from wrpCredential to deviceCredential
#       # (Optional)
#       inversed: true

#     -
#       name: "Devices with trust level > 999"
#       # For this check to succeed, the device should have connected with a JWT with the claims:
#       # {
#       # ...
#       #   "security": {
#       #     "trust": 1000
#       #   }
#       # ...
#       # }
#       #
#       #
#       deviceCredentialPath: "security,trust"
#       inputValue: 999
#       operation: gt

########################################
#   Service Discovery Configuration
########################################

# service configures the server for service discovery.
# defined https://godoc.org/github.com/xmidt-org/webpa-common/service/servicecfg#Options
# this is required, consul or fixed must be used.
service:
  # defaultScheme, used for the registered servers for communication.
  # (Optional) defaults to https
  defaultScheme: http

  # vnodeCount used for consistent hash calculation github.com/billhathaway/consistentHash.
  # number of virtual nodes. should be a prime number
  # it is a tradeoff of memory and ~ log(N) speed versus how well the hash spreads
  # (Optional) defaults to 211
  vnodeCount: 211

  # disableFilter disables filtering.
  # (Deprecated) does not do anything
  # disableFilter: false

  # fixed is the list of servers in the datacenter.
  # (Optional) default to empty list
  fixed:
    - http://talaria:6400
  # # consul configures consul for service discovery.
  # # defined https://godoc.org/github.com/xmidt-org/webpa-common/service/consul#Options
  # # (Optional) defaults define https://sourcegraph.com/github.com/hashicorp/consul/-/blob/api/api.go#L347
  # consul:
  #   # client is the configuration needed to connect to consul.
  #   # defined https://godoc.org/github.com/hashicorp/consul/api#Config
  #   client:
  #     # address is the address of the consul client or cluster.
  #     # (Optional) defaults to 127.0.0.1:8500
  #     address: "consul0:8500"
  #
  #     # scheme is the scheme to use for api calls to the consul agent.
  #     # (Optional) defaults to http
  #     scheme: "http"
  #
  #     # waitTime limits how long a Watch will block.
  #     # (Optional) defaults to 0s, aka wait forever before update
  #     waitTime: 30s
  #
  #   # DisableGenerateID disables consul from generating the id.
  #   # (Optional) defaults to false
  #   disableGenerateID: true
  #
  #   # DatacenterRetries is the number of attempts to get the datacenters
  #   # (Optional) defaults to 10
  #   datacenterRetries: 3
  #
  #   # Registrations is a list of service(s) to register with consul.
  #   # defined https://godoc.org/github.com/hashicorp/consul/api#AgentServiceRegistration
  #   # (Optional) defaults to empty list
  #   registrations:
  #     - # id is the unique id for the service registration.
  #       id: "talaria-1"
  #
  #       # name is the service name.
  #       name: "talaria"
  #
  #       # tags are a list of strings that others talking to consul can use to
  #       # filter services.  These are meant to help in grouping similar
  #       # services in consul.
  #       # (Optional) defaults to empty list
  #       tags:
  #         - "dev"
  #         - "docker"
  #         - "stage=dev"
  #         - "flavor=docker"
  #
  #       # address tells consul where to contact the service.
  #       address: "https://talaria-2"
  #
  #       # port tells consul what port to use to contact the service.  This is
  #       # used with the address for calls to this server.
  #       port: 6400
  #
  #       # checks is a list of checks to see if the service is healthy.
  #       # defined https://godoc.org/github.com/hashicorp/consul/api#AgentServiceCheck
  #       # (Optional) defaults to empty list
  #       # Warning: if there are no checks, this service will stay around even
  #       # when it is in a bad state.  This will cause other healthy servers
  #       # to send requests that will fail.
  #       checks:
  #         - # CheckID is a unique id for the check.
  #           checkID: "talaria-2:http"
  #
  #           # http tells consul to check via http rest request at the url
  #           # provided.
  #           http: "http://talaria-2:6221/health"
  #
  #           # interval is how often to check.
  #           interval: "30s"
  #
  #           # deregisterCriticalServiceAfter is how long to wait before this
  #           # service is considered bad.
  #           deregisterCriticalServiceAfter: "70s"
  #
  #   # Watches is a list of service(s) to watch from consul. The address of the
  #   # of the services are stored in memory. Upon update, the internal memory is
  #   # updated.
  #   # defined https://godoc.org/github.com/xmidt-org/webpa-common/service/consul#Watch
  #   # (Optional) defaults to empty list
  #   watches:
  #     - # service name to watch for updates.
  #       service: "talaria"
  #
  #       # tags is a list of strings that must be attached to the services
  #       # being watched.
  #       # (Optional) defaults to empty list
  #       tags:
  #         - "dev"
  #         - "docker"
  #
  #       # passingOnly determines if only services passing the consul check are returned.
  #       # (Optional) defaults to false
  #       passingOnly: true
  #
  #       # crossDatacenter determines if there is a watch for all datacenter changes, split by datacenter.
  #       # change this to have the devices hash across all datacenters instead of
  #       # the single datacenter. The datacenter is known by the consul agent who is
  #       # aware of which datacenter it is in.
  #       # It is recommended to keep this as false for talaria.
  #       # (Optional) defaults to false, aka only watch for services in the
  #       # current datacenter.
  #       crossDatacenter: false
  #
  #       # queryOptions are options for the consul query, used in conjunction
  #       # with passingOnly.
  #       # defined by https://godoc.org/github.com/hashicorp/consul/api#QueryOptions
  #       # (Optional) defaults to empty struct
  #       # queryOptions:
  #       #   useCache: true
  #     - # service name to watch for which caduceus to send events to.
  #       # NOTE: enableConsulRoundRobin must be set to true in order for this to work.
  #       service: "caduceus"
  #
  #       # tags is a list of strings that must be attached to the services
  #       # being watched.
  #       # (Optional) defaults to empty list
  #       tags:
  #         - "dev"
  #         - "docker"
  #
  #       # passingOnly determines if only services passing the consul check are returned.
  #       # (Optional) defaults to false
  #       passingOnly: true
  #
  #       # crossDatacenter determines if there is a watch for all datacenter changes, split by datacenter
  #       # change this to have the devices hash across all datacenters instead of
  #       # the single datacenter. The datacenter is known by the consul agent who is
  #       # aware of which datacenter it is in.
  #       # It is recommended to keep this as false for talaria.
  #       # (Optional) defaults to false, aka only watch for services in the
  #       # current datacenter.
  #       crossDatacenter: false
  #
  #       # queryOptions are options for the consul query, used in conjunction
  #       # with passingOnly.
  #       # defined by https://godoc.org/github.com/hashicorp/consul/api#QueryOptions
  #       # (Optional) defaults to empty struct
  #       # queryOptions:
  #       #   useCache: true
  #       #   datacenter: "dc1"

# tracing provides configuration around traces using OpenTelemetry.
# (Optional). By default, a 'noop' tracer provider is used and tracing is disabled.
tracing:
  # provider is the name of the trace provider to use. Currently, otlp/grpc, otlp/http, stdout, jaeger and zipkin are supported.
  # 'noop' can also be used as provider to explicitly disable tracing.
  provider: "noop"

  # skipTraceExport only applies when provider is stdout. Set skipTraceExport to true
  # so that trace information is not written to stdout.
  # skipTraceExport: true

  # endpoint is where trace information should be routed. Applies to otlp, zipkin, and jaegar. OTLP/gRPC uses port 4317 by default. 
  # OTLP/HTTP uses port 4318 by default.
  # endpoint: "http://localhost:9411/api/v2/spans"
  
  # ParentBased and NoParent dictate if and when new spans should be created.
  # ParentBased = "ignore" (default), tracing is effectively turned off and the "NoParent" value is ignored
  # ParentBased = "honor", the sampling decision is made by the parent of the span
  parentBased: ignore

  # NoParent decides if a root span should be initiated in the case where there is no existing parent
  # This value is ignored if ParentBased = "ignore"
  # NoParent = "never" (default), root spans are not initiated
  # NoParent = "always", roots spans are initiated
  noParent: never


zap:
  # OutputPaths is a list of URLs or file paths to write logging output to.
  outputPaths:
    - stdout
    # - /var/log/caduceus/caduceus.log

  # Level is the minimum enabled logging level. Note that this is a dynamic
  # level, so calling Config.Level.SetLevel will atomically change the log
  # level of all loggers descended from this config.
  level: debug

  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  errorOutputPaths:
    - stderr
    # - /var/log/caduceus/caduceus.log

  # DisableCaller stops annotating logs with the calling function's file
	# name and line number. By default, all logs are annotated.
  disableCaller: true
  
  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  encoderConfig:
    messageKey: message
    levelKey: key
    callerKey: caller
    levelEncoder: lowercase

  # Encoding sets the logger's encoding. Valid values are "json" and
  # "console", as well as any third-party encodings registered via
  # RegisterEncoder.
  encoding: json

#(Optional) failOpen determines if talaria should allow devices without authentication to connect or not
#default is to allow for fail open
failOpen: true

#scytale.yaml

# SPDX-FileCopyrightText: 2019 Comcast Cable Communications Management, LLC
# SPDX-License-Identifier: Apache-2.0
---
# The unique fully-qualified-domain-name of the server.  It is provided to
# the X-Scytale-Server header for showing what server fulfilled the request
# sent.
# (Optional)
server: "scytale"

########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# Provides this build number to the X-Scytale-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
build: "0.1.4"

# Provides the region information to the X-Scytale-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
region: "east"

# Provides the flavor information to the X-Scytale-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
flavor: "mint"

##############################################################################
# WebPA Service configuration
##############################################################################

# For a complete view of the service config structure,
# checkout https://godoc.org/github.com/xmidt-org/webpa-common/server#WebPA

########################################
#   primary endpoint Configuration
########################################

# primary defines the details needed for the primary endpoint.  The
# primary endpoint accepts the events from scytale (typically).
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Basic
primary:
  # address provides the port number for the endpoint to bind to.
  # ":443" is ideal, but may require some special handling due to it being
  # a reserved (by the kernel) port.
  address: ":6300"
  # HTTPS/TLS
  #
  # certificateFile provides the public key and CA chain in PEM format if
  # TLS is used.  Note: the certificate needs to match the fqdn for clients
  # to accept without issue.
  #
  # keyFile provides the private key that matches the certificateFile
  # (Optional)
  # certificateFile: "/etc/scytale/public.pem"
  # keyFile: "/etc/scytale/private.pem"

########################################
#   health endpoint Configuration
########################################

# health defines the details needed for the health check endpoint.  The
# health check endpoint is generally used by services (like AWS Route53
# or consul) to determine if this particular machine is healthy or not.
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Health
health:
  # address provides the port number for the endpoint to bind to.
  # ":80" is ideal, but may require some special handling due to it being
  # a reserved (by the kernel) port.
  address: ":6301"

  # logInterval appears to be present from before we had formal metrics
  # (Deprecated)
  # logInterval: "60s"
  # options appears to be present from before we had formal metrics
  # (Deprecated)
  # options:
  #  - "PayloadsOverZero"
  #  - "PayloadsOverHundred"
  #  - "PayloadsOverThousand"
  #  - "PayloadsOverTenThousand"

########################################
#   Debugging/pprof Configuration
########################################

# pprof defines the details needed for the pprof debug endpoint.
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Basic
# (Optional)
pprof:
  # address provides the port number for the endpoint to bind to.
  address: ":6302"

########################################
#   Metrics Configuration
########################################

# metric defines the details needed for the prometheus metrics endpoint
# define https://godoc.org/github.com/xmidt-org/webpa-common/server#Metric
# (Optional)
metric:
  # address provides the port number for the endpoint to bind to.  Port 6204
  # was chosen because it does not conflict with any of the other prometheus
  # metrics or other machines in the xmidt cluster.  You may use any port you
  # wish.
  address: ":6303"

  # metricsOptions provides the details needed to configure the prometheus
  # metric data.  Metrics generally have the form:
  #
  # {namespace}_{subsystem}_{metric}
  #
  # so if you use the suggested value below, your metrics are prefixed like
  # this:
  #
  # xmidt_scytale_{metric}
  #
  # (Optional)
  metricsOptions:
    # namespace is the namespace of the metrics provided
    # (Optional)
    namespace: "xmidt"
    # subsystem is the subsystem of the metrics provided
    # (Optional)
    subsystem: "scytale"

touchstone:
  # DefaultNamespace is the prometheus namespace to apply when a metric has no namespace
  defaultNamespace: "xmidt"
  # DefaultSubsystem is the prometheus subsystem to apply when a metric has no subsystem
  defaultSubsystem: "scytale"

########################################
#   Logging Related Configuration
########################################

# log configures the logging subsystem details
log:
  # file is the name of the most recent log file.  If set to "stdout" this
  # will log to os.Stdout.
  # (Optional) defaults to os.TempDir()
  # file: "/var/log/scytale/scytale.log"
  file: "stdout"

  # level is the logging level to use - INFO, DEBUG, WARN, ERROR
  # (Optional) defaults to ERROR
  level: "DEBUG"

  # maxsize is the maximum file size in MB
  # (Optional) defaults to max 100MB
  maxsize: 50

  # maxage is the maximum number of days to retain old log files
  # (Optional) defaults to ignore age limit (0)
  maxage: 30

  # maxbackups is the maximum number of old log files to retain
  # (Optional) defaults to retain all (0)
  maxbackups: 10

  # json is a flag indicating whether JSON logging output should be used.
  # (Optional) defaults to false
  json: true

zap:
  # OutputPaths is a list of URLs or file paths to write logging output to.
  outputPaths:
    - stdout
    # - /var/log/scytale/scytale.log

  # Level is the minimum enabled logging level. Note that this is a dynamic
  # level, so calling Config.Level.SetLevel will atomically change the log
  # level of all loggers descended from this config.
  level: DEBUG

  # DisableCaller stops annotating logs with the calling function's file
	# name and line number. By default, all logs are annotated.
  disableCaller: true

  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  errorOutputPaths:
    - stderr
    # - /var/log/scytale/scytale.log

  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  encoderConfig:
    messageKey: message
    levelKey: key
    callerKey: caller
    levelEncoder: lowercase

  # Encoding sets the logger's encoding. Valid values are "json" and
  # "console", as well as any third-party encodings registered via
  # RegisterEncoder.
  encoding: json

########################################
#   Fanout Related Configuration
########################################

# fanout describes the endpoints to fanout to and the http configuration to use for each fanout connection.
# defined https://godoc.org/github.com/xmidt-org/webpa-common/xhttp/fanout#Configuration
fanout:
#  # endpoints are the URLs for each endpoint to fan out to.
#  # if this is set, it overrides the service.fixed values.
#  # (Optional) if empty, endpoints are driven by service discovery
#  endpoints: ["http://petasos:6400/api/v2/device/send"]

  # pathPrefix is a string prepended to the beginning of the path being used for
  # fanout.
  pathPrefix: "/api/v3"

  # Authorization is the Basic Auth token to use for each request.
  # (Optional) defaults to no auth token
  # WARNING: This is an example auth token. DO NOT use this in production.
  authorization: dXNlcjpwYXNzs

  # transport is a way to overwrite the default golang http.Transport configuration.
  # defined  https://golang.org/pkg/net/http/#Transport
  # (Optional) defaults described below
  transport:
    # (Optional) defaults to 0, aka do not limit it
    maxIdleConns: 0
    # (Optional) defaults to 100
    maxIdleConnsPerHost: 100
    # (Optional) defaults to 0s, aka do not timeout while in idle
    idleConnTimeout: "120s"

  # fanoutTimeout is the timeout for the entire fanout operation.
  # fanoutTimeout should be the same or greater than the clientTimeout.
  # fanoutTimeout includes multiple http requests.
  # (Optional) defaults to 45s
  fanoutTimeout: "127s"

  # clientTimeout is the http.Client Timeout.
  # (Optional) defaults to 30s
  clientTimeout: "127s"

  # concurrency is the maximum number of concurrent fanouts allowed.
  # concurrency is managed by a semaphore described https://godoc.org/github.com/xmidt-org/webpa-common/xhttp#Busy.
  # (Optional) defaults to 1000
  concurrency: 10

  # maxRedirects defines the maximum number of redirects each fanout will allow.
  # (Optional) default to unlimited
  maxRedirects: 3

  # redirectExcludeHeaders are the headers that will *not* be copied on a redirect.
  # (Optional) defaults to copying all headers over.
  redirectExcludeHeaders:
    - X-Xmidt-Log-Level


########################################
#   Authorization Related Configuration
########################################

# Any combination of these configurations may be used for authorization.
# If ANY match, the request goes onwards.  If none are provided, no requests
# will be accepted.

# authHeader provides the list of basic auth headers that scytale will accept
# as authorization. Note: This is an example authHeader. Do not use this in production.
# (Optional)
authHeader: ["dXNlcjpwYXNz"]

# jwtValidator provides the details about where to get the keys for JWT
# kid values and their associated information (expiration, etc) for JWTs
# used as authorization
# (Optional)
jwtValidator:
  Config:
    Resolve:
      # Template is a URI template used to fetch keys.  This template may
	    # use a single parameter named keyID, e.g. http://keys.com/{keyID}.
      # This field is required and has no default.
      Template: "http://localhost/{keyID}"
    Refresh:
      Sources:
        # URI is the location where keys are served.  By default, clortho supports
        # file://, http://, and https:// URIs, as well as standard file system paths
        # such as /etc/foo/bar.jwk.
        #
        # This field is required and has no default.
        - URI: "http://localhost"

# capabilityCheck provides the details needed for checking an incoming JWT's
# capabilities.  If the type of check isn't provided, no checking is done.  The 
# type can be "monitor" or "enforce".  If it is empty or a different value, no 
# checking is done.  If "monitor" is provided, the capabilities are checked but 
# the request isn't rejected when there isn't a valid capability for the 
# request. Instead, a message is logged.  When "enforce" is provided, a request 
# that doesn't have the needed capability is rejected.
#
# The capability is expected to have the format:
#
# {prefix}{endpoint}:{method}
#
# The prefix can be a regular expression.  If it's empty, no capability check 
# is done.  The endpoint is a regular expression that should match the endpoint
# the request was sent to. The method is usually the method of the request, such as 
# GET.  The accept all method is a catchall string that indicates the capability 
# is approved for all methods.
# (Optional)
# capabilityCheck:
#   # type provides the mode for capability checking.
#   type: "enforce"
#   # prefix provides the regex to match the capability before the endpoint.
#   prefix: "prefix Here"
#   # acceptAllMethod provides a way to have a capability that allows all 
#   # methods for a specific endpoint.
#   acceptAllMethod: "all"
#   # endpointBuckets provides regular expressions to use against the request 
#   # endpoint in order to group requests for a metric label.
#   endpointBuckets:
#     - "hook\\b"
#     - "hooks\\b"
#     - "device/.*/stat\\b"
#     - "device/.*/config\\b"


# WRPCheck provides the details needed to authorize incoming WRP message
# requests from partners against their credentials. The type can be "monitor" or "enforce".
# If "monitor" is provided, requests are authorized even when the WRP message has invalid
# credentials. If "enforce" is provided, such requests are rejected. For either type, transaction
# metrics are collected. If no valid type is provided, no checks are provided.
# Note: Enabling this check requires that only JWT Authentication is enabled as the source of
# truth for the authorization comes from the JWT claims allowedResources.allowedPartners
# (Optional)
# WRPCheck:
#   type: "enforce"

########################################
#   Service Discovery Configuration
########################################

# service configures the server for service discovery.
# defined https://godoc.org/github.com/xmidt-org/webpa-common/service/servicecfg#Options
# (Optional) defaults to not set.
# If set consul or fixed must be used. Using fixed is the same as setting fanout.endpoints.
service:
  # defaultScheme, used for the registered servers for communication.
  # (Optional) defaults to https
  defaultScheme: http

  # vnodeCount used for consistent hash calculation github.com/billhathaway/consistentHash.
  # number of virtual nodes. should be a prime number
  # it is a tradeoff of memory and ~ log(N) speed versus how well the hash spreads
  # (Optional) defaults to 211
  vnodeCount: 211

  # disableFilter disables filtering.
  # (Deprecated) does not do anything
  # disableFilter: false

  # fixed is the list of servers in the datacenter.
  # (Optional) default to empty list
  fixed:
    - http://localhost:6400
  # # consul configures consul for service discovery.
  # # defined https://godoc.org/github.com/xmidt-org/webpa-common/service/consul#Options
  # # (Optional) defaults define https://sourcegraph.com/github.com/hashicorp/consul/-/blob/api/api.go#L347
  # consul:
  #   # client is the configuration needed to connect to consul.
  #   # defined https://godoc.org/github.com/hashicorp/consul/api#Config
  #   client:
  #     # address is the address of the consul client or cluster.
  #     # (Optional) defaults to 127.0.0.1:8500
  #     address: "consul0:8500"
  #
  #     # scheme is the scheme to use for api calls to the consul agent.
  #     # (Optional) defaults to http
  #     scheme: "http"
  #
  #     # waitTime limits how long a Watch will block.
  #     # (Optional) defaults to 0s, aka wait forever before update
  #     waitTime: 30s
  #
  #   chrysom:
  #     listen:
  #       # pullInterval is how often to call argus to update the list of 
  #       # inactive datacenters.
  #       pullInterval: "5s"
  #
  #     # bucket to store and retrieve inactive datacenters.
  #     bucket: "inactive-dcs"
  #
  #     # address is the location to talk to argus.
  #     address: "http://localhost:6600"
  #
  #     # auth the authentication method for argus.
  #     auth:
  #       # basic configures basic authentication for argus.
  #       # Must be of form: 'Basic xyz=='
  #       basic: ""
  #
  #       # jwt configures jwt style authentication for argus.
  #       jwt:
  #         # requestHeaders are added to the request for the token.
  #         # (Optional)
  #         # requestHeaders:
  #         #   "": ""
  #
  #         # authURL is the URL to access for the token.
  #         authURL: 
  #
  #         # timeout is how long the request to get the token will take before
  #         # timing out.
  #         timeout: "1m"
  #
  #         # buffer is the length of time before a token expires to get a new token.
  #         buffer: "2m"
  #   # DisableGenerateID disables consul from generating the id.
  #   # (Optional) defaults to false
  #   disableGenerateID: true
  #
  #   # DatacenterRetries is the number of attempts to get the datacenters
  #   # (Optional) defaults to 10
  #   datacenterRetries: 3
  #
  #   # DatacenterWatchInterval is the interval at which scytale checks for a change 
  #   # in active datacenters.
  #   datacenterWatchInterval: 10s
  #
  #   # Registrations is a list of service(s) to register with consul.
  #   # defined https://godoc.org/github.com/hashicorp/consul/api#AgentServiceRegistration
  #   # (Optional) defaults to empty list
  #   registrations:
  #     - # id is the unique id for the service registration.
  #       id: "scytale-1"
  #
  #       # name is the service name.
  #       name: "scytale"
  #
  #       # tags are a list of strings that others talking to consul can use to
  #       # filter services.  These are meant to help in grouping similar
  #       # services in consul.
  #       # (Optional) defaults to empty list
  #       tags:
  #         - "dev"
  #         - "docker"
  #         - "stage=dev"
  #         - "flavor=docker"
  #
  #       # address tells consul where to contact the service.
  #       address: "https://scytale"
  #
  #       # scheme tells consul what scheme to use to contact the service.
  #       # This is used with the address for calls to this server.
  #       scheme: "http"
  #
  #       # port tells consul what port to use to contact the service.  This is
  #       # used with the address for calls to this server.
  #       port: 6300
  #
  #       # checks is a list of checks to see if the service is healthy.
  #       # defined https://godoc.org/github.com/hashicorp/consul/api#AgentServiceCheck
  #       # (Optional) defaults to empty list
  #       # Warning: if there are no checks, this service will stay around even
  #       # when it is in a bad state.  This will cause other healthy servers
  #       # to send requests that will fail.
  #       checks:
  #         - # CheckID is a unique id for the check.
  #           checkID: "scytlae-1:http"
  #
  #           # http tells consul to check via http rest request at the url
  #           # provided.
  #           http: "http://scytale:6301/health"
  #
  #           # interval is how often to check.
  #           interval: "30s"
  #
  #           # deregisterCriticalServiceAfter is how long to wait before this
  #           # service is considered bad.
  #           deregisterCriticalServiceAfter: "70s"
  #
  #   # Watches is a list of service(s) to watch from consul. The address of the
  #   # of the services are stored in memory. Upon update, the internal memory is
  #   # updated.
  #   # defined https://godoc.org/github.com/xmidt-org/webpa-common/service/consul#Watch
  #   # (Optional) defaults to empty list
  #   watches:
  #     - # service name to watch for updates.
  #       service: "talaria"
  #
  #       # tags is a list of strings that must be attached to the services
  #       # being watched.
  #       # (Optional) defaults to empty list
  #       tags:
  #         - "dev"
  #         - "docker"
  #
  #       # passingOnly determines if only services passing the consul check are returned.
  #       # (Optional) defaults to false
  #       passingOnly: true
  #
  #       # crossDatacenter determines if a watch should be created for each known datacenter which allows
  #       # scytale's fannout hash to work across all datacenters.
  #       # (Optional) defaults to false meaning a single watch is setup with the current datacenter.
  #       crossDatacenter: true
  #
  #       # queryOptions are options for the consul query, used in conjunction
  #       # with passingOnly.
  #       # defined by https://godoc.org/github.com/hashicorp/consul/api#QueryOptions
  #       # (Optional) defaults to empty struct
  #       # queryOptions:
  #       #   useCache: true

# tracing provides configuration around traces using OpenTelemetry.
# (Optional). By default, a 'noop' tracer provider is used and tracing is disabled.
tracing:
  # provider is the name of the trace provider to use. Currently, otlp/grpc, otlp/http, stdout, jaeger and zipkin are supported.
  # 'noop' can also be used as provider to explicitly disable tracing.
  provider: "noop"

  # skipTraceExport only applies when provider is stdout. Set skipTraceExport to true
  # so that trace information is not written to stdout.
  # skipTraceExport: true

  # endpoint is where trace information should be routed. Applies to otlp, zipkin, and jaegar. OTLP/gRPC uses port 4317 by default.
  # OTLP/HTTP uses port 4318 by default.
  # endpoint: "localhost:4317"
  
  # ParentBased and NoParent dictate if and when new spans should be created.
  # ParentBased = "ignore" (default), tracing is effectively turned off and the "NoParent" value is ignored
  # ParentBased = "honor", the sampling decision is made by the parent of the span
  parentBased: ignore

  # NoParent decides if a root span should be initiated in the case where there is no existing parent
  # This value is ignored if ParentBased = "ignore"
  # NoParent = "never" (default), root spans are not initiated
  # NoParent = "always", roots spans are initiated
  noParent: never

# previousVersionSupport allows us to support two different major versions of
# the API at the same time from the same application.  When this is true,
# scytale will support both "/v2" and "/v3" endpoints.  When false, only "/v3"
# endpoints will be supported.
previousVersionSupport: true

#tr1d1um.yaml

---

########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# The unique fully-qualified-domain-name of the server.  It is provided to 
# the X-Tr1d1um-Server header for showing what server fulfilled the request 
# sent.
# (Optional)
server: "tr1d1um-local-instance-123.example.com"

# Provides this build number to the X-Tr1d1um-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
build: "0.1.3-434"

# Provides the region information to the X-Tr1d1um-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
region: "east"

# Provides the flavor information to the X-Tr1d1um-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
flavor: "mint"

prometheus:
  defaultNamespace: webpa
  defaultSubsystem: tr1d1um
  constLabels:
    development: "true"
  handler:
    maxRequestsInFlight: 5
    timeout: 5s
    instrumentMetricHandler: true

health:
  disableLogging: false
  custom:
    server: development

########################################
#   Primary Endpoint Configuration
########################################

servers:
  primary:
    address: :6100
    disableHTTPKeepAlives: true
    header:
      X-Midt-Server:
        - tr1d1um
      X-Midt-Version:
        - development
  alternate:
    address: :8090
    header:
      X-Midt-Server:
        - tr1d1um
      X-Midt-Version:
        - development
  metrics:
    address: :6101
    disableHTTPKeepAlives: true
    header:
      X-Midt-Server:
        - tr1d1um
      X-Midt-Version:
        - development
  health:
    address: :6102
    disableHTTPKeepAlives: true
    header:
      X-Midt-Server:
        - tr1d1um
      X-Midt-Version:
        - development
  pprof:
    address: :6103

########################################
#   Logging Related Configuration
########################################

logging:
  # OutputPaths is a list of URLs or file paths to write logging output to.
  outputPaths:
    - stdout
    # - /var/log/tr1d1um/tr1d1um.log

  # Level is the minimum enabled logging level. Note that this is a dynamic
  # level, so calling Config.Level.SetLevel will atomically change the log
  # level of all loggers descended from this config.
  level: debug

  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  errorOutputPaths:
    - stderr
    - denopink-tr1d1um.log

  # EncoderConfig sets options for the chosen encoder. See
  # zapcore.EncoderConfig for details.
  encoderConfig:
    messageKey: message
    levelKey: key
    levelEncoder: lowercase
  # reducedLoggingResponseCodes allows disabling verbose transaction logs for 
  # benign responses from the target server given HTTP status codes.
  # (Optional)
  # reducedLoggingResponseCodes: [200, 504]

  # Encoding sets the logger's encoding. Valid values are "json" and
  # "console", as well as any third-party encodings registered via
  # RegisterEncoder.
  encoding: json

##############################################################################
# Webhooks Related Configuration 
##############################################################################
# webhook provides configuration for storing and obtaining webhook
# information using Argus.
# Optional: if key is not supplied, webhooks would be disabled.
webhook:

  # disablePartnerIDs, if true, will allow webhooks to register without 
  # checking the validity of the partnerIDs in the request
  # Defaults to 'false'.
  disablePartnerIDs: false

  # validation provides options for validating the webhook's URL and TTL 
  # related fields. Some validation happens regardless of the configuration: 
  # URLs must be a valid URL structure, the Matcher.DeviceID values must 
  # compile into regular expressions, and the Events field must have at 
  # least one value and all values must compile into regular expressions. 
  validation:

    # url provides options for additional validation of the webhook's 
    # Config.URL, FailureURL, and Config.AlternativeURLs fields.
    url:
      # httpsOnly will allow only URLs with https schemes through if true.
      # (Optional). Defaults to 'false'.
      httpsOnly: false

      # allowLoopback will allow any canonical or IP loopback address if 
      # true. Otherwise, loopback addresses are considered invalid.
      # (Optional). Defaults to 'false'.
      allowLoopback: true

      # allowIP allows the different webhook URLs to have IP hostnames if set to true.
      # (Optional). Defaults to 'false'.
      allowIP: true

      # allowSpecialUseHosts allows URLs that include reserved domains if set to true.  
      # Read more here: https://en.wikipedia.org/wiki/Top-level_domain#Reserved_domains
      # (Optional). Defaults to 'false'.
      allowSpecialUseHosts: true

      # allowSpecialUseIPs, if set to true, allows URLs that contain or route to IPs that have 
      # been marked as reserved through various RFCs: rfc6761, rfc6890, rfc8190.
      # (Optional). Defaults to 'false'.
      allowSpecialUseIPs: true

      # invalidHosts is a slice that contains strings that we do not want 
      # allowed in URLs, providing a way to deny certain domains or hostnames.
      # (Optional). Defaults to an empty slice.
      invalidHosts: []

      # invalidSubnets is a list of IP subnets.  If a URL contains an 
      # IP or resolves to an IP in one of these subnets, the webhook is 
      # considered invalid.
      # (Optional). Defaults to an empty slice.
      invalidSubnets: []

    # ttl provides information for what is considered valid for time-related 
    # fields (Duration and Until) in the webhook.  A webhook set to expire 
    # too far in the future is considered invalid, while a time in the past 
    # is considered equivalent to a request to delete the webhook.  
    # Regardless of this configuration, either Until or Duration must have a 
    # non-zero value.
    ttl:
      # max is the length of time a webhook is allowed to live.  The Duration 
      # cannot be larger than this value, and the Until value cannot be set 
      # later than the current time + max + jitter.
      max: 1m

      # jitter is the buffer time added when checking that the Until value is 
      # valid.  If there is a slight clock skew between servers or some delay 
      # in the http request, jitter should help account for that when ensuring 
      # that Until is not a time too far in the future.
      jitter: 10s

  # JWTParserType establishes which parser type will be used by the JWT token
  # acquirer used by Argus. Options include 'simple' and 'raw'.
  # Simple: parser assumes token payloads have the following structure: https://github.com/xmidt-org/bascule/blob/c011b128d6b95fa8358228535c63d1945347adaa/acquire/bearer.go#L77
  # Raw: parser assumes all of the token payload == JWT token
  # (Optional). Defaults to 'simple'.
  JWTParserType: "raw"
  BasicClientConfig: 
    # listen is the subsection that configures the listening feature of the argus client
    # (Optional)
    listen:
      # pullInterval provides how often the current webhooks list gets refreshed.
      pullInterval: 5s

    # bucket is the partition name where webhooks will be stored.
    bucket: "webhooks"

    # address is Argus' network location.
    address: "http://localhost:6600"

    # auth the authentication method for argus.
    auth:
      # basic configures basic authentication for argus.
      # Must be of form: 'Basic xyz=='
      basic: "Basic dXNlcjpwYXNz"
  #
  #    # jwt configures jwt style authentication for argus.
  #    JWT:
  #      # requestHeaders are added to the request for the token.
  #      # (Optional)
  #      # requestHeaders:
  #      #   "": ""
  #
  #      # authURL is the URL to access the token.
  #      authURL: ""
  #
  #      # timeout is how long the request to get the token will take before
  #      # timing out.
  #      timeout: "1m"
  #
  #      # buffer is the length of time before a token expires to get a new token.
  #      buffer: "2m"


##############################################################################
# Authorization Credentials
##############################################################################
# jwtValidator provides Bearer auth configuration
jwtValidator:
  config:
    resolve:
      # Template is a URI template used to fetch keys.  This template may
      # use a single parameter named keyID, e.g. http://keys.com/{keyID}.
      # This field is required and has no default.
      template: "http://localhost/{keyID}"
    refresh:
      sources:
        # URI is the location where keys are served.  By default, clortho supports
        # file://, http://, and https:// URIs, as well as standard file system paths
        # such as /etc/foo/bar.jwk.
        #
        # This field is required and has no default.
        - uri: "http://localhost/available"
authx:
  inbound:
    # basic is a list of Basic Auth credentials intended to be used for local testing purposes
    # WARNING! Be sure to remove this from your production config
    basic: ["dXNlcjpwYXNz"]
# capabilityCheck provides the details needed for checking an incoming JWT's
# capabilities.  If the type of check isn't provided, no checking is done.  The 
# type can be "monitor" or "enforce".  If it is empty or a different value, no 
# checking is done.  If "monitor" is provided, the capabilities are checked but 
# the request isn't rejected when there isn't a valid capability for the 
# request. Instead, a message is logged.  When "enforce" is provided, a request 
# that doesn't have the needed capability is rejected.
#
# The capability is expected to have the format:
#
# {prefix}{endpoint}:{method}
#
# The prefix can be a regular expression.  If it's empty, no capability check 
# is done.  The endpoint is a regular expression that should match the endpoint
# the request was sent to. The method is usually the method of the request, such as 
# GET.  The accept all method is a catchall string that indicates the capability 
# is approved for all methods.
# (Optional)
# capabilityCheck:
#   # type provides the mode for capability checking.
#   type: "enforce"
#   # prefix provides the regex to match the capability before the endpoint.
#   prefix: "prefix Here"
#   # acceptAllMethod provides a way to have a capability that allows all 
#   # methods for a specific endpoint.
#   acceptAllMethod: "all"
#   # endpointBuckets provides regular expressions to use against the request 
#   # endpoint in order to group requests for a metric label.
#   endpointBuckets:
#     - "hook\\b"
#     - "hooks\\b"
#     - "device/.*/stat\\b"
#     - "device/.*/config\\b"


##############################################################################
# WRP and XMiDT Cloud configurations
##############################################################################

# targetURL is the base URL of the XMiDT cluster 
targetURL: http://localhost:6300/api/v3

# WRPSource is used as 'source' field for all outgoing WRP Messages
WRPSource: "dns:tr1d1um.example.com"

# supportedServices is a list of endpoints we support for the WRP producing endpoints 
# we will soon drop this configuration 
supportedServices:
  - "config"


##############################################################################
# HTTP Transaction Configurations
##############################################################################
# timeouts that apply to the Argus HTTP client.
# (Optional) By default, the values below will be used.
argusClientTimeout:
  # clientTimeout is the timeout for requests made through this 
  # HTTP client. This timeout includes connection time, any
  # redirects, and reading the response body.
  clientTimeout: 50s

  # netDialerTimeout is the maximum amount of time the HTTP Client Dialer will
  # wait for a connect to complete.
  netDialerTimeout: 5s

# timeouts that apply to the XMiDT HTTP client.
# (Optional) By default, the values below will be used.
xmidtClientTimeout:
  # clientTimeout is the timeout for the requests made through this 
  # HTTP client. This timeout includes connection time, any
  # redirects, and reading the response body.
  clientTimeout: 135s

  # requestTimeout is the timeout imposed on requests made by this client 
  # through context cancellation.
  # TODO since clientTimeouts are implemented through context cancellations,
  # we might not need this.
  requestTimeout: 129s

  # netDialerTimeout is the maximum amount of time the HTTP Client Dialer will
  # wait for a connect to complete.
  netDialerTimeout: 5s


# requestRetryInterval is the time between HTTP request retries against XMiDT 
requestRetryInterval: "2s"

# requestMaxRetries is the max number of times an HTTP request is retried against XMiDT in 
# case of ephemeral errors
requestMaxRetries: 2

# authAcquirer enables configuring the JWT or Basic auth header value factory for outgoing
# requests to XMiDT. If both types are configured, JWT will be preferred.
# (Optional)
# authAcquirer:
  # JWT: 
  #   # requestHeaders are added to the request for the token.
  #   # (Optional)
  #   # requestHeaders:
  #   #   "": "" 

  #   # authURL is the URL to access for the token.
  #   authURL: ""

  #   # timeout is how long the request to get the token will take before 
  #   # timing out.
  #   timeout: "1m"

  #   # buffer is the length of time before a token expires to get a new token.
  #   buffer: "2m"  
    
  # Basic: "" # Must be of form: 'Basic xyz=='


# tracing provides configuration around traces using OpenTelemetry.
# (Optional). By default, a 'noop' tracer provider is used and tracing is disabled.
tracing:
  # provider is the name of the trace provider to use. Currently, otlp/grpc, otlp/http, stdout, jaeger and zipkin are supported.
  # 'noop' can also be used as provider to explicitly disable tracing.
  provider: "noop"

  # skipTraceExport only applies when provider is stdout. Set skipTraceExport to true
  # so that trace information is not written to stdout.
  # skipTraceExport: true

  # endpoint is where trace information should be routed. Applies to otlp, zipkin, and jaegar. OTLP/gRPC uses port 4317 by default. 
  # OTLP/HTTP uses port 4318 by default.
  # endpoint: "localhost:4317"

  # ParentBased and NoParent dictate if and when new spans should be created.
  # ParentBased = "ignore" (default), tracing is effectively turned off and the "NoParent" value is ignored
  # ParentBased = "honor", the sampling decision is made by the parent of the span
  parentBased: ignore

  # NoParent decides if a root span should be initiated in the case where there is no existing parent
  # This value is ignored if ParentBased = "ignore"
  # NoParent = "never" (default), root spans are not initiated
  # NoParent = "always", roots spans are initiated
  noParent: never

# previousVersionSupport allows us to support two different major versions of
# the API at the same time from the same application.  When this is true,
# tr1d1um will support both "/v2" and "/v3" endpoints.  When false, only "/v3"
# endpoints will be supported.
previousVersionSupport: true

@denopink
Copy link
Contributor

denopink commented Apr 5, 2024

thx, I'll run a local cluster with these configurations in the coming days and let you know what I find

@VicLin66
Copy link
Author

VicLin66 commented Apr 8, 2024

@denopink OK, thanks in advance. And below is my talaria /scytale/tr1d1um config (.yaml) version, for you reference.
talaria: v0.9.2
scytale: v0.9.6
tr1d1um: v0.9.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants