diff --git a/CHANGELOG.md b/CHANGELOG.md index 83cbcb3..1aec77f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # CHANGELOG +## 0.12.1 + +- Key changes: + - Minor changes in CLI help; + - Improved docs. + ## 0.12.0 - Key changes: diff --git a/README.md b/README.md index 5cceca6..dbd9855 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ -# Label Filter Gateway (LFGW) +# Label Filter Gateway (lfgw) -LFGW is a trivial reverse proxy based on `httputil` and `VictoriaMetrics/metricsql` with a purpose of dynamically rewriting requests to Prometheus-like backends. +lfgw is a simple reverse proxy aimed at PromQL / MetricsQL metrics filtering based on OIDC roles. -More specifically, it manipulates label filters in metric expressions to reduce the scope of metrics exposed to an end user based on user's OIDC-roles. The process is described in more details [here](docs/filtering.md). +It relies on [VictoriaMetrics/metricsql](https://github.com/VictoriaMetrics/metricsql) for label filters manipulation in metric expressions (according to an [ACL](#acl)) before a request is proxied to Prometheus/VictoriaMetrics by [httputil](https://pkg.go.dev/net/http/httputil). The process is described in more details [here](docs/filtering.md). -Target setup: `grafana -> lfgw -> Prometheus/VictoriaMetrics`. +Target setup: `grafana (with OIDC integration) <-> lfgw <-> Prometheus/VictoriaMetrics`. ## Key features @@ -12,7 +12,7 @@ Target setup: `grafana -> lfgw -> Prometheus/VictoriaMetrics`. * a user can have multiple roles; * support for autoconfiguration in environments, where OIDC-role names match names of namespaces ("assumed roles" mode; thanks to [@aberestyak](https://github.com/aberestyak/) for the idea); * [automatic expression optimizations](https://pkg.go.dev/github.com/VictoriaMetrics/metricsql#Optimize) for non-full access requests; -* support for different headers with access tokens (`Authorization`, `X-Forwarded-Access-Token`, `X-Auth-Request-Access-Token`), which can be useful for [oauth2-proxy](https://github.com/oauth2-proxy/oauth2-proxy) or other tools; +* support for different headers with access tokens (`Authorization`, `X-Forwarded-Access-Token`, `X-Auth-Request-Access-Token`), which can be useful for tools like [oauth2-proxy](https://github.com/oauth2-proxy/oauth2-proxy); * requests to both `/api/*` and `/federate` endpoints are protected (=rewritten); * requests to sensitive endpoints are blocked by default; * compatible with both [PromQL](https://prometheus.io/docs/prometheus/latest/querying/basics/) and [MetricsQL](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/MetricsQL). @@ -27,6 +27,8 @@ Docker images are published on [ghcr.io/weisdd/lfgw](https://github.com/weisdd/l ## Configuration +Example of `keycloak + grafana + lfgw` setup is described [here](./docs/oidc.md). + ### Requirements for jwt-tokens * OIDC-roles must be present in `roles` claim; @@ -34,41 +36,32 @@ Docker images are published on [ghcr.io/weisdd/lfgw](https://github.com/weisdd/l ### Environment variables -| Module | Variable | Default Value | Description | -| -------------------- | --------------------------- | ------------- | ------------------------------------------------------------ | -| **General settings** | | | | -| | `ASSUMED_ROLES` | `false` | In environments, where OIDC-role names match names of namespaces, ACLs can be constructed on the fly (e.g. `["role1", "role2"]` will give access to metrics from namespaces `role1` and `role2`). The roles specified in `acl.yaml` are still considered and get merged with assumed roles. Role names may contain regular expressions, including the admin definition `.*`. | -| | `ENABLE_DEDUPLICATION` | `true` | Whether to enable deduplication, which leaves some of the requests unmodified if they match the target policy. Examples can be found in the "acl.yaml syntax" section. | -| | `OPTIMIZE_EXPRESSIONS` | `true` | Whether to automatically optimize expressions for non-full access requests. [More details](https://pkg.go.dev/github.com/VictoriaMetrics/metricsql#Optimize) | -| | `SET_GOMAXPROCS` | `true` | Automatically set `GOMAXPROCS` to match Linux container CPU quota. | -| | | | | -| **Logging** | | | | -| | `DEBUG` | `false` | Whether to print out debug log messages. | -| | `LOG_FORMAT` | `pretty` | Log format (`pretty`, `json`) | -| | `LOG_NO_COLOR` | `false` | Whether to disable colors for `pretty` format | -| | `LOG_REQUESTS` | `false` | Whether to log HTTP requests | -| | | | | -| **HTTP Server** | | | | -| | `PORT` | `8080` | Port the web server will listen on. | -| | `READ_TIMEOUT` | `10s` | `ReadTimeout` covers the time from when the connection is accepted to when the request body is fully read (if you do read the body, otherwise to the end of the headers). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) | -| | `WRITE_TIMEOUT` | `10s` | `WriteTimeout` normally covers the time from the end of the request header read to the end of the response write (a.k.a. the lifetime of the ServeHTTP). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) | -| | `GRACEFUL_SHUTDOWN_TIMEOUT` | `20s` | Maximum amount of time to wait for all connections to be closed. [More details](https://pkg.go.dev/net/http#Server.Shutdown) | -| | | | | -| **Proxy** | | | | -| | `UPSTREAM_URL` | | Prometheus URL, e.g. `http://prometheus.microk8s.localhost`. | -| | `SAFE_MODE` | `true` | Whether to block requests to sensitive endpoints like `/api/v1/admin/tsdb`, `/api/v1/insert`. | -| | `SET_PROXY_HEADERS` | `false` | Whether to set proxy headers (`X-Forwarded-For`, `X-Forwarded-Proto`, `X-Forwarded-Host`). | -| | | | | -| **OIDC** | | | | -| | `ACL_PATH` | `./acl.yaml` | Path to a file with ACL definitions (OIDC role to namespace bindings). Skipped if `ACL_PATH` is empty (might be useful when autoconfiguration is enabled through `ASSUMED_ROLES=true`). | -| | `OIDC_REALM_URL` | | OIDC Realm URL, e.g. `https://auth.microk8s.localhost/auth/realms/cicd` | -| | `OIDC_CLIENT_ID` | | OIDC Client ID (1*) | - -(1*): since it's grafana who obtains jwt-tokens in the first place, the specified client id must also be present in the forwarded token (the `aud` claim). To put it simply, better to use the same client id for both Grafana and LFGW. - -### acl.yaml syntax - -The file with ACL definitions has a simple structure: +| Variable | Default Value | Description | +| --------------------------- | ------------- | ------------------------------------------------------------ | +| `UPSTREAM_URL` | | Prometheus URL, e.g. `http://prometheus.localhost`. | +| `OIDC_REALM_URL` | | OIDC Realm URL, e.g. `https://keycloak.localhost/auth/realms/monitoring` | +| `OIDC_CLIENT_ID` | | OIDC Client ID (1*) | +| `ACL_PATH` | `./acl.yaml` | Path to a file with ACL definitions (OIDC role to namespace bindings). Skipped if `ACL_PATH` is empty (might be useful when autoconfiguration is enabled through `ASSUMED_ROLES=true`). | +| `ASSUMED_ROLES` | `false` | In environments, where OIDC-role names match names of namespaces, ACLs can be constructed on the fly (e.g. `["role1", "role2"]` will give access to metrics from namespaces `role1` and `role2`). The roles specified in `acl.yaml` are still considered and get merged with assumed roles. Role names may contain regular expressions, including the admin definition `.*`. | +| `ENABLE_DEDUPLICATION` | `true` | Whether to enable deduplication, which leaves some of the requests unmodified if they match the target policy. Examples can be found in the "acl.yaml syntax" section. | +| `OPTIMIZE_EXPRESSIONS` | `true` | Whether to automatically optimize expressions for non-full access requests. [More details](https://pkg.go.dev/github.com/VictoriaMetrics/metricsql#Optimize) | +| `SAFE_MODE` | `true` | Whether to block requests to sensitive endpoints like `/api/v1/admin/tsdb`, `/api/v1/insert`. | +| `SET_PROXY_HEADERS` | `false` | Whether to set proxy headers (`X-Forwarded-For`, `X-Forwarded-Proto`, `X-Forwarded-Host`). | +| `SET_GOMAXPROCS` | `true` | Automatically set `GOMAXPROCS` to match Linux container CPU quota. | +| `DEBUG` | `false` | Whether to print out debug log messages. | +| `LOG_FORMAT` | `pretty` | Log format (`pretty`, `json`) | +| `LOG_NO_COLOR` | `false` | Whether to disable colors for `pretty` format | +| `LOG_REQUESTS` | `false` | Whether to log HTTP requests | +| `PORT` | `8080` | Port the web server will listen on. | +| `READ_TIMEOUT` | `10s` | `ReadTimeout` covers the time from when the connection is accepted to when the request body is fully read (if you do read the body, otherwise to the end of the headers). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) | +| `WRITE_TIMEOUT` | `10s` | `WriteTimeout` normally covers the time from the end of the request header read to the end of the response write (a.k.a. the lifetime of the ServeHTTP). [More details](https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/) | +| `GRACEFUL_SHUTDOWN_TIMEOUT` | `20s` | Maximum amount of time to wait for all connections to be closed. [More details](https://pkg.go.dev/net/http#Server.Shutdown) | + +(1*): since it's grafana who obtains jwt-tokens in the first place, the specified client id must also be present in the forwarded token (the `aud` claim). + +### ACL + +The file with ACL definitions (`./acl.yaml` by default) has a simple structure: ```yaml role: namespace, namespace2 @@ -108,7 +101,7 @@ Note: a user is free to have multiple roles matching the contents of `acl.yaml`. * multiple roles, one of which gives full access => a prepopulated LF, corresponding to the full access role, is returned; * multiple "limited" roles - => definitions of all those roles are merged together, and then LFGW generates a new LF. The process is the same as if this meta-definition was loaded through `acl.yaml`. + => definitions of all those roles are merged together, and then lfgw generates a new LF. The process is the same as if this meta-definition was loaded through `acl.yaml`. ## Licensing diff --git a/cmd/lfgw/main.go b/cmd/lfgw/main.go index 8bb8826..57b5871 100644 --- a/cmd/lfgw/main.go +++ b/cmd/lfgw/main.go @@ -18,9 +18,8 @@ var ( func main() { app := &cli.App{ - Name: "lfgw", - Version: fmt.Sprintf("%s (commit: %s; runtime: %s)", version, commit, goVersion), - Compiled: time.Now(), + Name: "lfgw", + Version: fmt.Sprintf("%s (commit: %s; runtime: %s)", version, commit, goVersion), Authors: []*cli.Author{ { Name: "weisdd", @@ -52,13 +51,13 @@ func main() { Flags: []cli.Flag{ &cli.StringFlag{ Name: "upstream-url", - Usage: "Prometheus URL, e.g. http://prometheus.microk8s.localhost", + Usage: "Prometheus URL, e.g. http://prometheus.localhost", EnvVars: []string{"UPSTREAM_URL"}, Required: true, }, &cli.StringFlag{ Name: "oidc-realm-url", - Usage: "OIDC Realm URL, e.g. `https://auth.microk8s.localhost/auth/realms/cicd", + Usage: "OIDC Realm URL, e.g. `https://keycloak.localhost/auth/realms/monitoring", EnvVars: []string{"OIDC_REALM_URL"}, Required: true, }, diff --git a/docs/filtering.md b/docs/filtering.md index 43d3090..45a7cd7 100644 --- a/docs/filtering.md +++ b/docs/filtering.md @@ -4,4 +4,4 @@ By default, Grafana works with data sources in a so called [server mode](https:/ In an identity provider such as Keycloak, we can add custom client roles and pass them in, say, `roles` claim (claim name could be different, but lfgw does not currently allow any other name). That's where lfgw comes into play. By tying roles to a list of namespaces (either full names or regexps), we can tell lfgw which metric expressions have to be modified (to reduce the scope) and which are allowed to be passed as is. -When a metric expression is extracted from GET-parameters or a POST-form that Grafana sends, lfgw manipulates `namespace` label in each selector according to an acl. Once it's done, the updated request is forwarded to the Prometheus-like backend. Examples of ACL can be found in [README.md](../README.md#aclyaml-syntax). +When a metric expression is extracted from GET-parameters or a POST-form that Grafana sends, lfgw manipulates `namespace` label in each selector according to an ACL. Once it's done, the updated request is forwarded to the Prometheus-like backend. Examples of ACL can be found in [README.md](../README.md#aclyaml-syntax). diff --git a/docs/oidc.assets/image-20220427161123432.png b/docs/oidc.assets/image-20220427161123432.png new file mode 100644 index 0000000..4133e63 Binary files /dev/null and b/docs/oidc.assets/image-20220427161123432.png differ diff --git a/docs/oidc.assets/image-20220427161145312.png b/docs/oidc.assets/image-20220427161145312.png new file mode 100644 index 0000000..89c4388 Binary files /dev/null and b/docs/oidc.assets/image-20220427161145312.png differ diff --git a/docs/oidc.assets/image-20220427161159283.png b/docs/oidc.assets/image-20220427161159283.png new file mode 100644 index 0000000..ed6d817 Binary files /dev/null and b/docs/oidc.assets/image-20220427161159283.png differ diff --git a/docs/oidc.assets/image-20220427161235210.png b/docs/oidc.assets/image-20220427161235210.png new file mode 100644 index 0000000..a4f56e6 Binary files /dev/null and b/docs/oidc.assets/image-20220427161235210.png differ diff --git a/docs/oidc.assets/image-20220427161405774.png b/docs/oidc.assets/image-20220427161405774.png new file mode 100644 index 0000000..ed8f579 Binary files /dev/null and b/docs/oidc.assets/image-20220427161405774.png differ diff --git a/docs/oidc.assets/image-20220427161429694.png b/docs/oidc.assets/image-20220427161429694.png new file mode 100644 index 0000000..8700151 Binary files /dev/null and b/docs/oidc.assets/image-20220427161429694.png differ diff --git a/docs/oidc.assets/image-20220427161550269.png b/docs/oidc.assets/image-20220427161550269.png new file mode 100644 index 0000000..ea8f870 Binary files /dev/null and b/docs/oidc.assets/image-20220427161550269.png differ diff --git a/docs/oidc.assets/image-20220427161603039.png b/docs/oidc.assets/image-20220427161603039.png new file mode 100644 index 0000000..2e0305a Binary files /dev/null and b/docs/oidc.assets/image-20220427161603039.png differ diff --git a/docs/oidc.assets/image-20220427161630123.png b/docs/oidc.assets/image-20220427161630123.png new file mode 100644 index 0000000..bf350e8 Binary files /dev/null and b/docs/oidc.assets/image-20220427161630123.png differ diff --git a/docs/oidc.assets/image-20220427161720535.png b/docs/oidc.assets/image-20220427161720535.png new file mode 100644 index 0000000..4adbe31 Binary files /dev/null and b/docs/oidc.assets/image-20220427161720535.png differ diff --git a/docs/oidc.assets/image-20220427161804675.png b/docs/oidc.assets/image-20220427161804675.png new file mode 100644 index 0000000..00a7206 Binary files /dev/null and b/docs/oidc.assets/image-20220427161804675.png differ diff --git a/docs/oidc.assets/image-20220427162017107.png b/docs/oidc.assets/image-20220427162017107.png new file mode 100644 index 0000000..2262594 Binary files /dev/null and b/docs/oidc.assets/image-20220427162017107.png differ diff --git a/docs/oidc.assets/image-20220427162050451.png b/docs/oidc.assets/image-20220427162050451.png new file mode 100644 index 0000000..6df6496 Binary files /dev/null and b/docs/oidc.assets/image-20220427162050451.png differ diff --git a/docs/oidc.assets/image-20220427162126396.png b/docs/oidc.assets/image-20220427162126396.png new file mode 100644 index 0000000..0d1fa6c Binary files /dev/null and b/docs/oidc.assets/image-20220427162126396.png differ diff --git a/docs/oidc.assets/image-20220427163543913.png b/docs/oidc.assets/image-20220427163543913.png new file mode 100644 index 0000000..5927e0d Binary files /dev/null and b/docs/oidc.assets/image-20220427163543913.png differ diff --git a/docs/oidc.assets/image-20220427181051466.png b/docs/oidc.assets/image-20220427181051466.png new file mode 100644 index 0000000..5c8064e Binary files /dev/null and b/docs/oidc.assets/image-20220427181051466.png differ diff --git a/docs/oidc.assets/image-20220427181844852.png b/docs/oidc.assets/image-20220427181844852.png new file mode 100644 index 0000000..0ffbdf9 Binary files /dev/null and b/docs/oidc.assets/image-20220427181844852.png differ diff --git a/docs/oidc.md b/docs/oidc.md new file mode 100644 index 0000000..53f311a --- /dev/null +++ b/docs/oidc.md @@ -0,0 +1,190 @@ +# OIDC + +In the examples below, we'll have the following naming: + +- realm `monitoring`; +- URLs: + - [http://keycloak.localhost](http://keycloak.localhost); + - [http://grafana.localhost](http://grafana.localhost); + - [http://prometheus.localhost](http://prometheus.localhost); + - [http://lfgw](http://lfgw); +- client ID: `grafana`. + +## keycloak + +### Realm + +First, we need to create a Realm for our users: + +![image-20220427162126396](oidc.assets/image-20220427162126396.png) + +![image-20220427162050451](oidc.assets/image-20220427162050451.png) + +### Tokens + +By default, grafana and keycloak have different values for session duration. If a session in keycloak expires earlier than in grafana, a refresh token will be revoked, and grafana will fail to obtain a new access token. In this case, grafana will silently stop adding `Authorization` header to its requests, thus making data source authentication fail until the user logs out and logs in again. + +That's why it's important to tune `SSO Session Idle` and `SSO Session Max` (`Realm Settings` -> `Tokens`): + +![image-20220427181051466](oidc.assets/image-20220427181051466.png) + +NOTE: the particular durations are not set in stone. + +### Client + +#### Client creation + +Next, we need to create a client (`Clients` -> `Add Client`): + +![image-20220427161123432](oidc.assets/image-20220427161123432.png) + +NOTE: It's better not to specify `Root URL` if the same realm is reused across different grafana instances. + +In the client's settings (`Clients` -> `grafana`), `Access Type` should be changed to `confidential`: + +![image-20220427161145312](oidc.assets/image-20220427161145312.png) + +and `Valid Redirect URIs` to `http://grafana.localhost/*`: + +![image-20220427162017107](oidc.assets/image-20220427162017107.png) + +Then press `Save`. + +#### Credentials + +Copy `secret` (`Clients` -> `Credentials`), it will be needed for `client_secret` in grafana configuration: + +![image-20220427161159283](oidc.assets/image-20220427161159283.png) + +#### Mappers + +To expose client roles (used by both grafana and lfgw to limit user access) and token audience (used for token validation), we need to create two mappers (`Clients` -> `grafana` -> `Mappers`): + +-> Click `Create` and configure the mapper like on the screenshot below: + +![image-20220427161235210](oidc.assets/image-20220427161235210.png) + +Get back to Mappers and click `Add Builtin` and pick `client roles`: + +![image-20220427161405774](oidc.assets/image-20220427161405774.png) + +Click `Edit` next to `client roles`: + +![image-20220427161429694](oidc.assets/image-20220427161429694.png) + +Configure the mapper like on the screenshot below: + +![image-20220427181844852](oidc.assets/image-20220427181844852.png) + +#### Roles + +Now, we need to add some client roles (`Clients` -> `grafana` -> `Roles` -> `Add role`): + +![image-20220427161550269](oidc.assets/image-20220427161550269.png) + +![image-20220427161603039](oidc.assets/image-20220427161603039.png) + +#### Users and role assignments + +The last step would be to create a user and assign a role to him (`Users` -> `Add user`): + +![image-20220427161630123](oidc.assets/image-20220427161630123.png) + +NOTE: `Email` must always be defined. + +Set password (`Users` -> `test` -> `Credentials`): + +![image-20220427161720535](oidc.assets/image-20220427161720535.png) + +Pick a role and assign it (`Users` -> `test` -> `Role Mappings` -> `Client Roles` -> `grafana`): + +![image-20220427161804675](oidc.assets/image-20220427161804675.png) + +NOTE: A user may have many roles. + +There are two sets of roles that you need to have: + +- roles used by grafana itself (`grafana-admin` and `grafana-editor` in this case); +- roles used by lfgw. + +Those roles' names can either be the same or different, depending on what you want to achieve. For simplicity of the example, here they have the same names. + +## grafana + +### Config + +Below, you could find an example of how to configure OIDC integration between keycloak and grafana. + +```ini +[ auth ] +# Disable the default loging form (only OIDC users are allowed to login) +disable_login_form = true +# Automatically redirect to Keycloak +oauth_auto_login = true +# Redirect users to Keycloak logout page and then back to grafana (if not configured, keycloak session will not end upon logout from grafana) +signout_redirect_url = http://keycloak.localhost/auth/realms/monitoring/protocol/openid-connect/logout?redirect_uri=http://grafana.localhost +# The maximum lifetime (duration) an authenticated user can be inactive before being required to login at next visit. Default is 7 days (7d). This setting should be expressed as a duration, e.g. 5m (minutes), 6h (hours), 10d (days), 2w (weeks), 1M (month). The lifetime resets at each successful token rotation (token_rotation_interval_minutes). +# NOTE: Should be lower than "SSO Session Idle" in Keycloak. +login_maximum_inactive_lifetime_duration = 4h +# The maximum lifetime (duration) an authenticated user can be logged in since login time before being required to login. Default is 30 days (30d). This setting should be expressed as a duration, e.g. 5m (minutes), 6h (hours), 10d (days), 2w (weeks), 1M (month). +# NOTE: Should be lower than "SSO Session Max" in Keycloak. +login_maximum_lifetime_duration = 8h + +[ auth.generic_oauth ] +enabled = true +allow_sign_up = true +# The same client ID as added in Keycloak +client_id = grafana +# Client secret copied from Keycloak +client_secret = aqtmNg0G1hq5EWUu2HEuWJK93leVkBn2 +# Those links point to keycloak URLs tied to our realm (monitoring) +auth_url = http://keycloak.localhost/auth/realms/monitoring/protocol/openid-connect/auth +token_url = http://keycloak.localhost/auth/realms/monitoring/protocol/openid-connect/token +api_url = http://keycloak.localhost/auth/realms/monitoring/protocol/openid-connect/userinfo +scopes = email profile +# grafana-admin will give "Admin" role, grafana-editor - "Editor", in all other cases a user will become "Viewer" +role_attribute_path = "contains(roles[*], 'grafana-admin') && 'Admin' || contains(roles[*], 'grafana-editor') && 'Editor' || 'Viewer'" +``` + +NOTE: If `login_maximum_inactive_lifetime_duration` and `login_maximum_lifetime_duration` are not properly set, then authentication to a data source will eventually fail, and a user will be required to logout-login as a workaround. + +### Data source + +Point grafana at a lfgw instance and make sure access token is forwarded (`Forward Oauth Identity`, available only in `Server` mode) (`Configuration` -> `Data sources` -> ``): + +![image-20220427163543913](oidc.assets/image-20220427163543913.png) + +## lfgw + +### Environment variables + +```shell +# Prometheus / VictoriaMetrics base URL +export UPSTREAM_URL=http://prometheus.localhost +# Realm URL, lfgw uses the data exposed there to validate access tokens forwarded by grafana +export OIDC_REALM_URL=http://keycloak.localhost/auth/realms/monitoring +# Used to validate the Audience (aud) field in the tokens +export OIDC_CLIENT_ID=grafana +``` + +### ACL (acl.yaml) + +```yaml +# Gives full access to metrics +grafana-admin: .* +# Gives access only for metrics from these namespaces +grafana-editor: ku.*, min.*, monitoring +``` + +The ACL definitions are, essentially, role to namespace bindings. + +If "assumed roles" (autoconfiguration) functionality is enabled (`export ASSUMED_ROLES=true`), then unknown roles will be treated as ACL definitions. E.g. a role `.*` will give access to all metrics, `monitoring` - to those collected in the `monitoring` namespace. + +You can have a mix of pre-defined roles and assumed roles if needed. + +## Final notes + +Once all of that is configured, a user can login to the grafana instance and have: + +- a grafana role assigned depending on `role_attribute_path` in grafana settings (if there's no match, it'll be `Viewer`); +- metrics filtered based on lfgw ACL.