You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By default, this server binds to `localhost` and listens at port `8082`.
46
+
If you bind the HistoryServer to a network-facing address such as `0.0.0.0`, enable authentication or place it behind an authenticated gateway. Without authentication, any user with network access can inspect archived job metadata, execution plans, configuration values, timestamps, and configured log links.
46
47
47
48
Currently, you can only run it as a standalone process.
48
49
@@ -75,6 +76,33 @@ The contained archives are downloaded and cached in the local filesystem. The lo
75
76
76
77
Check out the configuration page for a [complete list of configuration options]({{< ref "docs/deployment/config" >}}#history-server).
77
78
79
+
## SPNEGO Authentication
80
+
81
+
The HistoryServer web UI and REST endpoints can require Kerberos/SPNEGO authentication. This is opt-in; the default `historyserver.web.authentication.type` is `NONE`.
82
+
83
+
To enable SPNEGO, configure an HTTP service principal and keytab for the HistoryServer process:
The principal must start with `HTTP/`. The `_HOST` placeholder is replaced with the local hostname, and `*` can be used to accept all `HTTP/` principals present in the configured keytab. If `historyserver.web.authentication.kerberos.name-rules` is not configured, the default Hadoop auth-to-local rule is used.
93
+
94
+
When authentication is enabled, requests without a valid cookie or `Authorization: Negotiate` header return `401` with `WWW-Authenticate: Negotiate`. A valid SPNEGO exchange issues a signed `hadoop.auth` cookie and then proceeds to the existing HistoryServer handlers. Invalid SPNEGO tokens return `403`.
95
+
96
+
Clients with Kerberos credentials can access the HistoryServer with:
If multiple HistoryServer instances serve the same endpoint, configure the same `historyserver.web.authentication.signature.secret` or `historyserver.web.authentication.signature.secret-file` on all instances so that authentication cookies are accepted consistently. If neither option is configured, a random process-local signing secret is generated at startup.
103
+
104
+
SPNEGO authenticates the caller only. It does not add per-user or per-group authorization, and it does not change JobManager or TaskManager REST authentication.
105
+
78
106
## Log Integration
79
107
80
108
Flink does not provide built-in methods for archiving logs of completed jobs.
Copy file name to clipboardExpand all lines: docs/content/docs/deployment/config.md
+3Lines changed: 3 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -161,11 +161,14 @@ You can configure checkpointing directly in code within your Flink job or applic
161
161
162
162
**Web UI**
163
163
164
+
-`web.authentication.type`: Controls authentication for the JobManager Web UI and REST endpoint *(NONE by default)*. Set this to `KERBEROS` to require SPNEGO authentication on all `8081` paths.
164
165
-`web.submit.enable`: Enables uploading and starting jobs through the Flink UI *(true by default)*. Please note that even when this is disabled, session clusters still accept jobs through REST requests (HTTP calls). This flag only guards the feature to upload jobs in the UI.
165
166
-`web.cancel.enable`: Enables canceling jobs through the Flink UI *(true by default)*. Please note that even when this is disabled, session clusters still cancel jobs through REST requests (HTTP calls). This flag only guards the feature to cancel jobs in the UI.
166
167
-`web.upload.dir`: The directory where to store uploaded jobs. Only used when `web.submit.enable` is true.
167
168
-`web.exception-history-size`: Sets the size of the exception history that prints the most recent failures that were handled by Flink for a job.
168
169
170
+
Exposing the JobManager Web UI / REST endpoint to an untrusted network without authentication allows unauthenticated job submission and control operations. Enable `web.authentication.type: KERBEROS` or place the endpoint behind an authenticated proxy before exposing port `8081`.
171
+
169
172
**Other**
170
173
171
174
-`io.tmp.dirs`: The directories where Flink puts local data, defaults to the system temp directory (`java.io.tmpdir` property). If a list of directories is configured, Flink will rotate files across the directories.
Copy file name to clipboardExpand all lines: docs/content/docs/deployment/security/security-kerberos.md
+29Lines changed: 29 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,6 +60,7 @@ The following services and connectors are supported for Kerberos authentication:
60
60
- HDFS
61
61
- HBase
62
62
- ZooKeeper
63
+
- JobManager Web UI / REST endpoint (SPNEGO)
63
64
64
65
Note that it is possible to enable the use of Kerberos independently for each service or connector.
65
66
For example, the user may enable Hadoop security without necessitating the use of Kerberos for ZooKeeper,
@@ -92,6 +93,34 @@ dynamic entries provided by this module.
92
93
This module configures certain process-wide ZooKeeper security-related settings, namely the ZooKeeper service name (default: `zookeeper`)
93
94
and the JAAS login context name (default: `Client`).
94
95
96
+
### JobManager Web UI SPNEGO Authentication
97
+
98
+
The JobManager Web UI and REST endpoint are unauthenticated by default. If port `8081` is reachable from an untrusted network, unauthenticated users can submit jobs and perform control operations. To require Kerberos/SPNEGO authentication on all JobManager WebMonitor paths, enable:
The principal must start with `HTTP/`. The `_HOST` placeholder is replaced from `rest.address`, then `rest.bind-address`, then the local canonical hostname. Use `*` to accept all `HTTP/` principals present in the configured keytab.
111
+
112
+
When enabled, requests without a valid authentication cookie or `Authorization: Negotiate` header receive `401` with `WWW-Authenticate: Negotiate`; malformed or invalid SPNEGO tokens receive `403`. A successful SPNEGO exchange issues a signed `hadoop.auth` cookie, so subsequent requests do not need to negotiate Kerberos again until the cookie expires.
113
+
114
+
Clients with Kerberos credentials can access the endpoint with:
For multiple JobManager instances behind the same address, configure a shared `web.authentication.signature.secret` or `web.authentication.signature.secret-file`; otherwise each process uses a random local signing secret and cannot verify cookies issued by another process.
121
+
122
+
SPNEGO authenticates the caller only. It does not add per-user authorization, user or group allowlists, Ranger policy enforcement, Knox integration, session ownership enforcement, or built-in Flink CLI / REST client SPNEGO support.
123
+
95
124
## Deployment Modes
96
125
Here is some information specific to each deployment mode.
The command starts the SQL Gateway with REST Endpoint that listens on the address localhost:8083. You can use the curl command to check
58
58
whether the REST Endpoint is available.
59
59
60
+
{{< hint warning >}}
61
+
The SQL Gateway REST endpoint does not authenticate requests by default. If it is bound to a network-facing address, any client with network access can create sessions and execute SQL unless you enable REST SPNEGO authentication or protect the endpoint with external access controls.
62
+
{{< /hint >}}
63
+
60
64
```bash
61
65
$ curl http://localhost:8083/v1/info
62
66
{"productName":"Apache Flink","version":"{{< version >}}"}
<td>Kerberos principal for accepting SPNEGO requests when REST authentication type is KERBEROS. The value supports HTTP/_HOST@REALM host replacement and * to load all HTTP principals from the keytab.</td>
<td>Shared secret used to sign REST authentication cookies. If neither this option nor signature.secret-file is configured, a random process-local secret is generated.</td>
<td>Path to a file containing the shared secret used to sign REST authentication cookies. Configure either signature.secret or signature.secret-file, but not both.</td>
138
+
</tr>
91
139
</tbody>
92
140
</table>
93
141
142
+
REST SPNEGO Authentication
143
+
----------------
144
+
145
+
By default, the REST endpoint does not authenticate requests. If the endpoint is reachable from an untrusted network, any client that can connect to it can create sessions, execute SQL, submit jobs, and access configured data sources. Bind the endpoint to a trusted interface, place it behind an authenticated gateway, or enable SPNEGO authentication before exposing it beyond a trusted boundary.
146
+
147
+
SPNEGO authentication is opt-in and applies to every SQL Gateway REST path, including `/v*/info`.
$ curl --negotiate -u : --request POST http://sql-gateway-host:8083/v1/sessions
163
+
```
164
+
165
+
Unauthenticated requests receive `401` with `WWW-Authenticate: Negotiate`. Malformed or invalid SPNEGO tokens receive `403`.
166
+
After a successful SPNEGO exchange, the REST endpoint issues a signed `hadoop.auth` cookie. Multi-instance or load-balanced SQL Gateway deployments must configure the same `sql-gateway.endpoint.rest.authentication.signature.secret` or `sql-gateway.endpoint.rest.authentication.signature.secret-file` on every instance; otherwise cookies issued by one instance will not be accepted by another.
167
+
168
+
The built-in Flink SQL Client and Flink JDBC Driver do not negotiate SPNEGO with the SQL Gateway REST endpoint in this change. Use an HTTP SPNEGO-capable client or put the SQL Gateway behind a separate authenticated gateway or proxy when those clients are required.
<td>Kerberos principal for HistoryServer SPNEGO authentication. Required when HistoryServer web authentication type is KERBEROS. The principal must start with HTTP/. The _HOST placeholder is replaced with the local hostname. Use * to accept all HTTP principals in the keytab.</td>
<td>Shared secret for signing HistoryServer SPNEGO authentication cookies. If neither this option nor the secret-file option is set, a random process-local secret is generated. Configure the same secret for all HistoryServer instances that should accept each other's authentication cookies.</td>
<td>File containing the shared secret for signing HistoryServer SPNEGO authentication cookies. Configure the same secret file for all HistoryServer instances that should accept each other's authentication cookies.</td>
<td>Authentication type for the HistoryServer web frontend and REST endpoints.<br/><br/>Possible values:<ul><li>"NONE"</li><li>"KERBEROS"</li></ul></td>
<td>Kerberos principal used by the SQL Gateway REST endpoint SPNEGO acceptor. The HTTP/_HOST@REALM pattern is supported, and '*' uses all HTTP principals from the keytab.</td>
40
+
</tr>
41
+
<tr>
42
+
<td><h5>authentication.signature.secret</h5></td>
43
+
<tdstyle="word-wrap: break-word;">(none)</td>
44
+
<td>String</td>
45
+
<td>Shared secret used to sign SQL Gateway REST authentication cookies. If neither this option nor authentication.signature.secret-file is configured, a random per-process secret is used.</td>
<td>File containing the shared secret used to sign SQL Gateway REST authentication cookies. If neither this option nor authentication.signature.secret is configured, a random per-process secret is used.</td>
52
+
</tr>
53
+
<tr>
54
+
<td><h5>authentication.token.validity</h5></td>
55
+
<tdstyle="word-wrap: break-word;">10 h</td>
56
+
<td>Duration</td>
57
+
<td>Validity period of the SQL Gateway REST authentication cookie.</td>
58
+
</tr>
59
+
<tr>
60
+
<td><h5>authentication.type</h5></td>
61
+
<tdstyle="word-wrap: break-word;">NONE</td>
62
+
<td><p>Enum</p></td>
63
+
<td>The authentication type for the SQL Gateway REST endpoint. Supported values are NONE and KERBEROS.<br/><br/>Possible values:<ul><li>"NONE"</li><li>"KERBEROS"</li></ul></td>
0 commit comments