🚧 Generate and validate JWT tokens from the Execution API #46981
Conversation
…I to use Since requests to Task Execution API can originate out-of-"cluser" so to speak, this PR re-works the JWTSigner class so that it is possible to use public/private keys (vs just a simple pre-shared secret). This is useful in many cloud environments where many companies have security requirements that ingress gateways must validate the JWT tokens on the way in, and the only way of doing this is with public keys So that we don't have two different ways of generating JWT tokens I have totally replaced the old "JWTSigner" class (which it turns out didn't have any unit test of its own, it was only tested indirectly through test_serve_logs etc). As part of this change I have also changed the JWT that was generated by the SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT) to use the offical `sub` (subject) clain to place the user identifer rather than a custom claim name. And although it might seem slightly strange at first, I have made the JWTValidator an async class internally. (Hence `avalidated_claims` -- the `a` prefix signifies async, much like `aclose` or `aread` on HTTPX async responses). This allows us to periodically refresh the JWK document if configured in the background, and using asgiref's async_to_sync means we only have one version. And also conviently, this is the same tech that FastAPI uses, which means when this is called from within a FastAPI app, rather than creating a new/nested event loop it will "bubble out" to the main async loop that FastAPI is using.
ashb
requested review from
jscheffl,
eladkal,
o-nikolas,
ephraimbuddy and
pierrejeambrun
as code owners
boring-cyborg
bot
added
area:API
|
(I've only done the first part of this PR, I haven't actually used it in the Task SDK yet, I wanted to test this full change) |
|
|
||
| if TYPE_CHECKING: | ||
| from airflow.auth.managers.base_auth_manager import ResourceMethod | ||
|
|
||
| oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token") | ||
|
|
||
|
|
||
| @cache | ||
| def get_signer() -> JWTSigner: |
There was a problem hiding this comment.
Nothing (I think) ever called this.
| Make sure that time on ALL the machines that you run airflow components on is synchronized | ||
| (for example using ntpd) otherwise you might get "forbidden" errors. | ||
| version_added: 3.0.0 | ||
| type: integer | ||
| example: ~ | ||
| default: "3600" | ||
| api_auth: |
There was a problem hiding this comment.
I decided to move the auth-related config to its own section, as there was starting to be a longer list of them, and since we don't have nested config sections this seemed like the next best option.
If anyone feels strongly the otherway I can move these back under api section
There was a problem hiding this comment.
I don't have a strong opinion on moving it under api_auth. It looks good. The separate heading would be more memorizable. Thanks!
Shouldn't we move auth_jwt_expiration_time and auth_jwt_cli_expiration_time since they relate to api_auth? Maybe not in this PR, though, since they are not directly related to this piece of work. I am asking out of curiosity. I can follow up with that piece of work too.
Since requests to Task Execution API can originate out-of-"cluser" so to
speak, this PR re-works the JWTSigner class so that it is possible to use
public/private keys (vs just a simple pre-shared secret). This is useful in
many cloud environments where many companies have security requirements that
ingress gateways must validate the JWT tokens on the way in, and the only way
of doing this is with public keys
So that we don't have two different ways of generating JWT tokens I have
totally replaced the old "JWTSigner" class (which it turns out didn't have any
unit test of its own, it was only tested indirectly through test_serve_logs
etc).
As part of this change I have also changed the JWT that was generated by the
SimpleAuthManager and the AwsAuthManager (the only two we have that use JWT)
to use the offical
sub(subject) clain to place the user identifer ratherthan a custom claim name.
And although it might seem slightly strange at first, I have made the
JWTValidator an async class internally. (Hence
avalidated_claims-- theaprefix signifies async, much like
acloseorareadon HTTPX asyncresponses). This allows us to periodically refresh the JWK document if
configured in the background, and using asgiref's async_to_sync means we only
have one version.
And also conviently, this is the same tech that FastAPI uses, which means when
this is called from within a FastAPI app, rather than creating a new/nested
event loop it will "bubble out" to the main async loop that FastAPI is using.
Closes #45107
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.