Sliding Token Clarification

[IvanFon]

Hi, I'm trying to use sliding tokens, but I think I need a bit of clarification. From what I understand, when I generate a sliding token, it can be used for authentication until it's expiration claim expires. After the expiration claim expires, I can still refresh the token until the refresh expiration claim expires.

The problem I'm running into is that I can only refresh the token while the auth expiration claim is valid. I'm not sure if this is intended or if I'm doing something wrong, but this makes it seem like there's no point to the refresh expiration claim.

My config is:

SIMPLE_JWT = {
    'AUTH_TOKEN_CLASSES': ('rest_framework_simplejwt.tokens.SlidingToken',),
    'SLIDING_TOKEN_LIFETIME': timedelta(minutes=1),
    'SLIDING_TOKEN_REFRESH_LIFETIME': timedelta(days=1),
}

I can obtain a token:

Request:
> POST /api/users/token/ HTTP/1.1
> Host: localhost:8000
> User-Agent: insomnia/6.6.2
> Content-Type: application/json
> Accept: */*
> Content-Length: 44

| {
| 	"username": "test",
| 	"password": "test"
| }

< HTTP/1.1 200 OK
< Date: Thu, 29 Aug 2019 16:56:17 GMT
< Server: WSGIServer/0.2 CPython/3.7.4
< Content-Type: application/json
< Vary: Accept
< Allow: POST, OPTIONS
< X-Frame-Options: SAMEORIGIN
< Content-Length: 252

| {
|  "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5NzgzNywianRpIjoiOWZkZDFkYTlkODI2NDAwNzkwNjRiYjM4Nzc5Y2FkMmQiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDE3NywidXNlcl9pZCI6Mn0.1eb5oFz6wwyjBI1wQrl5tPkkUogXWmWBBTWJn_xJC2k"
| }

I can use that token for authentication:

> GET /api/post/1 HTTP/1.1
> Host: localhost:8000
> User-Agent: insomnia/6.6.2
> Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5NzgzNywianRpIjoiOWZkZDFkYTlkODI2NDAwNzkwNjRiYjM4Nzc5Y2FkMmQiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDE3NywidXNlcl9pZCI6Mn0.1eb5oFz6wwyjBI1wQrl5tPkkUogXWmWBBTWJn_xJC2k
> Accept: */*

< HTTP/1.1 200 OK
< Date: Thu, 29 Aug 2019 16:57:06 GMT
< Server: WSGIServer/0.2 CPython/3.7.4
< Content-Type: application/json
< Vary: Accept
< Allow: GET, POST, HEAD, OPTIONS
< X-Frame-Options: SAMEORIGIN
< Content-Length: 140

| [correct response data...]

After SLIDING_TOKEN_LIFETIME (1 minute), trying to make authenticated requests gives a 401, which I would expect:

> GET /api/post/1 HTTP/1.1
> Host: localhost:8000
> User-Agent: insomnia/6.6.2
> Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5NzgzNywianRpIjoiOWZkZDFkYTlkODI2NDAwNzkwNjRiYjM4Nzc5Y2FkMmQiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDE3NywidXNlcl9pZCI6Mn0.1eb5oFz6wwyjBI1wQrl5tPkkUogXWmWBBTWJn_xJC2k
> Accept: */*

< HTTP/1.1 401 Unauthorized
< Date: Thu, 29 Aug 2019 17:00:27 GMT
< Server: WSGIServer/0.2 CPython/3.7.4
< Content-Type: application/json
< WWW-Authenticate: Bearer realm="api"
< Vary: Accept
< Allow: GET, POST, HEAD, OPTIONS
< X-Frame-Options: SAMEORIGIN
< Content-Length: 185

| {
|   "detail": "Given token not valid for any token type",
|   "code": "token_not_valid",
|   "messages": [
|     {
|       "token_class": "SlidingToken",
|       "token_type": "sliding",
|       "message": "Token is invalid or expired"
|     }
|   ]
| }

But when I try to refresh the token within SLIDING_TOKEN_REFRESH_LIFETIME:

> POST /api/users/token/refresh/ HTTP/1.1
> Host: localhost:8000
> User-Agent: insomnia/6.6.2
> Content-Type: application/json
> Accept: */*
> Content-Length: 256

| {
| 	"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5NzgzNywianRpIjoiOWZkZDFkYTlkODI2NDAwNzkwNjRiYjM4Nzc5Y2FkMmQiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDE3NywidXNlcl9pZCI6Mn0.1eb5oFz6wwyjBI1wQrl5tPkkUogXWmWBBTWJn_xJC2k"
| }

< HTTP/1.1 401 Unauthorized
< Date: Thu, 29 Aug 2019 17:02:53 GMT
< Server: WSGIServer/0.2 CPython/3.7.4
< Content-Type: application/json
< WWW-Authenticate: Bearer realm="api"
< Vary: Accept
< Allow: POST, OPTIONS
< X-Frame-Options: SAMEORIGIN
< Content-Length: 65

| {
|   "detail": "Token is invalid or expired",
|   "code": "token_not_valid"
| }

But if I try refreshing a token within SLIDING_TOKEN_LIFETIME, it works fine:

> POST /api/users/token/refresh/ HTTP/1.1
> Host: localhost:8000
> User-Agent: insomnia/6.6.2
> Content-Type: application/json
> Accept: */*
> Content-Length: 256

| {
| 	"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5ODM1NSwianRpIjoiNTg5ODhiMGFkMWE1NGU3MWFkOGU2NTMxZGI4ZTNhMTAiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDY5NSwidXNlcl9pZCI6Mn0.BW-qxTVupUlBogT4O-s_ySKjEdfPkl_zX7vw-d903iA"
| }

< HTTP/1.1 200 OK
< Date: Thu, 29 Aug 2019 17:05:06 GMT
< Server: WSGIServer/0.2 CPython/3.7.4
< Content-Type: application/json
< Vary: Accept
< Allow: POST, OPTIONS
< X-Frame-Options: SAMEORIGIN
< Content-Length: 252

| {
|   "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ0b2tlbl90eXBlIjoic2xpZGluZyIsImV4cCI6MTU2NzA5ODM2NiwianRpIjoiNTg5ODhiMGFkMWE1NGU3MWFkOGU2NTMxZGI4ZTNhMTAiLCJyZWZyZXNoX2V4cCI6MTU2NzE4NDY5NSwidXNlcl9pZCI6Mn0.nnBpI7FDiFZOa-f4fGQC5omtff2g6WFg5NKVlE6k0CM"
| }

So am I doing something wrong? What is SLIDING_TOKEN_REFRESH_LIFETIME for? The way I expected to use a sliding token would be to retrieve a token on login, and use it for requests. Once a request returns a 401 telling me the token expired, I refresh it to obtain a new token, and repeat the process.

Thanks! Aside from refreshing sliding tokens, this library has been super easy to use :)

[idoash4]

@IvanFon were you able to solve this?

[IvanFon]

No, unfortunately. I ended up using separate auth and refresh tokens, which works fine. It’s not a huge hassle to use two separate tokens, but in my case it would be slightly more convenient to use a single sliding token. Having separate tokens wouldn’t improve my security, both tokens are stored in memory/localStorage. If an attacker gets access to one, they have access to both.

…

On October 17, 2019 6:06:04 a.m. EDT, BlackXnt ***@***.***> wrote: @IvanFon were you able to solve this? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #154 (comment)

[decepulis]

I think I'm having the same problem...
If I understand the problem correctly, in the time after exp and before refresh_exp, we should be able to refresh the token with post('/api/tokens/refresh/', {'token': token}). But instead, we're getting 401 errors.

So I dug into the source a little to see if I could see what's going on.

TL;DR

A low-level verification function is checking if exp has passed (which it has) instead of refresh_exp (which hasn't). Because exp has passed, the functions are throwing the token out.

What Goes Wrong

Here's a nice, long stack-trace getting to the bottom of what's happening...
If you're interested like I was, grab a coffee and read on.

We'll start at the TokenRefreshSlidingSerializer, called with the refresh api, which looks like this:

class TokenRefreshSlidingSerializer(serializers.Serializer):
    token = serializers.CharField()

    def validate(self, attrs):
        token = SlidingToken(attrs['token'])

        # Check that the timestamp in the "refresh_exp" claim has not
        # passed
        token.check_exp(api_settings.SLIDING_TOKEN_REFRESH_EXP_CLAIM)

        # Update the "exp" claim
        token.set_exp()

        return {'token': str(token)}

The token gets rejected at the token = SlidingToken(attrs['token']) line, so I checked the SlidingToken.__init__ function, next:

class SlidingToken(BlacklistMixin, Token):
    token_type = 'sliding'
    lifetime = api_settings.SLIDING_TOKEN_LIFETIME

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        if self.token is None:
            # Set sliding refresh expiration claim if new token
            self.set_exp(
                api_settings.SLIDING_TOKEN_REFRESH_EXP_CLAIM,
                from_time=self.current_time,
                lifetime=api_settings.SLIDING_TOKEN_REFRESH_LIFETIME,
            )

Killed at the super(), so next stop, Token.__init__.

def __init__(self, token=None, verify=True):
        """
        !!!! IMPORTANT !!!! MUST raise a TokenError with a user-facing error
        message if the given token is invalid, expired, or otherwise not safe
        to use.
        """
        if self.token_type is None or self.lifetime is None:
            raise TokenError(_('Cannot create token with no type or lifetime'))

        self.token = token
        self.current_time = aware_utcnow()

        # Set up token
        if token is not None:
            # An encoded token was provided
            from .state import token_backend

            # Decode token
            try:
                self.payload = token_backend.decode(token, verify=verify)
            except TokenBackendError:
                raise TokenError(_('Token is invalid or expired'))

            if verify:
                self.verify()
        else:
            # New token.  Skip all the verification steps.
            self.payload = {api_settings.TOKEN_TYPE_CLAIM: self.token_type}

            # Set "exp" claim with default value
            self.set_exp(from_time=self.current_time, lifetime=self.lifetime)

            # Set "jti" claim
            self.set_jti()

self.payload = token_backend.decode(token, verify=verify) is the line where this dies. And what happens in there?

def decode(self, token, verify=True):
        """
        Performs a validation of the given token and returns its payload
        dictionary.

        Raises a `TokenBackendError` if the token is malformed, if its
        signature check fails, or if its 'exp' claim indicates it has expired.
        """
        try:
            return jwt.decode(token, self.verifying_key, algorithms=[self.algorithm], verify=verify)
        except InvalidTokenError:
            raise TokenBackendError(_('Token is invalid or expired'))

Yup that's right, it's only checking the exp key, not the refresh_exp key.

Just to be thorough, let's see what's going on in the jwt library.
Because verify=True, jwt.decode goes ahead and calls an internal function called self._validate_claims, which finally calls self._validate_exp, which, of course, gives us a nice ExpiredSignatureError.

A Brute-Force Solution

To fix this, one could change the SlidingToken initialization in the TokenRefreshSlidingSerializer from

token = SlidingToken(attrs['token'])

to

token = SlidingToken(attrs['token'], verify=False)

which I successfully tested, but I don't know the library well enough to understand the ramifications of that kind of action.

I hope this helps someone, somewhere?

[davesque]

Yeah, I included the whole sliding token thing to provide some kind of backwards compatibility with users of the legacy django-rest-framework-jwt library. But I honestly just don't like the idea and I kinda want to get rid of it. I need to look into how easy it would be to do this.

[EricLewe]

An alternative, as suggested @IvanFon is to use pair token. Dont forget to set 'ROTATE_REFRESH_TOKENS' to True, if you decide to go for the pair token approach. I forgot this flag existed which made me believe sliding tokens was the only alternative.

[mcsimps2]

Just wanted to add that I've also stumbled upon this issue. Thanks @decepulis for digging into the issue and finding a workaround in the meantime.

[Brecht-Pallemans]

I bumped in to this issue aswell since I wanted a fault proof system where only one valid token was present for a user. When using a pair, the access token is still valid for a certain time even though the refresh has been blacklisted. One additional step to add is that when overriding the serializer, you should also add token.check_blacklist() and token.blacklist() to make old tokens invalid and token.set_jti() for creating a new token.

[Kaspary]

I think I'm having the same problem...
If I understand the problem correctly, in the time after exp and before refresh_exp, we should be able to refresh the token with post('/api/tokens/refresh/', {'token': token}). But instead, we're getting 401 errors.

So I dug into the source a little to see if I could see what's going on.

TL;DR

A low-level verification function is checking if exp has passed (which it has) instead of refresh_exp (which hasn't). Because exp has passed, the functions are throwing the token out.

What Goes Wrong

Here's a nice, long stack-trace getting to the bottom of what's happening...
If you're interested like I was, grab a coffee and read on.

We'll start at the TokenRefreshSlidingSerializer, called with the refresh api, which looks like this:

class TokenRefreshSlidingSerializer(serializers.Serializer):
    token = serializers.CharField()

    def validate(self, attrs):
        token = SlidingToken(attrs['token'])

        # Check that the timestamp in the "refresh_exp" claim has not
        # passed
        token.check_exp(api_settings.SLIDING_TOKEN_REFRESH_EXP_CLAIM)

        # Update the "exp" claim
        token.set_exp()

        return {'token': str(token)}

The token gets rejected at the token = SlidingToken(attrs['token']) line, so I checked the SlidingToken.__init__ function, next:

class SlidingToken(BlacklistMixin, Token):
    token_type = 'sliding'
    lifetime = api_settings.SLIDING_TOKEN_LIFETIME

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        if self.token is None:
            # Set sliding refresh expiration claim if new token
            self.set_exp(
                api_settings.SLIDING_TOKEN_REFRESH_EXP_CLAIM,
                from_time=self.current_time,
                lifetime=api_settings.SLIDING_TOKEN_REFRESH_LIFETIME,
            )

Killed at the super(), so next stop, Token.__init__.

def __init__(self, token=None, verify=True):
        """
        !!!! IMPORTANT !!!! MUST raise a TokenError with a user-facing error
        message if the given token is invalid, expired, or otherwise not safe
        to use.
        """
        if self.token_type is None or self.lifetime is None:
            raise TokenError(_('Cannot create token with no type or lifetime'))

        self.token = token
        self.current_time = aware_utcnow()

        # Set up token
        if token is not None:
            # An encoded token was provided
            from .state import token_backend

            # Decode token
            try:
                self.payload = token_backend.decode(token, verify=verify)
            except TokenBackendError:
                raise TokenError(_('Token is invalid or expired'))

            if verify:
                self.verify()
        else:
            # New token.  Skip all the verification steps.
            self.payload = {api_settings.TOKEN_TYPE_CLAIM: self.token_type}

            # Set "exp" claim with default value
            self.set_exp(from_time=self.current_time, lifetime=self.lifetime)

            # Set "jti" claim
            self.set_jti()

self.payload = token_backend.decode(token, verify=verify) is the line where this dies. And what happens in there?

def decode(self, token, verify=True):
        """
        Performs a validation of the given token and returns its payload
        dictionary.

        Raises a `TokenBackendError` if the token is malformed, if its
        signature check fails, or if its 'exp' claim indicates it has expired.
        """
        try:
            return jwt.decode(token, self.verifying_key, algorithms=[self.algorithm], verify=verify)
        except InvalidTokenError:
            raise TokenBackendError(_('Token is invalid or expired'))

Yup that's right, it's only checking the exp key, not the refresh_exp key.

Just to be thorough, let's see what's going on in the jwt library.
Because verify=True, jwt.decode goes ahead and calls an internal function called self._validate_claims, which finally calls self._validate_exp, which, of course, gives us a nice ExpiredSignatureError.

A Brute-Force Solution

To fix this, one could change the SlidingToken initialization in the TokenRefreshSlidingSerializer from

token = SlidingToken(attrs['token'])

to

token = SlidingToken(attrs['token'], verify=False)

which I successfully tested, but I don't know the library well enough to understand the ramifications of that kind of action.

I hope this helps someone, somewhere?

I hope this helps someone, somewhere?

Have a security error in this solution. With verify=False on SlidingToken, the signature validate is don't verified, an is possible change the payload. Is possible make a test of this on debbuger of https://jwt.io/, changing the payload, with wrong key, and submit to refresh.

[Andrew-Chen-Wang]

@Kaspary You're probably using PyJWT 2.0.0. Ref #349 Once it's merged, it should solve your problem. Until then, downgrade to PyJWT 1.7.1. Sorry for the inconvenience!

[deepanshu-nickelfox]

one doubt, how do i create a sliding Token??

[Andrew-Chen-Wang]

Please don't use sliding tokens. They're confusing. (They came from migrating from drf-jwt). We may be deprecating is soon as migration from drf-jwt is much lower than before.

[deepanshu-nickelfox]

ok thanks @Andrew-Chen-Wang

[DBlek]

Sorry to resurect this thread @Andrew-Chen-Wang but I have stumbled on this myself and it bugs us.

Our use case is that we want to make sure that only one device is used at a time, and We could not achieve that with Access/refresh tokens + blacklist as only refresh tokens get blasklisted, so even if I blacklist all user tokens before login , I still have this access token that has 1 minute lifetime and a user can use the app on two devices, which we don't want.

We can achieve that with sliding tokens but then after the expire time refresh doesnt work so user get's returned to login, which is not the desired functionality...

Is it possible to do my use case with access/refresh tokens ? If not, is it a lot of effort to fix the issue that @decepulis described with changing the exp to refresh_exp ?

[Andrew-Chen-Wang]

Sorry to resurect this thread @Andrew-Chen-Wang but I have stumbled on this myself and it bugs us.

No problem. I am not going to be deprecating sliding tokens as I understand some people do like them, especially if they're coming from drf-jwt. Also, school has bogged me down, so I've been maintaining repos via PRs lately anyways.

Our use case is that we want to make sure that only one device is used at a time, and We could not achieve that with Access/refresh tokens + blacklist as only refresh tokens get blasklisted, so even if I blacklist all user tokens before login , I still have this access token that has 1 minute lifetime and a user can use the app on two devices, which we don't want.

We can achieve that with sliding tokens but then after the expire time refresh doesnt work so user get's returned to login, which is not the desired functionality...

Is it possible to do my use case with access/refresh tokens ? If not, is it a lot of effort to fix the issue that @decepulis described with changing the exp to refresh_exp ?

Sure. Like how we throttle users within the same IP address but we don't know the private IPs, we store several features of a device to make as unique of an identifier as possible. You can do two options (assuming not a malicious user obviously and this is just a regular ol joe):

1. Create a UUID on the device and store it. Send it to your server so you can get the access refresh tokens. Save the UUID in the token payload. Every request you send from the client now must include both the access token and the device's uuid in two separate headers for the server to compare. If the comparison fails between the two, then you should return a 401. Once a refresh token expires, it is up for grabs by any device. You may feel the desire to set a longer refresh token time. I think Microsoft sets it at 90 days. That security decision is up to you. Finally, determining whether to provide additional refresh tokens (in case of multiple devices where one has already gotten a refresh token) can be solved by db functionality. The first device authenticates via username/password, so now we know the user ID of the user. We store the uuid in a table with a OneToOneField primary key to the user ID with the exp timestamp. What I do in my native apps (assuming you're talking about iOS/Android) is I save the user credentials in Keychain. Every app launch, I grab a new refresh token. This updates the table's exp like the sliding token. Through this table, this way, we know if the token has expired, if another device with the same user ID sends a different UUID, we can reject, and if a device needs a refresh token and the previous device has not refreshed, we can override and give the new device a token pair because of the exp check. Using a sliding token works; just make sure to update exp in the table. Hitting the db when providing refresh tokens shouldn't be an issue since you don't do it often. Just make sure to throttle users.

2. Websockets. You can manage users in a live basis. I don't think this needs much explanation.
3. You can use device information and store in payload. Do similar comparisons serverside as to #\1.

Hope that helps!

[DBlek]

Thanks for your reply @Andrew-Chen-Wang but unfortunately we cannot store any information about the users.

If you don't have time maybe I can help with the issue? If i understand correctly, the decode function for sliding tokens is the same as for regular tokens but it should be a little different (incorporate refresh_exp)?

[Andrew-Chen-Wang]

If you're trying to use access/refresh, people above have resolved this using a setting for returning new token pairs.

I bumped in to this issue aswell since I wanted a fault proof system where only one valid token was present for a user. When using a pair, the access token is still valid for a certain time even though the refresh has been blacklisted. One additional step to add is that when overriding the serializer, you should also add token.check_blacklist() and token.blacklist() to make old tokens invalid and token.set_jti() for creating a new token.

I wouldn't mind a PR for the fix for sliding tokens. Just won't be online for the next few weeks.

[DBlek]

@Andrew-Chen-Wang Unfortunately this still leaves the time window when the access token can be used, as blacklist only affects refresh tokens (just tested it).

When using a pair, the access token is still valid for a certain time even though the refresh has been blacklisted.

I'll keep you posted whether I have time to help with this, as I'm working on other stuff right now.

[famdude]

@DBlek Have you solved the problem? I've got stuck with the same problem.

[DBlek]

@DBlek Have you solved the problem? I've got stuck with the same problem.

We have gone with a custom feature that blacklists all user tokens before giving a new one

[famdude]

@DBlek Have you solved the problem? I've got stuck with the same problem.

We have gone with a custom feature that blacklists all user tokens before giving a new one

You mean you blacklist all previous tokens of that specific user, or all tokens of all users?
If it's the first case, how do you get previous tokens of a user?

[DBlek]

only for the user

[famdude]

only for the user

So assume user has logged in before, and tries to login again with another device, while previous token is valid.
You say you blacklist first token and generate new token for the user.
Now the question is, how do you get first token of the user, by username or etc...? Because tokens aren't stored in database.

[DBlek]

We used OutstandingToken, filter out the ones with the user ID and add them to BlacklistedToken. Both can be stored in the DB. Periodically we delete all blacklisted and outstanding tokens

So to be crystal clear:
We made a custom TokenSerializer from TokenObtainSlidingSerializer, before we call the original get_token we use the custom function that blackslists all tokens for that particular user and then call the original get_token

[famdude]

@DBlek Brilliant! I could solve it, with your amazing help. But there is still this problem of not being able to limit users to use one token on multiple devices. If you find any solution, please let us know.

And BTW, what about this misconfiguration of sliding tokens refresh lifetime? Is there any stable solution to fix it?
@Andrew-Chen-Wang

[DBlek]

@famdude great u got it working - just saw ur msg :) All should be in docs for my solution so nothing new ;)