What are the x-ratelimit-remaining-tokens and the x-ratelimit-remaining-requests headers mean?

[honeming]

Select Topic Area

Question

Body

I just sent my first request to the gpt-4o-2024-08-06 model. The rate limits from the document are:

Rate limits	Value
Requests per minute	10
Requests per day	50
Tokens per request	8000 in, 4000 out

The request succeeded and I got following headers:

key	value
x-ratelimit-remaining-tokens	19999342
x-ratelimit-remaining-requests	199998

I think the headers have something to do with the rate limits but I don’t know what they exactly mean.

[rajbos]

The rate limits refer to the overview in the documentation:

You have 12k tokens (in+out) per request and you can make up to 150 requests a day (with the Free / Copilot Individual level). That amounts to 1.8 million tokens a day, which is a lot lower then you have in that number, so I am not sure how that is calculated.

Perhaps they have temporarily higher limits during the beta? I have the lower numbers by the way. I've used a PAT here:

key	value
x-ratelimit-remaining-tokens	1997968
x-ratelimit-remaining-requests	19998

I noticed this info in the response as well:

"usage": {
      "completion_tokens": 7,
      "prompt_tokens": 24,
      "total_tokens": 31
    }

[rajbos]

Same number size when using the GITHUB_TOKEN in the Codespace