API rate limit exceeded for installation

[gr2m]

I was hoping that rate limiting is now properly taken care of since we moved to Ocotkit via #487, but the error popped up again

https://github.com/semantic-release/commit-analyzer/actions/runs/5159013164/jobs/9293389281#step:6:126

We can tell from the user agent that the correct @semantic-release/github version was used

      'user-agent': '@semantic-release/github v8.1.0 octokit-core.js/4.2.1 Node.js/18.16.0 (linux; x64)',

What's odd: the request option does not have a retries key, it looks like it didn't attempt to retry at all:
https://github.com/semantic-release/commit-analyzer/actions/runs/5159013164/jobs/9293389281#step:6:138

I will look into it. Sorry, I'm not sure why this is happening and rate limit errors are hard to reproduce reliably, it will take me a moment to get to the bottom of this. If anyone is inclined to help out, you are very welcome to

[gr2m]

One thing I want to look into is to avoid using search in the first place, we can look up associated pull requests based on commits using GraphQL now, there is no need to use search for that.

[Kampfmoehre]

If you need any example failed pipeline you can look into this one.

[dhensby]

I got hit by this issue earlier today and have done some digging to see what is going on.

So far there I've found 1 reason that is potentially causing retries to fail: https://github.com/semantic-release/github/blob/v9.0.4/lib/octokit.js#L25-L48

In the setup of the Octokit class, the retry config is passed in via RETRY_CONF, where RETRY_CONF is:

{
  // By default, Octokit does not retry on 404s.
  // But we want to retry on 404s to account for replication lag.
  doNotRetry: [400, 401, 403, 422],
}

But in the onRetry function, RETRY_CONFIG.retries is used and is undefined, this means the onRetry function will never return true and so retries are not attempted (I assume).

I tried to write a test to cover off the rate limit scenario, but I also noticed that the TestOctokit doesn't use SemanticReleaseOctokit, so the test suite doesn't test the logic as it is when run outside the test suite (ie: no retry, no throttle checks, etc). I suspect this has been done to improve speed of the test suite as when I did use the SemanticReleaseOctokit the test suite did take much longer to complete.

[Kampfmoehre]

Here is another failed pipeline, I saw in the logs that the search request was already limited. We use dependabot to keep deps up to date, resulting in a lot of small merge requests. Combined with not so regularly "manual" updates this results in a lot of issues having to be updated after release. But before at least some issues got updated, now it seems it doesn't even update one issue because searching for affecting issues already fails.

https://github.com/droidsolutions/semantic-release-update-file/actions/runs/8229131488/job/22499768584

Since logs vanish after some time, here is the request from the log

   status: 403,
  response: {
    url: 'https://api.github.com/search/issues?q=repo%3Adroidsolutions%2Fsemantic-release-update-file+type%3Apr+is%3Amerged+203258a9db965ddd4c498d1256ba4609809d1c61+c0b8a0c51a5efab6453fdd7175ed3250301c8196+0e56bf1bb4622b6cb28b58afe8426d2c591eb194+bd8e51d5ebaae09faff3310464a865586daa70a8',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Mon, 11 Mar 2024 07:30:29 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-accepted-oauth-scopes': 'repo',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': 'E20E:33A877:280331C:48F2596:65EEB313',
      'x-oauth-scopes': 'repo, workflow',
      'x-ratelimit-limit': '30',
      'x-ratelimit-remaining': '0',
      'x-ratelimit-reset': '1710142229',
      'x-ratelimit-resource': 'search',
      'x-ratelimit-used': '30',
      'x-xss-protection': '0'
    },
    data: {
      message: 'API rate limit exceeded for user ID 14077931. If you reach out to GitHub Support for help, please include the request ID E20E:33A877:280331C:48F2596:65EEB313.',
      documentation_url: 'https://docs.github.com/rest/overview/rate-limits-for-the-rest-api'
    }
  },
  request: {
    method: 'GET',
    url: 'https://api.github.com/search/issues?q=repo%3Adroidsolutions%2Fsemantic-release-update-file+type%3Apr+is%3Amerged+203258a9db965ddd4c498d1256ba4609809d1c61+c0b8a0c51a5efab6453fdd7175ed3250301c8196+0e56bf1bb4622b6cb28b58afe8426d2c591eb194+bd8e51d5ebaae09faff3310464a865586daa70a8',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': '@semantic-release/github v9.0.2 octokit-core.js/4.2.1 Node.js/20.11.1 (linux; x64)',
      authorization: 'token [REDACTED]'
    },
    request: { agent: undefined, hook: [Function: bound bound register] }
  },
  pluginName: '@semantic-release/github'
}

[dhensby]

@Kampfmoehre your version of @semantic-release/github is v9.0.2, but fixes have come in since then (eg: my fix was released in v9.0.5.

Upgrade your version of the package and I suspect it won't happen anymore (I've not experienced any issues since v9.0.5)

[Kampfmoehre]

@dhensby thx, I'll give it a try. We don't have the plugin as extra dep though, was previously using the one that comes with semantic-release package so you might want to update the min version there.

[gr2m]

using the one that comes with semantic-release package

The semantic-release package defines a range, you should get all updates automatically. If you have it as a dev dependency, make sure to update your lock file. We recommend to run it with just npx semantic-release without adding it as dependency