chore: update lerna to v6.0.3

[AgentEnder]

Pre-flight checklist

• I have read the Contributing Guidelines on pull requests.
• not a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
• not a new API or substantial change: the PR has an accompanying issue (closes #0000) and the maintainers have approved on my working plan.

Motivation

@juristr talked over upgrading lerna to v6 with @slorber and @yangshun. v6 includes local caching and faster task execution via Nx by default 🎉 .

Configuration Explanation

This PR adds some configuration in nx.json, as well as website/package.json. I've copied the relevant sections here, with some explanation.

nx.json

{
  "tasksRunnerOptions": {
    "default": {
    /**
      * This tells Nx to use its default task runner. In the future, to add remote caching, this would change.
      * We can enable cloud at any time with `npx nx connect-to-nx-cloud`. The current setup should still
      * work with the local cache, and is a good start.
      */
      "runner": "nx/tasks-runners/default",
      "options": {
        /**
          * This tells Nx which scripts should be cached. The array of cacheable scripts 
          * is currently maintained only in Nx.json, so this should be updated if more tasks
          * (e.g. test, lint etc) were on the project level rather than at the workspace root. 
          */
        "cacheableOperations": [
          "build", // Every project + website has a build task
          "build:fast", // website has this task
          "build:baseUrl", // website has this task
          "build:blogOnly" // website has this task
        ]
      }
    }
  },
  // Think of this as a dictionary of values to use later
  "namedInputs": {
    // The projects primarily rely on any file under their root (i.e. the location of the package.json file).
    // However, they build to the lib folder. We can exclude it here, s.t. it doesn't get calculated into the hash.
    "default": ["{projectRoot}/**/*", "!{projectRoot}/lib/**/*", "!{projectRoot}/.docusaurus/**/*"]
  },
  // This gives us a spot to define some options for **_every_** instance of a script.
  "targetDefaults": {
   // We define the common elements of our build scripts here.
    "build": {
      // Build should take into account the default fileset of the project that is executing it (`default`),
      // as well as, the default fileset for each project that it depends on.
      "inputs": ["default", "^default"],
      // Our builds output to the lib directory, so we tell Nx about it so it can be cached.
      "outputs": ["{projectRoot}/lib"],
      // Run build script from deps before this target. 
      "dependsOn": ["^build"]
    }
  },
  "defaultBase": "main"
}

Test Plan

Played around a bit with this, running targets, deleting outputs, changing code and making sure it still worked etc. Here's some screenshots:

Test links

• https://blog.nrwl.io/lerna-reborn-whats-new-in-v6-10aec6e9091c

Deploy preview: https://deploy-preview-8318--docusaurus-2.netlify.app/ -- note: no code changes, so this is not changed.

Related issues/PRs

#7333 -- Lerna 6 enables Nx for caching by default, so I think this would close out.

[netlify]

✅ [V2]

Name	Link
🔨 Latest commit	3d4a7db
🔍 Latest deploy log	https://app.netlify.com/sites/docusaurus-2/deploys/637f50530d13ba0009dfb120
😎 Deploy Preview	https://deploy-preview-8318--docusaurus-2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[github-actions]

⚡️ Lighthouse report for the deploy preview of this PR

URL	Performance	Accessibility	Best Practices	SEO	PWA	Report
/	🟢 94	🟢 97	🟢 100	🟢 100	🟢 90	Report
/docs/installation	🟠 72	🟢 100	🟢 100	🟢 100	🟢 90	Report

[AgentEnder]

@slorber this PR should be in a good state now, apologies I meant to mark it as draft before commit 92c9e33. Let me know if there's anything else I can do on my side of things.

[AgentEnder]

@slorber Those e2e test failures don't appear to be related to these changes. Are they something that you have seen before?

[slorber]

Thanks for working on this.

Honestly, despite your comments, I'm still not 100% sure to understand everything of this config that doesn't really feel intuitive to me, and I don't really spend 1 day learning all this in-depth right now 😅

• Do you plan to support us in the long run, ie help the Docusaurus team troubleshoot problems?
• In case of problem, is it easy to revert to previous system? Is deleting nx.json enough for that?

It looks like it's indeed improving rebuild times.

Now what I also need is the dev experience. With yarn watch I can't really use TS incremental builds anymore, as logs do not print TS errors anymore.

If we adopt this, I need to keep a decent DX to work myself on Docusaurus, so that any change in any subpackage can be used as soon as possible on my local Docusaurus site (with hot reload for theme changes).

[AgentEnder]

Hey @slorber! Thanks for the questions. Here are some answers 😄

• You can definitely reach out if you have any issues on the Nx front pop up. I think @juristr has been in contact with folks already, feel free to reach out directly to him, or myself, or any other team member that you have contact info for. Additionally, there is a community slack that you can reach out for help on. Several of us keep an eye there, so you should receive an answer in a timely manner. There is a private channel on the slack for our community partners, that we would be able to invite you to after joining.
• Yeah, you can revert to previous behavior at any time. IIRC the only step you'd need to take is setting useNx to false in lerna.json. It would then make sense to also delete the nx.json file

Regarding the changed output from yarn watch, you may need to set the output style for Nx to stream. You can set this via an environment variable (NX_STREAM_OUTPUT=true). If invoking Nx directly, rather than through lerna, you can also do it by passing --output-style=stream to the command. I'm checking with a team member if there is an equivalent CLI flag for lerna.

[AgentEnder]

Following up on that last bit, you should be able to add --stream to the lerna commands to keep that output style.

[AgentEnder]

One other thing that you may run into with watch, is that by default when using Nx the maximum number of tasks ran in parallel is a bit lower. This is normally ideal for build and similar tasks, but if you are running watch scripts its not ideal as it would limit the number of tasks ran at the same time. You can add --concurrency 999, or any other number that's big enough, to override this limit where it makes sense.

[slorber]

Thanks

I tried yarn watch --stream --concurrency 999 and indeed it prints the logs similar to what it was before.

There's also warning also displaying temporarily that we didn't have before, and I don't know what's the impact of it:

(node:49771) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit

Will have to invest more time to review if things work properly.

---

Note: I'm not totally satisfied with our previous setup which consists of running each project independently in tsc watch mode. If you have better to suggest I'm open to change our maintainer workflow. I guess you have more experience on a TS repo build than me 😅

Using lerna + tsc --watch was just a quickfix I added 2 years ago, because at that time we only had full uncached (but incremental) tsc builds: not a great DX.

---

We have many e2e tests failing unfortunately.

That seems related to our tests that publish our packages to a local Verdaccio npm repository, using lerna publish

See e2e Github Actions, they run yarn test:build:website, which runs /admin/scripts/test-release.sh

I don't know how Lerna publish works with Nx and what makes it fail compared to before 😅

Note I'm also using Lerna to release Docusaurus so this part should rather work quite as before otherwise it might block me in an upcoming release

See our release process in admin/publish.md

I'm concerned by:

• yarn changelog --from v2.0.0-beta.0 (is lerna-changelog impacted? probably not)
• yarn lerna publish --exact 2.0.0-beta.0 (this is how we publish new versions so it should rather work as before)

We also have a GitHub action that publishes canary releases for each commit on main. It is a quite hacky setup to make Lerna work the way I wanted.

    "canary": "yarn canary:bumpVersion && yarn canary:publish",
    "canary:version": "echo 0.0.0-`git rev-list --count HEAD`+`git rev-parse --short HEAD`",
    "canary:bumpVersion": "yarn lerna version `yarn --silent canary:version` --exact --no-push --yes",
    "canary:publish": "yarn lerna publish from-package --dist-tag canary --yes --no-verify-access",

We should make sure it also keeps working.

---

I don't know if the failures are related to using Lerna v6, or the activation of Nx. Maybe it could be safer to upgrade Lerna without using Nx first?

I'd prefer to not troubleshoot 2 things at once 🤪

[AgentEnder]

Hey @slorber! Sorry for the longer lapse of communication there. Holidays + Illness had me knocked out for a bit. I hadn't noticed that the failures were related to publishing, that part should not have been affected, but I'll spend some time seeing whats going on if for some reason that was the case.

I can definitely split this up into either 2 or 3 PR's if you'd like. The 3 would look like:

• Bump to Lerna 6, Nx disabled
• Enable Nx
• Setup caching

If 2, the last 2 would be combined.

[slorber]

np, I'm also busy on my side ;)

The smaller the PRs, the more likely they are to pass CI and be merged asap 😄

[AgentEnder]

I'm going to close this one out, split into smaller PRs.