Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Automatic dedupe on install #4976

Open
1 of 2 tasks
kachkaev opened this issue Oct 19, 2022 · 6 comments
Open
1 of 2 tasks

[Feature] Automatic dedupe on install #4976

kachkaev opened this issue Oct 19, 2022 · 6 comments
Labels
enhancement New feature or request

Comments

@kachkaev
Copy link

kachkaev commented Oct 19, 2022

  • I'd be willing to implement this feature (contributing guide)
  • This feature is important to have in this repository; a contrib plugin wouldn't do

Describe the user story

As a project maintainer, I want all repo dependencies to be deduplicated at all times without having to call yarn dedupe.

Describe the solution you'd like

## .yarnrc.yml
dedupeStrategy: "latest"

(It can also be "fewer" instead of "latest" when #2297 is implemented)

If dedupeStrategy is defined, yarn dedupe is automatically called as part of yarn install, yarn add etc. Running yarn dedupe becomes a noop.

Describe the drawbacks of your solution

  • + 1 option in docs and config schema
  • potentially harmful package deduplication which leads to bugs (I haven’t encountered this in years while using Yarn 1 and yarn-deduplicate)

Describe alternatives you've considered

Alternative 1

Repo example: kachkaev/njt 🐸

I have these two scripts in package.json:

"fix:yarn-dedupe": "yarn dedupe",
"lint:yarn-dedupe": "yarn dedupe --check",

The latter one is called in .github/workflows/ci.yml, thus making sure that yarn.lock is deduplicated at all times.

If someone forgets to run yarn fix:dedupe locally, CI fails with this custom message:

ℹ️ ℹ️ ℹ️
Some dependencies can be deduplicated, which will make yarn.lock
lighter and potentially save us from unexplainable bugs.
Please run `yarn fix:yarn-dedupe` locally and commit yarn.lock.
ℹ️ ℹ️ ℹ️

A contributor needs to commit and push to resolve the issue, which is a pretty mechanical task. I need to maintain two additional package.json scripts and one additional CI task in all my repos.


Alternative 2

Back in Yarn 1 days, I experimented with running yarn-deduplicate as a pre-commit hook via husky. This led to slow git commit calls and some of my teammates were confused.


Alternative 3

When using Dependabot for automatic dependency updates, I saw CI failures for some of its PRs. Because Dependabot cannot be configured with custom post-run commands, this problem can only be solved with manual pushes or a custom CI pipeline that tracks new PRs. That’s quite tedious to setup.

Renovate supports postUpdateOptions and yarnDedupeHighest is available for Yarn 2+. Dependabot maintainers may also come up with something (dependabot/dependabot-core#5830), but they don’t want to introduce a new configuration option which makes sense. Until then, Renovate is a necessary alternative to Dependabot.

Allowing dedupeStrategy: "latest" in .yarnrc.yml can simplify Dependabot / Renovate logic and also align these bots with local yarn install calls. It seems to me that this new declarative approach has more pros than cons, but I might be missing something.

@RDIL
Copy link
Member

RDIL commented Oct 19, 2022

This could also be done as a plugin, just call dedupe in the afterAllInstalled hook. Just an idea though.

@kachkaev
Copy link
Author

kachkaev commented Oct 19, 2022

Yeah that could work locally, but I’m not sure if Dependabot will be able to use the plugin. They don’t run custom repo code for security reasons. A locally installed plugin would fall into this category.

@merceyz
Copy link
Member

merceyz commented Oct 19, 2022

Note that this has been requested before, see #2770.

@ambar
Copy link

ambar commented Oct 29, 2022

FYI, I created a dedupe-on-install script.

It's not very efficient, wish yarn would have an option to check and write at the same time: like yarn dedupe --check-and-write or yarn dedupe --check --write

@kachkaev
Copy link
Author

kachkaev commented Oct 29, 2022

Thanks @ambar! A plugin can definitely help in some scenarios but unfortunately it does not cover the whole problem space. Dependabot (or other hosted tools) need to be able to deduplicate dependencies without having to run repo code because doing so is unsafe. Any solution except dedupeStrategy: "latest" in .yarnrc.yml will require third-party hacks. In the end, the approach to deduplicating may become fragmented within the community.

@rtritto
Copy link

rtritto commented Jun 28, 2024

Any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants