Vercel build errors with Internal Error: EEXIST: file already exists during package installation in repo with submodule

[nickrttn]

Verify canary release

• I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/nickrttn/eexist-repro

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.0.4

Describe the Bug

On Vercel, with the following conditions:

• when using [email protected]
• in a monorepo that contains a git submodule
  • and uses packages from that git submodule
• has ENABLE_EXPERIMENTAL_COREPACK set to 1
• has "packageManager": "[email protected]+sha256.b6fd0bfda555e7e584ad7e56b30c68b01d5a04f9ee93989f4b93ca8473c49c74", in the root package.json (as set by corepack)

Vercel will throw an error during package installation with the following log:

[09:31:38.624] Running build in Washington, D.C., USA (East) – iad1
[09:31:38.800] Cloning github.com/nickrttn/eexist-repro (Branch: main, Commit: 6838abb)
[09:31:39.319] Previous build cache not available
[09:31:39.655] Cloning completed: 853.798ms
[09:31:40.404] Running "vercel build"
[09:31:41.082] Vercel CLI 34.2.7
[09:31:41.746] > Detected ENABLE_EXPERIMENTAL_COREPACK=1 and "[email protected]+sha256.b6fd0bfda555e7e584ad7e56b30c68b01d5a04f9ee93989f4b93ca8473c49c74" in package.json
[09:31:41.887] Running "install" command: `pnpm install`...
[09:31:42.198] Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.4.0.tgz.
[09:31:43.386] Scope: all 9 workspace projects
[09:31:43.412] Lockfile is up to date, resolution step is skipped
[09:31:43.421] Progress: resolved 1, reused 0, downloaded 0, added 0
[09:31:43.427] Packages: +2
[09:31:43.427] ++
[09:31:43.889] Progress: resolved 2, reused 0, downloaded 2, added 2, done
[09:31:43.923] 
[09:31:43.923] devDependencies:
[09:31:43.923] + turbo 2.0.4
[09:31:43.923] 
[09:31:43.929] Done in 1.1s
[09:31:44.296] 
[09:31:44.297] Attention:
[09:31:44.297] Turborepo now collects completely anonymous telemetry regarding usage.
[09:31:44.297] This information is used to shape the Turborepo roadmap and prioritize features.
[09:31:44.298] You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
[09:31:44.298] https://turbo.build/repo/docs/telemetry
[09:31:44.298] 
[09:31:44.320] • Packages in scope: app-a, app-b, pkg-a, pkg-b, submodule-pkg-a, submodule-pkg-b, submodule-tooling-config, tooling-config
[09:31:44.320] • Running build in 8 packages
[09:31:44.321] • Remote caching enabled
[09:31:44.482] submodule-pkg-b:prebuild: cache miss, executing cf524166d4e26742
[09:31:44.503] pkg-b:prebuild: cache hit, replaying logs 70a5a4407159e4a4
[09:31:44.503] pkg-b:prebuild: 
[09:31:44.504] pkg-b:prebuild: > pkg-b@ prebuild /vercel/path0/packages/pkg-b
[09:31:44.504] pkg-b:prebuild: > echo "Executing pre-build step..."
[09:31:44.504] pkg-b:prebuild: 
[09:31:44.504] pkg-b:prebuild: Executing pre-build step...
[09:31:44.518] submodule-pkg-a:build: cache miss, executing 052ae20334f71563
[09:31:44.553] app-b:build: cache hit, replaying logs 28f759b04c54ebd9
[09:31:44.553] app-b:build: 
[09:31:44.553] app-b:build: > app-b@ build /vercel/path0/apps/apps-b
[09:31:44.553] app-b:build: > mkdir -p dist && echo "Your application output!" > dist/app-output.txt && echo "Application B is built!"
[09:31:44.553] app-b:build: 
[09:31:44.553] app-b:build: Application B is built!
[09:31:44.645] pkg-b:build: cache hit, replaying logs 68b997ee68f5b253
[09:31:44.645] pkg-b:build: 
[09:31:44.645] pkg-b:build: > pkg-b@ prebuild /vercel/path0/packages/pkg-b
[09:31:44.645] pkg-b:build: > echo "Executing pre-build step..."
[09:31:44.645] pkg-b:build: 
[09:31:44.645] pkg-b:build: Executing pre-build step...
[09:31:44.645] pkg-b:build: 
[09:31:44.645] pkg-b:build: > pkg-b@ build /vercel/path0/packages/pkg-b
[09:31:44.645] pkg-b:build: > echo "Welcome to the Turboverse." > output-file.txt && cat output-file.txt
[09:31:44.645] pkg-b:build: 
[09:31:44.645] pkg-b:build: Welcome to the Turboverse.
[09:31:44.766] pkg-a:build: cache miss, executing d740213ed80c7662
[09:31:44.826] submodule-pkg-b:prebuild: Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.4.0.tgz.
[09:31:44.866] submodule-pkg-a:build: Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.4.0.tgz.
[09:31:44.941] pkg-a:build: Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.4.0.tgz.
[09:31:45.835] submodule-pkg-a:build: Internal Error: EEXIST: file already exists, rename '/vercel/.cache/node/corepack/v1/corepack-225-89898277.63c17' -> '/vercel/.cache/node/corepack/v1/pnpm/9.4.0'
[09:31:45.835] submodule-pkg-a:build: Error: EEXIST: file already exists, rename '/vercel/.cache/node/corepack/v1/corepack-225-89898277.63c17' -> '/vercel/.cache/node/corepack/v1/pnpm/9.4.0'
[09:31:45.847] submodule-pkg-a:build: ERROR: command finished with error: command (/vercel/path0/eexist-repro-submodule/packages/pkg-a) /vercel/path0/.vercel/cache/corepack/shim/pnpm run build exited (1)
[09:31:45.858] submodule-pkg-a#build: command (/vercel/path0/eexist-repro-submodule/packages/pkg-a) /vercel/path0/.vercel/cache/corepack/shim/pnpm run build exited (1)
[09:31:45.859] 
[09:31:45.859]   Tasks:    3 successful, 6 total
[09:31:45.859]  Cached:    3 cached, 6 total
[09:31:45.859]    Time:    1.557s 
[09:31:45.859] Summary:    /vercel/path0/.turbo/runs/2i2k9EarlCVE0k64r5vK6F6WUJw.json
[09:31:45.859]  Failed:    submodule-pkg-a#build
[09:31:45.859] 
[09:31:45.860]  ERROR  run failed: command  exited (1)
[09:31:45.920] Error: Command "turbo run build" exited with 1
[09:31:46.174] 

Expected Behavior

To install all packages without errors.

To Reproduce

Deploy the shared reproduction to Vercel

Additional context

This workflow used to work in [email protected]. We use it because we include branches of an open-source repository in a closed-source repository for testing and internal usage purposes, while also wanting to keep the OSS repo public for other implementers of our package.

[NicholasLYang]

Hi, in turbo 2.0, we changed the environment variable mode to be strict instead of loose by default. This means that environment variables are not passed to the tasks. This could be the source of your issue. Could you try adding ENABLE_EXPERIMENTAL_COREPACK to your environment variable config in either env or globalEnv? Alternatively, you could try running in loose mode by passing --env-mode=loose in your turbo command.

[nickrttn]

Could you try adding ENABLE_EXPERIMENTAL_COREPACK to your environment variable config in either env or globalEnv?

I just attempted this, but that didn't help.

Alternatively, you could try running in loose mode by passing --env-mode=loose in your turbo command.

This did help and builds are now completing. Are there any resources about going from --env-mode=loose to strict env mode?

[igloo1505]

Hi, in turbo 2.0, we changed the environment variable mode to be strict instead of loose by default. This means that environment variables are not passed to the tasks. This could be the source of your issue. Could you try adding ENABLE_EXPERIMENTAL_COREPACK to your environment variable config in either env or globalEnv? Alternatively, you could try running in loose mode by passing --env-mode=loose in your turbo command.

I encountered the same situation, but now I encountered:

No more than 12 Serverless Functions can be added to a Deployment on the Hobby plan. Create a team (Pro plan) to deploy more.

I'm not trying to be cheap, but I left software 2 years ago to write a couple papers related to my background in physics, and this app is the thing that's hopefully going to make me un-homeless now that they're both in peer review.

It is a rather large framework that I actually tried to reach out to part of the Vercel team about potentially partnering on because it aligns so closely with your platform and the path Vercel seems to be on, but I wasn't having this issue before this most recent update which involved almost 2 days of just trying to get things to build on Vercel due to different versioning issues that came out of nowhere.

The app I'm running on Vercel wouldn't be the actual framework itself, although the framework would allow user's to publish their own notes and research (potentially on Vercel) once the app is released in the next couple weeks, but just a documentation and configuration site. I can't imagine that there's significantly more going on with this app than most hobby level apps.

I'm all in on sending you guys $20 a month as soon as this app gets some users and a couple sponsors or supporters, but my bank account has had the same $0.24 in it for the past month or two.

Also... I'd still love to partner on the framework if your team is interested, it can bring an entirely new demographic to Vercel in students and academics that don't write web apps themselves, but most of all I just need the documentation site to build!