Add @next/vendored

[lfades]

This PR aims to replace the use of next/dist/compiled/* with @next/vendored/*.

Motivation

• Remove next/src/compiled from source. Currently Next.js clones its contents into next/dist/compiled but we can run the tasks that generate the modules fast enough with caching.
• The imports to next/dist/compiled/* will now to go @next/vendored/*, a separate package that has all the third party modules that were compiled by ncc. This allows Next.js to have exports inside package.json in a future update, which is currently highly difficult due to it having inner node packages (i.e nested modules that are resolved after reading a package.json).
• Move all of the related ncc tasks and compiled code out of the next repo.

Status

Some things that have been done so far:

• A custom task runner is being used instead of taskr. The reason is taskr doesn't work very well with ESM (The @taskr/esnext package is a hack and it's still commonjs), because of this resolving a package always sends its commonjs distribution to ncc. There are also some differences in how the tasks are declared, particularly I wanted to have a way to generate multiple tasks with a loop for the most common cases.
• The custom task runner is quite simple and it can also be made into its own package.

Todo

• Move all the related ncc tasks from next to @next/vendored
• Use Turborepo caching like in Turborepo for running packages/next tasks #64825

Testing locally

After running pnpm install, build @next/vendored with:

cd packages/next-vendored && pnpm dev

and then run the ncc tasks in @next/vendored:

pnpm task-runner ncc

Run pnpm dev at the root and the Next.js build should work. I'm testing it locally in an app from create-next-app and with pnpm next-with-deps

[eps1lon]

FYI we've replaced taskr is being replaced with using Turborepo: #64825

[lfades]

FYI we've replaced taskr is being replaced with using Turborepo: #64825

Thanks for sharing it. taskr is still being used there but it's being cached by Turborepo. I'll add that to this PR.

Note that I replaced taskr to resolve modules for ncc in a ESM environment. It's also a considerably old library that hasn't received an update in the last 7 years.