Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: rewrite makefile in python #52788

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 0 additions & 7 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,6 @@ insert_final_newline = true
trim_trailing_whitespace = true
quote_type = single

[vcbuild.bat]
end_of_line = crlf

[Makefile]
indent_size = 8
indent_style = tab

[{deps}/**]
charset = unset
end_of_line = unset
Expand Down
1 change: 0 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
test/fixtures/* -text
vcbuild.bat text eol=crlf
deps/npm/bin/npm text eol=lf
deps/npm/bin/npx text eol=lf
deps/corepack/shims/corepack text eol=lf
Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Before submitting a pull request, please read:
For code changes:
1. Include tests for any bug fixes or new features.
2. Update documentation if relevant.
3. Ensure that `make -j4 test` (UNIX), or `vcbuild test` (Windows) passes.
3. Ensure that `python3 build.py -j4 test` passes.

If you believe this PR should be highlighted in the Node.js CHANGELOG
please add the `notable-change` label.
Expand Down
2 changes: 1 addition & 1 deletion .github/label-pr-config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ subSystemLabels:
# Different variants of Makefile and build files
/^(?:tools\/)?(?:Makefile|BSDmakefile|create_android_makefiles)$/: build, needs-ci
/^tools\/(?:install\.py|getnodeversion\.py|js2c\.py|utils\.py|configure\.d\/.*)$/: build, python, needs-ci
/^vcbuild\.bat$/: build, windows, needs-ci
/^build\.py$/: build, needs-ci
/^(?:android-)?configure|node\.gyp|common\.gypi$/: build, needs-ci
# More specific tools
/^tools\/gyp/: tools, build, gyp, needs-ci
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/build-tarball.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:
export DISTTYPE=nightly
export DATESTRING=`date "+%Y-%m-%d"`
export COMMIT=$(git rev-parse --short=10 "$GITHUB_SHA")
./configure && make tar -j8 SKIP_XZ=1
./configure && python3 build.py tar -j8
mkdir tarballs
mv *.tar.gz tarballs
- name: Upload tarball artifact
Expand Down Expand Up @@ -90,8 +90,8 @@ jobs:
- name: Build
run: |
cd $TAR_DIR
make build-ci -j4 V=1
python3 build.py build-ci -j4
- name: Test
run: |
cd $TAR_DIR
make run-ci -j4 V=1 TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9"
TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9" python3 build.py run-ci -j4
2 changes: 1 addition & 1 deletion .github/workflows/build-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,4 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: ./vcbuild.bat
run: ./build.py
4 changes: 2 additions & 2 deletions .github/workflows/coverage-linux-without-intl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@ jobs:
- name: Install gcovr
run: pip install gcovr==4.2
- name: Build
run: make build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn --coverage --without-intl"
run: CONFIG_FLAGS="--error-on-warn --coverage --without-intl" python3 build.py build-ci -j4
# TODO(bcoe): fix the couple tests that fail with the inspector enabled.
# The cause is most likely coverage's use of the inspector.
- name: Test
run: NODE_V8_COVERAGE=coverage/tmp make test-cov -j4 V=1 TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9" || exit 0
run: TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9" NODE_V8_COVERAGE=coverage/tmp python3 build.py test-cov -j4 || exit 0
- name: Report JS
run: npx c8 report --check-coverage
env:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/coverage-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@ jobs:
- name: Install gcovr
run: pip install gcovr==4.2
- name: Build
run: make build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn --coverage"
run: V=1 CONFIG_FLAGS="--error-on-warn --coverage" python3 build.py build-ci -j4
# TODO(bcoe): fix the couple tests that fail with the inspector enabled.
# The cause is most likely coverage's use of the inspector.
- name: Test
run: NODE_V8_COVERAGE=coverage/tmp make test-cov -j4 V=1 TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9" || exit 0
run: V=1 TEST_CI_ARGS="-p dots --node-args='--test-reporter=spec' --measure-flakiness 9" NODE_V8_COVERAGE=coverage/tmp python3 build.py test-cov -j4 || exit 0
- name: Report JS
run: npx c8 report --check-coverage
env:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/coverage-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,11 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: ./vcbuild.bat
run: python3 build.py
# TODO(bcoe): investigate tests that fail with coverage enabled
# on Windows.
- name: Test
run: ./vcbuild.bat test-ci-js; node -e 'process.exit(0)'
run: python3 build.py test-ci-js; node -e 'process.exit(0)'
env:
NODE_V8_COVERAGE: ./coverage/tmp
- name: Report
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/daily-wpt-fyi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
# Node.js WPT Runner
- name: Run WPT and generate report
run: |
make test-wpt-report || true
python3 build.py test-wpt-report || true
if [ -e out/wpt/wptreport.json ]; then
echo "WPT_REPORT=$(pwd)/out/wpt/wptreport.json" >> $GITHUB_ENV
fi
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/doc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: NODE=$(command -v node) make doc-only
run: NODE=$(command -v node) python3 build.py doc-only
- uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1
with:
name: docs
path: out/doc
- name: Test
run: NODE=$(command -v node) make test-doc-ci TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
run: NODE=$(command -v node) python3 build.py test-doc-ci TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
24 changes: 12 additions & 12 deletions .github/workflows/linters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Lint addon docs
run: NODE=$(command -v node) make lint-addon-docs
run: NODE=$(command -v node) python3 build.py lint-addon-docs
lint-cpp:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
Expand All @@ -50,7 +50,7 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Lint C/C++ files
run: make lint-cpp
run: python3 build.py lint-cpp
format-cpp:
if: ${{ github.event.pull_request && github.event.pull_request.draft == false && github.base_ref == github.event.repository.default_branch }}
runs-on: ubuntu-latest
Expand All @@ -71,20 +71,20 @@ jobs:
run: npx envinfo
- name: Format C/C++ files
run: |
make format-cpp-build
# The `make format-cpp` error code is intentionally ignored here
python3 build.py format-cpp-build
# The `python3 build.py format-cpp` error code is intentionally ignored here
# because it is irrelevant. We already check if the formatter produced
# a diff in the next line.
# Refs: https://github.com/nodejs/node/pull/42764
CLANG_FORMAT_START="$(git merge-base HEAD refs/remotes/origin/$GITHUB_BASE_REF)" \
make format-cpp || true
python3 build.py format-cpp || true
git --no-pager diff --exit-code && EXIT_CODE="$?" || EXIT_CODE="$?"
if [ "$EXIT_CODE" != "0" ]
then
echo
echo 'ERROR: Please run:'
echo
echo " CLANG_FORMAT_START="$\(git merge-base HEAD ${GITHUB_BASE_REF}\)" make format-cpp"
echo " CLANG_FORMAT_START="$\(git merge-base HEAD ${GITHUB_BASE_REF}\)" python3 build.py format-cpp"
echo
echo 'to format the commits in your branch.'
exit "$EXIT_CODE"
Expand All @@ -103,15 +103,15 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Lint JavaScript files
run: NODE=$(command -v node) make lint-js
run: NODE=$(command -v node) python3 build.py lint-js
- name: Get release version numbers
if: ${{ github.event.pull_request && github.event.pull_request.base.ref == github.event.pull_request.base.repo.default_branch }}
id: get-released-versions
run: ./tools/lint-md/list-released-versions-from-changelogs.mjs >> $GITHUB_OUTPUT
- name: Lint markdown files
run: |
echo "::add-matcher::.github/workflows/remark-lint-problem-matcher.json"
NODE=$(command -v node) make lint-md
NODE=$(command -v node) python3 build.py lint-md
env:
NODE_RELEASED_VERSIONS: ${{ steps.get-released-versions.outputs.NODE_RELEASED_VERSIONS }}
lint-py:
Expand All @@ -129,8 +129,8 @@ jobs:
run: npx envinfo
- name: Lint Python
run: |
make lint-py-build
make lint-py
python3 build.py lint-py-build
python3 build.py lint-py
lint-yaml:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
Expand All @@ -146,8 +146,8 @@ jobs:
run: npx envinfo
- name: Lint YAML
run: |
make lint-yaml-build || true
make lint-yaml
python3 build.py lint-yaml-build || true
python3 build.py lint-yaml

lint-sh:
if: github.event.pull_request.draft == false
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/test-asan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,6 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: make build-ci -j4 V=1
run: python3 build.py build-ci -j4 V=1
- name: Test
run: make run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' -t 300 --measure-flakiness 9"
run: python3 build.py run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' -t 300 --measure-flakiness 9"
4 changes: 2 additions & 2 deletions .github/workflows/test-internet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: make build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
run: python3 build.py build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
- name: Test Internet
run: make test-internet -j4 V=1;
run: python3 build.py test-internet -j4 V=1;
4 changes: 2 additions & 2 deletions .github/workflows/test-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,6 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: make build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
run: python3 build.py build-ci -j4 V=1 CONFIG_FLAGS="--error-on-warn"
- name: Test
run: make run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
run: python3 build.py run-ci -j4 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
8 changes: 4 additions & 4 deletions .github/workflows/test-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,12 @@ jobs:
# The `npm ci` for this step fails a lot as part of the Test step. Run it
# now so that we don't have to wait 2 hours for the Build step to pass
# first before that failure happens. (And if there's something about
# `make run-ci -j3` that is causing the failure and the failure doesn't
# `python3 build.py run-ci -j3` that is causing the failure and the failure doesn't
# happen anymore running this step here first, that's also useful
# information.)
- name: tools/doc/node_modules workaround
run: make tools/doc/node_modules
run: python3 build.py tools/doc/node_modules
- name: Build
run: make build-ci -j$(getconf _NPROCESSORS_ONLN) V=1 CONFIG_FLAGS="--error-on-warn"
run: python3 build.py build-ci -j$(getconf _NPROCESSORS_ONLN) V=1 CONFIG_FLAGS="--error-on-warn"
- name: Test
run: make run-ci -j$(getconf _NPROCESSORS_ONLN) V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
run: python3 build.py run-ci -j$(getconf _NPROCESSORS_ONLN) V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' --measure-flakiness 9"
4 changes: 2 additions & 2 deletions .github/workflows/test-ubsan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,6 @@ jobs:
- name: Environment Information
run: npx envinfo
- name: Build
run: make build-ci -j2 V=1
run: python3 build.py build-ci -j2 V=1
- name: Test
run: make run-ci -j2 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' -t 300 --measure-flakiness 9"
run: python3 build.py run-ci -j2 V=1 TEST_CI_ARGS="-p actions --node-args='--test-reporter=spec' --node-args='--test-reporter-destination=stdout' -t 300 --measure-flakiness 9"
4 changes: 2 additions & 2 deletions .github/workflows/tools.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ jobs:
subsystem: deps
label: dependencies
run: |
make corepack-update
python3 build.py corepack-update
echo "NEW_VERSION=$(node deps/corepack/dist/corepack.js --version)" >> $GITHUB_ENV
- id: doc
subsystem: tools
Expand Down Expand Up @@ -210,7 +210,7 @@ jobs:
npm install --ignore-scripts $NEW_VERSION
npm install --ignore-scripts
cd ../..
make lint-md-rollup
python3 build.py lint-md-rollup
fi
- id: llhttp
subsystem: deps
Expand Down
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -146,9 +146,6 @@ tools/*/*.i.tmp
# Ignore dependencies fetched by tools/v8/fetch_deps.py
/deps/.cipd

# === Rules for Windows vcbuild.bat ===
/temp-vcbuild

# === Rules for CMake ===
cmake-build-debug/
CMakeCache.txt
Expand Down
2 changes: 1 addition & 1 deletion .gitpod.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Ref: https://github.com/gitpod-io/gitpod/issues/6283#issuecomment-1001043454
tasks:
- init: ./configure && timeout 50m make -j16 || true
- init: ./configure && timeout 50m python3 build.py all -j16 || true
- init: pnpm i -g @node-core/utils

# Ref: https://www.gitpod.io/docs/prebuilds#github-specific-configuration
Expand Down