Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: optimize Makefile and fix infinite loop #14000

Merged
merged 1 commit into from
Dec 16, 2024

Conversation

MasonM
Copy link
Member

@MasonM MasonM commented Dec 16, 2024

Motivation

This makes some improvements to the Makefile to speed up local development and avoid a common race condition when building.

Modifications

  1. Remove DOCKER_DESKTOP because nothing uses it

  2. Set GOPATH as a simply expanded variable so go env GOPATH isn't executed multiple times

  3. Don't run the commands associated with the ARGOEXEC_PKG_FILES/CLI_PKG_FILES/CONTROLLER_PKG_FILES variables unless the target(s) needs them, because they're very slow. This speeds up make by ~1.5s for me. Before.

    $ time make check-pwd 
    GIT_COMMIT=f796449df96888538af873bd6871374c2156db7b GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
    KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64 
    RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true make: Nothing to be done for 'check-pwd'.
    
    real    0m1.764s
    user    0m4.382s
    sys     0m0.857s
    

    After:

    $ time make check-pwd
    GIT_COMMIT=f796449df96888538af873bd6871374c2156db7b GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
    KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64
    RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
    make: Nothing to be done for 'check-pwd'.
    
    real    0m0.117s
    user    0m0.060s
    sys     0m0.073s
    
  4. Fix issue where pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go is deleted during a build and it causes Kit to enter an infinite loop. This is a race condition triggered when running something like kit pre-up. During that, as soon as util/telemetry/attributes.go is regenerated, this triggers Kit to abort the make ./dist/argo command. If that command is in the middle of regenerating pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go, then make deletes it:

    waiting for mutex "build"
    make: *** Deleting file 'pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go'
    make: *** [Makefile:657: pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go] Terminated
    signal: terminated
    

    This tells make not to delete it using the special .PRECIOUS target.

Verification

Ran kit pre-up and a bunch of other commands

This makes some improvements to the `Makefile`:
1. Remove `DOCKER_DESKTOP` because nothing uses it
2. Set `GOPATH` as a simply expanded variable so `go env GOPATH`
   isn't executed multiple times
3. Don't run the commands associated with the
   `ARGOEXEC_PKG_FILES`/`CLI_PKG_FILES`/`CONTROLLER_PKG_FILES` unless
   the target(s) needs them, because they're very slow. This speeds up
   `make` by ~1.5s for me. Before.
    ```
    $ time make check-pwd
    GIT_COMMIT=f796449df96888538af873bd6871374c2156db7b GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
    KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64
    RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
    make: Nothing to be done for 'check-pwd'.

    real    0m1.764s
    user    0m4.382s
    sys     0m0.857s
    ```
    After:
    ```
    $ time make check-pwd
    GIT_COMMIT=f796449df96888538af873bd6871374c2156db7b GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
    KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64
    RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
    make: Nothing to be done for 'check-pwd'.

    real    0m0.117s
    user    0m0.060s
    sys     0m0.073s
    ```
4. Fix issue where `pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go` is
   deleted during a build and it causes Kit to enter an infinite loop.
   This happens when running something like `kit pre-up` after
   `util/telemetry/attributes.go` is regenerated, which triggers it to
   abort the `make ./dist/argo` command. If that command is in the
   middle of regenerating
   `pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go`, then `make`
   deletes it:

    ```
    waiting for mutex "build"
    make: *** Deleting file 'pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go'
    make: *** [Makefile:657: pkg/apis/workflow/v1alpha1/zz_generated.deepcopy.go] Terminated
    signal: terminated
    ```

    This tells `make` not to delete it using the specia [.PRECIOUS
    target](https://www.gnu.org/software/make/manual/html_node/Special-Targets.html#index-precious-targets).

Signed-off-by: Mason Malone <[email protected]>
Copy link
Member

@terrytangyuan terrytangyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This should be fine as long as the build passes

@MasonM MasonM marked this pull request as ready for review December 16, 2024 03:52
@MasonM MasonM requested a review from terrytangyuan December 16, 2024 05:48
@terrytangyuan terrytangyuan merged commit e38f49b into argoproj:main Dec 16, 2024
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants