Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: upgrade kit #14105

Merged
merged 12 commits into from
Jan 28, 2025
4 changes: 2 additions & 2 deletions .devcontainer/pre-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ sudo mv ./kubectl /usr/local/bin/kubectl
kubectl cluster-info

# install kit
curl -q https://raw.githubusercontent.com/kitproj/kit/main/install.sh | sh
make kit

# install protocol buffer compiler (protoc)
sudo apt update
Expand All @@ -25,7 +25,7 @@ sudo chown vscode:vscode /home/vscode/go/src || true
sudo chown vscode:vscode /home/vscode/go/src/github.com || true

# download dependencies and do first-pass compile
CI=1 kit pre-up
kit build

# Patch CoreDNS to have host.docker.internal inside the cluster available
kubectl get cm coredns -n kube-system -o yaml | sed "s/ NodeHosts: |/ NodeHosts: |\n `grep host.docker.internal /etc/hosts`/" | kubectl apply -f -
11 changes: 1 addition & 10 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -532,16 +532,7 @@ dist/argosay:

.PHONY: kit
kit: Makefile
ifeq ($(shell command -v kit),)
ifeq ($(shell uname),Darwin)
brew tap kitproj/kit --custom-remote https://github.com/kitproj/kit
brew install kit
else
@echo "Downloading Kit"
curl -fsL --retry 99 "https://github.com/kitproj/kit/releases/download/v0.1.8/kit_0.1.8_$$(uname)_$$(uname -m | sed 's/aarch64/arm64/').tar.gz" | sudo tar -C /usr/local/bin -xzf - kit
endif
endif

go install github.com/kitproj/[email protected]

.PHONY: start
ifeq ($(RUN_MODE),local)
Expand Down
65 changes: 2 additions & 63 deletions hack/port-forward.sh
Original file line number Diff line number Diff line change
@@ -1,67 +1,6 @@
#!/usr/bin/env bash
set -eu -o pipefail

pf() {
set -eu -o pipefail
resource=$1
port=$2
dest_port=${3:-"$port"}
./hack/free-port.sh $port
echo "port-forward $resource $port"
kubectl -n argo port-forward "svc/$resource" "$port:$dest_port" &
until lsof -i ":$port" > /dev/null ; do sleep 1 ; done
}
go install github.com/kitproj/[email protected]
Copy link
Member

@MasonM MasonM Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I made my own attempt at getting rid of kubectl port-forward a little while ago: main...MasonM:argo-workflows:no-port-forward

The idea is to change all services to be of type LoadBalancer. With k3s, that means ServiceLB will automatically forward the port to the host. This works, but it kinda couples the dev environment to k3s, since some distributions (e.g. kind) don't come with a LoadBalancer out-of-the-box. It looks like kubeauto doesn't have that problem.


wait-for() {
set -eu -o pipefail
echo "wait-for $1"
kubectl -n argo wait --timeout 2m --for=condition=Available deploy/$1
}


dex=$(kubectl -n argo get pod -l app=dex -o name)
if [[ "$dex" != "" ]]; then
wait-for dex
pf dex 5556
fi

postgres=$(kubectl -n argo get pod -l app=postgres -o name)
if [[ "$postgres" != "" ]]; then
wait-for postgres
pf postgres 5432
fi

mysql=$(kubectl -n argo get pod -l app=mysql -o name)
if [[ "$mysql" != "" ]]; then
wait-for mysql
pf mysql 3306
fi

if [[ "$(kubectl -n argo get pod -l app=argo-server -o name)" != "" ]]; then
wait-for argo-server
pf argo-server 2746
fi

if [[ "$(kubectl -n argo get pod -l app=workflow-controller -o name)" != "" ]]; then
wait-for workflow-controller
pf workflow-controller-metrics 9090
if [[ "$(kubectl -n argo get svc workflow-controller-pprof -o name)" != "" ]]; then
pf workflow-controller-pprof 6060
fi
fi

if [[ "$(kubectl -n argo get pod -l app=prometheus -o name)" != "" ]]; then
wait-for prometheus
pf prometheus 9091 9090
fi

azurite=$(kubectl -n argo get pod -l app=azurite -o name)
if [[ "$azurite" != "" ]]; then
wait-for azurite
pf azurite 10000
fi

# forward MinIO last, so we can just wait for port 9000, and know that all ports are ready
wait-for minio
pf minio 9000
pf minio 9001
kubeauto -p 0
235 changes: 137 additions & 98 deletions tasks.yaml
Original file line number Diff line number Diff line change
@@ -1,98 +1,137 @@
apiVersion: kit/v1
kind: Tasks
metadata:
annotations:
help: |
Install `kit` by following https://github.com/kitproj/kit#install.

Run `kit up` to start argo.

- `env PROFILE=mysql kit up` to start with MySQL.
- `env PROFILE=plugins ARGO_EXECUTOR_PLUGINS=true kit up` to start with plugins.
- `env PROFILE=sso ARGO_AUTH_MODE=sso kit up` to start with SSO.

The app will be up-and-running between 15s and 1m later (if hot compiled or cold).
Any changes made to the source code will be automatically recompiled and the app restarted, typically within a few seconds.
name: argo-workflows
spec:
tasks:
- name: go-deps
command: go mod download
- name: install
command: sh -c "make install PROFILE=$PROFILE"
env:
- PROFILE=minimal
dependencies: go-deps
watch: manifests
mutex: docker
- name: build-controller
command: make ./dist/workflow-controller
watch: cmd/workflow-controller config errors persist pkg util workflow
dependencies: go-deps
mutex: build
- name: port-forward
command: ./hack/port-forward.sh
ports: 9000
dependencies: install
- name: controller
command: ./dist/workflow-controller
dependencies: install build-controller port-forward
env:
- ARGO_EXECUTOR_PLUGINS=false
- ARGO_NAMESPACE=argo
- ARGO_NAMESPACED=true
- ARGO_MANAGED_NAMESPACE=argo
- ARGO_LOGLEVEL=info
- ARGO_REMOVE_PVC_PROTECTION_FINALIZER=true
- ARGO_PROGRESS_PATCH_TICK_DURATION=7s
- DEFAULT_REQUEUE_TIME=1s
- LEADER_ELECTION_IDENTITY=local
- ALWAYS_OFFLOAD_NODE_STATUS=false
- OFFLOAD_NODE_STATUS_TTL=30s
- WORKFLOW_GC_PERIOD=30s
- UPPERIO_DB_DEBUG=1
- ARCHIVED_WORKFLOW_GC_PERIOD=30s
ports: "9090"
- name: build-argo
command: make ./dist/argo
dependencies: go-deps
env:
- STATIC_FILES=false
watch: cmd/argo config errors persist pkg util server workflow
mutex: build
- name: server
command: ./dist/argo server
dependencies: build-argo port-forward
env:
- ARGO_X_FRAME_OPTIONS=SAMEORIGIN
- ARGO_SECURE=false
- ARGO_NAMESPACE=argo
- ARGO_NAMESPACED=true
- ARGO_LOGLEVEL=info
- ARGO_AUTH_MODE=hybrid
- ARGO_MANAGED_NAMESPACE=argo
- UPPERIO_DB_DEBUG=1
ports: "2746"
- name: ui-deps
command: yarn install
workingDir: ui
watch: ui/package.json ui/yarn.lock
- name: ui
command: yarn start
workingDir: ui
dependencies: ui-deps
ports: "8080"
- name: executor
command: make argoexec-image
watch: cmd/argoexec config errors pkg util workflow
mutex: docker
- name: example
command: kubectl create -f examples/hello-world.yaml
dependencies: install
mutex: docker
- name: build
dependencies: build-controller build-argo
- name: pre-up
dependencies: build install executor example
- name: up
dependencies: pre-up controller server ui
tasks:
build:
dependencies:
- build-controller
- build-cli
- build-executor
build-cli:
command:
- make
- ./dist/argo
env:
STATIC_FILES: "false"
mutex: build
watch:
- cmd/argo
- config
- errors
- persist
- pkg
- util
- server
- workflow
build-controller:
command:
- make
- ./dist/workflow-controller
mutex: build
watch:
- cmd/workflow-controller
- config
- errors
- persist
- pkg
- util
- workflow
build-executor:
command:
- make
- argoexec-image
mutex: docker
watch:
- cmd/argoexec
- config
- errors
- pkg
- util
- workflow
controller:
command:
- ./dist/workflow-controller
dependencies:
- build-controller
- port-forward
env:
ALWAYS_OFFLOAD_NODE_STATUS: "false"
ARCHIVED_WORKFLOW_GC_PERIOD: 30s
ARGO_EXECUTOR_PLUGINS: "false"
ARGO_LOGLEVEL: info
ARGO_MANAGED_NAMESPACE: argo
ARGO_NAMESPACE: argo
ARGO_NAMESPACED: "true"
ARGO_PROGRESS_PATCH_TICK_DURATION: 7s
ARGO_REMOVE_PVC_PROTECTION_FINALIZER: "true"
DEFAULT_REQUEUE_TIME: 1s
LEADER_ELECTION_IDENTITY: local
OFFLOAD_NODE_STATUS_TTL: 30s
UPPERIO_DB_DEBUG: "1"
WORKFLOW_GC_PERIOD: 30s
ports:
- "9090"
example:
command:
- kubectl
- create
- -f
- examples/hello-world.yaml
dependencies:
- install
mutex: docker
install:
env:
PROFILE: minimal
mutex: docker
sh: make install PROFILE=$PROFILE
watch:
- manifests
port-forward:
command:
- ./hack/port-forward.sh
dependencies:
- install
ports:
- "9000"
server:
command:
- ./dist/argo
- server
dependencies:
- build-cli
- port-forward
env:
ARGO_AUTH_MODE: hybrid
ARGO_LOGLEVEL: info
ARGO_MANAGED_NAMESPACE: argo
ARGO_NAMESPACE: argo
ARGO_NAMESPACED: "true"
ARGO_SECURE: "false"
ARGO_X_FRAME_OPTIONS: SAMEORIGIN
UPPERIO_DB_DEBUG: "1"
ports:
- "2746"
ui:
command:
- yarn
- start
dependencies:
- ui-deps
ports:
- "8080"
watch:
- package.json
workingDir: ui
ui-deps:
command:
- yarn
- install
workingDir: ui
up:
command:
- sleep
- "999999"
dependencies:
- example
- controller
- server
- ui
- build-executor