Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI rework - use kind, validate->test->publish, contrib and release rework #1422

Merged
merged 77 commits into from
Oct 17, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
368751e
First cut of ci with kind
clkao Apr 30, 2019
09d566e
allow TEST_URL to be kubectl forwarded port to accommodate kind
clkao Apr 30, 2019
39fbedd
get correct KUBECONFIG during tests
clkao Apr 30, 2019
71b1bdd
only try to get IP for minikube
clkao May 6, 2019
484c1ea
Update kind to 0.3.0
clkao May 17, 2019
1f1cbed
Enable kind CI for 1.12 and 1.13
clkao May 23, 2019
eca8b54
bump kind to 0.4.0 and test kubernetes 1.15
clkao Jul 4, 2019
0528653
bump kind to 0.5.1
clkao Aug 26, 2019
aaed8e8
actually enable 1.15 kind-ci
clkao Aug 27, 2019
9ac3d4b
fix travis-run-kind
clkao Aug 27, 2019
2b369db
check why chartpress is not detecting existing images
clkao Aug 27, 2019
c36a004
Disable registry mirror to make chartpress cache check work
clkao Aug 27, 2019
1ba69aa
helm can be installed whilest the kind node gets ready
clkao Aug 27, 2019
204b7ce
Revert "check why chartpress is not detecting existing images"
clkao Aug 27, 2019
320470f
Big CI/CD and contributing docs rework
consideRatio Sep 23, 2019
162632a
Release instructions revised
consideRatio Sep 30, 2019
c89542a
Apply suggestions from code review
consideRatio Sep 30, 2019
8fc4a51
Correct spelling mistakes
consideRatio Oct 1, 2019
ca368ca
Document use of kind-load-docker-images.py
consideRatio Oct 1, 2019
fe81dba
Correct inline comment
consideRatio Oct 1, 2019
503badd
Merge dev-config-netpol into the dev-config
consideRatio Oct 1, 2019
361ec24
Black autoformatting on tests/
consideRatio Oct 1, 2019
55b5da1
Fail quick on missing HUB_API_URL
consideRatio Oct 1, 2019
55fbee0
Cleanup comments
consideRatio Oct 1, 2019
0982d2d
Consistently use yaml.safe_load over yaml.full_load
consideRatio Oct 1, 2019
4be955c
Apply suggestions from code review
consideRatio Oct 2, 2019
633cf66
Re-fix MacOS DNS with @manics' still relevant fix
consideRatio Oct 3, 2019
0109246
Align with chartpress --reset's default value
consideRatio Oct 5, 2019
bc9a535
Bump chartpress for --reset
consideRatio Oct 6, 2019
b990d68
Initial step from bash to python scripts
consideRatio Oct 6, 2019
f3f81c4
Add decorator for binaries/envs dependencies
consideRatio Oct 7, 2019
523a343
Bulk work on the Python ./dev script
consideRatio Oct 7, 2019
ff689e8
start-k8s / lint-and-validate with ./dev.py
consideRatio Oct 7, 2019
25b341a
Add colorama and python-dotenv
consideRatio Oct 7, 2019
81fe82e
Add missed dependencies
consideRatio Oct 7, 2019
aa1760b
Support Python 3.6
consideRatio Oct 7, 2019
810c70b
Lint/Validate does not require kubectl
consideRatio Oct 7, 2019
5f0627a
Fail early in TravisCI
consideRatio Oct 7, 2019
7674559
More Python 3.6 compability
consideRatio Oct 7, 2019
1ddda02
Even more Python 3.6 compability
consideRatio Oct 7, 2019
fea69df
Even more and more Python 3.6 compability
consideRatio Oct 7, 2019
1c7a570
Bugfix ./dev.py upgrade
consideRatio Oct 8, 2019
d44095c
Fixes...
consideRatio Oct 8, 2019
60e7b31
Attempt on port-forwarding from Python script
consideRatio Oct 8, 2019
a8177d2
Kube 1.16.1 etc.
consideRatio Oct 8, 2019
a6a48f4
Disable k8s 1.16 until calico fix, etc.
consideRatio Oct 8, 2019
700d21c
Fix syntax bugs
consideRatio Oct 8, 2019
45bff5c
Deleted replaced bash scripts
consideRatio Oct 8, 2019
6f28cb8
Add automatic restart of dormant kind clusters
consideRatio Oct 10, 2019
c61e921
Make the calico install up to date
consideRatio Oct 10, 2019
44f0f87
Remove old bash script (dev)
consideRatio Oct 11, 2019
9b612ef
Rename dev.py to dev
consideRatio Oct 11, 2019
e0cd9a2
Rewrite of CONTRIBUTING.md etc.
consideRatio Oct 12, 2019
6e0f758
Rewrite of CONTRIBUTING.md etc. again
consideRatio Oct 13, 2019
68cda07
Remove all remnants of chartpress --commit-range flag
consideRatio Oct 13, 2019
327759b
Add back required daemonset/calico-node fix
consideRatio Oct 13, 2019
a0ee570
Add back K8s 1.16 with fixed calico
consideRatio Oct 13, 2019
e02387a
Helm 2.15 is required for k8s 1.16
consideRatio Oct 13, 2019
f86e7c7
Support helm 2.15 requirements
consideRatio Oct 13, 2019
41beb01
Disable k8s 1.16 again and use old helm again
consideRatio Oct 13, 2019
75761a3
Improve docstrings in test_spawn.py
consideRatio Oct 14, 2019
95f9ce0
--context and --namespace enforced
consideRatio Oct 14, 2019
d0d0904
Fix --context --namespace bugs and port-forwarding
consideRatio Oct 14, 2019
30302ee
Converting comment to dosctring
consideRatio Oct 14, 2019
c4e2acb
Systematic use of port 8080
consideRatio Oct 14, 2019
c9c7ad6
Py37 for Windows, make ./dev port-forwarding provide info
consideRatio Oct 15, 2019
9a0d469
Return to Python 3.6 since it was troublesome with the VM
consideRatio Oct 17, 2019
047daec
Update debugging notes of port clashes
consideRatio Oct 17, 2019
a619310
Get Python 3.6 right again
consideRatio Oct 17, 2019
dc1a3d8
Prefix with Z2JH_ for our env vars
consideRatio Oct 17, 2019
599cd9c
Selective environment override
consideRatio Oct 17, 2019
bc7f593
Add note about DNS failure
consideRatio Oct 17, 2019
752dc83
Add warning when non Z2JH_ prefixed envs are overridden
consideRatio Oct 17, 2019
08e302a
Cleanup and test k8s 1.16 with allow_failures
consideRatio Oct 17, 2019
5328d5a
Optimize CI build order for speed
consideRatio Oct 17, 2019
48bd4f4
Bump chartpress to 0.4.0
consideRatio Oct 17, 2019
a29b987
Bump chartpress to 0.4.1
consideRatio Oct 17, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .binder/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# What is this folder about?

This folder contains the dependency information required by [a notebook](doc/ntbk/draw_function.ipynb) [referenced](doc/source/cost.rst) in our documentation to run on mybinder.org. This dependency information allows a click of [a link](http://mybinder.org/v2/gh/jupyterhub/zero-to-jupyterhub-k8s/master?filepath=doc/ntbk/draw_function.ipynb) to create the rendered notebook.
3 changes: 0 additions & 3 deletions binder/environment.yml → .binder/environment.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# Dependencies for the guides cloud cost estimator notebook
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# http://mybinder.org/v2/gh/jupyterhub/zero-to-jupyterhub-k8s/master?filepath=doc/ntbk/draw_function.ipynb

# Need to install bqplot with conda so it calls the javascript extension.
name: bqplot
channels:
Expand Down
8 changes: 8 additions & 0 deletions .circleci/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# What is this folder about?

We use CircleCI to build documentation previews for PRs, as configured through
[.circleci/config.yml], this allows us to easily preview documentation changes
in a PR in its final form before the PR is merged.
consideRatio marked this conversation as resolved.
Show resolved Hide resolved

When a PR is merged, the configuration in [readthedocs.yml](readthedocs.yml) will help ReadTheDocs
build the docs and publish to https://z2jh.jupyter.org.
3 changes: 0 additions & 3 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
# This CircleCI build lets us preview the documentation inside PRs before they
# are merged! And when they are, the readthedocs.yml file will help ReadTheDocs
# build and publish it on https://z2jh.jupyter.org.
version: 2
jobs:
build_docs:
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
### Zero to JupyterHub Kubernetes ###
tools/templates/rendered-templates/
bin/
ci/.vagrant
.vagrant/
tools/github.sqlite

.vscode
Expand Down
102 changes: 88 additions & 14 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,99 @@
dist: xenial
dist: bionic
language: python
python:
- 3.6
git:
## depth set to false overrides travis default behavior to use shallow clones
## with depth 50 that can cause issues
consideRatio marked this conversation as resolved.
Show resolved Hide resolved
##
## ref: https://github.com/jupyterhub/chartpress#shallow-clones
##
depth: false
services:
- docker
## stages declares and orders stages
##
## ref: https://docs.travis-ci.com/user/build-stages/#build-stages-and-deployments
##
stages:
- name: lint and validate
- name: test
- name: publish
## if conditions gives us control if the stage should run
##
## ref: https://docs.travis-ci.com/user/conditions-v1
##
if: >
branch in (master) AND
type in (push)
install:
- set -e
- pip3 install --no-cache-dir -r dev-requirements.txt
- . "ci/minikube-${SCENARIO}.env"
- ./ci/install.sh
script:
- ./ci/travis-script.sh
- . ci/common

stage: test
script:
- setup_kubectl
- setup_kind
- setup_helm
- ./ci/travis-docker-fix
- ./dev kind create
- ./dev upgrade
- ./dev port-forward
- ./dev test
env:
# Different scenarios described in
# /ci/minikube-${SCENARIO}.env
matrix:
- SCENARIO=1.13-default
- SCENARIO=1.12-netpol
- SCENARIO=1.11-default
global:
- secure: jpFpbMccpjGP+otWH2Z03VFdtR9AAu2vzrNxsoZ3IvJvrO4MfzYJ3uSCDQuB0NG9gBgaAscpTJtliPTEi7njXHLcsFeWXLUmeBEHLozYxfzDQzMvW3EYdNWcC7oVAAt3de0i0ojw9rGswiofhbu2dAe+Xd2bejv1+PVJcEC3SRPGy17kb6bme6gD3zty5ft4VpzP0nomUNqfZBRLUYxSZuKlHJaZ6Nuq434rKmXrcN6uy+eEWDorTbjyM22IIYgUmrhg++Qtu/MBR7/rriPhyRltCU14361bcxqyq2Hw+HNG8D3hsqo5TiEiYwxOQcXRgddL+Ci6/y0L1EvqOQc+1V8ycwNs2oNicwNgSn5A+9HpF495Kae039hGtj2Gpt4IbplSYwKFq/sFTq+CekxdD2YVQmGvsjep4bNVL66o2RSZVAW1Bg/G8/sSe3BwgD8IToy9+1NHPPuaVupeukRqNyUDcVvWH8hdb8AkXYY87+546etYDpn91GQnhTEberKbXX4UCmpKNXpXoprLE8nQLGb6TIoHPTyA+RRNQ4erDzMjqF43UVmhOZTtkGaRgIWK7vDAKpLUnuOguuhJUNpYpRggGQsMV8cZnaCumy5OFUf6i6rfN0Ru6a+/Bm7grJiAcnZlU7igaxgI38QaJgCKcqqzIImdcRYNQC74/Ok/1oM=
- secure: BK++GwKVPoS0iG8aB7wQ13daTgJR9MifHA+l9xr/tSZ3SUL6nc7kjxLbliRQJCqT9lcOODsd+v2u9PziEzBp0CCh67ftFxJw8riP2+FgdmHTK4yav9QpSwoBJHhV2SgBMGlXiqdUVC7wpgjzzK63V8abvzAhXkthWPl3kYpUI//xGYyuBNXVHEOImHB3F1M5bn90lflFtRfq2iH5FigGesMi2BFfTVeqvbzZVZrAs0E1/NRdO+/cRq0c9aRpNLkh254k1tcKbUvULQq1iLQuHN2Ramn3NgNnx93sbwp1e7ZjmETbjr9cwMIDg5mh25H0Rjf2Nn8cqHbBCWzoMkjZW097HRVDYht2kJZQIbQcaxX38DW6vykUwGWSBAWbtvCUwYwU57s/dIbSYUTQErkYYmhiq52cdOtnxZ2/ULoElCVyR8lTmQuANJrq9YFC9q1ly69YuMWWnFgwxWpK1JCgAJGELgj5EvcghEtNmkEFh5f6pmbKBE7PKQPTovzNKcdRauR/L+MsmhVYukCfNZq57LrruIQIX1GQNw9w3Ck8P4EPtNjdI4umCSy6nZSyTevWgVTmIP9EwXa5Cap32ZU+iDtw+wUBAr3sjROJOYGKlL/ktWsWbjog5hIG0rrb8PbgOfbLRZSEYGL9sYsyXXyW5oI37lB7AqG6D7vOA4TdmTQ=
## NOTE: The environment variables will be expanded to multiple jobs. For
## additional individual jobs, only the first entry is used.
##
## ref: https://docs.travis-ci.com/user/build-stages/#build-stages-and-build-matrix-expansion
##
##
## Z2JH_KUBE_VERSION should match a released kindest/node image tag, but they
## are currently not automatically published.
##
## ref: https://hub.docker.com/r/kindest/node/tags
## ref: https://github.com/kubernetes-sigs/kind/issues/197
##
- Z2JH_KUBE_VERSION=1.15.3
- Z2JH_KUBE_VERSION=1.14.6
- Z2JH_KUBE_VERSION=1.13.10
- &allow_failure Z2JH_KUBE_VERSION=1.16.1 Z2JH_HELM_VERSION=2.15.0-rc.2

jobs:
## allow experimental setups to fail
##
## ref: https://docs.travis-ci.com/user/customizing-the-build/#rows-that-are-allowed-to-fail
##
allow_failures:
- env: *allow_failure
## don't wait for the jobs that are allowed to fail to report success
##
## ref: https://docs.travis-ci.com/user/customizing-the-build/#fast-finishing
##
fast_finish: true

## include additional individual jobs
##
include:
- stage: lint and validate
script:
- setup_helm
- setup_kubeval
- ./dev check templates
env:
- Z2JH_VALIDATE_KUBE_VERSIONS=1.11.0,1.12.0,1.13.0,1.14.0,1.15.0,1.16.0
- stage: publish
script:
- setup_helm
- setup_git_crypt
- ./ci/travis-docker-fix
- ./ci/publish
env:
## encrypted environment variables, used on push to master in the
## publish script to in turn decrypt a SSH key
##
## ref: https://docs.travis-ci.com/user/environment-variables/#encrypting-environment-variables
- secure: jpFpbMccpjGP+otWH2Z03VFdtR9AAu2vzrNxsoZ3IvJvrO4MfzYJ3uSCDQuB0NG9gBgaAscpTJtliPTEi7njXHLcsFeWXLUmeBEHLozYxfzDQzMvW3EYdNWcC7oVAAt3de0i0ojw9rGswiofhbu2dAe+Xd2bejv1+PVJcEC3SRPGy17kb6bme6gD3zty5ft4VpzP0nomUNqfZBRLUYxSZuKlHJaZ6Nuq434rKmXrcN6uy+eEWDorTbjyM22IIYgUmrhg++Qtu/MBR7/rriPhyRltCU14361bcxqyq2Hw+HNG8D3hsqo5TiEiYwxOQcXRgddL+Ci6/y0L1EvqOQc+1V8ycwNs2oNicwNgSn5A+9HpF495Kae039hGtj2Gpt4IbplSYwKFq/sFTq+CekxdD2YVQmGvsjep4bNVL66o2RSZVAW1Bg/G8/sSe3BwgD8IToy9+1NHPPuaVupeukRqNyUDcVvWH8hdb8AkXYY87+546etYDpn91GQnhTEberKbXX4UCmpKNXpXoprLE8nQLGb6TIoHPTyA+RRNQ4erDzMjqF43UVmhOZTtkGaRgIWK7vDAKpLUnuOguuhJUNpYpRggGQsMV8cZnaCumy5OFUf6i6rfN0Ru6a+/Bm7grJiAcnZlU7igaxgI38QaJgCKcqqzIImdcRYNQC74/Ok/1oM=
- secure: BK++GwKVPoS0iG8aB7wQ13daTgJR9MifHA+l9xr/tSZ3SUL6nc7kjxLbliRQJCqT9lcOODsd+v2u9PziEzBp0CCh67ftFxJw8riP2+FgdmHTK4yav9QpSwoBJHhV2SgBMGlXiqdUVC7wpgjzzK63V8abvzAhXkthWPl3kYpUI//xGYyuBNXVHEOImHB3F1M5bn90lflFtRfq2iH5FigGesMi2BFfTVeqvbzZVZrAs0E1/NRdO+/cRq0c9aRpNLkh254k1tcKbUvULQq1iLQuHN2Ramn3NgNnx93sbwp1e7ZjmETbjr9cwMIDg5mh25H0Rjf2Nn8cqHbBCWzoMkjZW097HRVDYht2kJZQIbQcaxX38DW6vykUwGWSBAWbtvCUwYwU57s/dIbSYUTQErkYYmhiq52cdOtnxZ2/ULoElCVyR8lTmQuANJrq9YFC9q1ly69YuMWWnFgwxWpK1JCgAJGELgj5EvcghEtNmkEFh5f6pmbKBE7PKQPTovzNKcdRauR/L+MsmhVYukCfNZq57LrruIQIX1GQNw9w3Ck8P4EPtNjdI4umCSy6nZSyTevWgVTmIP9EwXa5Cap32ZU+iDtw+wUBAr3sjROJOYGKlL/ktWsWbjog5hIG0rrb8PbgOfbLRZSEYGL9sYsyXXyW5oI37lB7AqG6D7vOA4TdmTQ=
on:
tags: true
Loading