Merge pull request #6313 from berkeley-dsep-infra/staging

merging 6281/6312 to prod

README.md

@@ -1,5 +1,3 @@
-[![CircleCI](https://dl.circleci.com/status-badge/img/gh/berkeley-dsep-infra/datahub/tree/staging.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/berkeley-dsep-infra/datahub/tree/staging)
-
 # Berkeley JupyterHubs 
 
 Contains a fully reproducible configuration for JupyterHub on datahub.berkeley.edu,
@@ -9,6 +7,13 @@ as well as the single user images.
 
 [UC Berkeley CDSS](https://cdss.berkeley.edu)
 
+## Single-user server images
+All user images are located in their own repositories located in the
+[Berkeley DSEP infra repo](https://github.com/berkeley-dsep-infra).  You can
+find them either by [searching there](https://github.com/orgs/berkeley-dsep-infra/repositories?language=&q=image&sort=&type=all)
+or from links in the deployment's `image/README.md`
+([eg: Datahub's](https://github.com/berkeley-dsep-infra/datahub/tree/staging/deployments/datahub/image)).
+
 ## Branches
 
 The `staging` branch always reflects the state of the [staging JupyterHub](http://staging.datahub.berkeley.edu),
@@ -107,7 +112,7 @@ branch of this repo while the choice for `head` is your fork.
 
 Once this is complete and if there are no problems, you can request that
 someone review the PR before merging, or you can merge yourself if you are
-confident. This merge will trigger a CircleCI process which upgrades the
+confident. This merge will trigger a Github Actions workflow which upgrades the
 helm deployment on the staging site. When this is complete, test your
 changes there. For example if you updated a library, make sure that a new
 user server instance has the new version. If you spot any problems you can

deployments/biology/hubploy.yaml

@@ -1,5 +1,5 @@
 images:
-  image_name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/biology-user-image:c6aa3725c360
+  image_name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/biology-user-image:a81b11cbb998
 
 cluster:
   provider: gcloud

docs/.gitignore

@@ -0,0 +1,2 @@
+/.quarto/
+_site

docs/admins/howto/clusterswitch.qmd

@@ -10,8 +10,8 @@ You might find it easier to switch to a new cluster if you're running a [very ol
 1. Create a new cluster using the specified [configuration](../cluster-config.qmd).
 2. Set up helm on the cluster according to the instructions here:  
    http://z2jh.jupyter.org/en/latest/setup-helm.html
-     - Make sure the version of helm you're working with matches the version CircleCI is using.  
-       For example:  https://github.com/berkeley-dsep-infra/datahub/blob/staging/.circleci/config.yml#L169
+     - Make sure the version of helm you're working with matches the version Github Actions is using.
+       For example:  https://github.com/berkeley-dsep-infra/datahub/blob/staging/.github/workflows/deploy-support.yaml#L66
 3. Re-create all existing node pools for hubs, support and prometheus deployments in the new cluster.  If the old cluster is still up and running, you will probably run out of CPU quota, as the new node pools will immediately default to three nodes.  Wait ~15m for the new pools to wind down to zero, and then continue.
 
 ## Setting the 'context' for kubectl and work on the new cluster.
@@ -110,13 +110,21 @@ for x in $(cat hubs.txt); do hubploy deploy ${x} hub prod; done
 
 When done, add the modified configs to your feature branch (and again, don't push yet).
 
-## Update CircleCI
-Once you've successfully deployed the clusters manually via `hubploy`, it's time to update CircleCI to point to the new cluster.
+## Update Github Actions
+Once you've successfully deployed the clusters manually via `hubploy`, it's time to update the Github Actions to point to the new cluster.
 
-All you need to do is `grep` for the old cluster name in `.circleci/config.yaml` and change this to the name of the new cluster.  There should just be four entries:  two for the `gcloud get credentials <cluster-name>`, and two in comments.  Make these changes and add them to your existing feature branch, but don't commit yet.
+All you need to do is `grep` for the old cluster name in `.github/workflows/` and change this to the name of the new cluster.
+There should just be two entries, one each in the support and node placeholder deploy workflows.
+Make these changes and add them to your existing feature branch, but don't commit yet.
+
+```
+$ grep -ir spring .github/workflows
+.github/workflows/deploy-node-placeholder.yaml:            get-credentials spring-2024
+.github/workflows/deploy-support.yaml:            get-credentials spring-2024
+```
 
 ## Create and merge your PR!
-Now you can finally push your changes to github.  Create a PR, merge to `staging` and immediately kill off the deploy jobs for `node-placeholder`, `support` and `deploy`.
+Now you can finally push your changes to github.  Create a PR, merge to `staging` and immediately kill off the [deploy jobs](https://github.com/berkeley-dsep-infra/datahub/actions) for `node-placeholder`, `support` and `deploy`.
 
 Create another PR to merge to `prod` and that deploy should work just fine.
 

docs/admins/howto/delete-hub.qmd

@@ -28,8 +28,7 @@ gcloud filestore backups create <hubname>-backup-YYYY-MM-DD --file-share=shares
 ```
 4.  Log in to `nfsserver-01` and unmount filestore from nfsserver:
     `sudo umount /export/<hubname>-filestore`
-5.  Comment out the hub build steps out in `.circleci/config.yaml`
-    (deploy and build steps)
+5.  Comment out the hub's image repo entry (if applicable) in `scripts/user-image-management/repos.txt`
 6.  Comment out GitHub label action for this hub in
     `.github/labeler.yml`
 7.  Comment hub entries out of `datahub/node-placeholder/values.yaml`

docs/admins/howto/documentation.qmd

@@ -5,7 +5,7 @@ title: Documentation
 ## Overview
 
 Documentation is managed under the `docs/` folder, and is generated with
-[Quarto](https://quarto/). It is published to this site,
+[Quarto](https://quarto.org/). It is published to this site,
 <https://docs.datahub.berkeley.edu>, hosted at GitHub Pages. Content is written
 in [markdown](https://quarto.org/docs/authoring/markdown-basics.html).
 

docs/admins/howto/new-hub.qmd

@@ -35,7 +35,7 @@ Proper access to the following systems:
 
   -   Google Cloud IAM: *owner*
   -   Write access to the [datahub repo](https://github.com/berkeley-dsep-infra/datahub)
-  -   CircleCI account linked to our GitHub organization.
+  -   Owner or admin access to the [berkeley-dsep-infra organization](https://github.com/berkeley-dsep-infra/)
 
 ## Configuring a New Hub
 
@@ -247,71 +247,48 @@ bcourses.
 Please reach out to Jonathan Felder to set this up, or
 <[email protected]> if he is not available.
 
-### CircleCI
+### CI/CD and single-user server image
 
-The CircleCI configuration file `.circleci/config.yml` will need to
-include directives for building and deploying your new hub at several
-phases of the CircleCI process. Generally speaking, an adequate manual
-strategy for this is to pick the name of an existing hub, find each
-occurrence of that name, and add analogous entries for your new hub
-alongside your example existing hub. Please order new entries for your
-new hub in alphabetical order amongst the entries for existing hubs.
+CI/CD is managed through Github Actions, and the relevant workflows are located
+in `.github/workflows/`.  Deploying all hubs are managed via Pull Request
+Labels, which are applied automatically on PR creation.
 
-Here is a partial (but incomplete) sampling of some of the relevant
-sections of the CircleCI configuration file:
+To ensure the new hub is deployed, all that needs to be done is add a new entry
+(alphabetically) in `.github/labeler.yml` under the `# add hub-specific labels
+for deployment changes` stanza:
 
 ``` yaml
-- run:
-    name: Deploy <hubname>
-      command: |
-        hubploy deploy <hubname> hub ${CIRCLE_BRANCH}
-
-- hubploy/build-image:
-    deployment: <hubname>
-    name: <hubname> image build
-    filters:
-      branches:
-        ignore:
-          - staging
-          - prod  
-
-
-  - hubploy/build-image:
-      deployment:  <hubname>
-      name:  <hubname> image build
-      push: true
-      filters:
-        branches:
-          only:
-            - staging
-
-
-    -  <hubname> image build
+"hub: <hubname>":
+  - "deployments/<hubname>/**"
 ```
 
-Review hubploy.yaml file inside your project directory and update the
-images section. Example from a11y hub:
+#### Hubs using a custom single-user server image
 
-``` yaml
-images:
-  images:
-    - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/a11y-user-image
-      path: image/
-      repo2docker:
-        base_image: docker.io/library/buildpack-deps:jammy
-```
+If this hub will be using its own image, then follow the
+[instructions here](https://docs.datahub.berkeley.edu/admins/howto/new-image.html)
+to create the new image and repository.  In this case, the image tag will be
+`PLACEHOLDER` and will be updated AFTER your PR to datahub is merged.
+
+*NOTE:* The changes to the `datahub` repo are required to be merged BEFORE the new
+image configuration is pushed to `main` in the image repo.  This is due to
+the image building/pushing workflow requiring this deployment's
+`hubploy.yaml` to be present in the `deployments/<hubname>/` subdirectory, as
+it updates the image tag.
+
+#### Hubs inheriting an existing single-user server image
 
-### Add hub to the github labeler workflow
+If this hub will inherit an existing image, you can just copy `hubploy.yaml`
+from an existing deployment which will contain the latest image hash.
 
-The new hub will now need to be added to the github labeler workflow.
+#### Review the deployment's `hubploy.yaml`
 
-Edit the file `.github/labeler.yml` and add an entry for this hub
-(alphabetically) in the
-`# add hub-specific labels for deployment changes` block:
+Next, review `hubploy.yaml` inside your project directory to confirm that
+looks cromulent.  An example from the `a11y` hub:
 
 ``` yaml
-"hub: <hubname>":
-  - "deployments/<hubname>/**"
+images:
+  images:
+    - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/a11y-user-image:<image tag OR "PLACEHOLDER">
 ```
 
 ### Create placeholder node pool
@@ -399,25 +376,37 @@ events](calendar-scaler.qmd).
 This is useful for large courses which can have placeholder nodes set
 aside for predicatable periods of heavy ramp up.
 
-### Commit and deploy staging
+### Commit and deploy to `staging`
 
 Commit the hub directory, and make a PR to the the `staging` branch in
-the GitHub repo. Once tests pass, merge the PR to get a working staging
-hub! It might take a few minutes for HTTPS to work, but after that you
+the GitHub repo. 
+
+#### Hubs using a custom single-user server image
+
+If this hub is using a custom image, and you're using `PLACEHOLDER` for the
+image tag in `hubploy.yaml`, be sure to remove the hub-specific Github
+label that is automatically attached to this pull request.  It will look
+something like `hub: <hubname>`.  If you don't do this the deployment will
+fail as the image sha of `PLACEHOLDER` doesn't exist.
+
+After this PR is merged, perform the `git push` in your image repo.  This will
+trigger the workflow that builds the image, pushes it to the Artifact Registry,
+and finally creates a commit that updates the image hash in `hubploy.yaml` and
+pushes to the datahub repo.  Once this is merged in to `staging`, the
+deployment pipeline will run and your hub will finally be deployed.
+
+#### Hubs inheriting an existing single-user server image
+
+Your hub's deployment will proceed automatically through the CI/CD pipeline.
+
+It might take a few minutes for HTTPS to work, but after that you
 can log into it at <https://>\<hub_name\>-staging.datahub.berkeley.edu.
 Test it out and make sure things work as you think they should.
 
-1.  Make a PR from the `staging` branch to the `prod` branch. When this
-    PR is merged, it'll deploy the production hub. It might take a few
-    minutes for HTTPS to work, but after that you can log into it at
-    <https://>\<hub_name\>.datahub.berkeley.edu. Test it out and make
-    sure things work as you think they should.
-2.  You may want to customize the docker image for the hub based on your
-    unique requirements. Navigate to deployments/'Project Name'/image
-    and review environment.yml file and identify packages that you want
-    to add from the `conda repository` \<<https://anaconda.org/>\>. You
-    can copy the image manifest files from another deployment. It is
-    recommended to use a repo2docker-style image build, without a
-    Dockerfile, if possible. That format will probably serve as the
-    basis for self-service user-created images in the future.
-3.  All done.
+### Commit and deploy to `prod`
+
+Make a PR from the `staging` branch to the `prod` branch. When this
+PR is merged, it'll deploy the production hub. It might take a few
+minutes for HTTPS to work, but after that you can log into it at
+<https://>\<hub_name\>.datahub.berkeley.edu. Test it out and make
+sure things work as you think they should.