Skip to content

Commit

Permalink
add code for gcp
Browse files Browse the repository at this point in the history
  • Loading branch information
eeholmes committed May 8, 2024
1 parent 21d1458 commit 63e8353
Show file tree
Hide file tree
Showing 5 changed files with 92 additions and 7 deletions.
8 changes: 8 additions & 0 deletions docs/posts/JHub-User-Guide.html
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ <h2 id="toc-title">On this page</h2>
<li><a href="#lets-choose-jupyterlab" id="toc-lets-choose-jupyterlab" class="nav-link" data-scroll-target="#lets-choose-jupyterlab">Let’s choose JupyterLab</a></li>
<li><a href="#stop-your-server" id="toc-stop-your-server" class="nav-link" data-scroll-target="#stop-your-server">Stop your server</a></li>
<li><a href="#using-your-own-conda-environment" id="toc-using-your-own-conda-environment" class="nav-link" data-scroll-target="#using-your-own-conda-environment">Using your own conda environment</a></li>
<li><a href="#trouble-shooting" id="toc-trouble-shooting" class="nav-link" data-scroll-target="#trouble-shooting">Trouble-shooting</a></li>
</ul>
<div class="toc-actions"><ul class="collapse"><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/edit/main/posts/JHub-User-Guide.Rmd" class="toc-action"><i class="bi bi-github"></i>Edit this page</a></li><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/issues/new" class="toc-action"><i class="bi empty"></i>Report an issue</a></li><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/blob/main/posts/JHub-User-Guide.Rmd" class="toc-action"><i class="bi empty"></i>View source</a></li></ul></div></nav>
</div>
Expand Down Expand Up @@ -207,6 +208,7 @@ <h1 class="title">JHub User Instructions</h1>
</header>


<p><a href="https://medium.com/@magstherdev/jupyterhub-on-kubernetes-c51953ae9ecd">Intro article</a></p>
<p>I have set us up a JupyterHub/RStudio cloud-computing hub on Azure. It’s on Kubernetes and will spin up VMs as needed. The VMs are not huge: 2CPU &amp; 8 Gig RAM.</p>
<p><a href="https://jhub.opensci.live/hub/login" class="uri">https://jhub.opensci.live/hub/login</a></p>
<p>Login authentication is via GitHub. Only members of the JHub GitHub team can log on. This is a testing environment. Contact Eli if you want to test it out.</p>
Expand Down Expand Up @@ -316,6 +318,12 @@ <h2 class="anchored" data-anchor-id="using-your-own-conda-environment">Using you
jupyter kernelspec remove myenv</code></pre>
<p>You can see the kernels with</p>
<pre><code>jupyter kernelspec list</code></pre>
</section>
<section id="trouble-shooting" class="level2">
<h2 class="anchored" data-anchor-id="trouble-shooting">Trouble-shooting</h2>
<ul>
<li>Debugging https://test-zerotojh.readthedocs.io/en/latest/administrator/debug.html</li>
</ul>


</section>
Expand Down
50 changes: 48 additions & 2 deletions docs/posts/set-up-jh-gcp.html
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ <h2 id="toc-title">On this page</h2>

<ul class="collapse">
<li><a href="#background" id="toc-background" class="nav-link active" data-scroll-target="#background">Background</a></li>
<li><a href="#just-show-me-the-code" id="toc-just-show-me-the-code" class="nav-link" data-scroll-target="#just-show-me-the-code">Just show me the code!</a></li>
<li><a href="#set-up-google-cloud" id="toc-set-up-google-cloud" class="nav-link" data-scroll-target="#set-up-google-cloud">Set-up Google Cloud</a></li>
<li><a href="#set-up-kubernetes-cluster" id="toc-set-up-kubernetes-cluster" class="nav-link" data-scroll-target="#set-up-kubernetes-cluster">Set-up Kubernetes cluster</a></li>
<li><a href="#install-helm-3" id="toc-install-helm-3" class="nav-link" data-scroll-target="#install-helm-3">install helm 3</a></li>
Expand All @@ -180,6 +181,7 @@ <h2 id="toc-title">On this page</h2>
<li><a href="#set-up-authentication" id="toc-set-up-authentication" class="nav-link" data-scroll-target="#set-up-authentication">Set up authentication</a></li>
<li><a href="#deleting-the-kubernetes-cluster" id="toc-deleting-the-kubernetes-cluster" class="nav-link" data-scroll-target="#deleting-the-kubernetes-cluster">Deleting the Kubernetes cluster</a></li>
<li><a href="#post-installation-checklist" id="toc-post-installation-checklist" class="nav-link" data-scroll-target="#post-installation-checklist">Post-installation checklist</a></li>
<li><a href="#just-the-code-to-set-up" id="toc-just-the-code-to-set-up" class="nav-link" data-scroll-target="#just-the-code-to-set-up">Just the code to set up</a></li>
</ul>
<div class="toc-actions"><ul class="collapse"><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/edit/main/posts/set-up-jh-gcp.Rmd" class="toc-action"><i class="bi bi-github"></i>Edit this page</a></li><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/issues/new" class="toc-action"><i class="bi empty"></i>Report an issue</a></li><li><a href="https://github.com/nmfs-opensci/nmfs-jhub/blob/main/posts/set-up-jh-gcp.Rmd" class="toc-action"><i class="bi empty"></i>View source</a></li></ul></div></nav>
</div>
Expand Down Expand Up @@ -219,6 +221,10 @@ <h2 class="anchored" data-anchor-id="background">Background</h2>
</ul>
<p>See examples of full <code>config.yaml</code> files in the <code>config</code> directory in the <a href="https://github.com/nmfs-opensci/nmfs-jhub">nmfs-opensci/nmfs-jhub</a> GitHub repo.</p>
</section>
<section id="just-show-me-the-code" class="level2">
<h2 class="anchored" data-anchor-id="just-show-me-the-code">Just show me the code!</h2>
<p>Scroll to the bottom see the short version just to get the JHub set-up.</p>
</section>
<section id="set-up-google-cloud" class="level2">
<h2 class="anchored" data-anchor-id="set-up-google-cloud">Set-up Google Cloud</h2>
<ol type="1">
Expand Down Expand Up @@ -284,9 +290,9 @@ <h2 class="anchored" data-anchor-id="install-jupyterhub">Install JupyterHub</h2>
</section>
<section id="connect-to-the-jupyterhub" class="level2">
<h2 class="anchored" data-anchor-id="connect-to-the-jupyterhub">Connect to the JupyterHub</h2>
<p>Now it is running. Let’s try connecting via IP address</p>
<p>Now it is running. Let’s try connecting via the External IP address</p>
<pre><code>kubectl --namespace jhubk8 get service proxy-public</code></pre>
<p>Will show you the public IP address. You should be able to go to that and log in with any username (no password).</p>
<p>It will show you the public (external) IP address. You should be able to go to that with <code>http://</code> and you will see the log in page. Note you will not be able to log in as we did not create any users in the hub node (where the JupyterHub is running).</p>
</section>
<section id="set-up-https" class="level2">
<h2 class="anchored" data-anchor-id="set-up-https">Set-up https</h2>
Expand Down Expand Up @@ -358,6 +364,46 @@ <h2 class="anchored" data-anchor-id="post-installation-checklist">Post-installat
<li>Verify an external IP is provided for the k8s Service proxy-public. <code>kubectl --namespace=jhubk8 get service proxy-public</code></li>
<li>If the external ip remains <pending>, diagnose with: <code>kubectl --namespace=jhubk8 describe service proxy-public</code></pending></li>
</ul>
</section>
<section id="just-the-code-to-set-up" class="level2">
<h2 class="anchored" data-anchor-id="just-the-code-to-set-up">Just the code to set up</h2>
<pre><code># Open a Cloud Shell and run these lines of code
gcloud container clusters create \
--machine-type n1-standard-2 \
--num-nodes 2 \
--zone us-west1-a \
--cluster-version latest \
jhub

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole=cluster-admin \
[email protected]

gcloud beta container node-pools create user-pool \
--machine-type n1-standard-2 \
--num-nodes 0 \
--enable-autoscaling \
--min-nodes 0 \
--max-nodes 3 \
--node-labels hub.jupyter.org/node-purpose=user \
--node-taints hub.jupyter.org_dedicated=user:NoSchedule \
--zone us-west1-a \
--preemptible \
--cluster jhub

curl https://raw.githubusercontent.com/helm/helm/HEAD/scripts/get-helm-3

helm repo add jupyterhub https://hub.jupyter.org/helm-chart/
helm repo update

helm upgrade --cleanup-on-fail \
--install jhub1 jupyterhub/jupyterhub \
--namespace jhubk8 \
--create-namespace \
--version=3.3.4 \
--values config.yaml

# DONE!!! Now edit config.yaml by looking at some of the examples in the config directory in this repo.</code></pre>


</section>
Expand Down
34 changes: 32 additions & 2 deletions docs/search.json
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,16 @@
"Set-up JHub on Google Cloud"
]
},
{
"objectID": "posts/set-up-jh-gcp.html#just-show-me-the-code",
"href": "posts/set-up-jh-gcp.html#just-show-me-the-code",
"title": "Set up JupyterHub on GCP",
"section": "Just show me the code!",
"text": "Just show me the code!\nScroll to the bottom see the short version just to get the JHub set-up.",
"crumbs": [
"Set-up JHub on Google Cloud"
]
},
{
"objectID": "posts/set-up-jh-gcp.html#set-up-google-cloud",
"href": "posts/set-up-jh-gcp.html#set-up-google-cloud",
Expand Down Expand Up @@ -174,7 +184,7 @@
"href": "posts/set-up-jh-gcp.html#connect-to-the-jupyterhub",
"title": "Set up JupyterHub on GCP",
"section": "Connect to the JupyterHub",
"text": "Connect to the JupyterHub\nNow it is running. Let’s try connecting via IP address\nkubectl --namespace jhubk8 get service proxy-public\nWill show you the public IP address. You should be able to go to that and log in with any username (no password).",
"text": "Connect to the JupyterHub\nNow it is running. Let’s try connecting via the External IP address\nkubectl --namespace jhubk8 get service proxy-public\nIt will show you the public (external) IP address. You should be able to go to that with http:// and you will see the log in page. Note you will not be able to log in as we did not create any users in the hub node (where the JupyterHub is running).",
"crumbs": [
"Set-up JHub on Google Cloud"
]
Expand Down Expand Up @@ -219,6 +229,16 @@
"Set-up JHub on Google Cloud"
]
},
{
"objectID": "posts/set-up-jh-gcp.html#just-the-code-to-set-up",
"href": "posts/set-up-jh-gcp.html#just-the-code-to-set-up",
"title": "Set up JupyterHub on GCP",
"section": "Just the code to set up",
"text": "Just the code to set up\n# Open a Cloud Shell and run these lines of code\ngcloud container clusters create \\\n --machine-type n1-standard-2 \\\n --num-nodes 2 \\\n --zone us-west1-a \\\n --cluster-version latest \\\n jhub\n\nkubectl create clusterrolebinding cluster-admin-binding \\\n --clusterrole=cluster-admin \\\n [email protected]\n\ngcloud beta container node-pools create user-pool \\\n --machine-type n1-standard-2 \\\n --num-nodes 0 \\\n --enable-autoscaling \\\n --min-nodes 0 \\\n --max-nodes 3 \\\n --node-labels hub.jupyter.org/node-purpose=user \\\n --node-taints hub.jupyter.org_dedicated=user:NoSchedule \\\n --zone us-west1-a \\\n --preemptible \\\n --cluster jhub\n\ncurl https://raw.githubusercontent.com/helm/helm/HEAD/scripts/get-helm-3\n\nhelm repo add jupyterhub https://hub.jupyter.org/helm-chart/\nhelm repo update\n\nhelm upgrade --cleanup-on-fail \\\n --install jhub1 jupyterhub/jupyterhub \\\n --namespace jhubk8 \\\n --create-namespace \\\n --version=3.3.4 \\\n --values config.yaml\n\n# DONE!!! Now edit config.yaml by looking at some of the examples in the config directory in this repo.",
"crumbs": [
"Set-up JHub on Google Cloud"
]
},
{
"objectID": "posts/Setup-Notes.html",
"href": "posts/Setup-Notes.html",
Expand Down Expand Up @@ -431,7 +451,7 @@
"href": "posts/JHub-User-Guide.html",
"title": "JHub User Instructions",
"section": "",
"text": "I have set us up a JupyterHub/RStudio cloud-computing hub on Azure. It’s on Kubernetes and will spin up VMs as needed. The VMs are not huge: 2CPU & 8 Gig RAM.\nhttps://jhub.opensci.live/hub/login\nLogin authentication is via GitHub. Only members of the JHub GitHub team can log on. This is a testing environment. Contact Eli if you want to test it out.\nInstructions: It should be pretty self-explanatory.",
"text": "Intro article\nI have set us up a JupyterHub/RStudio cloud-computing hub on Azure. It’s on Kubernetes and will spin up VMs as needed. The VMs are not huge: 2CPU & 8 Gig RAM.\nhttps://jhub.opensci.live/hub/login\nLogin authentication is via GitHub. Only members of the JHub GitHub team can log on. This is a testing environment. Contact Eli if you want to test it out.\nInstructions: It should be pretty self-explanatory.",
"crumbs": [
"JHub User Guide"
]
Expand Down Expand Up @@ -506,6 +526,16 @@
"JHub User Guide"
]
},
{
"objectID": "posts/JHub-User-Guide.html#trouble-shooting",
"href": "posts/JHub-User-Guide.html#trouble-shooting",
"title": "JHub User Instructions",
"section": "Trouble-shooting",
"text": "Trouble-shooting\n\nDebugging https://test-zerotojh.readthedocs.io/en/latest/administrator/debug.html",
"crumbs": [
"JHub User Guide"
]
},
{
"objectID": "posts/Set-up-centos-tljh.html",
"href": "posts/Set-up-centos-tljh.html",
Expand Down
6 changes: 3 additions & 3 deletions docs/sitemap.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/set-up-jh-gcp.html</loc>
<lastmod>2024-03-31T16:10:36.737Z</lastmod>
<lastmod>2024-05-08T17:13:58.184Z</lastmod>
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/Setup-Notes.html</loc>
Expand All @@ -22,15 +22,15 @@
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/JHub-User-Guide.html</loc>
<lastmod>2024-04-13T00:03:11.894Z</lastmod>
<lastmod>2024-04-13T21:40:30.764Z</lastmod>
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/Set-up-centos-tljh.html</loc>
<lastmod>2024-03-31T15:55:59.619Z</lastmod>
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/Set-up-daskhub.html</loc>
<lastmod>2024-03-31T16:02:35.659Z</lastmod>
<lastmod>2024-04-14T00:36:04.527Z</lastmod>
</url>
<url>
<loc>https://nmfs-opensci.github.io/nmfs-jhub/posts/set-up-authentication.html</loc>
Expand Down
1 change: 1 addition & 0 deletions posts/set-up-jh-gcp.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -232,5 +232,6 @@ helm upgrade --cleanup-on-fail \
--values config.yaml
# DONE!!! Now edit config.yaml by looking at some of the examples in the config directory in this repo.
```


0 comments on commit 63e8353

Please sign in to comment.