Jupytehrhub not able to talk to notebook running in the compute node (SLURM) #233
Comments
|
Were you able to eventually solve it? If so, how? |
|
no :-( |
|
Thanks for letting me know. In the half day since I asked, I found that the problem for me was that the batch nodes were trying to connect to the hub on the batch nodes themselves, rather than on the actual server where it is running. Setting the correct IP with |
|
what is the value, then? cause in my case it was the DNS alias without port/protocol. Maybe that's my issue, too. |
|
I don't have the port/protocol there either. In my case the hub_ip is simply the hostname in the same way that I would use to ssh there, namely As a separate setting, I have I suggest you try a separate install/configuration starting from zero and adding things one at the time. I found that often stuff that you think is unrelated instead has unintended consequences, so adding one thing at the time and trying helped me. |
|
I have added the port but it still does not work. Would you mind sharing your conf with me, please? |
|
Sure. I will do that later today. |
|
This is what I have Note the apparently useless |
|
Thanks. Seems quute like mine, maybe some fw... i hate proxies.
El dv., 29 de jul. 2022, 21:11, Davide ***@***.***> va
escriure:
… This is what I have
c.JupyterHub.port = 443
c.JupyterHub.proxy_class = 'jupyterhub_traefik_proxy.TraefikTomlProxy'
c.JupyterHub.spawner_class = 'batchspawner.SlurmSpawner'
import batchspawner
c.SlurmSpawner.req_prologue = """
hostname
source /home/sw/modules.sh
spack env activate JH
"""
c.JupyterHub.hub_ip = "co35svhead01"
c.JupyterHub.ssl_cert = 'cert.pem'
c.JupyterHub.ssl_key = 'key.pem'
c.Spawner.default_url = '/lab'
Note the apparently useless import batchspawner per their instructions
—
Reply to this email directly, view it on GitHub
<#233 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAYX2LS2FSUBKLE2HAUEABLVWQUEXANCNFSM5SFROYAA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Yes, it's a lot of moving parts and dependencies... I used traefik because I could not get |
|
@pelacables Did you get yours working? I'm having the same issue. worker notifies hub, but then nothing… |
|
Unfortunately not. Let me know if you manage to solve the issue, please. |
|
Hi, I believe to have encounter the same issue. I cannot make the process jupyterhub-singleuser to connect to the JupyterHub after it has been successfully spawned and is running. The batch process just times out and peacefully stops. I tried to connect to the Notebook directly in browser with the address given above just to be redirected to address I noticed that when the batch job is running, a connection to another randomly generated port appears. And the two ports do not match. Could this be the reason why the services do not communicate? |
|
I also see the 404, but what I do not see is the serevr talking to the client. That's why I did not pay attention to the 404. but at this point I'm not sure about anything :-) |
|
I was getting this same issue and #251 fixed it for me |
|
thanks ! that fixed the issue for me, too. |
Bug description
When I strart a server the jobs gets scheduled and successfully starts but the hub cannot talk to it.
Expected behaviour
The hub knows about the server running in the node and the server starts.
Actual behaviour
From the job logs:
From the hub logs:
How to reproduce
Request a job using the batchspawner.
Your personal set up
Configuration
Logs
Hub logs (148.191.97.103 resolves to computenode).
Job logs:
The text was updated successfully, but these errors were encountered: