Have server open a file descriptor as a unix socket for systemd socket activation.

[TomHodson]

Problem

I would like to start a t server using socket activation. The way that works is that systemd listens on a socket or port and when it gets a connection on it, systemd starts the target process.

Annoyingly, what systemd wants the process to do is then open file descriptor (fd) 3 (depending on configuration) with something like:

sock = socket.fromfd(fd, socket.AF_UNIX, socket.SOCK_STREAM) 

So the gist of this feature request is: a way to tell Jupyter server I want it to open a specific fd as a unix socket and listen on it.

Proposed Solution

If you run Jupyter lab —sock /dev/fd/3 and trace the code, execution goes through two places:

jupyter_server/serverapp.2663

sock = bind_unix_socket(self.sock, mode=int(self.sock_mode.encode(), 8))

Here self.sock == “/dev/fd/3” and bind_unix_socket is a function imported from tornado.netutils.

The relevant section of tornado/netutil.py:204 looks like:

        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        try:
            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        except OSError as e:
            if errno_from_exception(e) != errno.ENOPROTOOPT:
                # Hurd doesn't support SO_REUSEADDR
                raise
        sock.setblocking(False)
        # File names comprising of an initial null-byte denote an abstract
        # namespace, on Linux, and therefore are not subject to file system
        # orientated processing.
        if not file.startswith("\0"):
            try:
                st = os.stat(file)
            except FileNotFoundError:
                pass
            else:
                if stat.S_ISSOCK(st.st_mode):
                    os.remove(file)
                else:
                    raise ValueError("File %s exists and is not a socket", file)
            sock.bind(file)
            os.chmod(file, mode)
        else:
            sock.bind(file)
        sock.listen(backlog)
        return sock

What I want to do can be achieved by identifying at one of those two levels that the user is asking to open a file descriptor and creating the socket like this instead:

sock = socket.fromfd(fd, socket.AF_UNIX, socket.SOCK_STREAM) 

See https://docs.python.org/3/library/socket.html#socket.fromfd

NB: doing socket.bind(“/dev/fd/3”) doesn’t seem to work sadly.

I’ve currently hacked it together to work by inserting an extra if statement in tornado/netutil.py to see if the user has passed a string like “/dev/fd/{int}” and then using socket.fromfd

My questions are:

1. Is there scope for this feature in Jupyter server if I wrote the PR for it? I am conscious of the additional maintenance and documentation burden imposed by each additional feature so would understand a no here.

2. If yes, how should the user indicate to Jupyer that they wish it to listen on a file descriptor, a separate cmd line argument like —file-descriptor 3 or by passing something like —sock /dev/fd/3 or perhaps just—sock 3. I’m not sure if there is an existing convention to denote listening on a file descriptor.

3. You could argue this should instead be an issue/PR on tornado, which I appreciate, however the format would still need to pass through Jupyter before getting to bind_unix_socket so I thought I’d ask for opinions here first.

[TomHodson]

I just found out that uvicorn uses —fd as an alternative to —host —port and —uds so perhaps I could write a PR that adds a —fd argument to Jupyter that causes it to open the given file descriptor. The logic could go somewhere in serverapp.py