Use random local ports when doing automatic port forwarding or reuse existing tunnels

[andreyorst]

I often have to connect to a remote REPL running behind an SSH. CIDER has this custom option nrepl-use-ssh-fallback-for-remote-hosts which I relied on for quite a long time until recently.

The problem is when I use cider-connect-clj, and give it a remote host and port it will open an SSH tunnel to that host and port, which by itself is fine unless you want to connect to the same port from multiple projects, which I often have to do. It is problematic because CIDER creates a port forwarding from the given port to the same port, e.g. from localhost:1234 to user@remote:1234. Thus, when I go into the next project and try to do the same thing, CIDER tries to set up port forwarding from 1234 again, but it is already taken.

There are two possible solutions, I think:

1. Use a random local port, e.g. when a user says M-x cider-connect-clj RET user@host RET 1234 RET, bind localhost:46573 to user@host:1234 instead of localhost:1234. It is similar to how CIDER currently works when using cider-jack-in-clj.
2. Reuse existing tunnels - check if CIDER already set up a port forwarding for this particular port, and don't try to bind it again.

As an alternative, I've been using a custom ssh-tunnel function I've made for myself:

(defvar-local ssh-tunnel-port nil)
(put 'ssh-tunnel-port 'safe-local-variable #'numberp)

(defun ssh-tunnel (host port &optional local-port)
  "Start an SSH tunnel from localhost to HOST:PORT.
If LOCAL-PORT is nil, PORT is used as local port."
  (interactive (list (read-string "host: " nil 'ssh-host-history)
                     (read-number "port: " ssh-tunnel-port 'ssh-port-history)
                     (when current-prefix-arg
                       (read-number "local port: " ssh-tunnel-port 'ssh-port-history))))
  (let ((name (if (and local-port (not (= local-port port)))
                  (format "*ssh-tunnel:%s:%s:%s" local-port host port)
                (format "*ssh-tunnel:%s:%s" host port))))
    (async-shell-command
     (format "ssh -4 -N -L %s:localhost:%s %s" (or local-port port) port host)
     (concat " " name))))

It's also handy to have the ability to fix a specific remote port in a dir local variable, like I do above, so I could connect to a specific port immediately, instead of remembering what port this particular service uses every time.

[vemv]

Hi @andreyorst, thanks for the detailed issue!

I'd favor 1. because it would imply that CIDER has less state to keep track of - every connection would be independent can can be closed without having to worry about its neighbors.

(oftentimes, choosing port 0 results in a random port being chosen for us)

Would you be interested in opening a PR?

[vemv]

instead of remembering what port this particular service uses every time.

Probably if we gave 'contexts' as described here #3400 this problem wouldn't exist

[andreyorst]

Would you be interested in opening a PR?

I can do that but probably a bit later

[vemv]

You can run (nrepl--ssh-tunnel-command (executable-find "ssh") "/ssh:[email protected]#12345:" "1234") to see the ssh command that we produce, namely:

 "/usr/bin/ssh -v -N -L 1234:localhost:1234 -l 'cider-devs' '192.168.50.9' -p '12345' "

Apparently, specifying 0:localhost:1234 would work. Inspecting the -voutput would tell us the chosen port.

Maybe we don't even need to keep track of what port was chosen - maybe it would just work, and every REPL process would get its own ssh tunnel with a different local port.

I'd appreciate if you could try it out at some point.

Cheers - V

[vemv]

Hi @andreyorst , did you have the chance to try port 0?

[andreyorst]

not yet, but hopefully this week. Renovation is a huge timesink :)

[andreyorst]

@vemv sorry it took so long, the end of the year is a busy time

Unfortunately the -L switch doesn't allow for using 0:localhost:remote-port. I've tried the recipe and indeed it fails.

So I tried to use this kind of trick:

(defun nrepl--ssh-tunnel-command (ssh dir port)
  "Command string to open SSH tunnel to the host associated with DIR's PORT."
  (with-parsed-tramp-file-name dir v
    ;; this abuses the -v option for ssh to get output when the port
    ;; forwarding is set up, which is used to synchronise on, so that
    ;; the port forwarding is up when we try to connect.
    (format-spec
     "%s -v -N -L %l:localhost:%p %u'%h' %n"
     `((?s . ,ssh)
       (?l . ,(+ 1024 (random 54511)))
       (?p . ,port)
       (?h . ,v-host)
       (?u . ,(if v-user (format "-l '%s' " v-user) ""))
       (?n . ,(if v-port (format "-p '%s' " v-port) ""))))))

It works but is unreliable, as the port is randomly chosen and not checked to be available. Perhaps, we can simply loop until the valid port pops up, this is suggested on the internet, and I couldn't find any other way to do it. Or maybe there's a Elisp function to get a free local port, but I didn't dig deeper into this yet.