Socket timeout is ignored when changing directory

[cd-ppi]

I'm calling ChannelSftp.cd(Path) on a session that has a timeout of five seconds, then make sure that the SFTP-Server does not answer, but also does not break the connection. The execution of the client hangs in the read loop of the PipedInputStream:
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Object.java:-1)
at java.io.PipedInputStream.read(PipedInputStream.java:326)
at java.io.PipedInputStream.read(PipedInputStream.java:377)
at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2866)
at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2894)
at com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2149)
at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:331)
at my.App.changeDirViaSftp(SftpProvider.java:621)

(You mostly catch it in the wait if you pause the thread, but it really loops the whole time). It seems that it can't be easily helped, since the PipedInputStream does not implement any timeout.

I would expect the timeout to have some effect

[norrisjeremy]

Hi @cd-ppi,

I believe the socket timeout should cause the connection to fail assuming the server actually stops responding with any TCP packets completely.
Can you describe what exactly you are doing to make sure that the SFTP-Server does not answer, but also does not break the connection?

Thanks,
Jeremy

[cd-ppi]

Hi Jeremy,
I'm starting the SFTP-Server in debugger mode and let it run into a breakpoint, so that it won't send anything back.
It's been a while, but I think I remember that the connection does indeed break after the timeout, however this is not propagated through the PipedInputStream.
Thanks for looking into it!
Curd

[norrisjeremy]

Hi @cd-ppi,

Can you provide a step-by-step set of instructions for how we can reproduce this locally?
Otherwise it will be difficult for us to investigate & try to remediate.

Thanks,
Jeremy

[cd-ppi]

Hi @norrisjeremy,

what server do you run tests against? I was using Apache Mina SSHD to have a Java application that I can pause with the debugger.

Thanks again,
Curd

[norrisjeremy]

Hi @cd-ppi,

We have a wide battery of integration tests, that for the most part focus on testing against OpenSSH.
That said: I'm not sure why you think stopping Apache Mina SSHD in a debugger should induce socket timeouts on the connection from JSch.
Stopping it in the debugger only prevents the Apache Mina SSHD from sending any responses over the socket back to JSch, but the operating system will still hold the TCP socket open and respond to traditional TCP keepalives, and thus the socket timeout would never be triggered.

If you want to ensure SSH application level continuity over the socket, I suggest you research the Session.setServerAliveInterval() and Session.setServerAliveCountMax() methods in JSch.
These should instruct JSch to periodically send SSH session messages over the socket to confirm the server is actually responsive (at a periodic interval, aka ServerAliveInterval), and if more than X consecutive messages are sent without a response (aka ServerAliveCountMax), then JSch should cause the session to then fail.

Thanks,
Jeremy

[cd-ppi]

Hi @norrisjeremy,

thank you for the explanation. I just checked by wiretapping the traffic with WireShark, and I can confirm that you were right and that the way I disabled the SFTP-Server did not prevent it from answering to keep alive messages. So, the socket stays open, and the socket timeout has no reason to trigger. My idea of how my problem is caused obviously was wrong.

My problem being: We are encountering a situation where the cd hangs. Might be a fault in the server, might be something wrong on the network, however the thread is looping in the methods mentioned above - I think. I don't have any evidence from the 'real world' case, only from my tries at recreating it.

In another attempt, I made my test server drop the response to package type 16. Then again, cd is looping forever, so simular, even if the stacktrace up to the read on the PipedInputStream is different. Using Session.setServerAliveInterval() and Session.setServerAliveCountMax() does not seem to change that. I understand that this is not an issue with the socket timeout, but is there any other way to avoid the cd being stuck forever?

Thank you very much,

Curd

[norrisjeremy]

Hi @cd-ppi,

Unfortunately, I'm not sure how easily we will be able to help to diagnose and fix the issue you are experiencing with hangs when sending a cd command.

Unless you can document an easy method in which we can locally reproduce the issue (and thus then be able to try and debug it locally), or you can debug it yourself and better identify the source of the hang and propose a potential fix for it, this issue will likely just languish unfortunately.

Thanks,
Jeremy