Skip to content

Ssh command deadlock with a bad host connection #688

Description

@aniknaemmm
using (var command = client.CreateCommand(CommandText))
{
command.CommandTimeout = TimeSpan.FromMinutes(2);
command.Execute();
}

I have problems with this code. Sometimes a thread with this part of the code may hang. I began to research this problem and found that the problem happens when I had a bad connection with the host (about 60-70 pocket loss).

Deep in the library code, I found this method with infinite wait time:

 private static int TrySocketRead(Socket socket, byte[] buffer, int offset, int length)
{
return SocketAbstraction.Read(socket, buffer, offset, length, InfiniteTimeSpan);
}

And inside of this method in SocketAbstraction.cs

public static int Read(Socket socket, byte[] buffer, int offset, int size, TimeSpan timeout)
{
var totalBytesRead = 0;
var totalBytesToRead = size;
socket.ReceiveTimeout = (int) timeout.TotalMilliseconds;
do
{
try
{
var bytesRead = socket.Receive(buffer, offset + totalBytesRead, totalBytesToRead - totalBytesRead, SocketFlags.None);
if (bytesRead == 0)
return 0;
totalBytesRead += bytesRead;
}

We will have socket.Recive() call with infinity timeout.

Therefore, in case of a bad connection, we will hang in the Receive method, and in another thread, we will call Dispose of SshCommand and hang when _socketReadLock is locked because this lock object is used when we call TrySocketRead in Sesseion.cs

private bool IsSocketConnected()
{
lock (_socketDisposeLock)
{
#if FEATURE_SOCKET_POLL
if (!_socket.IsConnected())
{
return false;
}
lock (_socketReadLock)
{
var connectionClosedOrDataAvailable = _socket.Poll(0, SelectMode.SelectRead);
return !(connectionClosedOrDataAvailable && _socket.Available == 0);
}
#else
return _socket.IsConnected();
#endif // FEATURE_SOCKET_POLL
}
}

I found 2 workarounds for this case:

  1. We can set a timeout for this method from Session.cs
 private static int TrySocketRead(Socket socket, byte[] buffer, int offset, int length)
{
return SocketAbstraction.Read(socket, buffer, offset, length, InfiniteTimeSpan);
}
  1. We can disable FEATURE_SOCKET_POLL

What do you think about this?
Can you explain why FEATURE_SOCKET_POLL is necessary or what is bad if disable FEATURE_SOCKET_POLL?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions