[vlc-devel] [PATCH 0/1] Improve SFTP network performance with buffer

[Ilkka Ollakka]

On Sun, Feb 18, 2018 at 04:23:17PM +0100, Martin Disch wrote:
> On Sun, Feb 18, 2018 at 12:48 PM, Ilkka Ollakka <ileoo at videolan.org> wrote:
> 
> > So it might be that the prefetch-filter starves for some reason, meaning buffer
> > is full and doesn't get into refilling when it's consumed in pace that would
> > prevent stalling the playback.
> >
> > If it would be possible to check if tuning the prefetch-buffer-size helps
> > instead of tuning the prefetch-read-size?
> 
> In the meantime, I was able to confirm that an increased prefetch read
> size directly affects the read size used in the SFTP module. That's no
> surprise, after all this is what should happen. With the default
> prefetch read size of 16384 bytes, that's the maximum amount of bytes
> that VLC asks the SFTP module to provide in the Read() function. With
> the prefetch read size increased to 262144 (which I found to work well
> on my system during testing), the SFTP module received requests for up
> to 262144 bytes and sometimes libssh2_sftp_read() was actually able to
> return that many in one call.

> This means that increasing the prefetch read size to a very high value
> has the same effect as my little sftp buffer hack, which makes the
> SFTP module request as many bytes from libssh2 as it can provide in
> one call without blocking the socket. Thus, it also explains why I get
> the same result (smooth playback through good bandwidth utilization)
> from both of these methods.

Hi,

I quickly checkend, and seems that libssh2 has same buffer-size (16k) as
what prefetch uses as default read-size, so might be that libssh2
handles some stuff in non-optimal way when in blocking mode on those
cases.

I tested quickly sftp access-module with non-blocking mode, but didn't
have time to test it excessively. For my simple test-case and debugging
measurement (sample size of 2-3 runs) it seemed to get data with
littlebit lower latency on non-blocking cases. If you want to test the
patch is hacked quickly like this:

diff --git a/modules/access/sftp.c b/modules/access/sftp.c
index 1219f573a1..c6939e570e 100644
--- a/modules/access/sftp.c
+++ b/modules/access/sftp.c
@@ -520,7 +520,14 @@ static ssize_t Read( stream_t *p_access, void *buf,
size_t len )
 {
     access_sys_t *p_sys = p_access->p_sys;
 
+    /* Set the socket in non-blocking mode */
+    libssh2_session_set_blocking( p_sys->ssh_session, 0 );
+
     ssize_t val = libssh2_sftp_read(  p_sys->file, buf, len );
+    if( val == LIBSSH2_ERROR_EAGAIN )
+    {
+        return -1;
+    }
     if( val < 0 )
     {
         msg_Err( p_access, "read failed" );

-- 
Ilkka Ollakka