[vlc-devel] [PATCH 3/3] access/sftp: include key-type in flag to libssh2_knownhost_check

[Filip Roséen]

> typemask is a bitmask that specifies format and info about the data
> passed to this function. Specifically, it details what format the
> host name is, what format the key is and what key type it is."

The above is the documentation associated with the 5th argument to
libssh2_knownhost_check available at the below resource:

  - https://www.libssh2.org/libssh2_knownhost_check.html

Given that the previous implementation did not properly supply the
flag associated with a certain key-type, libssh2_knownhost_check would
fail (since it compares the found key-type to the one set to look for).

--

See the below for further information:

    - https://github.com/libssh2/libssh2/blob/7b9a9a782d69c097f69a1992023bde9e4ead046f/src/knownhost.c#L444
---
 modules/access/sftp.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/modules/access/sftp.c b/modules/access/sftp.c
index d6cea71..7a48d6d 100644
--- a/modules/access/sftp.c
+++ b/modules/access/sftp.c
@@ -213,10 +213,29 @@ static int Open( vlc_object_t* p_this )
 
     const char *fingerprint = libssh2_session_hostkey( p_sys->ssh_session, &i_len, &i_type );
     struct libssh2_knownhost *host;
+    int knownhost_fingerprint_algo;
+
+    switch( i_type )
+    {
+        case LIBSSH2_HOSTKEY_TYPE_RSA:
+            knownhost_fingerprint_algo = LIBSSH2_KNOWNHOST_KEY_SSHRSA;
+            break;
+
+        case LIBSSH2_HOSTKEY_TYPE_DSS:
+            knownhost_fingerprint_algo = LIBSSH2_KNOWNHOST_KEY_SSHDSS;
+            break;
+
+        default:
+            msg_Err( p_access, "Host uses unrecognized session-key algorithm" );
+            goto error;
+
+    }
+
     int check = libssh2_knownhost_check( ssh_knownhosts, url.psz_host,
                                          fingerprint, i_len,
                                          LIBSSH2_KNOWNHOST_TYPE_PLAIN |
-                                         LIBSSH2_KNOWNHOST_KEYENC_RAW,
+                                         LIBSSH2_KNOWNHOST_KEYENC_RAW |
+                                         knownhost_fingerprint_algo,
                                          &host );
 
     libssh2_knownhost_free( ssh_knownhosts );
-- 
2.9.0