[vlc-commits] linux: vlc_memfd: fallback to POSIX implementation

[Thomas Guillem]

vlc | branch: master | Thomas Guillem <thomas at gllm.fr> | Fri May  3 11:33:40 2019 +0200| [3f5661b9444ff1f6287ae4b22b2c48d46a7a2b94] | committer: Thomas Guillem

linux: vlc_memfd: fallback to POSIX implementation

>From man (2) open:

O_TMPFILE  requires  support by the underlying filesystem; only a subset of
Linux filesystems provide that support.  In the initial implementation, support
was provided in  the  ext2,  ext3,  ext4,  UDF, Minix,  and shmem filesystems.
Support for other filesystems has subsequently been added as follows: XFS
(Linux 3.15); Btrfs (Linux 3.16); F2FS (Linux 3.16); and ubifs (Linux 4.9)

One must check for two different error codes, EISDIR and ENOENT, when trying to
determine whether the kernel supports O_TMPFILE functionality.

Fixes #22250

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=3f5661b9444ff1f6287ae4b22b2c48d46a7a2b94
---

 src/linux/filesystem.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/src/linux/filesystem.c b/src/linux/filesystem.c
index 6634c3d374..caa97fbf3f 100644
--- a/src/linux/filesystem.c
+++ b/src/linux/filesystem.c
@@ -36,11 +36,22 @@
 
 int vlc_memfd(void)
 {
+    int fd;
 #ifdef HAVE_MEMFD_CREATE
-    int fd = memfd_create(PACKAGE_NAME"-memfd",
-                          MFD_CLOEXEC | MFD_ALLOW_SEALING);
+    fd = memfd_create(PACKAGE_NAME"-memfd", MFD_CLOEXEC | MFD_ALLOW_SEALING);
     if (fd != -1 || errno != ENOSYS)
         return fd;
 #endif
-    return open("/tmp", O_RDWR | O_CLOEXEC | O_TMPFILE, S_IRUSR | S_IWUSR);
+
+    /* Fallback to open with O_TMPFILE, */
+    fd = open("/tmp", O_RDWR | O_CLOEXEC | O_TMPFILE, S_IRUSR | S_IWUSR);
+    if (fd != -1 || (errno != EISDIR && errno != ENOENT))
+        return fd;
+
+    /* Fallback to POSIX implementation if O_TMPFILE is not supported */
+    char bufpath[] = "/tmp/"PACKAGE_NAME"XXXXXX";
+    fd = vlc_mkstemp(bufpath);
+    if (fd != -1)
+        unlink(bufpath);
+    return fd;
 }