[vlc-commits] youtube.lua: work around VLC API limitations on very long lines

[Pierre Ynard]

vlc/vlc-3.0 | branch: master | Pierre Ynard <linkfanel at yahoo.fr> | Tue Sep  8 06:30:54 2020 +0200| [19987b81fc65bc225bf242c499ba71f3ea4bb902] | committer: Pierre Ynard

youtube.lua: work around VLC API limitations on very long lines

The main configuration line is such a very long line, and has been
growing longer recently, frequently hitting the VLC core limit at
200 kB. This caused readline() to fail to return any data, and stop
parsing of the web page short, preventing playback as the stream URLs
were in that line that was never returned.

Instead this relies on peek() and sized read() calls to parse and
recover that line by hand. This effectively bumps things up to up to
1 MB of usable configuration data.

Fixes #24957

(cherry picked from commit e3378ac3a892728bb8775d1bd243b31fa721ba49)
Signed-off-by: Pierre Ynard <linkfanel at yahoo.fr>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-3.0.git/?a=commit;h=19987b81fc65bc225bf242c499ba71f3ea4bb902
---

 share/lua/playlist/youtube.lua | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/share/lua/playlist/youtube.lua b/share/lua/playlist/youtube.lua
index 7b282bd6f1..b262d0ec64 100644
--- a/share/lua/playlist/youtube.lua
+++ b/share/lua/playlist/youtube.lua
@@ -314,11 +314,26 @@ function parse()
         -- fmt is the format of the video
         -- (cf. http://en.wikipedia.org/wiki/YouTube#Quality_and_formats)
         fmt = get_url_param( vlc.path, "fmt" )
-        local lastline
         while true do
             local line = vlc.readline()
             if not line then break end
-            lastline = line
+
+            -- The next line is the major configuration line that we need.
+            -- It is very long and readline() is likely to fail on it due
+            -- to #24957, so we need this instead.
+            if string.match( line, '<div id="player%-api">' ) then
+                if not vlc.peek( 1 ) then break end
+                local eol
+                local pos = 0
+                local len = 32768
+                repeat
+                    len = len * 2
+                    line = vlc.peek( len )
+                    eol = string.find( line, "\n", pos + 1 )
+                    pos = len
+                until eol or len >= 1024 * 1024
+                line = vlc.read( eol or len )
+            end
 
             -- Try to find the video's title
             if string.match( line, "<meta property=\"og:title\"" ) then
@@ -417,10 +432,6 @@ function parse()
             end
         end
 
-        if not path and lastline and string.match( lastline, '<div id="player%-api">' ) then
-            vlc.msg.err( "YouTube web page truncated at very long line, please check https://trac.videolan.org/vlc/ticket/24957 for updates to this issue" )
-        end
-
         if not path then
             local video_id = get_url_param( vlc.path, "v" )
             if video_id then
@@ -445,9 +456,9 @@ function parse()
         return { { path = path; name = name; description = description; artist = artist; arturl = arturl } }
 
     elseif string.match( vlc.path, "/get_video_info%?" ) then -- video info API
-        local line = vlc.readline() -- data is on one line only
+        local line = vlc.read( 1024*1024 ) -- data is on one line only
         if not line then
-            vlc.msg.err( "YouTube API output missing: probably rejected as very long line, please check https://trac.videolan.org/vlc/ticket/24957 for updates to this issue" )
+            vlc.msg.err( "YouTube API output missing" )
             return { }
         end