[vlc-devel] [PATCH] livehttp - keep index file up to date when doing non-live transcoding on Windows

[Rob]

Hi Remi,

the point is to make the content useable before the whole transcode has  
finished.

at the moment on windows, the process for transcoding a movie is something  
like:

1) livehttp outputs first segment, and index.m3u8 referencing segment 1
2) outputs all the other segments, updating index.m3u8.tmp
(step 2 might take 30mins or so for a movie)
3) index.m3u8 is with the index.m3u8.tmp

This means that until step 3 - a client can't watch more than the first  
segment (because the index file only knows about segment 1).
In my streamer app, we work around this by custom handling in the http  
server so that if the user asks for index.m3u8 - they get index.m3u8.tmp
(there are hundreds of thousands of people using this already)

this of course completely destroys any advantage there ever was to keeping  
the .tmp file 'safe' from the webserver.

(incidentally - in the live streaming case, livehttp.c already writes  
directly to the index.m3u8 file.)

the patch gets rid of the facade of the .tmp file.

thoughts?

Rob



On Thu, 17 Feb 2011 17:36:43 -0000, Rémi Denis-Courmont <remi at remlab.net>  
wrote:

> Le jeudi 17 février 2011 19:11:01 Rob, vous avez écrit :
>> previously, the live http encoding made all changes after the first  
>> write
>> to index.m3u8 to index.m3u8.tmp when doing non-live encoding. This patch
>> gets rid of the .tmp and just keeps the index.m3u8 up to date. This  
>> makes
>> the stream viewable even when it isn't finished.
>
> The point of rename is to make the file update atomic, so that an HTTP  
> client
> will either get the old or the new M3U8 file, and never an incomplete  
> file.
>
>> this follows the author's comment with the initial livehttp.c
>> 'I'm not sure I am doing the right thing with the Win32 rename function.
>> Linux (I believe) allows me to rename a file over an existing file, even
>> if the existing file is in use.  Win32 is not so friendly.  This ability
>> is useful for updating the index file at same time it may be currently
>> being read by the HTTP server serving the files.'
>
> On Windows, vlc_rename() erases the old file before renaming. This is not
> atomic, but:
> - the window of opportunity for failure is still much smaller,
> - the client never gets an incomplete file; at worse, an error,
> - it does not break working POSIX systems for the sake of Windows.
>
> I don't really see the point of this patch.
>
>> http://techblog.unwiredappeal.com/2010/03/vlc-http-live-streaming-module-pa
>> tch.html it brings functionality in line with linux/mac.
>>
>> tested on winxp, win7