<!DOCTYPE html><html><head><title></title><style type="text/css">
p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>Here is an idea to improve our caching system:<br></div><div><br></div><div>Demuxers should tell the stream (via a control) if they are reading for playback or probing/seeking.<br></div><div>If the demuxer is reading for playback, then the cache (prefetch) should always read the maximum.<br></div><div>If the demuxer is reading for probing, then the cache should read only what is requested (or cache a little like cache_read).<br></div><div><br></div><div>We could even use cache_read/block for probing and prefetch for playback.<br></div><div>That way, we can benefits from the best of all.<br></div><div><br></div><div>Is this idea valid ?<br></div><div><br></div><div>On Thu, Feb 6, 2020, at 15:02, Rémi Denis-Courmont wrote:<br></div><blockquote type="cite" id="qt"><div>Hi,<br></div><div><br></div><div>I don't follow how that's a protocol rather than implementation issue. And yes, reducing the read size would reopen 21470, 19806 and 19988 - depending on round-trip time. I don't have anything to add to what's already been written on Trac and here.<br></div><div><br></div><div class="qt-gmail_quote"><div>Le 6 février 2020 15:03:44 GMT+02:00, Thomas Guillem <thomas@gllm.fr> a écrit :<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div>On Thu, Feb 6, 2020, at 13:47, Rémi Denis-Courmont wrote:<br></div><blockquote id="qt-qt" type="cite"><div>Hi,<br></div><div><br></div><div>A plugin should wait for data if it has none available, and return whatever data is available up to the requested length.<br></div></blockquote><div><br></div><div>OK, I guess we should put that on streams.h documentation.<br></div><div><br></div><blockquote id="qt-qt" type="cite"><div><br></div><div>Of course if SMB2 behaves as I suspect it does, capping the read size will break streaming high rate media on high RTT networks... But it's necessary to stream media on low bandwidth networks. That's why I wrote that you've already lost if you care about the read size.<br></div></blockquote><div><br></div><div>This seems to be a limitation of the protocol, not our libs (libsmb2 or libsmbclient), cf. <a href="https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/4801c3e5-dd73-4e1c-b2ba-8ebf73642227">https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/4801c3e5-dd73-4e1c-b2ba-8ebf73642227</a><br></div><div>The client has to send read requests of a fixed size. This make impossible to behave as you said.<br></div><div><br></div><div>Still, there is something I don't understand, your commits (the 3.0 one) say it fix #19988 and #21470, that are SMB bugs. Putting a read cap in smb2.c like you said will just make these bugs reappear, no ?<br></div><div><br></div><blockquote id="qt-qt" type="cite"><div><br></div><div class="qt-qt-gmail_quote"><div>Le 6 février 2020 14:36:18 GMT+02:00, Thomas Guillem <thomas@gllm.fr> a écrit :<br></div><blockquote class="qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><div><br></div><div>On Thu, Feb 6, 2020, at 13:22, Rémi Denis-Courmont wrote:<br></div><blockquote type="cite" id="qt-qt-qt"><div>Hi,<br></div><div><br></div><div>I don't think SMB streaming (as opposed to downloading) is a lost cause. I do think SMB streaming with libsmbclient implementation is a lost cause.<br></div><div><br></div><div>The prefetch plugin already gives maximum flexibility, within its own constraints, to the underlying access to choose how many bytes it returns in a given read, as does the core.<br></div></blockquote><div><br></div><div>Then, we have to update the vlc_stream.h pf_read() documentation. Nothing says that access modules *should* (and not could) return less than the requested size in some case.<br></div><div><br></div><div>I still don't know how I should cap the read size for the smb2 access plugin.<br></div><div>If I cap the smb2 read size using the same old value, it will just re-open <a href="https://trac.videolan.org/vlc/ticket/19988">https://trac.videolan.org/vlc/ticket/19988</a> and <a href="https://trac.videolan.org/vlc/ticket/21470">https://trac.videolan.org/vlc/ticket/21470</a> ?<br></div><div><br></div><div>Or I missed something ?<br></div><div><br></div><blockquote type="cite" id="qt-qt-qt"><div><br></div><div>It can't get any better without increasing the prefetch buffer size (and we have a setting for that already).<br></div></blockquote><div><br></div><div><br></div><div><br></div><blockquote type="cite" id="qt-qt-qt"><div><br></div><div class="qt-qt-qt-gmail_quote"><div>Le 6 février 2020 14:00:55 GMT+02:00, Thomas Guillem <thomas@gllm.fr> a écrit :<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-qt-qt-gmail_quote"><div>I quote you:<br></div><div><br></div><div> "Badly behaving streams wait for the whole requested amount and do not<br></div><div> perform any pipelining/buffering (e.g. CIFS, probably SFTP)."<br></div><div><br></div><div>Why SMB, NFS, SFTP are badly behaving streams ? For me it is just like local I/O. You read only what you request.<br></div><div><br></div><div>On Thu, Feb 6, 2020, at 12:28, Rémi Denis-Courmont wrote:<br></div><blockquote id="qt-qt-qt-qt" type="cite"><div>Hi,<br></div><div><br></div><div>The access is free to return any non-zero size up to the requested read size, so the requested read size can never be too big. It can only ever be too small, and that's exactly why this patch made sense and still makes sense.<br></div></blockquote><div><br></div><div>At the VideoLabs office with 1TB network, I don't care, but at home with a shitty 2MB wifi, reading chunks by 16MB is terrible. Specially when 99% is trashed since the demuxer will only ask few kB and seek.<br></div><div><br></div><blockquote id="qt-qt-qt-qt" type="cite"><div><br></div><div>But if you need to worry about the read size, you've already lost.<br></div></blockquote><div><br></div><div>Why am I lost ? Are you saying SMB streaming is a lost cause ? <br></div><div><br></div><blockquote id="qt-qt-qt-qt" type="cite"><div>Small read size will cause bandwidth problems and large read size will cause latency problems.<br></div></blockquote><div><br></div><div>Can we find a solution for both cases in the core or the cache modules ? I really don't see why I should cap the read size of all "badly behaving" streams.<br></div><div>If I need to do that, what size should I use ? Should I add one more option (likely the same that you removed in prefetch) ?<br></div><div><br></div><blockquote id="qt-qt-qt-qt" type="cite"><div><br></div><div class="qt-qt-qt-qt-gmail_quote"><div>Le 6 février 2020 12:33:55 GMT+02:00, Thomas Guillem <thomas@gllm.fr> a écrit :<br></div><blockquote class="qt-qt-qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><pre class="qt-qt-qt-qt-k9mail"><div>Hello,<br></div><div><br></div><div>This commit and this one on vlc-3.0 : <a href="https://git.videolan.org/?p=vlc/vlc-3.0.git;a=commit;h=3dfa6d4ddd0922552e6575ded416f62b8ce828f5">https://git.videolan.org/?p=vlc/vlc-3.0.git;a=commit;h=3dfa6d4ddd0922552e6575ded416f62b8ce828f5</a> are making the smb2 module ultra slow, around 10 times slower.<br></div><div><br></div><div>The read size requested by prefetch is really big. This make probing / seek ultra slow since a lot of small read() are executed (reproduced with mkv and mp4).<br></div><div><br></div><div>So, I have one big question: Is the smb2 module supposed to not listen to the size argument and always read a smaller size ?<br></div><div>How can I decide the read size then ? Should this size be decided by the core of cache modules instead (like before) ?<br></div><div><br></div><div>PS: There is not the same problem with the libdsm module since the max read size is caped inside libdsm.<br></div><div><br></div><div>On Fri, Apr 20, 2018, at 15:45, Thomas Guillem wrote:<br></div><div>> <br></div><blockquote class="qt-qt-qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(114, 159, 207);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><div>On Fri, Apr 20, 2018, at 15:05, Rémi Denis-Courmont wrote:<br></div><blockquote class="qt-qt-qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><div>And I don't really want to hear a commit is dangerous from somebody that <br></div><div>backports stuff as lightly and quickly as you have lately.<br></div></blockquote><div>That is a bit close to false accusation IMHO.<br></div><div><br></div><blockquote class="qt-qt-qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;">This was sent for review, and it was *not* backported.<br></blockquote><div>Yes and I was OK with it. It's just lately that I found out that this <br></div><div>commit might be dangerous.<br></div><div>Dangerous as changing the behavior of some access modules and maybe <br></div><div>induce some new bugs (that are access related).<br></div><div><br></div><blockquote class="qt-qt-qt-qt-gmail_quote" style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;"><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser <br></div><div>ma brièveté.<hr>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div><a href="https://mailman.videolan.org/listinfo/vlc-devel">https://mailman.videolan.org/listinfo/vlc-devel</a><br></div></blockquote><div><hr>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div><a href="https://mailman.videolan.org/listinfo/vlc-devel">https://mailman.videolan.org/listinfo/vlc-devel</a><br></div></blockquote><div><hr>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div><a href="https://mailman.videolan.org/listinfo/vlc-devel">https://mailman.videolan.org/listinfo/vlc-devel</a><br></div></pre></blockquote></div><div><br></div><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. <br></div><div>_______________________________________________<br></div><div>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div>https://mailman.videolan.org/listinfo/vlc-devel<br></div></blockquote><div><br></div></blockquote></div><div><br></div><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. <br></div><div>_______________________________________________<br></div><div>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div>https://mailman.videolan.org/listinfo/vlc-devel<br></div></blockquote><div><br></div></blockquote></div><div><br></div><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. <br></div><div>_______________________________________________<br></div><div>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div>https://mailman.videolan.org/listinfo/vlc-devel<br></div></blockquote><div><br></div></blockquote></div><div><br></div><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. <br></div><div>_______________________________________________<br></div><div>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div>https://mailman.videolan.org/listinfo/vlc-devel<br></div></blockquote><div><br></div></body></html>