<!DOCTYPE html><html><head><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div><br></div><div><br></div><div>On Wed, Jul 31, 2019, at 08:57, Rémi Denis-Courmont wrote:<br></div><blockquote type="cite" id="qt"><div>Hi,<br></div><div><br></div><div>In my understanding, the filter input pictures are allocated by upstream as before. This already followed the push model from the beginning.<br></div><div><br></div><div>And filter output pictures are the responsibility of the filter to allocate. For preexisting in-place/destructive filters, that means no changes from before. For non-destructive filters, that means allocating in whatever way they see fit, with the potential use of the video context.<br></div><div><br></div><div>filter_NewPicture() should be just a compatibility alias for picture_NewFromFormat(). Hardware-driving filters should use custom allocators like VDPAU already does, IIRC.<br></div></blockquote><div><br></div><div>By the way, should we use a CPU picture pool in that case ? Even, if it can be slightly slower from my tests on Linux, using a pool won't cause performance/memory issues that are OS/runtime dependent.<br></div><div><br></div><div>If we want a CPU pool, should the filter decide the number of pictures from the beginning or should we use a dynamic pool (allocating a picture if there is no free pics in the pool) ?<br></div><div><br></div><blockquote type="cite" id="qt"><div><br></div><div>tl;dr: push model for picture allocation<br></div><div><br></div><div class="qt-gmail_quote"><div>Le 30 juillet 2019 23:40:48 GMT+03:00, Alexandre Janniaux <ajanni@videolabs.io> a écrit :<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:0pt;margin-left:0.8ex;border-left-color:rgb(204, 204, 204);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><pre class="qt-k9mail"><div>Hi,<br></div><div><br></div><div>On Tue, Jul 30, 2019 at 08:12:47PM +0300, Rémi Denis-Courmont wrote:<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(114, 159, 207);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div>Le tiistaina 30. heinäkuuta 2019, 10.14.17 EEST Steve Lhomme a écrit :<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div>On 2019-07-29 21:18, Rémi Denis-Courmont wrote:<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(138, 226, 52);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote">Le maanantaina 29. heinäkuuta 2019, 22.08.58 EEST Alexandre Janniaux a<br></blockquote></blockquote><div>écrit :<br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(138, 226, 52);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(252, 175, 62);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div>While I agree that we probably don't need flow control and that image<br></div><div>number is probably dominated, the starvation issue seems a real issue to<br></div><div>tackle. What are the solution against starvation on Windows ? Should we<br></div><div>just fail if we got starved ?<br></div></blockquote><div> In general, it is the halting problemm which is to say that the general<br></div><div> case cannot be solved and assumptions have to be made.<br></div><div><br></div><div> If we need to make assumptions, we might just as well assume that filters<br></div><div> are well-behaved, as indeed they have so far been. Ideally, a filter that<br></div><div> needs to allocate surfaces (rather than modify input surface in-place)<br></div><div> should allocate a suitably large pool for itself. And that turns out<br></div><div> incompatible with filter_NewPicture().<br></div></blockquote><div>I don't see how. This particular patch adds the possibility for filters<br></div><div>to provide their own (output) allocator. A filter could very well create<br></div><div>a pool and have its allocator pick pictures in that pool. That still<br></div><div>goes through filter_NewPicture, because the filter doesn't know if it<br></div><div>has to use pictures from the outside or it can use its own.<br></div></blockquote></blockquote><div><br></div><div>I don't really understand how can you use picture from the outside in a GPU<br></div><div>filter ? It doesn't seem to fit the push model too.<br></div><div><br></div><div>><br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(114, 159, 207);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div> That makes no sense. The "outside" cannot know if the filter allocates or not.<br></div><div> Already now, both cases have to work - it has more or less always been that<br></div><div> way with filters.<br></div><div><br></div><div> So far only converters could be assumed to allocate pictures - and in the push<br></div><div> model, that would be from the video context, not from the downstream/<br></div><div> filter_NewPicture().<br></div></blockquote><div><br></div><div>Just to clarify, do we agree that it has to allocate pictures thanks to the<br></div><div>video context, but not from it ? Meaning the video context can only provide a<br></div><div>rendering-like context for rendering-like API in the case of filter (be it GPU<br></div><div>or CPU, the CPU case being "allocate your memory that way (shm) or that way<br></div><div>(dynamic memory chunk)") and the memory is allocated by the filter through this<br></div><div>context with the rendering API, which should match the context ?<br></div><div><br></div><div>For instance, even if I'm not really fond of this model, it would be an openGL<br></div><div>context as a video context, and the filter would use glGenTextures ? Or would<br></div><div>the video context provide a `picture_t pf_AllocatePicture(some args)` ?<br></div><div><br></div><div>> ><br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(114, 159, 207);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(173, 127, 168);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(138, 226, 52);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div>Since the filter chain is (partially) dynamic, we cannot even rely on<br></div><div>filters telling how many "extra" surfaces they need - the total value<br></div><div>could change in the middle of the stream. Instead, the filter ought to<br></div><div>allocate its surfaces during initialization, and potentially refuse to<br></div><div>start if it cannot succeed - but not break the whole pipeline.<br></div></blockquote><div>I'm not sure we can estimate this amount easily. For example what would<br></div><div>be the amount an adjust filter has to allocate ?<br></div></blockquote><div> The adjust filter needs 0 surfaces since it operates in place.<br></div><div><br></div><div> Point being anyway that only the filter can know that, if anything. Since the<br></div><div> filter chain is dynamic, it cannot be taken into account when the decoder and<br></div><div> display are set up. It may be that even the filter does not know how many<br></div><div> pictures it needs, but then there is nothing to fix; it will just maybe or<br></div><div> maybe not work by allocating on the fly.<br></div></blockquote><div><br></div><div>I don't really agree, doing things in place seems to go against the usual<br></div><div>practice with filters. Or do we change this model to add copy filters ?<br></div><div><br></div><div>In the mean time, if filters are still processed synchronously, every filters<br></div><div>would need only a fixed number of pictures, or at least a number of pictures<br></div><div>known by the filter itself, except maybe the last one which have to take into<br></div><div>account the pictures allocated by the display. Copies can be made by the filters<br></div><div>itself if they absolutely need to keep a picture, and optimization could be made<br></div><div>so as to do late copies later only if the image is to be reused.<br></div><div><br></div><div>Remi, is the last-filter special-case what you mean with the fact that only<br></div><div>converters are allocating pictures or is it more complex than that ?<br></div><div><br></div><div>Regards,<br></div><div>--<br></div><div>Alexandre Janniaux<br></div><div>VideoLabs<br></div><div><br></div><div><br></div><div>><br></div><blockquote style="margin-top:0pt;margin-right:0pt;margin-bottom:1ex;margin-left:0.8ex;border-left-color:rgb(114, 159, 207);border-left-style:solid;border-left-width:1px;padding-left:1ex;" class="qt-gmail_quote"><div> --<br></div><div> レミ・デニ-クールモン<br></div><div> <a href="http://www.remlab.net/">http://www.remlab.net/</a><hr> vlc-devel mailing list<br></div><div> To unsubscribe or modify your subscription options:<br></div><div> <a href="https://mailman.videolan.org/listinfo/vlc-devel">https://mailman.videolan.org/listinfo/vlc-devel</a><br></div></blockquote><div><hr>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div><a href="https://mailman.videolan.org/listinfo/vlc-devel">https://mailman.videolan.org/listinfo/vlc-devel</a><br></div></pre></blockquote></div><div><br></div><div>-- <br></div><div>Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. <br></div><div>_______________________________________________<br></div><div>vlc-devel mailing list<br></div><div>To unsubscribe or modify your subscription options:<br></div><div>https://mailman.videolan.org/listinfo/vlc-devel<br></div></blockquote><div><br></div></body></html>