[vlc-devel] [PATCH 5a/5] stream_ReadLine: control maximum length with object variable

[Pierre Ynard]

> And the caller can retain many line buffers around, e.g. the subtitle
> filter.

Okay, I think I get it now. Good point, thanks. It sounds like subtitles
could indeed use a lower limit: can you even print 200 kB of text over
a video?... Really, the subtitle parser has the knowledge to manage
this: it knows how many subtitle lines it's currently handling, for how
much time each of them needs to be kept around, and I guess also the
expectable displayable length because it knows the font size... Even
dynamically, if under memory pressure, it could crank down the limit on
what it accepts. So for all that it makes more sense if it can control
the limit itself.

But even if you put forward valid ways how many lines will be kept
around, it doesn't mean that everything is passed down and stored
indiscriminately. As I mentioned, the caller has the knowledge and
opportunity to exercise discretion. Subtitle lines may get parsed,
and edited to format them; so it's not like it's always the raw lines
returned by stream_ReadLine() that get passed down untouched.

The caller is also the one who knows the number and lifecycle of lines
kept around. And it varies indeed. For playlist parsers, most of the
lines will yield a significant portion of various pieces and fields,
that will get stored into the playlist, maybe permanently. For subtitle
parsers, most lines will retain a comparable length and will be kept
around for a few seconds, a few of them at a time. For lua playlist
parsers, 99% of the lines will be thrown out right away, and only small
portions of the relevant ones will be kept. So again, we don't have
consistent behavior patterns, and to me it sure looks like each user
should be able to set suited limits to what it knows best.

But finally, maybe this is all somewhat moot and still in an order
of magnitude that remains a non-issue; as I was saying about the
performance and weight of web pages... have you seen many playlist or
subtitle files that weighed 16 MB? And then times several lines?...

> If so, you're welcome to fix it. That's the code that hasn't seen
> revectoring in the longest of the entire stream.c file.
>
> You will note that this makes the function ever more certainly
> quadratic in complexity,

Not really if the search position is preserved across iterations. The
size increase can also be made exponential instead of linear.

> and imposes on memory allocation within the generic stream
> (non-line-based) code, so you will have to the line size even lower
> than it already is...

We can lower the line length limit as low as we want once it's added as
a parameter to the API.

-- 
Pierre Ynard
"Une âme dans un corps, c'est comme un dessin sur une feuille de papier."