[vlc-devel] Decoder.c memory leak patch

[Mark Moriarty]

Hi --
I submitted an e-mail earlier today, not sure if it made it through to
vlc-devel properly.

I traced the memory leak I am seeing using the screen: input function (and
probably the one I saw earlier with my Osprey DirectShow input device on a
slower CPU).

What's occuring is that if the raw device input is feeding data faster than
can be processed, memory use balloons.  Need to adjust function
input_DecoderDecode, in \src\input\decoder.c to do a release_Block on input
packets if the FIFO isn't being processed.  Otherwise VLC consumes all
system memory.

I tried the following, using a slow (800MHz PIII) PC with 1280x1024 screen:
input at 5 fps.  CPU utilization still pegged, but the memory leak
disappeared -- no leak over 20 minutes when without the patch it would
consume > 600 MB over a few minutes. VLC's max ram use stayed reasonable,
and dropped back properly when I stopped the screen: input.

I'd appreciate it if someone with SVN privileges could take a look at the
suggested change, test and submit.  No idea if limiting i_depth to 10 is
"reasonable", perhaps VOUT_MAX_PICTURES (8) would be a better thing to use.
The patch consists of the 8 deeply-indented lines in the following code
snippet.

The possible change I suggested yesterday is unnecessary if this change is
implemented, I believe.  This change definitely stops the gross leak seen in
Windows.

Thanks.

void input_DecoderDecode( decoder_t * p_dec, block_t *p_block )
{
    if( p_dec->p_owner->b_own_thread )
    {
		    //MFM -- patch for memory leak if FIFO not being
processed fast enough
               if (p_dec->p_owner->p_fifo->i_depth > 10)
		   {
	            block_Release( p_block );
		   }
		   else
               {
        block_FifoPut( p_dec->p_owner->p_fifo, p_block );

        if( p_dec->p_owner->p_input->b_out_pace_control )
        {
            /* FIXME !!!!! */
            while( !p_dec->b_die && !p_dec->b_error &&
                   p_dec->p_owner->p_fifo->i_depth > 10 )
            {
                msleep( 1000 );
            }
        }
               }



-----Original Message-----
From: vlc-devel-bounce at videolan.org [mailto:vlc-devel-bounce at videolan.org]
On Behalf Of Mark Moriarty
Sent: Tuesday, July 27, 2004 7:16 AM
To: vlc-devel at videolan.org
Subject: [vlc-devel] Memory leaks

In Src\misc\block.c
Function block_FifoPut

Variable i_depth already keeps track of blocks allocated, just no test to
see if it's getting large.

Suggest updating it to:
    int i_size = 0;
		if (p_fifo->i_depth < 60)
		{
    vlc_mutex_lock( &p_fifo->lock );
	......
    vlc_mutex_unlock( &p_fifo->lock );
		}

The "60" isn't a magic number, but putting some kind of limit would keep it
from expanding to infinity Unfortunately, this will not clear the memory
growth I'm seeing; there's either an alternate buffer being filled or
something else is allocating memory without freeing it.

Block_Release gets called in block_FifoEmpty, but perhaps there is something
else needed in decoderdecode of decode.c?

 

-----Original Message-----
From: vlc-devel-bounce at videolan.org [mailto:vlc-devel-bounce at videolan.org]
On Behalf Of Gildas Bazin
Sent: Monday, July 26, 2004 11:28 AM
To: vlc-devel at videolan.org
Subject: [vlc-devel] Re: vlc: svn commit r8284 (gbazin)

On Monday 26 July 2004 16:13, Mark Moriarty wrote:
>  WinXP, fully patched, current SVN build, 2.6GHz Intel P4.
> I tried using the screen: function, UDP output.
> High CPU utilization, variable but generally stayiung between 45 - 80%.
>

I did the win32 part on Linux so couldn't test it but I did today and the
performance is indeed abysmal. That partly comes from the fact that I've
been using the GDI api and this one totally locks the display while a frame
is being grabbed.
I think I'll investigate doing this with directx. It should be a lot
quicker.

> Very rapid, very large, memory leak -- hundreds of megabytes growth 
> per minute.
> 
> This is similar to what I saw with an Osprey framegrabber in a slower 
> 800 MHz Win2K PC, a separate thread series.
> 
> Is there some chance that there is a "marshmallow mountain" effect, 
> that
if
> frames are being served to the ffmpeg CODEC engine faster than they 
> can be processed a queue, just gets larger and larger?
> 

Yes, this is what happens.
The input thread is only putting data in a fifo that the decoder / stream
output thread empties as it processes the data. So if the decoder thread
doesn't process the data quickly enough, the fifo will grow. We could have a
mechanism to limit the size of the fifo but that's not been done yet.

> Anyhow, for Windows screen, perhaps an option to decimate the input, a 
> straight /2 in both width and height, would cut the input by a factor 
> of
4,
> perhaps help a large amount.
> 
> The GPL'd RealVNC does pretty well on Windows, screen 
> capture/manipulation functions in it might map well to VLC, handles 
> different screen bits-per-pixel, and I believe has things related to 
> multi-monitor implementations (they've had several years experience, a 
> lot of lessons-learned).  The VNC windows source code germane to this 
> is in sdisplay and deviceframebuffer.
> 

I doubt I'll be able to re-use the VNC code because they are likely working
by only catching regions that have been updated. This would be way too much
work/code for VLC and I'm not sure of the benefit because the most CPU
intensive task here will be the encoding anyway. I'll have a quick look at
their code anyway.

--
Gildas

--
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/ To
unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>

--
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/ To
unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>

-- 
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
To unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>