[vlc-devel] You should test with Guard Malloc on Mac OS X

[Rippit the Ogg Frog]

Greetings, I've been lurking for eons without actually ever posting.

I just signed up at your Trac bug database as user Rippit.  I have a 
repeatible crash on the Macintosh that I'd like to report - could 
someone validate my account?

Apple's Xcode development system comes with a memory stress test tool 
called Guard Malloc.  I'd like to recommend that you test your Mac 
builds thoroughly under Guard Malloc - you'll make a lot of crashes 
immediate and completely repeatible, that would otherwise be hard to find.

Not just my crash...

If you're running VLC from within the Xcode IDE, select Enable Guard 
Malloc from the Debug menu, then run VLC under the debugger.

If you're running the GNU Debugger from within the Terminal, give GDB 
the following command before you run:

    set env DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib

Note that "set env" is TWO words when used within GDB.

If you're running from the shell prompt, give the following command to 
bash just before you run:

    export DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib

There are some options you can set with other environment variables - 
see "man libgmalloc".

But note that the MALLOC_STRICT_SIZE option will cause trouble for code 
within OS X itself; I was advised on Apple's Xcode mailing list not to 
use it with GUI applications.

What MALLOC_STRICT_SIZE does is place new allocations immediately at the 
end of a virtual memory page, with the page just after being unmapped. 
A buffer overrun of even just one byte will cause a crash, even if 
you're just reading the memory and not writing.

Unfortunately, it can cause allocations to be unaligned; some of Apple's 
code assumes that certain allocations are always aligned, so the 
unaligned allocs introduce bugs that won't be VLC's fault.

In any case, you don't need to use MALLOC_STRICT_SIZE to get lots of 
crashes from VLC - just run Guard Malloc with the default options - you 
shouldn't need to set any options explicitly.

I hope this helps...

-- 
Michael D. Crawford
aka Rippit the Ogg Frog
rippit at oggfrog.com
http://www.oggfrog.com/