[x264-devel] Trouble keeping 8 cores busy

[Ben Rosenblum]

Hi All,

About 18 months ago I ran into the same issue on my server (Dual X5355 
Quad-Core Xeon's).  After some playing around by Loren we determined 
that it was being caused by the way x264 handles its threading and not 
by the encode itself.  We found that the time it took x264 to create a 
thread and then subsequently destroy the thread once it was done doing 
its business was killing a significant amount of time that could be 
better spent actually doing something useful.  To that extent he created 
the thread pool patch.  Instead of creating and destroying threads it 
simply created all of the necessary threads at the beginning and just 
kept using the same ones over and over.  The last diff I have that 
applied cleanly was on revision 860 or so (don't remember the exact 
number but its close to that).  The ultimate results was that on dual 
cores there was somewhere between no result and a slight slow down, quad 
cores showed very minimal gains and octa-cores showed significant 
gains.  Because of the fact that there was the slow downs on the dual 
cores Loren chose not to apply the patch to the repository. 

If you want to read through the 11 pages of posts and all the logs from 
the testing I did you can find them all at:
http://forum.doom9.org/showthread.php?t=124557

If someone were to take one of the older diff's and adjust it to apply 
cleanly now I would bet that you would still notice a significant gain 
on an octa-core system.  To that point it probably would show even more 
of a gain now that x264 has been further optimized since we did our 
testing.  I'd also be curious if it were possibly to adjust the patch  
in such a way so that you could have a --thread-pool switch or something 
that would allow people with the higher core systems to enable this if 
they want. 

Hope that helps a bit...

-Ben Rosenblum