[x264-devel] Re: Scalability

[Alex Izvorski]

On Tue, 2007-02-27 at 19:27 +0100, Mathieu Monnier wrote:
> > The new threading code is indeed better than the old one, but x264 still
> > has problems scaling beyond 8 CPUs. Are there any known reasons for
> > that?
> 
> Yes. X264 encodes several frames in parallel. That means a frame is 
> encoded while its reference(s) aren't yet completely encoded. That works 
> because x264 checks for the current frame that the motion vector doesn't 
> go in the area of the reference frame that hasn't been encoded yet.
> 
> And that is done by waiting that at least a few row(s) of macroblock in 
> the reference frame is encoded before starting a new frame. The 
> consequence is that if your video has only 36 rows ( 576p ), you can 
> only have at most 36 concurrent threads ( and actually half less, since 
> you must at least wait 2 rows ).
> 
> So 576p won't benefit from more than 18 threads.


Mathieu - there is more to it than that.  Loren is the real threading
expert, but until he steps in, I'll explain what I understand.  There is
a certain minimal offset between the rows or macroblocks the threads for
dependent frames are working on, yes - it is set I believe by the
mvrange-thread option.  However, that says nothing about encoding
*independent* frames in parallel - the code is smart enough to do both
bframes in parallel (with an offset from the frames they depend on, as
usual) and frames after a future keyframe in parallel.  So just looking
at the available rows will grossly underestimate the available level of
parallelism.  In fact, except for memory use and encoding delay issues,
I do not believe there is an upper limit ;)

Update: I ran some tests with short keyint and large numbers of threads,
the results appear to indicate that threading independent frames across
keyframes does not work as suggested above.  I'm not entirely sure if it
is supposed to work in the current implementation.

Regards,
--Alex


-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html