[x264-devel] Threading/parallel processing question

[tsuraan]

If I understand correctly, x264 currently does frame-based
multithreading, where work on adjacent frames is done in parallel with
frames being able to do motion vectors on the completed parts of
previous frames, and the not-yet-completed parts are hopefully not
relevant.

I was wondering, at least in the case of encoding a completed file to
h.264 (which I think is probably pretty common), would it be possible
to do parallel encoding of large chunks of the stream, such as the
chunks between iframes?  My thought is that it seems like it would be
possible to have a single thread read through the on-disk video stream
that's going to be re-encoded, find the frames that will be h.264
iframes, and record their positions.  Then, for each chunk of frames
between two iframes, a task is created to encode that region of the
video.  A final concatenation run would create the final product with
all the encoded chunks of video.

Is this something that makes sense for the h.264 codec?  How are
iframes chosen?  Are they time-based, or do they have to be chosen
during the creation of pframes, determined by accumulated error or
something?  I didn't have any luck looking through the source to
figure out what the process is for choosing them.

If the chunked approach is possible, I think it would be pretty simple
to create a clustered version of x264 for re-encoding video using
multiple machines in parallel.  Since I always use x264 to compress my
DVDs, and I have a lot of computers laying around, I think that would
be a pretty nice thing to have.

Any thoughts, corrections, whatever would be appreciated.  Thanks!