[vlc-devel] Bug 2 in VLC theora: theora_info.aspect and offset_y

[Kevin H. Patterson]

Hi,

I'm new to this list, so please forgive me if there's a better way to 
mention this...

I'm writing a complex streaming media server app and it includes a 
database-driven encoder module. The encoded files are in 
ogg+theora+vorbis format, and VLC seems to play them well (from disk or 
from the server via http), except for a few problems:

* Theora's pixel aspect ratio is used incorrectly when the file is 
masked (cropped). Consider the following (hypothetical) encoded file:

width=640, height=480
frame_width=640, frame_height=240
offset_x=0, offset_y=240
aspect_numerator=1, aspect_denominator=2

What is this? Say we want to store interlaced video in theora, in a 
backwards-compatible way. We separate the "odd" and "even" fields and 
store the odd field in the top half of the frame, and the even field in 
the bottom half. Then we encode using theora. We want to only show one 
of the two fields for "legacy" players, and make the file compatible and 
viewable for them:

So we mask (crop) off the top half of the picture (so the odd field is 
hidden), and make the pixel aspect ratio twice as high, so the picture 
(bottom half of the original frame) will be stretched vertically 2x, and 
appear at the correct aspect. This also has the effect of deinterlacing 
by discarding one field, and no changes have to be made to existing 
theora-compliant players to view the file. Custom players that 
understand the interlaced format can re-integrate the odd and even 
fields and ignore the crop settings. (cropping is accomplished in the 
encoded theora file by setting theora_info.frame_height and 
theora_info.offset_y)

The problem is that VLC is not handling the theora pixel aspect ratio 
correctly. VLC seems to take the size of the original (uncropped) frame, 
and through some calculation, resize it to match the theora pixel aspect 
ratio (theora_info.aspect_numerator and theora_info.aspect_denominator). 
This is OK if the frame is not cropped, BUT if the frame is cropped it 
is not correct. Observe:

What theora currently does is, it takes the original picture size and 
modifies it using the aspect ratio. So 640x480 becomes 640x960 (2:1) 
ratio. Then it crops the frame, and stretches it to fit the calculated 
size. So we have 640x240 (cropped) frame stretched to 640x960! (2x 
taller than it should be! and 4x taller than the original pixels.)
 
What VLC SHOULD be doing is this: Crop the frame first. 640x480 => 
640x240. Then use the pixel aspect ratio to stretch it (2:1) so, 640x240 
=> 640x480. (also remember that this is PIXEL aspect ratio, so a 640x480 
file (which is a 4:3 frame aspect ratio) actually has a 1:1 pixel aspect 
ratio in theora, because in a 640x480 4:3 image the pixels are SQUARE.)

I've read through the theora specs many times, and I'm pretty sure this 
is how it should work. Otherwise, your pixel aspect ratio becomes way 
off if any cropping is used in the frame. It allows me to do a nice 
backwards-compatible hack for interlaced video (even if theora isn't 
designed to support it and existing players don't understand it.)

I have some example files that I can put up on the web if anyone want to 
test/observe this effect.

-Kevin H. Patterson

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3245 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20060504/eba88e38/attachment.bin>