[x264-devel] Re: x264_center_filter_mmxext: Use of uninitialised value

[Mathieu Monnier]

Hi everyone,

Sorry for the late answer, but I haven't got time to investigate the 
issue at hand before today.

I'll explain more in details how the function works, and what i think is 
happening.

The function compute both vertical and diagonal hpel interpolation. It 
works one row at a time, and processes a row in two steps. First, it 
computes all the vertical values of the line, storing the results before 
division into a temporary buffer, and the result after division where it 
belongs, i.e in the vertical hpel plane. Then, once the whole line has 
been processed, the diagonal hpel value is computed from the data stored 
into the temporary buffer, and stored into the diagonal hpel plane.

So far, so good.

The temporary buffer is created on the stack ( beginning of the function 
: sub esp, 2 * stride ), and that adress is stored into [buffer], which 
is declared into a .data section.

The unitialized access happens when I access the data stored into the 
temporary buffer. Valgrind reports more than 30000 uninitialized 
accesses, which means all the pixels are in fact computed from values 
non initialized, which is rather strange, since the function does work. 
So i'd say that valgrind seems to have some trouble with the way i 
declared my buffer.

I must say that it's the first time i need to work with a temporary 
buffer of varaible size in pure asm, so my way of declaring it is 
perhaps not the good one. Since users are reporting issues with the way 
the .data section is accessed ( for example, somebody got that error 
with mingw / nasm on linux  : i586-mingw32msvc/bin/ld: 
libx264.a(mc-a2.o): bad reloc address 0x4a in section `.text'. , and 
others got issues with nasm 0.98.38 too ), i guess that that part too 
isn't following the 'good way' either. There again, I must acknowledge 
that I'm rather clueless on how to properly use the .data / .rodata 
section too.

For the buffer, I think I'll give to the function an additionnal 
parameter, which will be the buffer. I have yet to decide where to store 
the buffer ( i think i'll add a member to the x264_frame_t structure, 
which will point to a new member of the x264_t structure, what do you 
think of that ? ).

For the .data / .rodata / .section thing, I hope somebody who knows how 
assembler works will make the necessary modifications ( i did some 
googling, but found nothing )

Regards,

Mathieu

-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html