[x265] source/common/intrapred.cpp -- C version of angular intra performs double transpose

[Matt Johnson]

(sorry to break this up into so many separate emails :( )

The behavior has been present since the all_angs function was created in 
source/common/IntraPred.cpp in revision 2456 (6/18/13).  At that time, 
the function was called xPredIntraAngs4().

I can't find a place where the horizontal mode predicted pixels are 
transposed a third time (to get the correct value), so it seems like an 
incorrect SATD value and so forth will be computed.

-Matt

On 12/13/2013 02:54 PM, Matt Johnson wrote:
> It also looks like the assembly/SSE variants must be doing the same
> thing, since commenting out the transpose in all_angs_pred_c() results
> in a failure in the TestBench binary.
>
> On 12/13/2013 01:40 PM, Matt Johnson wrote:
>> Hi all,
>>      It looks like the allangs C variant of angular intra prediction
>> transposes the predicted pixel matrix twice; the intra_pred_ang_c()
>> function, which does a single mode prediction, transposes the predicted
>> pixel matrix if necessary, then the wrapper all_angs_pred_c() transposes
>> it again.  It seems like the all_angs_pred_c() transpose should be
>> removed.
>>
>>      As a side note, should the planad_pred_c() templated function be
>> renamed planar_pred_c()?
>>
>> Thanks,
>> Matt
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel