[x264-devel] -nan value for variable in x264?!

[BugMaster]

On Thu, 1 Mar 2012 16:51:51 +0800 (CST), 敏河 王 wrote:
> hi.

> I am working on x264 (Copyright (c) 2003-2008 x264 project) for
> Encrypting videos, and get a mistake unsolvable. Would you please give me some tips?

> What I have done is described as follow:

> 1. In function x264_macroblock_write_cavlc(...) , I change the code for I_4x4 pred from
>             bs_write( s, 4, i_mode - (i_mode > i_pred) );
> to
>             bs_write( s, 4, EncryptPred(i_mode - (i_mode > i_pred), 8, nFirst + i, nSec) );
> 2. The implement of function EncryptPred(...) is very simple:
>             static inline int EncryptPred(int nVal, int nMod, int nSeedFirst, int nSeedSec)
>             {return (nVal ^ GetKey(nMod, nSeedFirst, nSeedSec));}
> 3. Part of the function GetKey(...) is:
>             static int GetKey(int nMod, int nSeedFirst, int nSeedSec)
>             {
>                 const double LAMBDA = 3.987;
>                 const double SALT = 0.000321;
>                 const double FEEDBACK = g_fSeed;
>                 const int OFFSET = 100000000;

>                 /*double fPad = (double)nSeedFirst / ((double)nSeedSec + FEEDBACK);*/
>                 double fSeed = (double)nSeedFirst / ((double)nSeedSec + FEEDBACK);
>                 //...
>             }
> 4. Then comes the problem that I get a -nan value for fSeed when
> disable fPad, and get a right value when enable fPad. And it's a
> unsolvable problem for me. Wish your help.

> Thank you so much.
> Best regards.

As you use double/float arithmetic in your calculation than you
probably need to call x264_emms somewhere before that (you probably
can do it at the start of x264_macroblock_write_cavlc before you use
GetKey or inside it) because asm functions before that can break x87
state by using MMX.