[x264-devel] Problem on transform_bypass

[Jason Garrett-Glaser]

On Mon, Oct 18, 2010 at 8:16 AM, Jin Lei <jinleicm at yahoo.com.cn> wrote:
> Dear all:
>
>         I want to apply transfrom_bypass mode to a part of macroblocks in a
> video(with several sequences concluded), but x264 only support compressing
> all the macroblocks of all frames in lossless mode. Therefore, I make some
> modification on x264. I try a few methods only to get a yuv output that is
> decoded incorrectly. I use JM17.2 and ffmpeg（the version supports high444)
> to test my encoded videos. Both of them cannot give the output expected.
>
>        The methods I tried are:
>        1. set h->sps->b_qpprime_y_zero_transform_bypass=1 in
> x264_sps_init(set.c).
>            set set h->mb.i_qp=0 at the end of
> x264_macroblock_analyse(analyse.c), if some conditions are meet.
>            And according to the values of the variables metioned above(
> if(h->sps->b_qpprime_y_zero_transform_bypass==1&&set h->mb.i_qp==0)), it
> is   judged that whether h->mb.b_lossless=1 or not, just after the statement
> "h->mb.i_qp=0"(analyse.c).
>
>
>        2. set h->sps->b_qpprime_y_zero_transform_bypass=1 in
> x264_sps_init(set.c).
>            set h->mb.i_qp=0 in x264_macroblock_analyse before
> x264_mb_analyse_init(analyse.c), if some conditions are meet.
>            The x264 program contains serveral statements like
> if(h->mb.b_lossless) x264_predict_lossless_x*x() or  if(h->mb.b_lossless)
> then the operations of DCT and quantization are skipped(these results can be
> get by seaching reference in vs2005). I add the condition that
> "||(h->mb.i_qp==0 && h->sps->b_qpprime_y_zero_transform_bypass==1" into
> these IFs.
>
>       Both of the methods above cannot get the correct result.
>
>       Would you please give me some ideas of the right way of achieving the
> good results.
>       Thanks a lot!

I've done this in the past: basically you have to vary
h->mb.b_lossless on a per-macroblock basis.

DO NOTE the comment in encoder/macroblock.c next to the prediction
functions!  You will have to slightly modify them, as they assume that
the neighboring macroblocks are also lossless.

Dark Shikari