[x265] [PATCH] slicetype: fix bug causing encoder crash in b-adapt 2 flow.

[Steve Borho]

On Jan 24, 2014, at 6:59 AM, Aarthi Priya Thirumalai <aarthi at multicorewareinc.com> wrote:

> 
> # HG changeset patch
> # User Aarthi Thirumalai
> # Date 1390567762 -19800
> #      Fri Jan 24 18:19:22 2014 +0530
> # Node ID 118235a34a09be7a1a3b15aee18a6d5c08a3a71d
> # Parent  807495b7a9fc918b78ac664b273e973f03668818
> slicetype: fix bug causing encoder crash in b-adapt 2 flow.
> 
> set initial value of best_cost to MAX_COST as frameCost will never be less than 0.
> 
> diff -r 807495b7a9fc -r 118235a34a09 source/encoder/slicetype.cpp
> --- a/source/encoder/slicetype.cpp	Fri Jan 24 11:50:23 2014 +0530
> +++ b/source/encoder/slicetype.cpp	Fri Jan 24 18:19:22 2014 +0530
> @@ -637,7 +637,7 @@
>  {
>      char paths[2][X265_LOOKAHEAD_MAX + 1];
>      int num_paths = X265_MIN(cfg->param.bframes + 1, length);
> -    uint64_t best_cost = (uint64_t)-1;
> +    uint64_t best_cost = (uint64_t)MotionEstimate::COST_MAX;

on the default branch, I think this logic was ok.  It was a problem on stable where these uint64_t's were changed into signed int64_t's.  We upgraded these frame cost variables to 64bits because 32bits was not enough for some 4k files, so the 32bit COST_MAX from MotionEstimate is not a safe default value for best_cost either.

I've fixed this by making the loop always take the cost in the first pass.

--
Steve