[vlc-devel] [PATCH] check for crazy values in rtsp npt range

[Tobias Güntner]

Am 25.03.2012 09:21, schrieb Tobias Güntner:
> * Perhaps checking if a value >= INT64_MAX (instead of > INT64_MAX) is
> sufficient?

I'm reasonably sure this will work.

* If INT64_MAX is rounded towards infinity, it is rounded to the nearest 
representable value, i.e., there is no double between the actual 
INT64_MAX and (double)INT64_MAX. Therefore (double)INT64_MAX is the very 
first "invalid" value which must be replaced.

* If INT64_MAX has an exact representation, replacing INT64_MAX with 
INT64_MAX is obviously correct. ;)

* If INT64_MAX is rounded towards zero (and I doubt this will ever 
happen in practice), the rounded double value is still within INT64 
limits, i.e., the cast is safe.

(Analogously for INT64_MIN, INT32_MIN/MAX, etc.)

I attached a patch and a separate test program (because it was easier to 
test this way).

Regards,
Tobias
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Improve-range-check.patch
Type: text/x-patch
Size: 1078 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20120330/be2774bc/attachment.bin>
-------------- next part --------------
#undef NDEBUG
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <limits.h>
#include <math.h>
#include <fenv.h>
#include <assert.h>
#include <float.h>

/** Safely truncates floating point values to integers */
int64_t vlc_trunc64( double value )
{
    /* Use >= instead of > because INT64_MAX could be rounded
     * up when converted to a double for comparison. */
    if( value >= INT64_MAX )
        return INT64_MAX;
    else if( value <= INT64_MIN )
        return INT64_MIN;
    else if( isnan( value ) )
        return 0;
    else
        return (int64_t)value;
}

#if !defined(math_errhandling) || !( math_errhandling & MATH_ERREXCEPT )
#error This test requires floating point exceptions!
#endif

int64_t test_truncate( double value )
{
    feclearexcept( FE_ALL_EXCEPT );
    int64_t i = vlc_trunc64( value );
    int exc = fetestexcept( FE_ALL_EXCEPT );
    printf( "%f -> %"PRIi64" %i\n", value, i, exc );
    assert( ( exc & ~FE_INEXACT ) == 0 );
    return i;
}

#define testx( value, result ) \
    assert( test_truncate( value ) result )

#define test( value, result ) \
    testx( value, == ( result ) )

int main()
{
    double max = INT64_MAX;
    double min = INT64_MIN;

    /* make sure errors can be detected */
    feclearexcept( FE_ALL_EXCEPT );
    assert( !fetestexcept( FE_ALL_EXCEPT ) );
    int64_t i = (int64_t)( max - min );
    assert( fetestexcept( FE_INVALID ) );
    printf( "%"PRIi64"\n", i );

    assert( +DBL_MAX > INT64_MAX );
    assert( -DBL_MAX < INT64_MIN );

    test( +INFINITY, INT64_MAX );
    test( -INFINITY, INT64_MIN );

    test( +HUGE_VAL, INT64_MAX );
    test( -HUGE_VAL, INT64_MIN );

    test( max * 2, INT64_MAX );
    test( min * 2, INT64_MIN );

    test( nextafter( max, +INFINITY ), INT64_MAX );
    test( nextafter( min, -INFINITY ), INT64_MIN );
    
    test( max, INT64_MAX );
    test( min, INT64_MIN );

    testx( nextafter( max, 0 ), < INT64_MAX );
    testx( nextafter( min, 0 ), > INT64_MIN );

    test( +INT64_C(0x4000000000000000), +INT64_C(0x4000000000000000) );
    test( -INT64_C(0x4000000000000000), -INT64_C(0x4000000000000000) );

    test( +123.456, +123 );
    test( -123.456, -123 );

    test( +0.999999, 0 );
    test( -0.999999, 0 );

    test( +0.0, 0 );
    test( -0.0, 0 );

    test( +NAN, 0 );
    test( -NAN, 0 );

    printf("OK!\n");
    return 0;
}