[vlc-commits] picture_pool: Fix potential undefined bitshift operation

[Hugo Beauzée-Luyssen]

vlc | branch: master | Hugo Beauzée-Luyssen <hugo at beauzee.fr> | Tue Feb 14 16:06:00 2017 +0100| [52e2cc6fd283a24baf04a63c0874a25560e715ab] | committer: Hugo Beauzée-Luyssen

picture_pool: Fix potential undefined bitshift operation

If the pool size is 64, then the bit shifting operation yields an
undefined behavior
CID #1402610

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=52e2cc6fd283a24baf04a63c0874a25560e715ab
---

 src/misc/picture_pool.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/src/misc/picture_pool.c b/src/misc/picture_pool.c
index 348d136..fce7dc1 100644
--- a/src/misc/picture_pool.c
+++ b/src/misc/picture_pool.c
@@ -127,7 +127,10 @@ picture_pool_t *picture_pool_NewExtended(const picture_pool_configuration_t *cfg
     pool->pic_unlock = cfg->unlock;
     vlc_mutex_init(&pool->lock);
     vlc_cond_init(&pool->wait);
-    pool->available = (1ULL << cfg->picture_count) - 1;
+    if (cfg->picture_count == pool_max)
+        pool->available = ~0ULL;
+    else
+        pool->available = (1ULL << cfg->picture_count) - 1;
     atomic_init(&pool->refs,  1);
     pool->picture_count = cfg->picture_count;
     memcpy(pool->picture, cfg->picture,