diff options
| author | Jingning Han <jingning@google.com> | 2014-07-24 15:31:32 -0700 |
|---|---|---|
| committer | Jingning Han <jingning@google.com> | 2014-07-24 15:35:51 -0700 |
| commit | 53844275e9e19d0af6f6e5bda98d884cf09aad5d (patch) | |
| tree | b5fbfd26fd0f4339e8ed27fc5a49a286dd70d223 /vp9/common | |
| parent | 7112d70f24719278c1f5e53d50636a1ce9bc552b (diff) | |
| download | libvpx-53844275e9e19d0af6f6e5bda98d884cf09aad5d.tar libvpx-53844275e9e19d0af6f6e5bda98d884cf09aad5d.tar.gz libvpx-53844275e9e19d0af6f6e5bda98d884cf09aad5d.tar.bz2 libvpx-53844275e9e19d0af6f6e5bda98d884cf09aad5d.zip | |
Fix potential ioc issue in vp9_get_prob for 4K above sizes
This commit turns on the existing vp9_get_prob function using
64 bit in the intermediate step. It fixes the ioc issue for 4K
above frame sizes (issue 828).
Change-Id: I9f627f3beca2c522f73b38fd2a3e7eefdff01a7c
Diffstat (limited to 'vp9/common')
| -rw-r--r-- | vp9/common/vp9_prob.h | 9 |
1 files changed, 0 insertions, 9 deletions
diff --git a/vp9/common/vp9_prob.h b/vp9/common/vp9_prob.h index f36148035..3920619d8 100644 --- a/vp9/common/vp9_prob.h +++ b/vp9/common/vp9_prob.h @@ -47,18 +47,9 @@ static INLINE vp9_prob clip_prob(int p) { return (p > 255) ? 255u : (p < 1) ? 1u : p; } -// int64 is not needed for normal frame level calculations. -// However when outputting entropy stats accumulated over many frames -// or even clips we can overflow int math. -#ifdef ENTROPY_STATS static INLINE vp9_prob get_prob(int num, int den) { return (den == 0) ? 128u : clip_prob(((int64_t)num * 256 + (den >> 1)) / den); } -#else -static INLINE vp9_prob get_prob(int num, int den) { - return (den == 0) ? 128u : clip_prob((num * 256 + (den >> 1)) / den); -} -#endif static INLINE vp9_prob get_binary_prob(int n0, int n1) { return get_prob(n0, n0 + n1); |