diff options
author | paulwilkins <paulwilkins@google.com> | 2016-09-01 10:01:33 +0100 |
---|---|---|
committer | paulwilkins <paulwilkins@google.com> | 2016-09-01 16:10:12 +0100 |
commit | 3e9e77008c631091c9b6ee51d03b414c334e6480 (patch) | |
tree | 8a5c89e23d27576f359a6c2195a1d8f10ed55247 /vp9/encoder/vp9_encodeframe.c | |
parent | 113f9721d144a1a143a77036db97549b73c66fe9 (diff) | |
download | libvpx-3e9e77008c631091c9b6ee51d03b414c334e6480.tar libvpx-3e9e77008c631091c9b6ee51d03b414c334e6480.tar.gz libvpx-3e9e77008c631091c9b6ee51d03b414c334e6480.tar.bz2 libvpx-3e9e77008c631091c9b6ee51d03b414c334e6480.zip |
Casts to remove some warnings.
Added casts to remove warnings:
BUG=webm:1274
In regards to the safety of these casts they are of two types:-
- Normalized bits per (16x16) MB stored in a 32 bit int (This is safe as bits
per MB even with << 9 normalization cant overflow 32 bits. Even raw 12
bits hdr source even would only be 29 bits :- (4+4+12+9) and the encoder
imposes much stricter limits than this on max bit rate.
- Cast as part of variance calculations. There is an internal cast up to 64 bit
for the Sum X Sum calculation, but after normalization dividing by the number
of points the result will always be <= the SSE value.
Change-Id: I4e700236ed83d6b2b1955e92e84c3b1978b9eaa0
Diffstat (limited to 'vp9/encoder/vp9_encodeframe.c')
-rw-r--r-- | vp9/encoder/vp9_encodeframe.c | 5 |
1 files changed, 4 insertions, 1 deletions
diff --git a/vp9/encoder/vp9_encodeframe.c b/vp9/encoder/vp9_encodeframe.c index d323a17a6..b853dcd08 100644 --- a/vp9/encoder/vp9_encodeframe.c +++ b/vp9/encoder/vp9_encodeframe.c @@ -1788,7 +1788,10 @@ static void set_source_var_based_partition(VP9_COMP *cpi, d32[i].sum += d16[j]->sum; } - d32[i].var = d32[i].sse - (((int64_t)d32[i].sum * d32[i].sum) >> 10); + d32[i].var = + (unsigned int)(d32[i].sse - + (unsigned int)(((int64_t)d32[i].sum * d32[i].sum) >> + 10)); index = coord_lookup[i * 4].row * mis + coord_lookup[i * 4].col; mi_8x8[index] = mi_upper_left + index; |