diff options
| author | Johann <johannkoenig@google.com> | 2017-08-16 10:22:48 -0700 |
|---|---|---|
| committer | Johann <johannkoenig@google.com> | 2017-08-17 12:34:28 -0700 |
| commit | 7f602d6114ed0544a2f8526f8660488e92b4038e (patch) | |
| tree | 06cb8390bd8d0f51c93ff993610d878e719b8c66 /vp9/encoder/vp9_quantize.c | |
| parent | f95686895b041555d533840aed8928a08f2f1de0 (diff) | |
| download | libvpx-7f602d6114ed0544a2f8526f8660488e92b4038e.tar libvpx-7f602d6114ed0544a2f8526f8660488e92b4038e.tar.gz libvpx-7f602d6114ed0544a2f8526f8660488e92b4038e.tar.bz2 libvpx-7f602d6114ed0544a2f8526f8660488e92b4038e.zip | |
quantize: normalize intermediate types
Despite abs_coeff being a positive value, all the other implementations
treat it as signed which simplifies restoring the sign.
HBD builds cast qcoeff to avoid a visual studio warning. Match
vp9_quantize.c style of casting the entire expression.
Change-Id: I62b539b8df05364df3d7644311e325288da7c5b5
Diffstat (limited to 'vp9/encoder/vp9_quantize.c')
| -rw-r--r-- | vp9/encoder/vp9_quantize.c | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/vp9/encoder/vp9_quantize.c b/vp9/encoder/vp9_quantize.c index da8ca4c08..56bcd60f7 100644 --- a/vp9/encoder/vp9_quantize.c +++ b/vp9/encoder/vp9_quantize.c @@ -78,7 +78,7 @@ void vp9_highbd_quantize_fp_c(const tran_low_t *coeff_ptr, intptr_t count, const int abs_coeff = (coeff ^ coeff_sign) - coeff_sign; const int64_t tmp = abs_coeff + round_ptr[rc != 0]; const int abs_qcoeff = (int)((tmp * quant_ptr[rc != 0]) >> 16); - qcoeff_ptr[rc] = (tran_low_t)((abs_qcoeff ^ coeff_sign) - coeff_sign); + qcoeff_ptr[rc] = (tran_low_t)(abs_qcoeff ^ coeff_sign) - coeff_sign; dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0]; if (abs_qcoeff) eob = i; } |