diff options
| author | Deb Mukherjee <debargha@google.com> | 2014-10-03 10:22:21 -0700 |
|---|---|---|
| committer | Deb Mukherjee <debargha@google.com> | 2014-10-03 10:22:21 -0700 |
| commit | 431cdc33eee902bbc1c4272a8551e3734fe4a16f (patch) | |
| tree | 6d3858940800c174023fa93d2ec2cd939f1a9710 /vp9/encoder | |
| parent | ba172ff0a587d8f47587c478b368c1ea33829a79 (diff) | |
| download | libvpx-431cdc33eee902bbc1c4272a8551e3734fe4a16f.tar libvpx-431cdc33eee902bbc1c4272a8551e3734fe4a16f.tar.gz libvpx-431cdc33eee902bbc1c4272a8551e3734fe4a16f.tar.bz2 libvpx-431cdc33eee902bbc1c4272a8551e3734fe4a16f.zip | |
Prevent negative cost for highbitdepth
Adds proper scaling for highbitdepth in a rdopt cost.
Change-Id: I066694799a7f491b830945ef1c66eb202071c355
Diffstat (limited to 'vp9/encoder')
| -rw-r--r-- | vp9/encoder/vp9_rdopt.c | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/vp9/encoder/vp9_rdopt.c b/vp9/encoder/vp9_rdopt.c index b14ff0822..0ae0d5757 100644 --- a/vp9/encoder/vp9_rdopt.c +++ b/vp9/encoder/vp9_rdopt.c @@ -491,7 +491,9 @@ static void block_rd_txfm(int plane, int block, BLOCK_SIZE plane_bsize, if (x->plane[plane].eobs[block]) { int64_t dc_correct = coeff[0] * coeff[0] - (coeff[0] - dqcoeff[0]) * (coeff[0] - dqcoeff[0]); - +#if CONFIG_VP9_HIGHBITDEPTH + dc_correct >>= ((xd->bd - 8) * 2); +#endif if (tx_size != TX_32X32) dc_correct >>= 2; |