diff options
| author | paulwilkins <paulwilkins@google.com> | 2016-06-30 13:31:39 +0100 |
|---|---|---|
| committer | paulwilkins <paulwilkins@google.com> | 2016-06-30 13:34:11 +0100 |
| commit | f9a3d08f1b4c1a41d87d9d74b640d68e9dacda8d (patch) | |
| tree | 7896c163ec56add5cd0595753a8875b7cac751ae | |
| parent | d004c640133b5def534b219cbf66fce9784581e6 (diff) | |
| download | libvpx-f9a3d08f1b4c1a41d87d9d74b640d68e9dacda8d.tar libvpx-f9a3d08f1b4c1a41d87d9d74b640d68e9dacda8d.tar.gz libvpx-f9a3d08f1b4c1a41d87d9d74b640d68e9dacda8d.tar.bz2 libvpx-f9a3d08f1b4c1a41d87d9d74b640d68e9dacda8d.zip | |
Fix error in get_smooth_intra_threshold() for 10/12 bit.
This function seems to scale the threshold for testing an
SSE value in the wrong direction for 10 and 12 bit inputs.
Also for a true SSE the scalings should probably be << 4 and 8
Change-Id: Iba8047b3f70d04aa46d9688a824f3d49c1c58e90
| -rw-r--r-- | vp9/encoder/vp9_firstpass.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/vp9/encoder/vp9_firstpass.c b/vp9/encoder/vp9_firstpass.c index 53a3ec7de..11b397cd6 100644 --- a/vp9/encoder/vp9_firstpass.c +++ b/vp9/encoder/vp9_firstpass.c @@ -532,10 +532,10 @@ static int get_smooth_intra_threshold(VP9_COMMON *cm) { ret_val = SMOOTH_INTRA_THRESH; break; case VPX_BITS_10: - ret_val = SMOOTH_INTRA_THRESH >> 2; + ret_val = SMOOTH_INTRA_THRESH << 4; break; case VPX_BITS_12: - ret_val = SMOOTH_INTRA_THRESH >> 4; + ret_val = SMOOTH_INTRA_THRESH << 8; break; default: assert(0 && "cm->bit_depth should be VPX_BITS_8, " |