diff options
| author | Yunqing Wang <yunqingwang@google.com> | 2016-08-11 17:34:20 -0700 |
|---|---|---|
| committer | Yunqing Wang <yunqingwang@google.com> | 2016-08-12 09:27:58 -0700 |
| commit | a413dbe594062788335d9d6b726548a664ad58fa (patch) | |
| tree | f7f218b1e5561eee4cf5ffd1b62252467970221e /vp9/encoder/vp9_rdopt.c | |
| parent | f1e12c1bf3b24e67c8c1542c87c0cd84f85b88fc (diff) | |
| download | libvpx-a413dbe594062788335d9d6b726548a664ad58fa.tar libvpx-a413dbe594062788335d9d6b726548a664ad58fa.tar.gz libvpx-a413dbe594062788335d9d6b726548a664ad58fa.tar.bz2 libvpx-a413dbe594062788335d9d6b726548a664ad58fa.zip | |
Fix another motion vector out of range bug
This patch fixed a motion vector out of range bug:
vpxenc: ../libvpx/vp9/encoder/vp9_mcomp.c:69:
mv_cost: Assertion `mv->col >= -((1 << (11 + 1 + 2)) - 1) &&
mv->col < ((1 << (11 + 1 + 2)) - 1)' failed.
For blocks that returned without having full-pixel search, the original
MV limits were not restored, which caused the failure. Moved the set
MV limit function down to fix the bug.
Change-Id: Id7d798fc7214e95c6e4846c588f0233fcf1a4223
Diffstat (limited to 'vp9/encoder/vp9_rdopt.c')
| -rw-r--r-- | vp9/encoder/vp9_rdopt.c | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/vp9/encoder/vp9_rdopt.c b/vp9/encoder/vp9_rdopt.c index 307ce668c..8cb90e893 100644 --- a/vp9/encoder/vp9_rdopt.c +++ b/vp9/encoder/vp9_rdopt.c @@ -2347,8 +2347,6 @@ static void single_motion_search(VP9_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bsize, vp9_setup_pre_planes(xd, 0, scaled_ref_frame, mi_row, mi_col, NULL); } - vp9_set_mv_search_range(&x->mv_limits, &ref_mv); - // Work out the size of the first step in the mv step search. // 0 here is maximum length first step. 1 is VPXMAX >> 1 etc. if (cpi->sf.mv.auto_mv_step_size && cm->show_frame) { @@ -2396,6 +2394,10 @@ static void single_motion_search(VP9_COMP *cpi, MACROBLOCK *x, BLOCK_SIZE bsize, } } + // Note: MV limits are modified here. Always restore the original values + // after full-pixel motion search. + vp9_set_mv_search_range(&x->mv_limits, &ref_mv); + mvp_full = pred_mv[x->mv_best_ref_index[ref]]; mvp_full.col >>= 3; |