diff options
author | Johann <johannkoenig@google.com> | 2016-09-21 15:55:45 -0700 |
---|---|---|
committer | Johann <johannkoenig@google.com> | 2016-09-21 16:17:07 -0700 |
commit | 2bed8b6acd60f6e3db768e06170364c43a92faa2 (patch) | |
tree | 54b0762e25f6676784be3eacf6036d59bdecd849 /vp8/common/onyxc_int.h | |
parent | 35ebc1cddf3542692acf3690302dd2028ce251fb (diff) | |
download | libvpx-2bed8b6acd60f6e3db768e06170364c43a92faa2.tar libvpx-2bed8b6acd60f6e3db768e06170364c43a92faa2.tar.gz libvpx-2bed8b6acd60f6e3db768e06170364c43a92faa2.tar.bz2 libvpx-2bed8b6acd60f6e3db768e06170364c43a92faa2.zip |
Keep vp8 sixtap read within bounds
When filtering it needs 6 pixels: 2 prior to the source, the source, and
3 after the source.
When filtering 16 wide, that means 21. To accomplish this the SSE2 reads
[-2] to [5], [6] to [13], and [14] to [21], a total of 24 bytes (reading
in groups of 8 is easy)
The filter then shifts this last set to the top half of the register and
uses 'or' to combine it with the previous set.
Valgrind detected an issue reading pixels [19], [20] and [21]:
Address 0x7f581c2 is 434 bytes inside a block of size 441 alloc'd
Note: we only need pixels [16], [17], and [18] as context for [15].
To fix this, it now reads 8 bytes starting at [11], which re-loads [11]
through [13], but stops at [18] and does not over-read any values.
This is shifted by 5 and 'or'd with xmm1. Although the lower bits are
not cleared, they overlap directly with [11] through [13], so 'or'
produces the correct results.
Change-Id: I0c89c03afa660fc9b0108ac055d7bd403e493320
Diffstat (limited to 'vp8/common/onyxc_int.h')
0 files changed, 0 insertions, 0 deletions