summaryrefslogtreecommitdiff
path: root/vpx_dsp/variance.c
AgeCommit message (Collapse)Author
2017-07-18variance: call C comp_avg_predJohann
Keep optimized code out of the reference implementation. This matches the style of the other sub calls. Change-Id: I3da6acd4f2c647b029c420e22ac9410a18259689
2017-05-30comp_avg_pred: alignmentJohann
x86 requires 16 byte alignment for some vector loads/stores. arm does not have the same requirement. The asserts are still in avg_pred_sse2.c. This just removes them from the common code. Change-Id: Ic5175c607a94d2abf0b80d431c4e30c8a6f731b6
2017-04-17re-enable vpx_comp_avg_pred_sse2Johann
Buffers on 32 bit x86 builds only guaranteed 8 byte alignment. Fixed with "AvgPred test: use aligned buffers" and "sad avg: align intermediate buffer" Also re-enable asserts on the C version. BUG=webm:1390 Change-Id: I93081f1b0002a352bb0a3371ac35452417fa8514
2017-04-14Disable vpx_comp_avg_pred_sse2Johann
Failures on windows: unknown file: error: SEH exception with code 0xc0000005 thrown in the test body. Alignment check errors on linux: test_libvpx: ../libvpx/vpx_dsp/variance.c:230: void vpx_comp_avg_pred_c(uint8_t *, const uint8_t *, int, int, const uint8_t *, int): Assertion `((intptr_t)comp_pred & 0xf) == 0' failed. BUG=webm:1390 Change-Id: I5eed5381c0f1a8fe594a128eb415e77232f544ea
2017-04-13vpx_comp_avg_pred: sse2 optimizationJohann
Provides over 15x speedup for width > 8. Due to smaller loads and shifting for width == 8 it gets about 8x speedup. For width == 4 it's only about 4x speedup because there is a lot of shuffling and shifting to get the data properly situated. BUG=webm:1390 Change-Id: Ice0b3dbbf007be3d9509786a61e7f35e94bdffa8
2017-04-05Resolve -Wshorten-64-to-32 in highbd variance.James Zern
For 8-bit the subtrahend is small enough to fit into uint32_t. This is the same that was done for: c0241664a Resolve -Wshorten-64-to-32 in variance. For 10/12-bit apply: 63a37d16f Prevent negative variance Change-Id: Iab35e3f3f269035e17c711bd6cc01272c3137e1d
2016-08-23Remove halfpix specializationJohann
This function only exists as a shortcut to subpixel variance with predefined offsets. xoffset = 4 for horizontal, yoffset = 4 for vertical and both for "hv" Removing this allows the existing optimizations for the variance functions to be called. Instead of having only sse2 optimizations, this gives sse2, ssse3, msa and neon. BUG=webm:1273 Change-Id: Ieb407b423b91b87d33c4263c6a1ad5e673b0efd6
2016-07-28Resolve -Wshorten-64-to-32 in variance.Alex Converse
The subtrahend is small enough to fit into uint32_t. Change-Id: Ic4d7128aaa665eaf6b25d562610ba8942c46137f
2016-07-25vpx_dsp: apply clang-formatclang-format
Change-Id: I3ea3e77364879928bd916f2b0a7838073ade5975
2016-06-22Prevent negative varianceYaowu Xu
Due to rounding used computation, HDB variance computation may produce slightly negative values. This commit adds clamping to make sure output variance values for 10 and 12 to be non-negative. Change-Id: Id679aa55a4c201958c4c7d28cd8733b9246a71c8
2016-06-16vpx_dsp/variance.c: change to use correct typeYaowu Xu
This commit change to use int64_t to represent the sum of pixel differences, which can be negative. This fixes a number of ubsan warnings. BUG=webm:1219 Change-Id: I885f245ae895ab92ca5f3b9848d37024b07aac98
2016-01-13Revert "Merge "Change highbd variance rounding to prevent negative variance.""Alex Converse
This reverts commit ea48370a500537906d62544ca4ed75301d79e772, reversing changes made to 15939cb2d76c773950cda40988ede89e111872ea. The commit was insufficiently tested and causes failures. Change-Id: I623d6fc2cd3ae6fd42d0abab1f8eada465ae57a7
2015-11-24Change highbd variance rounding to prevent negative variance.Alex Converse
Always round sum error and sum square error toward zero in variance calculations. This prevents variance from becoming negative. Avoiding rounding variance at all might be better but would be far more invasive. Change-Id: Icf24e0e75ff94952fc026ba6a4d26adf8d373f1c
2015-07-07Move sub pixel variance to vpx_dspJohann
Change-Id: I66bf6720c396c89aa2d1fd26d5d52bf5d5e3dff1
2015-05-26Move variance functions to vpx_dspJohann
subpel functions will be moved in another patch. Change-Id: Idb2e049bad0b9b32ac42cc7731cd6903de2826ce