diff options
| author | Alex Converse <aconverse@google.com> | 2015-11-20 13:48:51 -0800 |
|---|---|---|
| committer | Alex Converse <aconverse@google.com> | 2015-11-24 16:32:01 -0800 |
| commit | 022c848b4d04a2c1f56a54748eee503767269023 (patch) | |
| tree | 3a27a7582fb2719361e679c5ad738c69db8193cb /.gitignore | |
| parent | 64a60ce3ba78c33a69e0adf23f68638d644015e3 (diff) | |
| download | libvpx-022c848b4d04a2c1f56a54748eee503767269023.tar libvpx-022c848b4d04a2c1f56a54748eee503767269023.tar.gz libvpx-022c848b4d04a2c1f56a54748eee503767269023.tar.bz2 libvpx-022c848b4d04a2c1f56a54748eee503767269023.zip | |
Change highbd variance rounding to prevent negative variance.
Always round sum error and sum square error toward zero in variance
calculations. This prevents variance from becoming negative.
Avoiding rounding variance at all might be better but would be far
more invasive.
Change-Id: Icf24e0e75ff94952fc026ba6a4d26adf8d373f1c
Diffstat (limited to '.gitignore')
0 files changed, 0 insertions, 0 deletions