summaryrefslogtreecommitdiff
path: root/vp8/common/idctllm.c
AgeCommit message (Collapse)Author
2012-10-10Merge of the TX_16X16 experimentDeb Mukherjee
Change-Id: I22aa803ffff330622cdb77277e7b196a9766f882
2012-10-05Fix SIMD unsafe use of floating point.Paul Wilkins
This commit fixes unsafe simd / floating point interactions arising from the current hybrid and 16x16 transform implementation. These led to a raft of bugs and issues when the project was built using VS2008 for Win32 though they did not show up with the unix builds. Gerrit makes a meal out of presenting the fix but all I have actually done is indent the body of each function that uses floating point by one level and bracket with emms instructions using the function vp8_clear_system_state(). See below. function () { vp8_clear_system_state(); { ... function body } vp8_clear_system_state(); } This is almost certainly over the top in terms of number of emms instructions but is a temporary measure pending implementation of integer variants of each function to replace the floating point. Limited testing suggests that this fixes the problems that arose for Win32 VS2008 when the hybrid or 16x16 transforms were enabled. Change-Id: I7c9a72bd79315246ed880578dec51e2b7c178442
2012-08-30hybrid transform of 16x16 dimensionJingning Han
Enable ADST/DCT of dimension 16x16 for I16X16 modes. This change provides benefits mostly for hd sequences. Set up the framework for selectable transform dimension. Also allowing quantization parameter threshold to control the use of hybrid transform (This is currently disabled by setting threshold always above the quantization parameter. Adaptive thresholding can be built upon this, which will further improve the coding performance.) The coding performance gains (with respect to the codec that has all other configuration settings turned on) are derf: 0.013 yt: 0.086 hd: 0.198 std-hd: 0.501 Change-Id: Ibb4263a61fc74e0b3c345f54d73e8c73552bf926
2012-08-21hybridtransform8x8: fix compilationJohn Koleszar
corrected ifdefs to include CONFIG_HYBRIDTRANSFORM8X8 for missing definitions Change-Id: I96cd08c98d74b5ba10866f3fa30252073ce92fa5
2012-08-13Combine optimize_b and optimize_b_8x8Daniel Kang
The 16x16 has not been done yet for speed reasons. Change-Id: I066e369a065674d58986e937344c9303fed19450
2012-08-07A bit of temporary relief on encoder slownessYaowu Xu
The commit replaces run-time initialization of cosine constants with static constant values, which provides ~30% relief on slow speed. The real solution, however will be to implement integer versions of those functions that current use float/double. Change-Id: Ie3ff1793509653d78dd1aeaf88cc6737da1bc55f
2012-08-06Refactoring hybrid transform codingJingning Han
The forward and inverse hybrid transforms are now performed using single function modules, where the dimension is sent as argument. Added an inline function clip8b to clip the reconstruction pixels into range of 0-255. Change-Id: Id7d870b3e1aefc092721c80c0af6f641eb5f3747
2012-08-03Replacing the 8x8 DCT with 8x8 ADST/DCT for I8x8Jingning Han
Fixed the code review comments. Under the htrans8x8 experiment the 8X8 DCT in the I8X8 mode is replaced with a combination of 8X8 ADST and DCT. Overall coding gains with the htrans8x8 experiment are: derf: 0.486 std-hd: 1.040 hd: 1.063 yt: 0.506 Note that part of the gain comes from bigger transforms (8x8 instead of 4x4) and part comes from replacing the DCT wth the ADST. Change-Id: I92ca6bbfce11b4165d612b81d9adfad4d010c775
2012-08-0216x16 DCT blocks.Daniel Kang
Set on all 16x16 intra/inter modes Features: - Butterfly fDCT/iDCT - Loop filter does not filter internal edges with 16x16 - Optimize coefficient function - Update coefficient probability function - RD - Entropy stats - 16x16 is a config option Have not tested with experiments. hd: 2.60% std-hd: 2.43% yt: 1.32% derf: 0.60% Change-Id: I96fb090517c30c5da84bad4fae602c3ec0c58b1c
2012-07-19Adds hybrid transformJingning Han
Adds ADST/DCT hybrid transform coding for Intra4x4 mode. The ADST is applied to directions in which the boundary pixels are used for prediction, while DCT applied to directions without corresponding boundary prediction. Adds enum TX_TYPE in b_mode_infor to indicate the transform type used. Make coding style consistent with google style. Fixed the commented issues. Experimental results in terms of bit-rate reduction: derf: 0.731% yt: 0.982% std-hd: 0.459% hd: 0.725% Will be looking at 8x8 transforms next. Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-07-17Restyle codeJohn Koleszar
Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-06-28Add lossless compression mode.Hui Su
This commit adds lossless compression capability to the experimental branch. The lossless experiment can be enabled using --enable-lossless in configure. When the experiment is enabled, the encoder will use lossless compression mode by command line option --lossless, and the decoder automatically recognizes a losslessly encoded clip and decodes accordingly. To achieve the lossless coding, this commit has changed the following: 1. To encode at lossless mode, encoder forces the use of unit quantizer, i.e, Q 0, where effective quantization is 1. Encoder also disables the usage of 8x8 transform and allows only 4x4 transform; 2. At Q 0, the first order 4x4 DCT/IDCT have been switched over to a pair of forward and inverse Walsh-Hadamard Transform (http://goo.gl/EIsfy), with proper scaling applied to match the range of the original 4x4 DCT/IDCT pair; 3. At Q 0, the second order remains to use the previous walsh-hadamard transform pair. However, to maintain the reversibility in second order transform at Q 0, scaling down is applied to first order DC coefficients prior to forward transform, and scaling up is applied to the second order output prior to quantization. Symmetric upscaling and downscaling are added around inverse second order transform; 4. At lossless mode, encoder also disables a number of minor features to ensure no loss is introduced, these features includes: a. Trellis quantization optimization b. Loop filtering c. Aggressive zero-binning, rounding and zero-bin boosting d. Mode based zero-bin boosting Lossless coding test was performed on all clips within the derf set, to verify that the commit has achieved lossless compression for all clips. The average compression ratio is around 2.57 to 1. (http://goo.gl/dEShs) Change-Id: Ia3aba7dd09df40dd590f93b9aba134defbc64e34
2012-06-18Fix a bunch of warnings.Christian Duvivier
Change-Id: I883b4f63559c1cf66a25016e946b1b1ae98a20da
2012-05-10Reversible WHT pairYaowu Xu
This commit changed the forward and the inverse 4x4 Walsh Hadamard transform to a new pair, where the inverse transform can pefectly reconstuct the input to forward transform. It also does so without changing the input and output value range. Even more, it does not change the complexity of the transforms. While it was not expected to improve the results of our current test, it does improve std-hd set by 0.2% on all metrics. No change on derf. Change-Id: Ie4f23ddd3a0f3c5fbe97fb58399f860031f99337
2012-03-01Merge t8x8 experimentsYaowu Xu
Change-Id: I8e9b6b154e1a0d0cb42d596366380d69c00ac15f
2012-02-16a bit code clean-upYaowu Xu
Removed some transform code that is not in use. Change-Id: I9489af7e23d9d7fe052feb6c8bbafa62ebbda39c
2012-02-16optmized rounding for transformsYaowu Xu
the changes are still temporary, the final transforms, especially inverse ones should take in account both accuracy, complexity, and sign-bias, which should be decided at a later time. Change-Id: I116b0c70b25f5ee324ae5713d4564f5d0aa27151
2012-02-16re-scaled 2nd order haar transformYaowu Xu
During the work of extend_qrange, we have rolled a factor of 2 from quantization/dequatnization into 2nd order walsh-hadamard transform. This commit does the same for the 2nd order haar transform. so they can share the same quantizaiton process as the 2nd order WHT. Change-Id: I734af4a20ea8149a01b5b1971a065092977dfe33
2012-02-16moved scaling from dequantization to inverse transform for T8x8Yaowu Xu
Previously, the scaling related to extended quantize range happens in dequantization stage, which implies the coefficients form forward transform are in different scale(4x) from dequantization coefficients This worked fine when there was not distortion computation done based on 8x8 transform, but it completely wracked the distortion estimation based on transform coefficients and dequantized transform coefficients introduced in commit f64725a00 for macroblocks using 8x8 transform. This commit fixed the issue by moving the scaling into the stage of inverse 8x8 transform. TODO: Test&Verify the transform/quantization pipeline accuracy. Change-Id: Iff77b36a965c2a6b247e59b9c59df93eba5d60e2
2012-02-09Merge Extended Q experiment.Paul Wilkins
Merge the extended Q experiment as indicated by the Change-Id: I02d9e654fff9998cc7e9e2f1f5cd838dad8fb431
2011-11-08Attempt to fix an issue related to 8x8 transform and segfeatureYaowu Xu
logically this commit should NOT change anything, but seems to help revert the 3DB loss on bowing in the following commit: https://on2-git.corp.google.com/g/#change,6193 This is still debugging in progress. Need further investigation to understand the root cause of the issue. Change-Id: I0b49d1ef3a311dfff58c6acd3eaebdb3bda6257c
2011-10-24added a last stage rounding for 8x8 inverse dctYaowu Xu
Prior to the added rounding, tests on randomly generated data showed that forward-inverse transform round trip errors are about 3.02/block for input range [-10,10] and 2.68/block for input range [-256, 255]. The added rounding reduced the errors to 0.031/block for input range [-10,10] and 0.037/block for input range [-256, 255]. Maximum round trip error on for any pixel position is 1. The average errors are calculated based on 100,000 blocks of randomly with the specified ranges. Paul mentioned in discussion that the change was not clear on why we need change the rounding, so Patch 2 intends to make the rationale obvious in code, it merged the two separate shifts into one, and the two separate rounding factors into one. Patch 1 and 2 have same numerical test results. Change-Id: Ic5e2f5463de17253084d8b2398c4a210194b20de
2011-10-14fixed the wrong rounding in inverse haar transformYaowu Xu
Given the current forward haar transform: f0 = I0 + I1 + I2 + I3 f1 = I0 + I1 - I2 - I3 f2 = I0 - I1 + I2 - I3 f3 = I0 - I1 - I2 + I3 the output of the inverse haar prior rounding: i0 = f0 + f1 + f2 + f3 = I0 * 4; i1 = f0 + f1 - f2 - f3 = I1 * 4; i2 = f0 - f1 + f2 - f3 = I2 * 4; i3 = f0 - f1 - f2 + f3 = I3 * 4; As all the numbers are 4 multiples, simply >>2 always produces prefect results in term of forward-inverse transform round trip error. Change-Id: Id6658b00ea819ee61cfeef8c5985d4cd3e77f44e
2011-07-21fix more merge issuesYaowu Xu
With this fix, the experimental branch now builds and encodes correctly with the following two configure options respectively: --enable-experimental --enable-t8x8 --enable-experimental Change-Id: I3147c33c503fe713a85fd371e4f1a974805778bf
2011-07-20Add 8x8 transform to experimental branchDeb Mukherjee
Please refer to previous commit messages for detailed info: https://on2-git.corp.google.com/g/#change,5940 https://on2-git.corp.google.com/g/#change,6045 Change-Id: I8b16992f2f69c5a808ad40a3e32ef589cce7c59d
2011-01-19experiment extending the quantizer rangeYaowu Xu
Prior to this change, VP8 min quantizer is 4, which caps the highest quality around 51DB. This experimental change extends the min quantizer to 1, removes the cap and allows the highest quality to be around ~73DB, consistent with the fdct/idct round trip error. To test this change, at configure time use options: --enable-experimental --enable-extend_qrange The following is a brief log of changes in each of the patch sets patch set 1: In this commit, the quantization/dequantization constants are kept unchanged, instead scaling factor 4 is rolled into fdct/idct. Fixed Q0 encoding tests on mobile: Before: 9560.567kbps Overall PSNR:50.255DB VPXSSIM:98.288 Now: 18035.774kbps Overall PSNR:73.022DB VPXSSIM:99.991 patch set 2: regenerated dc/ac quantizer lookup tables based on the scaling factor rolled in the fdct/idct. Also slightly extended the range towards the high quantizer end. patch set 3: slightly tweaked the quantizer tables and generated bits_per_mb table based on Paul's suggestions. patch set 4: fix a typo in idct, re-calculated tables relating active max Q to active min Q patch set 5: added rdmult lookup table based on Q patch set 6: fix rdmult scale: dct coefficient has scaled up by 4 patch set 7: make transform coefficients to be within 16bits patch set 8: normalize 2nd order quantizers patch set 9: fix mis-spellings patch set 10: change the configure script and macros to allow experimental code to be enabled at configure time with --enable-extend_qrange patch set 11: rebase for merge Change-Id: Ib50641ddd44aba2a52ed890222c309faa31cc59c
2010-09-09Use WebM in copyright notice for consistencyJohn Koleszar
Changes 'The VP8 project' to 'The WebM project', for consistency with other webmproject.org repositories. Fixes issue #97. Change-Id: I37c13ed5fbdb9d334ceef71c6350e9febed9bbba
2010-07-23Combine idct and reconstruction stepsJeff Muizelaar
This moves the prediction step before the idct and combines the idct and reconstruction steps into a single step. Combining them seems to give an overall decoder performance improvement of about 1%. Change-Id: I90d8b167ec70d79c7ba2ee484106a78b3d16e318
2010-06-18cosmetics: trim trailing whitespaceJohn Koleszar
When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-04LICENSE: update with latest textJohn Koleszar
Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-05-18Initial WebM releaseJohn Koleszar