summaryrefslogtreecommitdiff
path: root/vp8/encoder/dct.c
AgeCommit message (Collapse)Author
2012-10-22Merging the hybrid transform experimentsDeb Mukherjee
Change-Id: I99f1982b30a630a9a070a8326d83b34a33cba14c
2012-10-19Some cleanups and fixes.Deb Mukherjee
Separates the logic on transform type selection previously spread out over a number of files into a separate function. Currently the tx_type field in b_mode_info is not used, but still left in there to eventually use for signaling the transform type in the bitstream. Also, now for tx_type = DCT_DCT, the regular integer DCT is used, as opposed to the floating point DCT used in conjuction with hybrid transform. Results change somewhat due to the transform change, but are within reasonable limits. The hd/std-hd sets are slightly up, while derf/yt are slightly down. Change-Id: I5776840c2239ca2da31ca6cfd7fd1148dc5f9e0f
2012-10-10Merge of the TX_16X16 experimentDeb Mukherjee
Change-Id: I22aa803ffff330622cdb77277e7b196a9766f882
2012-10-05Fix SIMD unsafe use of floating point.Paul Wilkins
This commit fixes unsafe simd / floating point interactions arising from the current hybrid and 16x16 transform implementation. These led to a raft of bugs and issues when the project was built using VS2008 for Win32 though they did not show up with the unix builds. Gerrit makes a meal out of presenting the fix but all I have actually done is indent the body of each function that uses floating point by one level and bracket with emms instructions using the function vp8_clear_system_state(). See below. function () { vp8_clear_system_state(); { ... function body } vp8_clear_system_state(); } This is almost certainly over the top in terms of number of emms instructions but is a temporary measure pending implementation of integer variants of each function to replace the floating point. Limited testing suggests that this fixes the problems that arose for Win32 VS2008 when the hybrid or 16x16 transforms were enabled. Change-Id: I7c9a72bd79315246ed880578dec51e2b7c178442
2012-09-27Entropy coding for hybrid transformDeb Mukherjee
Separates the entropy coding context models for 4x4, 8x8 and 16x16 ADST variants. There is a small improvement for HD (hd/std-hd) by about 0.1-0.2%. Results on derf/yt are about the same, probably because there is not enough statistics. Results may improve somewhat once the initial probability tables are updated for the hybrid transforms which is coming soon. Change-Id: Ic7c0c62dacc68ef551054fdb575be8b8507d32a8
2012-08-30hybrid transform of 16x16 dimensionJingning Han
Enable ADST/DCT of dimension 16x16 for I16X16 modes. This change provides benefits mostly for hd sequences. Set up the framework for selectable transform dimension. Also allowing quantization parameter threshold to control the use of hybrid transform (This is currently disabled by setting threshold always above the quantization parameter. Adaptive thresholding can be built upon this, which will further improve the coding performance.) The coding performance gains (with respect to the codec that has all other configuration settings turned on) are derf: 0.013 yt: 0.086 hd: 0.198 std-hd: 0.501 Change-Id: Ibb4263a61fc74e0b3c345f54d73e8c73552bf926
2012-08-21hybridtransform8x8: fix compilationJohn Koleszar
corrected ifdefs to include CONFIG_HYBRIDTRANSFORM8X8 for missing definitions Change-Id: I96cd08c98d74b5ba10866f3fa30252073ce92fa5
2012-08-13Combine optimize_b and optimize_b_8x8Daniel Kang
The 16x16 has not been done yet for speed reasons. Change-Id: I066e369a065674d58986e937344c9303fed19450
2012-08-07A bit of temporary relief on encoder slownessYaowu Xu
The commit replaces run-time initialization of cosine constants with static constant values, which provides ~30% relief on slow speed. The real solution, however will be to implement integer versions of those functions that current use float/double. Change-Id: Ie3ff1793509653d78dd1aeaf88cc6737da1bc55f
2012-08-06Refactoring hybrid transform codingJingning Han
The forward and inverse hybrid transforms are now performed using single function modules, where the dimension is sent as argument. Added an inline function clip8b to clip the reconstruction pixels into range of 0-255. Change-Id: Id7d870b3e1aefc092721c80c0af6f641eb5f3747
2012-08-03Replacing the 8x8 DCT with 8x8 ADST/DCT for I8x8Jingning Han
Fixed the code review comments. Under the htrans8x8 experiment the 8X8 DCT in the I8X8 mode is replaced with a combination of 8X8 ADST and DCT. Overall coding gains with the htrans8x8 experiment are: derf: 0.486 std-hd: 1.040 hd: 1.063 yt: 0.506 Note that part of the gain comes from bigger transforms (8x8 instead of 4x4) and part comes from replacing the DCT wth the ADST. Change-Id: I92ca6bbfce11b4165d612b81d9adfad4d010c775
2012-08-0216x16 DCT blocks.Daniel Kang
Set on all 16x16 intra/inter modes Features: - Butterfly fDCT/iDCT - Loop filter does not filter internal edges with 16x16 - Optimize coefficient function - Update coefficient probability function - RD - Entropy stats - 16x16 is a config option Have not tested with experiments. hd: 2.60% std-hd: 2.43% yt: 1.32% derf: 0.60% Change-Id: I96fb090517c30c5da84bad4fae602c3ec0c58b1c
2012-07-19fixed a compiling issueYaowu Xu
Change-Id: I9ccbea76eaeb81ce7b833dd92216734725796e84
2012-07-19Adds hybrid transformJingning Han
Adds ADST/DCT hybrid transform coding for Intra4x4 mode. The ADST is applied to directions in which the boundary pixels are used for prediction, while DCT applied to directions without corresponding boundary prediction. Adds enum TX_TYPE in b_mode_infor to indicate the transform type used. Make coding style consistent with google style. Fixed the commented issues. Experimental results in terms of bit-rate reduction: derf: 0.731% yt: 0.982% std-hd: 0.459% hd: 0.725% Will be looking at 8x8 transforms next. Change-Id: I46dbd7b80dbb3e8856e9c34fbc58cb3764a12fcf
2012-07-17removed floating point version 8x8 fdctYaowu Xu
the integer version has very good precision, the float version is no longer useful. this commit also removes the experiment option from configure script. Change-Id: Ibb92e63c9f5083357cdf89c559d584a7deb3353f
2012-07-17Restyle codeJohn Koleszar
Approximate the Google style guide[1] so that that there's a written document to follow and tools to check compliance[2]. [1]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml [2]: http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py Change-Id: Idf40e3d8dddcc72150f6af127b13e5dab838685f
2012-06-28Add lossless compression mode.Hui Su
This commit adds lossless compression capability to the experimental branch. The lossless experiment can be enabled using --enable-lossless in configure. When the experiment is enabled, the encoder will use lossless compression mode by command line option --lossless, and the decoder automatically recognizes a losslessly encoded clip and decodes accordingly. To achieve the lossless coding, this commit has changed the following: 1. To encode at lossless mode, encoder forces the use of unit quantizer, i.e, Q 0, where effective quantization is 1. Encoder also disables the usage of 8x8 transform and allows only 4x4 transform; 2. At Q 0, the first order 4x4 DCT/IDCT have been switched over to a pair of forward and inverse Walsh-Hadamard Transform (http://goo.gl/EIsfy), with proper scaling applied to match the range of the original 4x4 DCT/IDCT pair; 3. At Q 0, the second order remains to use the previous walsh-hadamard transform pair. However, to maintain the reversibility in second order transform at Q 0, scaling down is applied to first order DC coefficients prior to forward transform, and scaling up is applied to the second order output prior to quantization. Symmetric upscaling and downscaling are added around inverse second order transform; 4. At lossless mode, encoder also disables a number of minor features to ensure no loss is introduced, these features includes: a. Trellis quantization optimization b. Loop filtering c. Aggressive zero-binning, rounding and zero-bin boosting d. Mode based zero-bin boosting Lossless coding test was performed on all clips within the derf set, to verify that the commit has achieved lossless compression for all clips. The average compression ratio is around 2.57 to 1. (http://goo.gl/dEShs) Change-Id: Ia3aba7dd09df40dd590f93b9aba134defbc64e34
2012-05-15Changed to use integer 8x8 dctYaowu Xu
The commit added an integer version of 8x8 forward DCT, based on the orginal forward DCT from VP6. The constants, roundings, and shifts were adjusted to improve the accuracy. The latest patch has a very similar accuracy in term of round trip error against the floating point version. It should be noted here that the purpose of the patch is to help encoding speed and facilitate all other experiments. There will be futher review in combination with inverse DCT before finalization. configure with "--enable--int_8x8fdct" to use the integer version Change-Id: I5a4f80507429f0e07cf02a13768ec81cbfddc5bc
2012-05-10Reversible WHT pairYaowu Xu
This commit changed the forward and the inverse 4x4 Walsh Hadamard transform to a new pair, where the inverse transform can pefectly reconstuct the input to forward transform. It also does so without changing the input and output value range. Even more, it does not change the complexity of the transforms. While it was not expected to improve the results of our current test, it does improve std-hd set by 0.2% on all metrics. No change on derf. Change-Id: Ie4f23ddd3a0f3c5fbe97fb58399f860031f99337
2012-03-01Merge t8x8 experimentsYaowu Xu
Change-Id: I8e9b6b154e1a0d0cb42d596366380d69c00ac15f
2012-02-16optmized rounding for transformsYaowu Xu
the changes are still temporary, the final transforms, especially inverse ones should take in account both accuracy, complexity, and sign-bias, which should be decided at a later time. Change-Id: I116b0c70b25f5ee324ae5713d4564f5d0aa27151
2012-02-16re-scaled 2nd order haar transformYaowu Xu
During the work of extend_qrange, we have rolled a factor of 2 from quantization/dequatnization into 2nd order walsh-hadamard transform. This commit does the same for the 2nd order haar transform. so they can share the same quantizaiton process as the 2nd order WHT. Change-Id: I734af4a20ea8149a01b5b1971a065092977dfe33
2012-02-09Merge Extended Q experiment.Paul Wilkins
Merge the extended Q experiment as indicated by the Change-Id: I02d9e654fff9998cc7e9e2f1f5cd838dad8fb431
2011-11-11Make 8x8 and extend_qrange to work togetherYaowu Xu
This commit added scaling factors to 8x8 transform, quant, dequant and inverse transform pipeline to make 8x8 transform to work when configed with enable-extend_qrange. This commit also disabled the trellis-quant when extend_qrange is configured. Change-Id: Icfb3192e4746f70a4bb35ad18b7b47705b657e52
2011-07-21fix more merge issuesYaowu Xu
With this fix, the experimental branch now builds and encodes correctly with the following two configure options respectively: --enable-experimental --enable-t8x8 --enable-experimental Change-Id: I3147c33c503fe713a85fd371e4f1a974805778bf
2011-07-20Add 8x8 transform to experimental branchDeb Mukherjee
Please refer to previous commit messages for detailed info: https://on2-git.corp.google.com/g/#change,5940 https://on2-git.corp.google.com/g/#change,6045 Change-Id: I8b16992f2f69c5a808ad40a3e32ef589cce7c59d
2011-01-19experiment extending the quantizer rangeYaowu Xu
Prior to this change, VP8 min quantizer is 4, which caps the highest quality around 51DB. This experimental change extends the min quantizer to 1, removes the cap and allows the highest quality to be around ~73DB, consistent with the fdct/idct round trip error. To test this change, at configure time use options: --enable-experimental --enable-extend_qrange The following is a brief log of changes in each of the patch sets patch set 1: In this commit, the quantization/dequantization constants are kept unchanged, instead scaling factor 4 is rolled into fdct/idct. Fixed Q0 encoding tests on mobile: Before: 9560.567kbps Overall PSNR:50.255DB VPXSSIM:98.288 Now: 18035.774kbps Overall PSNR:73.022DB VPXSSIM:99.991 patch set 2: regenerated dc/ac quantizer lookup tables based on the scaling factor rolled in the fdct/idct. Also slightly extended the range towards the high quantizer end. patch set 3: slightly tweaked the quantizer tables and generated bits_per_mb table based on Paul's suggestions. patch set 4: fix a typo in idct, re-calculated tables relating active max Q to active min Q patch set 5: added rdmult lookup table based on Q patch set 6: fix rdmult scale: dct coefficient has scaled up by 4 patch set 7: make transform coefficients to be within 16bits patch set 8: normalize 2nd order quantizers patch set 9: fix mis-spellings patch set 10: change the configure script and macros to allow experimental code to be enabled at configure time with --enable-extend_qrange patch set 11: rebase for merge Change-Id: Ib50641ddd44aba2a52ed890222c309faa31cc59c
2010-09-09Use WebM in copyright notice for consistencyJohn Koleszar
Changes 'The VP8 project' to 'The WebM project', for consistency with other webmproject.org repositories. Fixes issue #97. Change-Id: I37c13ed5fbdb9d334ceef71c6350e9febed9bbba
2010-06-28Improve the accuracy of forward walsh-hadamard transformYaowu Xu
Besides the slight improvement in round trip error. This also fixes a sign bias in the forward transform, so the round trip errors are evenly distributed between +1s and -1s. The old bias seemed to work well with the dc sign bias in old fdct, which no longer exist in the improved fdct. Change-Id: I8635e7be16c69e69a8669eca5438550d23089cef
2010-06-24Redo the forward 4x4 dctYaowu Xu
The new fdct lowers the round trip sum squared error for a 4x4 block ~0.12. or ~0.008/pixel. For reference, the old matrix multiply version has average round trip error 1.46 for a 4x4 block. Thanks to "derf" for his suggestions and references. Change-Id: I5559d1e81d333b319404ab16b336b739f87afc79
2010-06-18cosmetics: trim trailing whitespaceJohn Koleszar
When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-04LICENSE: update with latest textJohn Koleszar
Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-05-18Initial WebM releaseJohn Koleszar