summaryrefslogtreecommitdiff
path: root/vp8/encoder/quantize.c
AgeCommit message (Collapse)Author
2013-03-14Removed shadow warnings : bitstream.c encodeframe.c onyx_if.cScott LaVarnway
ratectrl.c and quantize.c Adding -Wshadow to CFLAGS generated a bunch of warnings. This patch removes these warnings. Change-Id: I8c8faa9fde57c1c49662d332a90bc8d9a0f4a2ce
2013-02-28Use intrinsics for sse2 regular quantizeJohann
Remove dependency of this function on asm_offsets. ssse3/sse4 next. Change quant_shift calculation so it be done using SIMD. Pre-calculate as much as possible to simplify EOB selection. Take advantage of qcoeff being zero'd by tying the if statements together. Speed parity with previous implementation with gcc x86_64 linux Change-Id: Ife97556a1eca3a74b09def1a3d04084974dff1fb
2012-12-10Moved zbin_mode_boost to macroblock structScott LaVarnway
Fixing unsafe threading in VP8 encoder. Change-Id: Ibf4c89a2043654834747811bc11eb283de0bb830
2012-12-10Moved zbin_over_quant to macroblock structScott LaVarnway
Change-Id: I76fe20ade099573997404b8733cf7f79e82fb21e WIP: Fixing unsafe threading in VP8 encoder.
2012-06-11Fix pedantic compiler warningsJohn Koleszar
Allows building the library with the gcc -pedantic option, for improved portabilty. In particular, this commit removes usage of C99/C++ style single-line comments and dynamic struct initializers. This is a continuation of the work done in commit 97b766a46, which removed most of these warnings for decode only builds. Change-Id: Id453d9c1d9f44cc0381b10c3869fabb0184d5966
2012-01-30RTCD: add quantizer functionsJohn Koleszar
This commit continues the process of converting to the new RTCD system. Change-Id: Iba9df4c03a508e51c37201c621be43523fae87d9
2012-01-06Reduced the size of Y1Dequant and friends to [128][2]Scott LaVarnway
This patch removes the local copies of the dequantize constants and implements John's idea as described in "Make a local copy of the dequantized data" commit. Change-Id: Ic6b7d681f00bf63263f71ff1e39ab2f80729e8b2
2011-12-28Improve vp8cx_init_quantizer()Yunqing Wang
Except zrun_zbin_boost, 15 AC values are the same for all other parameters. Removed unneccessary calculation. Change-Id: I6101c0fe8080bd2b4387c3b04d7ddedbf6010409
2011-11-28Populate q_index in multi-thread encodingYunqing Wang
This value needs to be copied to each thread's data structure. This fixed artifact problem in multi-thread encoder. Change-Id: Iab6d9745a1d44846aa503184705376f63a505597
2011-11-08Fix checks in MB quantizer initializationYunqing Wang
vp8cx_mb_init_quantizer() needs to be called at least once to get all values calculated. This change added one check to decide if we could skip initialization or not. Change-Id: I3f65eb548be57580a61444328336bc18c25c085b
2011-11-04Merge "Add checks in MB quantizer initialization"Yunqing Wang
2011-11-03Change use of eob in the encoderTero Rintaluoma
Changed 'int eob' to 'char *eob' in BLOCKD so that both encoder and decoder will use eobs[25] array from MACROBLOCKD structure. In future, this will enable use of the decoder side IDCT in the encoder. Change-Id: I6e1c011628cb8864fd4a0b80f0279ce16a5ca978
2011-11-01Add checks in MB quantizer initializationYunqing Wang
In some situations (f.g. error-resilient is turned on), vp8cx_mb _init_quantizer() was called once per macroblock. Added checks to avoid calculations when there is no change. Change-Id: Ie4f0a5ade2202041254990a4e9d5b03bd1ac5aea
2011-06-01neon fast quantize block pairTero Rintaluoma
vp8_fast_quantize_b_pair_neon function added to quantize two adjacent blocks at the same time to improve performance. - Additional 3-6% speedup compared to neon optimized fast quantizer (Tanya VGA@30fps, 1Mbps stream, cpu-used=-5..-16) Change-Id: I3fcbf141e5d05e9118c38ca37310458afbabaa4e
2011-05-19Move quantizer init functions to quantize.cJohn Koleszar
Group related functions together. Change-Id: I92fd779225b75a7204650f1decb713142c655d71
2011-04-13store quant_shift as an unsigned charJohann
in encodframe.c, quant_shift is set to 0 or 1 in vp8cx_invert_quant only use 8 bits to store this, instead of 16. will allow saving an xmm register in an updated version of the regular quantize Change-Id: Ie88c47fe2aff5af0283dab1147fb2791e4b12f90
2011-02-18remove unused vp8_predict_dc functionJohn Koleszar
Change-Id: I64fa47889c54cfed094a674c49ef0996d49bdd42
2011-02-10Fix relative include pathsJohn Koleszar
Allow compiling without adding vp8/{common,encoder,decoder} to the include paths. Change-Id: Ifeb5dac351cdfadcd659736f5158b315a0030b6c
2011-01-14update sse2 regular quantizerJohann
about ~5% gain on 32bit. disabled for 64bit unset executable bit on ssse3 version (cosmetic) Change-Id: I1a5860839eb294ce4261f819caea2dcfa78e57ca
2011-01-11Remove unused local variablesHenrik Lundin
Removing unused local variables causing compiler warnings in Visual Studio. Change-Id: I0e2096303be1fdbc01428a6e57cca9796bb32c8a
2010-12-28Use the fast quantizer for inter mode selectionScott LaVarnway
Use the fast quantizer for inter mode selection and the regular quantizer for the rest of the encode for good quality, speed 1. Both performance and quality were improved. The quality gains will make up for the quality loss mentioned in I9dc089007ca08129fb6c11fe7692777ebb8647b0. Change-Id: Ia90bc9cf326a7c65d60d31fa32f6465ab6984d21
2010-11-11quantizer: fix assertion in fast quantizer pathJohn Koleszar
The fast quantizer assembly code has not been updated to match the new exact quantizer, which was made the default in commit 6adbe09. Specifically, they are not aware of the potential for the coefficient to be scaled, which results in the quantized result exceeding the range of the DCT. This patch restores the previous behavior of using the non-shifted coefficients when in the fast quantizer code path, but unfortunately requires rebuilding the tables when switching between the two. Change-Id: I0a33f5b3850335011a06906f49fafed54dda9546
2010-11-10Tuning for the more exact quantizer.Paul Wilkins
Small changes to the default zero bin and rounding tables. Though the tables are currently the same for the Y1 and Y2 cases I have left them as separate tables in case we want to tune this later. There is now some adjustment of the zbin based on the prediction mode. Previously this was restricted to an adjustment for gf/arf 0,0 MV. The exact quantizer now marginal outperforms and is the default. The overall average gain is about 0.5% Change-Id: I5e4353f3d5326dde4e86823684b236a1e9ea7f47
2010-10-21Convert [4][4] matrices to [16] arrays.Timothy B. Terriberry
Most of the code that actually uses these matrices indexes them as if they were a single contiguous array, and coverity produces reports about the resulting accesses that overflow the static bounds of the first row. This is perfectly legal in C, but converting them to actual [16] arrays should eliminate the report, and removes a good deal of extraneous indexing and address operators from the code. Change-Id: Ibda479e2232b3e51f9edf3b355b8640520fdbf23
2010-10-12Centralize mb skip state calculationJohn Koleszar
This patch moves the scattered updates to the mb skip state (mode_info_context->mbmi.mb_skip_coeff) to vp8_tokenize_mb. Recent changes to the quantizer exposed a bug where if a macroblock could be coded as a skip but isn't, the encoder would run the loopfilter but the decoder wouldn't, causing a reference buffer mismatch. The loopfilter is controlled by a flag called dc_diff. The decoder looks at the number of decoded coefficients when setting this flag. The encoder sets this flag based on the skip state, since any skippable macroblock should be transmitted as a skip. The coefficient optimization pass (vp8_optimize_b()) could change the coefficients such that a block that was not a skip becomes one. The encoder was not updating the skip state in this situation for intra coded blocks. The underlying issue predates it, but this bug was recently triggered by enabling trellis quantization on the Y2 block in commit dcd29e3, and by changing the quantizer range control in commit 305be4e. Change-Id: I5cce5da0dbc2d22f7d79ee48149f01e868a64802
2010-10-11Move vp8_strict_quantize_b inside EXACT_QUANT #define.Timothy B. Terriberry
There is currently no inexact version of this function, so do not even compile it without EXACT_QUANT. This will prevent someone from inadvertently trying to use it without the proper EXACT_QUANT setup. Change-Id: Ia13491e0128afb281c05c9222ee5987101e4010d
2010-10-06optimize fast_quantizer c versionYaowu Xu
As the zbin and rounding constants are normalized, rounding effectively does the zbinning, therefore the zbin operation can be removed. In addition, the memset on the two arrays are no longer necessary. Change-Id: If39c353c42d7e052296cb65322e5218810b5cc4c
2010-09-09Use WebM in copyright notice for consistencyJohn Koleszar
Changes 'The VP8 project' to 'The WebM project', for consistency with other webmproject.org repositories. Fixes issue #97. Change-Id: I37c13ed5fbdb9d334ceef71c6350e9febed9bbba
2010-09-02Whitespace: nuke CRLFsJohn Koleszar
Change-Id: I8b9fdf9875a8fcff4cb49a3357ce44f18108c2e7
2010-08-12Removed unnecessary MB_MODE_INFO copiesScott LaVarnway
These copies occurred for each macroblock in the encoder and decoder. Thetemp MB_MODE_INFO mbmi was removed from MACROBLOCKD. As a result, a large number compile errors had to be fixed. Change-Id: I4cf0ffae3ce244f6db04a4c217d52dd256382cf3
2010-08-10Add trellis quantization.Timothy B. Terriberry
Replace the exponential search for optimal rounding during quantization with a linear Viterbi trellis and enable it by default when using --best. Right now this operates on top of the output of the adaptive zero-bin quantizer in vp8_regular_quantize_b() and gives a small gain. It can be tested as a replacement for that quantizer by enabling the call to vp8_strict_quantize_b(), which uses normal rounding and no zero bin offset. Ultimately, the quantizer will have to become a function of lambda in order to take advantage of activity masking, since there is limited ability to change the quantization factor itself. However, currently vp8_strict_quantize_b() plus the trellis quantizer (which is lambda-dependent) loses to vp8_regular_quantize_b() alone (which is not) on my test clip. Patch Set 3: Fix an issue related to the cost evaluation of successor states when a coefficient is reduced to zero. With this issue fixed, now the trellis search almost exactly matches the exponential search. Patch Set 2: Overall, the goal of this patch set is to make "trellis" search to produce encodings that match the exponential search version. There are three main differences between Patch Set 2 and 1: a. Patch set 1 did not properly account for the scale of 2nd order error, so patch set 2 disable it all together for 2nd blocks. b. Patch set 1 was not consistent on when to enable the the quantization optimization. Patch set 2 restore the condition to be consistent. c. Patch set 1 checks quantized level L-1, and L for any input coefficient was quantized to L. Patch set 2 limits the candidate coefficient to those that were rounded up to L. It is worth noting here that a strategy to check L and L+1 for coefficients that were truncated down to L might work. (a and b get trellis quant to basically match the exponential search on all mid/low rate encodings on cif set, without a, b, trellis quant can hurt the psnr by 0.2 to .3db at 200kbps for some cif clips) (c gets trellis quant to match the exponential search to match at Q0 encoding, without c, trellis quant can be 1.5 to 2db lower for encodings with fixed Q at 0 on most derf cif clips) Change-Id: Ib1a043b665d75fbf00cb0257b7c18e90eebab95e
2010-07-28Enable the switch between two versions of quantizerYaowu Xu
To facilitate more testing related to quantizer and rate control, the old version quantizer is added back. old and new quantizer can be switched back and forth by define or un-define the macro "EXACT_QUANT". Change-Id: Ia77e687622421550f10e9d65a9884128a79a65ff
2010-07-23Make the quantizer exact.Timothy B. Terriberry
This replaces the approximate division-by-multiplication in the quantizer with an exact one that costs just one add and one shift extra. The asm versions have not been updated in this patch, and thus have been disabled, since the new method requires different multipliers which are not compatible with the old method. Change-Id: I53ac887af0f969d906e464c88b1f4be69c6b1206
2010-06-18cosmetics: trim trailing whitespaceJohn Koleszar
When the license headers were updated, they accidentally contained trailing whitespace, so unfortunately we have to touch all the files again. Change-Id: I236c05fade06589e417179c0444cb39b09e4200d
2010-06-08minor cleanup of quantizer and fdct codeYaowu Xu
Change-Id: I7ccc580410bea096a70dce0cc3d455348d4287c5
2010-06-07Remove duplicate and unused functionsYaowu Xu
Change-Id: I944035e720ef834561a9da0d723879a4f787312c
2010-06-04LICENSE: update with latest textJohn Koleszar
Change-Id: Ieebea089095d9073b3a94932791099f614ce120c
2010-05-18Initial WebM releaseJohn Koleszar