summaryrefslogtreecommitdiff
path: root/vp9/encoder/vp9_quantize.h
AgeCommit message (Collapse)Author
2015-07-17Migrate quantization functions from vp9/ to vpx_dsp/Yunqing Wang
The following quantization functions were moved: vp9_quantize_b vp9_quantize_b_32x32 vp9_highbd_quantize_b vp9_highbd_quantize_b_32x32 vp9_quantize_dc vp9_quantize_dc_32x32 vp9_highbd_quantize_dc vp9_highbd_quantize_dc_32x32 The purpose of doing that was to allow these functions to be shared by multiple codecs. Change-Id: Id8ab939f283353cdd07bd930d47db3d932a5d87f
2015-03-03dc quantizer fix for 32x32 transformsDeb Mukherjee
The rounding factor needs to be scaled down by a factor of 2. Also, the quantized and dequantized coefficients are memset to 0 when dc quantizer is used. Change-Id: Ifa68bab02addbf1b83d249c5b4cbd5cda796b1cf
2014-12-22Revert "Revert "Removal of legacy zbin_extra / zbin_oq_value.""Jingning Han
This reverts commit 9946ee23e0a4c158e26a505b162a072f81b8a3be. Fix the ssse3 asm function. Change-Id: I07f77a63aa98087626e45c4e87aa5dcafc0b0b07
2014-12-19Revert "Removal of legacy zbin_extra / zbin_oq_value."Paul Wilkins
This reverts commit e9b586e21bb899e247346e82bccf5afb42604910. Change-Id: I5b36e6727da6c05278d97e2c37b80c109f79bed4
2014-12-18Removal of legacy zbin_extra / zbin_oq_value.Paul Wilkins
zbin extra / zbin_oq_value was widely passed around, hence removal touches a lot of code. Change-Id: Idc94359735b60c38a160e4385ae09d5ca8b6b8e5
2014-12-18Remove mode dependent zbin boost.Paul Wilkins
Initial patch to remove get_zbin_mode_boost() and cpi->zbin_mode_boost. For now sets a dummy value of 0 for zbin extra pending a further clean up patch. Change-Id: I64a1e1eca2d39baa8ffb0871b515a0be05c9a6af
2014-10-09Rename highbitdepth functions to use highbd prefixDeb Mukherjee
Uses highbd_ prefix convention consistently. Change-Id: I58f7f799a7ff8e32701bcd71c955bcf1cdd4581e
2014-09-11Adds high bitdepth transform functions and testsDeb Mukherjee
Adds various high bitdepth transform functions and tests. Much of the changes are related to using typedefs tran_low_t and tran_high_t for the final transform cofficients and intermediate stages of the transform computation respectively rather than fixed types int16_t/int. When vp9_highbitdepth configure flag is off, these map tp int16_t/int32_t, but when the flag is on, they map to int32_t/int64_t to make space for needed extra precision. Change-Id: I3c56de79e15b904d6f655b62ffae170729befdd8
2014-07-21Remove unfinished VP9 alpha channel.Alex Converse
Change-Id: Ic5d3a3a0dac10b49495771886a31e793bb78b5ca
2014-07-01Re-design quantization processJingning Han
This commit re-designs the quantization process for transform coefficient blocks of size 4x4 to 16x16. It improves compression performance for speed 7 by 3.85%. The SSSE3 version for the new quantization process is included. The average runtime of the 8x8 block quantization is reduced from 285 cycles -> 255 cycles, i.e., over 10% faster. Change-Id: I61278aa02efc70599b962d3314671db5b0446a50
2014-06-12Fast computation path for forward transform and quantizationJingning Han
This commit enables a fast path computational flow for forward transformation. It checks the sse and variance of prediction residuals and decides if the quantized coefficients are all zero, dc only, or more. It then selects the corresponding coding path in the forward transformation and quantization stage. It is currently enabled in rtc coding mode. Will do it for rd coding mode next. In speed -6, the runtime for pedestrian_area 1080p at 1000 kbps goes down from 14234 ms to 13704 ms, i.e., about 4% speed-up. Overall coding performance for rtc set is changed by -0.18%. Change-Id: I0452da1786d59bc8bcbe0a35fdae9f623d1d44e1
2014-04-22Fix the CONFIG_ALPHA build.Alex Converse
Change-Id: Ib89fe34812c17cd6294ce3c38f87d43a79abb16f
2014-04-09Moving q_trans[] table to vp9_quantize.{c, h}.Dmitry Kovalev
Change-Id: I1324c339815a47004ddccdaf651d24c60382b92f
2014-03-28Moving encoder quantization parameters into separate struct.Dmitry Kovalev
Change-Id: I2a169535489aeda3943fb5a46ab53e7a12abaa36
2014-02-28Cleaning up vp9_quantize.c.Dmitry Kovalev
Change-Id: I9a38af32f16f196b83dd69755eafb9543edf5691
2014-01-23vp9/encoder: add extern "C" to headersJames Zern
Change-Id: I4f51ce859a97bf1b8fd2b37ac585b7c643232b69
2013-12-02Remove plane_block_idx.Alex Converse
Its last remaining caller can be passed its results directly without any additional work. Also, it's not non-4:2:0 safe. Change-Id: Ia5089ba5f7f66c7617270483c619c9271aefd868
2013-10-28Cleaning up vp9_regular_quantize_b_4x4.Dmitry Kovalev
Passing scan & iscan as parameters, adding useful local variables. Change-Id: Ia2a87906941db9557350d273669ce5c3cdb7235d
2013-10-22Removing quantize_b_4x4 function pointer.Dmitry Kovalev
The pointer was asigned only once with vp9_regular_quantize_b_4x4, calling this function directly now. Also removing unused declarations: prototype_quantize_block prototype_quantize_block_pair prototype_quantize_mb vp9_regular_quantize_b_4x4_pair vp9_regular_quantize_b_8x8 Change-Id: I14325bc2f082336820671eafbc06126651b79f73
2013-09-24Removing redundant 'extern' keyword.Dmitry Kovalev
Change-Id: Ie51306689c0dc527a8aa12d3984389dd8f360dea
2013-07-15Inline vp9_quantize() in xform_quant().Ronald S. Bultje
Cycle times: 4x4: 151 to 131 cycles (15% faster) 8x8: 334 to 306 cycles (9% faster) 16x16: 1401 to 1368 cycles (2.5% faster) 32x32: 7403 to 7367 cycles (0.5% faster) Total encode time of first 50 frames of bus @ 1500kbps (speed 0) goes from 1min39.2 to 1min38.6, i.e. a 0.67% overall speedup. Change-Id: I799a49460e5e3fcab01725564dd49c629bfe935f
2013-05-03Separate transform and quant from vp9_encode_sbJohn Koleszar
This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-02Create common vp9_encode_sb{,y}John Koleszar
Creates a common encode (subtract, transform, quantize, optimize, inverse transform, reconstruct) function for all sb sizes, including the old 16x16 path. Change-Id: I964dff1ea7a0a5c378046a069ad83495f54df007
2013-04-30Remove unused quantize optimizations.Johann
Files were copied from vp8 and never maintained. Change-Id: I9659a8755985da73e8c19c3c984423b6666d8871
2013-04-25quantize: make 4x4, 8x8 common with larger transformsJohn Koleszar
There were 4 variants of the quantize loop in vp9_quantize.c, now there is 1. Change-Id: Ic853393411214b32d46a6ba53769413bd14e1cac
2013-04-11Remove unused macroblock versions of reconstruction functions.Ronald S. Bultje
More specifically, remove vp9_quantize_mb*, vp9_optimize_mb*, vp9_inverse_transform_mb* and vp9_transform_mb*. Instead, use the generic _sb* functions that take a size argument, and call them with BLOCK_SIZE_MB16X16. Change-Id: I33024afea95d3a23ffbc1df7da426e4645110f29
2013-04-09Make SB coding size-independent.Ronald S. Bultje
Merge sb32x32 and sb64x64 functions; allow for rectangular sizes. Code gives identical encoder results before and after. There are a few macros for rectangular block sizes under the sbsegment experiment; this experiment is not yet functional and should not yet be used. Change-Id: I71f93b5d2a1596e99a6f01f29c3f0a456694d728
2013-04-04Move EOB to per-plane dataJohn Koleszar
Continue migrating data from BLOCKD/MACROBLOCKD to the per-plane structures. Change-Id: Ibbfa68d6da438d32dcbe8df68245ee28b0a2fa2c
2013-03-26Add col/row-based coefficient scanning patterns for 1D 8x8/16x16 ADSTs.Ronald S. Bultje
These are mostly just for experimental purposes. I saw small gains (in the 0.1% range) when playing with this on derf. Change-Id: Ib21eed477bbb46bddcd73b21c5c708a5b46abedc
2013-03-07Re-add support for ADST in superblocks.Ronald S. Bultje
This also changes the RD search to take account of the correct block index when searching (this is required for ADST positioning to work correctly in combination with tx_select). Change-Id: Ie50d05b3a024a64ecd0b376887aa38ac5f7b6af6
2013-03-04Make superblocks independent of macroblock code and data.Ronald S. Bultje
Split macroblock and superblock tokenization and detokenization functions and coefficient-related data structs so that the bitstream layout and related code of superblock coefficients looks less like it's a hack to fit macroblocks in superblocks. In addition, unify chroma transform size selection from luma transform size (i.e. always use the same size, as long as it fits the predictor); in practice, this means 32x32 and 64x64 superblocks using the 16x16 luma transform will now use the 16x16 (instead of the 8x8) chroma transform, and 64x64 superblocks using the 32x32 luma transform will now use the 32x32 (instead of the 16x16) chroma transform. Lastly, add a trellis optimize function for 32x32 transform blocks. HD gains about 0.3%, STDHD about 0.15% and derf about 0.1%. There's a few negative points here and there that I might want to analyze a little closer. Change-Id: Ibad7c3ddfe1acfc52771dfc27c03e9783e054430
2013-02-27Move eob from BLOCKD to MACROBLOCKD.Ronald S. Bultje
Consistent with VP8. Change-Id: I8c316ee49f072e15abbb033a80e9c36617891f07
2013-02-15Remove some Y2-related code.Ronald S. Bultje
Change-Id: I4f46d142c2a8d1e8a880cfac63702dcbfb999b78
2013-01-10Merge tx32x32 experiment.Ronald S. Bultje
Change-Id: I615651e4c7b09e576a341ad425cf80c393637833
2013-01-08Merge superblocks (32x32) experiment.Ronald S. Bultje
Change-Id: I0df99742029834a85c4933652b0587cf5b6b2587
2012-12-18Use standard integer types for pixel values and coefficients.Ronald S. Bultje
For coefficients, use int16_t (instead of short); for pixel values in 16-bit intermediates, use uint16_t (instead of unsigned short); for all others, use uint8_t (instead of unsigned char). Change-Id: I3619cd9abf106c3742eccc2e2f5e89a62774f7da
2012-12-0732x32 transform for superblocks.Ronald S. Bultje
This adds Debargha's DCT/DWT hybrid and a regular 32x32 DCT, and adds code all over the place to wrap that in the bitstream/encoder/decoder/RD. Some implementation notes (these probably need careful review): - token range is extended by 1 bit, since the value range out of this transform is [-16384,16383]. - the coefficients coming out of the FDCT are manually scaled back by 1 bit, or else they won't fit in int16_t (they are 17 bits). Because of this, the RD error scoring does not right-shift the MSE score by two (unlike for 4x4/8x8/16x16). - to compensate for this loss in precision, the quantizer is halved also. This is currently a little hacky. - FDCT and IDCT is double-only right now. Needs a fixed-point impl. - There are no default probabilities for the 32x32 transform yet; I'm simply using the 16x16 luma ones. A future commit will add newly generated probabilities for all transforms. - No ADST version. I don't think we'll add one for this level; if an ADST is desired, transform-size selection can scale back to 16x16 or lower, and use an ADST at that level. Additional notes specific to Debargha's DWT/DCT hybrid: - coefficient scale is different for the top/left 16x16 (DCT-over-DWT) block than for the rest (DWT pixel differences) of the block. Therefore, RD error scoring isn't easily scalable between coefficient and pixel domain. Thus, unfortunately, we need to compute the RD distortion in the pixel domain until we figure out how to scale these appropriately. Change-Id: I00386f20f35d7fabb19aba94c8162f8aee64ef2b
2012-12-03Remove ARM optimizations from VP9Johann
Change-Id: I9f0ae635fb9a95c4aa1529c177ccb07e2b76970b
2012-11-30google style guide include guardsJim Bankoski
Change-Id: I2c252f3ddcc99e96c1f5d3dab8bcb25a2a3637ea
2012-11-28fixed includes to be fully specifiedJim Bankoski
Change-Id: Ia1cce221f8511561b9cbd8edb7726fbc286ff243
2012-11-27Add vp9_ prefix to all vp9 filesJohn Koleszar
Support for gyp which doesn't support multiple objects in the same static library having the same basename. Change-Id: Ib947eefbaf68f8b177a796d23f875ccdfa6bc9dc