summaryrefslogtreecommitdiff
path: root/vp9/encoder/vp9_quantize.c
AgeCommit message (Collapse)Author
2013-11-12Moving q_index from MACROBLOCKD to MACROBLOCK.Dmitry Kovalev
Moving because q_index is used only by encoder. Change-Id: I0b96175614ed4fd3d76ee56a0ba36258e1e896f6
2013-11-05Cleaning up vp9_quantize_b_c() function.Dmitry Kovalev
Change-Id: I42c75530a8c9cff68480657f074131e6b60d9fca
2013-10-29Adding const to vp9_quantize_b_{32x32,} parameters.Dmitry Kovalev
Change-Id: I56f8c50ac382202f66040cd9cfaa05d889572fc7
2013-10-28Cleaning up vp9_regular_quantize_b_4x4.Dmitry Kovalev
Passing scan & iscan as parameters, adding useful local variables. Change-Id: Ia2a87906941db9557350d273669ce5c3cdb7235d
2013-10-16Get rid of "this_mi", use "mi_8x8[0]" everywhere insteadGuillaume Martres
The only case where they were intentionally pointing to different structures was in mbgraph, and this didn't have the expected behavior because both of these pointers are used interchangeably through the code Change-Id: I979251782f90885fe962305bcc845bc05907f80c
2013-10-16Implement variance-based adaptive quantizationGuillaume Martres
This should be similar to what x264 does with --aq-mode 1. It works well with clips like parkjoy and touhou (http://x264.nl/developers/Dark_Shikari/LosslessTouhou.mkv). At low bitrates, the segmentation signaling overhead may negate the benefits of this feature. (PGW) Default changed to feature OFF to allow provisional merge. Change-Id: I938abf9bb487e1d4ad3b0264ea03d9826275c70b
2013-09-24Removing redundant 'extern' keyword.Dmitry Kovalev
Change-Id: Ie51306689c0dc527a8aa12d3984389dd8f360dea
2013-09-13Merge "New mode_info_context storage -- undo revert"Scott LaVarnway
2013-09-11New mode_info_context storage -- undo revertScott LaVarnway
mode_info_context was stored as a grid of MODE_INFO structs. The grid now constists of pointers to MODE_INFO structs. The MODE_INFO structs are now stored as a stream (decoder only), eliminating unnecessary copies and is a little more cache friendly. Change-Id: I031d376284c6eb98a38ad5595b797f048a6cfc0d
2013-09-10Remove redundant condition check in 32x32 quantJingning Han
The c code implementation of 32x32 quantization does the zbin check of all coefficients prior to the quant/dequant loop, hence removing the redundant zbin check inside the loop. This only affects the c code version. SSSE3 version does not separate the zbin check out. Change-Id: Ic197a7d61d0b25fcac3cc092987651378cb56e4e
2013-09-09Merge "Revert "New mode_info_context storage""James Zern
2013-09-09Revert "New mode_info_context storage"James Zern
This reverts commit dae17734ece414091ba1184f7becd0aa6c0004f1 Encode crashes, leaks and increases integer overflow errors. Change-Id: I595aa2649bb8d0b6552ff91652837a74c103fda2
2013-09-08Merge "New mode_info_context storage"Jim Bankoski
2013-09-06Fix overflow issue in 16x16 quantization SSSE3Jingning Han
The 16x16 transform unit test suggested that the peak coefficient value can reach 32639. This could cause potential overflow issue in the SSSE3 implmentation of 16x16 block quantization. This commit fixes this issue by replacing addition with saturated addition. Change-Id: I6d5bb7c5faad4a927be53292324bd2728690717e
2013-09-06New mode_info_context storageScott LaVarnway
mode_info_context was stored as a grid of MODE_INFO structs. The grid now constists of a pointer to a MODE_INFO struct and a "in the image" flag. The MODE_INFO structs are now stored as a stream, eliminating unnecessary copies and is a little more cache friendly. For the test clips used, the decoder performance improved by ~4.3% (1080p) and ~9.7% (720p). Patch Set 2: Re-encoded clips with latest. Now ~1.7% (1080p) and 5.9% (720p). Change-Id: I846f29e88610fce2523ca697a9a9ef2a182e9256
2013-09-05Use saturated addition in SSSE3 of 32x32 quantJingning Han
The 32x32 forward transform can potentially reach peak coefficient value close to 32700, while the rounding factor can go upto 610. This could cause overflow issue in the SSSE3 implementation of 32x32 quantization process. This commit resolves this issue by replacing the addition operations with saturated addition operations in 32x32 block quantization. Change-Id: Id6b98996458e16c5b6241338ca113c332bef6e70
2013-08-29Fix overflow issue in SSSE3 32x32 quantizationJingning Han
The 32x32 quantization process can potentially have the intermediate stacks over 16-bit range, thereby causing enc/dec mismatch. This commit fixes this overflow issue in the SSSE3 implementation, as well as the prototype, of 32x32 quantization. This fixes issue 607 from webm@googlecode. Change-Id: I85635e6ca236b90c3dcfc40d449215c7b9caa806
2013-08-19Moving plane_block_idx from vp9_blockd.h to vp9_quantize.c.Dmitry Kovalev
Change-Id: Ib8af21f2e7f603c2fb407e5d15a3bba64b545b49
2013-08-15Moving segmentation struct from MACROBLOCKD to VP9_COMMON.Dmitry Kovalev
VP9_COMMON is the right place to segmentatation struct because it has global segmentation parameters, not something specific to macroblock processing. Change-Id: Ib9ada0c06c253996eb3b5f6cccf6a323fbbba708
2013-08-12Quantization code cleanup.Dmitry Kovalev
Change-Id: I77b42418b852093f79260cbd880533a0bd86678f
2013-08-09Inlining 16 as a stride for BLOCK_OFFSET macro.Dmitry Kovalev
Change-Id: I7f23d174eb089e5500f268a10db09648634c1b82
2013-07-15Inline vp9_quantize() in xform_quant().Ronald S. Bultje
Cycle times: 4x4: 151 to 131 cycles (15% faster) 8x8: 334 to 306 cycles (9% faster) 16x16: 1401 to 1368 cycles (2.5% faster) 32x32: 7403 to 7367 cycles (0.5% faster) Total encode time of first 50 frames of bus @ 1500kbps (speed 0) goes from 1min39.2 to 1min38.6, i.e. a 0.67% overall speedup. Change-Id: I799a49460e5e3fcab01725564dd49c629bfe935f
2013-07-11Moving segmentation related vars into separate struct.Dmitry Kovalev
Adding segmentation struct to vp9_seg_common.h. Struct members are from macroblockd and VP9Common structs. Moving segmentation related constants and enums to vp9_seg_common.h. Change-Id: I23fabc33f11a359249f5f80d161daf569d02ec03
2013-07-01Update quantize SSSE3 SIMD to cover 32x32 transform case also.Ronald S. Bultje
Encode time of bus (speed 0) 50 frames @ 1500kbps goes from 2min14.4 to 2min10.1, i.e. a 2.3% overall speed increase. Change-Id: I3699580e74ec26c7d24e03681bc47ba25ee1ee87
2013-07-01Quantize (64-bit only, for now) SSSE3 SIMD.Ronald S. Bultje
Total encoding time for first 50 frames of bus (speed 0) @ 1500kbps goes 2min34.8 to 2min14.4, i.e. a 10.4% overall speedup. The code is x86-64 only, it needs some minor modifications to be 32bit compatible, because it uses 15 xmm registers, whereas 32bit only has 8. Change-Id: I2df53770c2e850813ffa713e1a91b45b0082b904
2013-06-28Make coefficient skip condition an explicit RD choice.Ronald S. Bultje
This commit replaces zrun_zbin_boost, a method of biasing non-zero coefficients following runs of zero-coefficients to be rounded towards zero, with an explicit skip-block choice in the RD loop. The logic is basically that if individual coefficients should be rounded towards zero (from a RD point of view), the trellis/optimize loop should take care of it. If whole blocks should be zero (from a RD point of view), a single RD check is much more efficient than a complete serialization of the quantization loop. Quality change: derf +0.5% psnr, +1.6% ssim; yt +0.6% psnr, +1.1% ssim. SIMD for quantize will follow in a separate patch. Results for other test sets pending. Change-Id: Ife5fa641163ac5150ac428011e87188f1937c1f4
2013-06-27Inline quantize so idiv instruction gets removed from inner loop.Ronald S. Bultje
Encoding time of first 50 frames of bus @ 1500kbps (speed 0) goes from 3min15.0 to 3min10.9, i.e. 2.1% faster overall. Change-Id: If592ee99be09bcd34a7c8498347f44e7305e982c
2013-06-19Add two-pass quantizationYunqing Wang
Optimized the quantization function by making it a two-pass process. The first pass does a quick checking of the transform coefficients against the base ZBIN, and only keep the good enough set of coefficients for quantization. A skipping check is added. If all coefficients are within the base ZBIN, no quantization is needed. The second pass is the actual quantization pass, which only processes the coefficient subset determined in first pass. This reduces the computation. Furthermore, an alternitive method is used for large transform size, which often has sparse nonzero quantized coefficients. Overall, the encoder speedup is about 4%. The quantization function itself gets 20% faster. Change-Id: I3a9dd0da6db030260b6d9c314a9fa48ecae89f22
2013-05-23Merge Scatter Scan experiment.Paul Wilkins
Removal from under configure flag. A bit renaming Change-Id: I2213229dfe852001dfec16b149f47c52ce88f3aa
2013-05-16Initial version of alpha channel supportJohn Koleszar
This is a mostly-working implementation of an extra channel in the bitstream. Configure with --enable-alpha to test. Notable TODOs: - Add extra channel to all mismatch tests, PSNR, SSIM, etc - Configurable subsampling - Variable number of planes (currently always uses all 4) - Loop filtering - Per-plane lossless quantizer - ARNR support This implementation just uses the same contents as the Y channel for the A channel, due to lack of content and general pain in playing back 4 channel content. A later patch will use the actual alpha channel passed in from outside the codec. Change-Id: Ibf81f023b1c570bd84b3064e9b4b8ae52e087592
2013-05-07Renaming Y1 and UV quant prefixes to y_ and uv_.Dmitry Kovalev
Change-Id: If57e360c187a475fc90edb8c7170f498efcb31a5
2013-05-07Adjust q rangePaul Wilkins
Skip Q values between the q.0 mode and a real q of 2.0 as these are not valuable from an RD perspective. Change-Id: I110c4858c57f97315953f4d88a2596d4764360df
2013-05-07Merge SB8X8 into the codebaseJingning Han
Pull sb8x8 out of experimental list. verified via borg run tests. Fixed unit test failures. Change-Id: I12a4bbd17395930580c048ab68becad1ffe46e76
2013-05-03Merge "Separate transform and quant from vp9_encode_sb" into experimentalJohn Koleszar
2013-05-03Separate transform and quant from vp9_encode_sbJohn Koleszar
This allows removing a large number of transform size specific functions, as well as supporting 444/alpha by routing all code through the subsampling-aware path. Change-Id: Ieb085cebe9f37f24fc24de179898b22abfda08a4
2013-05-03Merge "Create common vp9_encode_sb{,y}" into experimentalJohn Koleszar
2013-05-03Merge "Reduced y_dequant, uv_dequant size" into experimentalScott LaVarnway
2013-05-02Create common vp9_encode_sb{,y}John Koleszar
Creates a common encode (subtract, transform, quantize, optimize, inverse transform, reconstruct) function for all sb sizes, including the old 16x16 path. Change-Id: I964dff1ea7a0a5c378046a069ad83495f54df007
2013-05-01Reduced y_dequant, uv_dequant sizeScott LaVarnway
Currently, only two values are used. Removed the unused values. Change-Id: Idc5b8be354d84ffc68df39ea3e45f9f50d977b35
2013-05-01Merge "Adding vp9_get_qindex function." into experimentalDmitry Kovalev
2013-04-30sb8x8 integration in rd loop.Ronald S. Bultje
Work-in-progress, not yet ready for review. TODO items: - bitstream writing (encoder) and reading (decoder) - decoder reconstruction Change-Id: I5afb7284e7e0480847b47cd0097cb469433c9081
2013-04-30Adding vp9_get_qindex function.Dmitry Kovalev
Moving common code from encoder and decoder to vp9_get_qindex function. Also moving quant-related constants from vp9_onyxc_int.h to vp9_quant_common.h. Change-Id: I70c5bfbaa1c8bf00fde0bfc459d077f88b6d46c8
2013-04-26Consistent names for quant-related functions and variables.Dmitry Kovalev
Change-Id: I3a6d601e90e8740b9c26dd0afbfe9d467b75d367
2013-04-26Merge "quantize: make 4x4, 8x8 common with larger transforms" into experimentalJohn Koleszar
2013-04-25quantize: make 4x4, 8x8 common with larger transformsJohn Koleszar
There were 4 variants of the quantize loop in vp9_quantize.c, now there is 1. Change-Id: Ic853393411214b32d46a6ba53769413bd14e1cac
2013-04-25Use b_width/height_log2 instead of mb_ where appropriate.Ronald S. Bultje
Basic assumption: when talking about transform units, use b_; when talking about macroblock indices, use mb_. Change-Id: Ifd163f595d4924ff892de4eb0401ccd56dc81884
2013-04-25Move dequant from BLOCKD to per-plane MACROBLOCKDJohn Koleszar
This data can vary per-plane, but not per-block. Change-Id: I1971b0b2c2e697d2118e38b54ef446e52f63c65a
2013-04-23Move skip_block from BLOCK to MACROBLOCKJohn Koleszar
This data is fixed at the MB level, so move it to the common part of MACROBLOCK. Change-Id: Idd8c87118e501cdf0a202bd84c28b502a8234edf
2013-04-23Move quantizer data from BLOCK to MACROBLOCKJohn Koleszar
Quantizers can vary per plane, but not per block. Move these values to the per-plane part of MACROBLOCK. Change-Id: I320a55e38b7b28b29aec751a4aca5ccd0c9b9326
2013-04-23Merge "Convert coeff to per-plane MACROBLOCK data" into experimentalJohn Koleszar
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-11 07:28:38 +0000