summaryrefslogtreecommitdiff
path: root/vp9/encoder/vp9_pickmode.c
AgeCommit message (Collapse)Author
2019-05-10Cast buffer offset to int64_tJerome Jiang
To prevent integer overflow with extreme frame sizes. Change-Id: Ib77f1c11f0264257d9e6c162f474d637592e7b09
2019-05-10Reland "vp9-rtc: tx_size selection for intra mode in nonrd"Jerome Jiang
Reland this change since tsan failure is fixed. Change-Id: I20e3d3d23e34befabb43a36d491d27dfc2a908b6
2019-05-09Fix tsan failure in webrtc test.Jerome Jiang
plane block size is used when computing model rd for uv. However, it iterates thru sub-blocks based on tx size on uv planes and plane block size could be bigger than that, which leads to reading beyond tile boundary when the block is on it. BUG=b/131414589 Change-Id: I362091484b1325b89d2175039323b235a06ebffc
2019-05-06Revert "vp9-rtc: tx_size selection for intra mode in nonrd"Jerome Jiang
This reverts commit cdd40d1cd042d8c139b3426f311b8b5722d696bb. Cause tsan failure in webrtc tests. BUG=b/131414589 Change-Id: I04f98153bc1f9d013d3d1eb8d06df312fe12f8b4
2019-04-25vp9-rtc: Adjust thresh for 4x4 tx selectionMarco Paniconi
For screen content nonrd_pickmode: reduce threshold to select 4x4 tx_size, under certain conditions. Change-Id: If68c30172272868033f0e3011e53c76b4e7c48b6
2019-04-25vp9-rtc: Fix int conversion error in nonrd_pickmode.Marco Paniconi
Change-Id: I1be775d8c11f530ff26121f1ffaf1dae100b2510
2019-04-24vp9-rtc: Pass source variance and mode to select txMarco Paniconi
For nonrd-pickmode: pass the source variance and the mode (intra/inter) to select tx_size, for better tuning. Neutral change for video mode, speed 7. Some quality improvement for screen content. Change-Id: I53336f23fa4f14076aa1cdf8036e9af73c43060a
2019-04-23vp9-rtc: tx_size selection for intra mode in nonrdMarco Paniconi
In nonrd_pickmode for intra modes: add tx_size selection based on Y prediction signal for the bsize. The tx selection is done in model_rd, same as inter-modes. Existing code for intra mode was first setting a tx_size based only on the bsize, and then in some cases in block_yrd (during the loop over bsize in units of tx_size) the tx_size may be set again if model_rd is called in block_yrd. This CL separates out the tx_size setting (based on Y channel prediction via model_rd), and then block_yrd is called once for whole bsize. This allows for better tuning of the tx selection for intra modes in future change. Adjust threshold in svc datarate test. Negligible/neutral change in psnr/ssim metrics for speed 7 and 8, 1 layer and SVC mode. Change-Id: I33bc8447afdc3785482e13aac5c3636e13c59644
2019-04-19Merge "vp9: Use model rd for large blocks in filter search."Jerome Jiang
2019-04-18vp9: Use model rd for large blocks in filter search.Jerome Jiang
Neutral on speed and quality. Change-Id: Ia1d3929124bb57e31bbab516a994734d2fd51891
2019-04-19Merge "vp9-rtc: Use correct plane for UV in estimate_intra"Marco Paniconi
2019-04-18vp9-rtc: Use correct plane for UV in estimate_intraMarco Paniconi
For nonrd-pickmode: some PSNR increase observed on screen content/scroll clips. Change-Id: Idf1bce9dd434e33d7c35dbeb59e02e2e58ea1aaa
2019-04-18vp9-rtc: Move setting for use_model_rd_largeMarco Paniconi
Move the setting to just before the inter-mode loop, as for screen content the value may change due to reset of segment. No change in behavior except for screen-content. Change-Id: I256795b581ceda352e57b88eba2e86aa18b0fdc4
2019-04-17Merge "vp9: refactor condtions for model rd for large blocks."Jerome Jiang
2019-04-16vp9: refactor condtions for model rd for large blocks.Jerome Jiang
Change-Id: If474273642b68e29abacb7b87cbb6e3c91bb93a4
2019-04-16vp9-rtc: Add speed feature to force SMOOTH filterMarco Paniconi
Add speed feature for real-time to always force SMOOTH filter for subpel motion. Can be useful in some cases for noisy content or high motion at low bitrate. Also some speedup in avoiding the checking of two filters. Keep it off always for now. Change-Id: I843d79aaddef75f9c6ded60906cc75c279a6e37a
2019-04-10vp9-rtc: Fix to re-eval zero-mv for denoisingMarco Paniconi
Change-Id: I3bb0646661efa06c8d1d688c746e41855c99f408
2019-04-09vp9-rtc: Speed feature for filter_search in nonrd_pickmode.Marco Paniconi
Use chessboard search only for certain speeds/resolns (speed >= 8) for real-time speed features. Disable chessboard search for speeds <= 7. ~2.5 gain on rtc set for speed 7. ~1% slowdown. Change-Id: Ic6898aa475817e128154f691413c73f65306e2a8
2019-04-09vp9-rtc: Fix to rate cost for switchable filterMarco Paniconi
Add consistent switchable rate cost, which should be only when non-integer motion mode is tested. Neutal/negligible change in metrics. Also diable the re-evaluation of ZEROMV mode after denoising feature, as this rate cost fix exposed an exsting issue with this feature. Change-Id: I9e5479281810a392b9a409e238c564b2def8e546
2019-03-24Use high bd path to substract blocks in hbd build.Jerome Jiang
BUG=webm:1609 Change-Id: Ifc15d616e7cfb247b399def64ef7691589d90075
2019-03-21Merge "vp9: remove condition on high bitdepth using simple block yrd."Jerome Jiang
2019-03-20vp9-rtc: Fixes for low-resolnMarco Paniconi
The force smooth_filter should only be used for noisy content, so for now keep it off and add TODO. Also fix/adjust low-resoln condition and threshold in cyclic refresh. Change-Id: I6c456dc9f23daabba20badd65a2f7ee6c5e259c4
2019-03-19vp9-rtc: Adjustments for base layer in screen.Marco Paniconi
On scene/content changes for base layer of screen: reduce 32x32 split threshold, bias rdcost for flat blocks if sse_y is non-zero, and avoid early exit on intra-check. Reduce artifacts in scroll content. Change-Id: I144357a61462351173af900e0b8a47dac4aad6ca
2019-03-19vp9: remove condition on high bitdepth using simple block yrd.Jerome Jiang
Speed is similar between non HBD vs 8bit with HBD build. BUG=webm:1541 Change-Id: I8b5f7eff87ec7dc4710d31744155a60e50b0f0a9
2019-03-15vp9-rtc: Some adjustments for low-resolns real-timeMarco Paniconi
Force smooth_interpol filter for low resolutions at high Q, avoid the loopfilter strength reduction for similar conditon, and reduce thresh_motion for cyclic refresh turnoff. Change-Id: I4e9121d1cdc7d1b04992c741dc4f0cec281592f7
2019-03-10vp9-screen: Rework the mode skip logic for screenMarco Paniconi
Don't force skip of zero-golden reference when zero_temp_sad_source = 0, as it be may the inter-layer reference. And remove the flatness conditon when superblock is static. Change-Id: I6b4b6eac0f6a2abc862c23d0e5467c7cf61995ef
2019-03-08vp9-screen: Fix to screen wth layered encodingMarco Paniconi
zero_temp_sad_source is only computed when compute_source_sad_onepass and sf->use_source_sad are on, which currently is only for the top layer of the layered encoding. So qualify the usage of zero_temp_sad_source on those flags. This affects the quality/speed of the lower layers of screen content mode when SVC (quality layers) are used. Change-Id: I54167265a05a4b918ce015931375aa42d3e75cf5
2019-03-06vp9-rtc: Improve mode check on flat blocks in screen modeMarco Paniconi
For nonrd-pickmode in screen content mode: modify logic for inter and intra mode check for spatially flat blocks. Condition skip of non-zero/zero inter mode based on zero_temp_sad_source, and force intra/DC check regardless. Reduces artifacts in scrolling motion. Change-Id: Iee75cd19d03296afeb649c5bce628806103769ae
2019-03-06vp9-denoiser: Bias to last for golden long termMarco Paniconi
If golden referene is selected as long-term reference, bias the denoiser filter to use last reference. Fixes visual artifact. And reduce the thresh_svc_golden, which was used to reduce the artifact occurrence. Change-Id: I08f24160ca11bd8f5f70acaefe989d5f92988132
2019-02-28vp9-rtc: Reduce thresholds for skip goldenMarco Paniconi
For nonrd-pickmode CBR mode: reduce the skip golden ref thresholds, to reduce some psnr regression in some clips, while still effectively reducing flashing block artifact occurrence. Change-Id: I468dcf5354411aeb54ac3ef56c6fb73267d93fde
2019-02-27Merge "vp9-rtc: Add cyclic_refresh condition to segment reset"Marco Paniconi
2019-02-27vp9-rtc: Add cyclic_refresh condition to segment resetMarco Paniconi
In non-rd pickmode for screen content: this logic to reset segment should only be for cyclic_refresh mode on, so add that condition explicitly. There may be other uses of segments, like ROI, so we should condition this reset logic on cyclic_refresh, as it was intended for that mode only. Change-Id: I954e6cee968fbca35b34286c4a7ca2531c8e9823
2019-02-27vp9-rtc: Modify skip golden mode check for 1 layerMarco Paniconi
For real-time CBR mode: golden reference mode testing is skipped under certain conditons based on sse of zero-last mode. This was done for svc mode. Here we add similar condition for non-svc/1 layer encoding. Reduces flashing block artifacts that can occur in background areas with noise. Change-Id: I93f71ea9507af8c9153fc6c0ba7dcc7a0fa8810d
2019-01-31vp9: Adjust intra check for short_circuit_flat_blocksMarco Paniconi
For non-rd pickmode: include H and V intra mode check for spatially flat blocks when the sf->short_circuit_flat_blocks speed feature is set. Small improvement on screen content tests. Change-Id: I3391d02cce6a46160be6ccc8a1e33fd8547eb467
2019-01-30vp9-svc: Fix to non-rd pickmode for screen contentMarco Paniconi
For screen content mode: always force intra check for spatially flat blocks that have moved. Also adjust/fix condition for forcing check of zeromv-golden for quality layers. Reduces artifacts in screensharing tests. Change-Id: Iafd62fb24a4e05f5b12af663dde2805fdb4c7b36
2019-01-29vp9-svc: Modify early breakout for non-rd pickmodeMarco Paniconi
Modify early breakout condition for non-rd pickmode for quality layers: when lower layer has lower QP force test of zeromv on golden (lower layer reference) before breakout due to skip. Reduce artifacts, observed in cases of scrolling content. Change-Id: Id834b1eb024a4c97f0e74d8b7f7a0351459e088f
2018-12-17Localize x->encode_breakout setup to non-rd mode searchJingning Han
The breakout speed feature is currently only used by the non-rd mode search path. Localize it to simplify set_offset() logic. Change-Id: I27e7519c987a7caac2e4bd6be0ede1b9c8320e55
2018-12-07Add enum definition for subpel search precisionHui Su
To improve readability. Change-Id: Idc08b2068c7d8ba9dadc0d559a3b4d61c2a88c94
2018-12-04vp9: force refresh of long term ref when denoiser reset.Jerome Jiang
This will allocate extra frame buffer if long term temporal reference is used and denoiser is enabled on non-key frame. Add test. Change-Id: I0e8d1fdb9a2d697a8eed7fe6206bcb362e69f1c8
2018-11-26vp9-svc: Put check on usage of long term temporal ref.Marco Paniconi
If the scale factor of the golden long term reference is different from the last reference then disable usage of long term reference. This should not happen, but add this as a check against some possibly incorrect update of the svc configuration. Change-Id: Ic1062d4384e005007d8c922813fa8ad188d8fa98
2018-10-05Changes to facilitate accurate sub-pel motion searchYunqing Wang
This patch included changes to facilitate accurate sub-pel motion search. More patch will follow to turn on accurate sub-pel motion search. Change-Id: I224c28c338353fe5c7609372162f79885c54248f
2018-09-21Improve subpel MV search for speed 1Hui Su
Do one more subpel MV search each round. This improves coding efficiency slightly: lowres 0.12% midres 0.11% hdres 0.13% Also renames the control flag for subpel MV search quality. Encoding speed loss is less than 1%. This only affects speed 1. Change-Id: I3aecd25342f2dcacea6c143db494f7db6282cb92
2018-08-02vp9: new struct BEST_PICKMODE containing search results.Jerome Jiang
One of steps to refactor the nonrd pickmode. Simplify arguments list for functions. Change-Id: I5f23375caa36be2ae0fbd2ff851b303150a7aa8f
2018-08-01vp9: Refactor nonrd pickmode: new mv searchJerome Jiang
Move new mv search to a separate function. Change-Id: I6ef22d03ccad7b87cb5cd611094de204d508f63e
2018-08-01vp9: Refactor nonrd pickmode: interp filter.Jerome Jiang
Move interp filter search to new function. Change-Id: I6ac57d5b3800c9944732a84a4d4a825a6c0f4c35
2018-08-01vp9: Refactor nonrd pickmode: tx_size.Jerome Jiang
Remove duplicated code to calculate transform size. Change-Id: Id71772607eea911f24b59168c0629ba5ff891afb
2018-07-31vp9: Clamp tx_size in model_rd_largeMarco Paniconi
For nonrd_pickmode: add clamp/check to make sure tx_size is not set to lower than 8X8, for the model_rd_large function (which is only called for big block sizes). No change in behavior. Change-Id: I9c6093068e406ac16cfd6784ba75868906225378
2018-07-31vp9: Remove assert from model_rd in non-rd pickmode.Marco Paniconi
The assert checks for tx_size >= 8x8, but 4x4 can be set in some cases. Change-Id: I8bf9683e1add768becaa1208e1709ad0470e3850
2018-07-26vp9: 4x4 tx_size for nonrd-pickmode for screen contentMarco Paniconi
Force 4x4 transform size under some conditions for real-time screen-content mode. Improvemet on text in some screen clips. Change-Id: I77cafa23ea1060ef4334dc07eac53189bf80e0ec
2018-07-26Merge "vp9: Modify condition for force test of intra"Marco Paniconi
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-12 05:23:45 +0000