summaryrefslogtreecommitdiff
path: root/vp8
AgeCommit message (Collapse)Author
2011-11-21Decoder segmentation bug.Paul Wilkins
Fix decoder segmentation bug for temporal coding where the segment map was first initialized on a key frame. in vp8_kfread_modes() after reading the segment id it must be written to the pbi->segmentation_map[] for use in temporal coding on subsequent frames. Change-Id: I1489305efc376564e734a216f69c2844646ee3d3
2011-11-21CONFIG_T8X8 experiment.:Paul Wilkins
Block the selection of 4x4 modes in key frames if 8x8 is selected. Change-Id: Ie5729ec22a999d9a1996f020bd4b941e29514992
2011-11-17Clip buffer level to the maximum buffer size in CBRAdrian Grange
The buffer level was able to increase indefinitely rather than being clipped to the maximum buffer size specified by the user. This change checks the buffrer level and prevents it from going beyond the upper limit of the buffer. Change-Id: Ifff55f79d3c018e4d3d77e554b11ada543cc1654
2011-11-17Merge "fixed the scaling in 8x8 trellis quant" into experimentalYaowu Xu
2011-11-16fixed the scaling in 8x8 trellis quantYaowu Xu
This commit has a few minor fixes to the 8x8 trellis quant, so to make it work regardless if extend_qrange is enabled or not. It also borrowed adaptive RDMULT constants from 4x4 trellis that was missed in the 8x8 trellis quant. Change-Id: I60d7769071f102c699b5084597e62bca87a1f759
2011-11-16Header inclusion for Unix buildPaul Wilkins
Explicit inclusion of limits.h to satisfy unix build for definition of INT_MAX. Some commented out code removed. Change-Id: I5b5980dfaa9b4d2d12bfd729cfd35bd982106908
2011-11-15Merge CONFIGURE_SEGMENTATION experiment.Paul Wilkins
Removal of CONFIGURE_SEGMENTATION ifdefs. Removal of legacy support code fo the old coding mechanism. Use local reference "xd" for MACROBLOCKD structure in encode_frame_to_data_rate() Moved call to choose_segmap_coding_method() out of encode loop as the cost of segmentation is not properly accounted in the loop anyway. If this is desirable in the future it can be moved back. The use of this function to do all the analysis and set the probabilities also removes the need to track segment useage in threading code. Change-Id: I85bc8fd63440e7176c73d26cb742698f9b70cade
2011-11-15Further clean up of Segmentation experiment codePaul Wilkins
Changed name and sense of segment_flag to "seg_id_predicted" Added some additional comments and retested. I also did some experimentation with a spatial prediction option using a similar strategy to the temporal mode implemented. This helps in some cases where temporal prediction is bad but I suspect there is more overlap here with work on a larger scale block structure and spatial correlation will likely be better handled through that mechanism. Next check in will remove #ifdefs and legacy mode code. Change-Id: I3b382b65ed2a57bd7775ac0f3a01a9508a209cbc
2011-11-15Further work on Segmentation Experiment:Paul Wilkins
This check in includes quite a lot of clean up and refactoring. Most of the analysis and set up for the different coding options for the segment map (currently simple distribution based coding or temporaly predicted coding), has been moved to one location (the function choose_segmap_coding_method() in segmenation.c). This code was previously scattered around in various locations making integration with other experiments and modification / debug more difficult. Currently the functionality is as it was with the exception that the prediction probabilities are now only transmitted when the temporal prediction mode is selected. There is still quite a bit more clean up work that will be possible when the #ifdef is removed. Also at that time I may rename and alter the sense of macroblock based variable "segment_flag" which indicates (1 that the segmnet id is not predicted vs 0 that it is predicted). I also intend to experiment with a spatial prediction mode that can be used when coding a key frame segment map or in cases where temporal prediction does not work well but there is spatial correlation. In a later check in when the ifdefs have gone I may also move the call to choose_segmap_coding_method() to just before where the bitsream is packed (currently it is in vp8_encode_frame()) to further reduce the possibility of clashes with other experiments and prevent it being called on each itteration of the recode loop. Change-Id: I3d4aba2a2826ec21f367678d5b07c1d1c36db168
2011-11-11Segmentation experiment:Paul Wilkins
Added last_segmentation_map[] structure to keep track of what we had before when doing temporal prediction. With this change the existing code does once again appear to be giving a decodable bitstream for both temporal and standard prediction modes. However, it is still somewhat messy and confused and there is no option to take advantage of spatial prediction so it could do with further work. Some housekeeping / clean out. Change-Id: I368258243f82127b81d8dffa7ada615208513b47
2011-11-11SEGMENTATION experiment:Paul Wilkins
Some initial cleanup to aid testing and debug. Pull code to choose temporal or spatial encoding out of encodeframe.c into a dedicated function in segmentation.c. For now disable broken temporal mode. Move the coding of "temporal_update" flag and only transmit if segment map update is indicated. Rename the functions read_mb_features() and write_mb_features() to read_mb_segid() and read_mb_segid() as they only read and write the macroblock segment id not any of the features. Change-Id: Ib75118520b1144c24d35fdfc6ce46106803cabcf
2011-11-11changed function name for clarityYaowu Xu
The dequantizer functions for 2nd order haar block had confusing 8x8 in their names. this commit fixed their name to avoid confusion. Change-Id: I6ae4e7888330865f831436313637d4395b1fc273
2011-11-11Merge "fixed the decoder when using 8x8 transform" into experimentalYaowu Xu
2011-11-11Make 8x8 and extend_qrange to work togetherYaowu Xu
This commit added scaling factors to 8x8 transform, quant, dequant and inverse transform pipeline to make 8x8 transform to work when configed with enable-extend_qrange. This commit also disabled the trellis-quant when extend_qrange is configured. Change-Id: Icfb3192e4746f70a4bb35ad18b7b47705b657e52
2011-11-10fixed the decoder when using 8x8 transformYaowu Xu
updated the decode_macroblock logic to reflect that 8x8 transform is not used for "SPLITMV". Also fixed an issue where 2nd order haar block has wrong dequant/idct process. Change-Id: I1e373f6535c009dfec503b6362c8a5cfc196e1da
2011-11-10Merge "scaled the threshold for 2nd order coefficient reset" into experimentalYaowu Xu
2011-11-10scaled the threshold for 2nd order coefficient resetYaowu Xu
extend_qrange introduces a different scaling factor, this commit takes the scaling difference into account for reset 2nd order coefficients. Change-Id: Ie58bca9f52698fa759e3f88da2aa4d82630fa91a
2011-11-10Merge "T8x8 experiment merge." into experimentalPaul Wilkins
2011-11-10Merge "fixed an encoder bug" into experimentalYaowu Xu
2011-11-09fixed an encoder bugYaowu Xu
the bug caused the encoder to produce invalid bitstream when configured with enable_extend_qrange. Change-Id: I1e81c48b13359d0043cbbd480e679380a2da117c
2011-11-09T8x8 experiment merge.Paul Wilkins
For ease of testing and merging experiments I have removed in line code in encode_frame() that assigns MBs to be t8x8 or t4x4 coded segments and have moved the decision point and segment setup to the init_seg_features0 test function. Keeping everything in one place helps make sure for now that experiments using segmentation are not fighting each other. Also made sure mode selection code can't choose 4x4 modes if t8x8 is selected. Patch2: In init_seg_features() add checks for SEG_LVL_TRANSFORM active. Change-Id: Ia1767edd99b78510011d4251539f9bc325842e3a
2011-11-09Merging and testing of SEGMENTATION experiment.Paul Wilkins
Removed code in #if CONFIG_SEGMENTATION that enables segmentation and creates a test segmentation map, to avoid conflicts with the other segmentation test code, Change-Id: I7a21a44ed188b814cd80b30dd628c62474eba730
2011-11-09Segmentation feature logic fix.Paul Wilkins
Bug fix to logic in vp8_pick_inter_mode() and vp8_rd_pick_inter_mode(). The block on the use of segment features for the cm->refresh_alt_ref_frame case was just for testing and is not correct. The special case code for alt ref can be re-enabled as an else clause. Change-Id: Ic9b57cdb5f04ea7737032b8fb953d84d7717b3ce
2011-11-08make debug match release build on win32 with 8x8 transform enabledYaowu Xu
The 8x8 forward transform makes use of floating operations, therefore requires emms call to reset mmx registers to correct state. Without the resets, the 8x8 forward transform results are indefinite on win32 platform. Change-Id: Ib5b71c3213e10b8a04fe776adf885f3714e7deb1
2011-11-08Attempt to fix an issue related to 8x8 transform and segfeatureYaowu Xu
logically this commit should NOT change anything, but seems to help revert the 3DB loss on bowing in the following commit: https://on2-git.corp.google.com/g/#change,6193 This is still debugging in progress. Need further investigation to understand the root cause of the issue. Change-Id: I0b49d1ef3a311dfff58c6acd3eaebdb3bda6257c
2011-11-08Segment signaling of TX sizePaul Wilkins
Initial attempt at using new segment feature signaling to indicate 4x4 or 8x8 transform. needs --enable-experimental --enable-t8x8 Note this is work in progress. Change-Id: Ib160d46a5d810307bfcbc79853ce1a65b5b870b7
2011-11-04Merge "Added context reset when 2nd order coefficients are cleared" into ↵Yaowu Xu
experimental
2011-11-04Segment Features with 8x8DCT.Paul Wilkins
Temporary check in to turn off other segment features tests when #if CONFIG_T8X8 is set as the assignment of MBs to differnt segments in each case will conflict. The 8x8 code will be modified to use the new segment feature method properly in a later check in. Increase bits allowed for EOB end stop marker to 6 ready for 8x8. Change-Id: I4835bc8d3bf98e1775c3d247d778639c90b01f7f
2011-11-04Segment Feature Data AccessPaul Wilkins
No change to functionality or output. Updates to the segment feature data structure now all done through functions such as set_segdata() and get_segdata() in seg_common.c. The reason for this is to make changing the structures (if needed) and debug easier. In addition it provides a single location for subsequent addition of range and validity checks. For example valid combination of mode and reference frame. Change-Id: I2e866505562db4e4cb6f17a472b25b4465f01add
2011-11-03make uv intra mode coding adaptive to Y modeYaowu Xu
This commit tries to do UV intra mode coding adaptive to Y intra mode. Entropy context is defined as conditional PDF of uv intra mode given the Y mode. All constants are normalized with 256 to be fit in 8 bits. This provides further coding efficiency beyond the quantizer adaptive y intra mode coding. Consistent gains were observed on all clips and all bit rates for HD all key encoding tests. To test, configure with --enable-experimental --enable-uvintra Change-Id: I2d78d73f143127f063e19bd0bac3b68c418d756a
2011-11-03Added context reset when 2nd order coefficients are clearedYaowu Xu
As discovered in path 10 of Change Ia12acd2f, reset 2nd order coeffs without reset of above and left coding context may have introduced problem that causes encoder/decoder mismatching. This commit added update to coding context when the 2nd coefficients are cleared. In addition, this commit also introduced early breakout in the checks to speed up when coefficients are too significant to be cleared. Change-Id: I85322a432b11e8af85001525d1e9dc218f9a0bd6
2011-11-03Segment Features. Removal of #ifdefsPaul Wilkins
Removal of configure #ifdefs so that segment features always available. Removal of code supporting old segment feature method. Still a good deal of tidying up to do. Change-Id: I397855f086f8c09ab1fae0a5f65d9e06d2e3e39f
2011-11-03Merge "Segmentation: Reference frames" into experimentalPaul Wilkins
2011-11-02Merge "Change to prevent encoding of effect-less 2nd order coefficients" ↵Paul Wilkins
into experimental
2011-11-02Segmentation: Reference framesPaul Wilkins
Modify reference frame segmentation so that ONE or MORE reference frames may be marked as a available for a given segment. Fixed bugs relating to segment coding of INTRA and some INTER modes at the segment level. Modified Q boost for static areas based on ambient average Q. Strong results now on clips with significant static areas. (some data points in derf set as high as 9% and some static & slide show type content in YT set > 20%) Change-Id: Ia79f912efa84b977f35a23683ae3643251e24f0c
2011-11-01Deleted repeated code blockAdrian Grange
The block of code skipped testing the current mode if the reference frame is AltRef, the mv is not (0,0) and ARNR filtering is disabled. This block of code has already been tested above if the macro CONFIG_SEGFEATURES is set to 0. Change-Id: I3f5710bb8270caad06c9a0eee59fa0daf1f70776
2011-10-31Fixed this_mode used before set in vp8_pick_inter_modeAdrian Grange
The variable this_mode was being used before it had been initialized. Moved the line that sets-up this_mode toward the top of the enclosing loop, prior to its first use. The bug would result in tests in the loop lagging the mode that was expected to be tested. Change-Id: If4e51600449ce6b4285f112da17a44c24b4a19fb
2011-10-31Segmentation Entropy and tweaks.Paul Wilkins
Some correction for entropy impact of segment signaled (EOB and ref frame) Other slight tweaks. Derf VBR average gain now over 1% (best over 7%) One YT test clip has gains of circa 30% (VBR) There is still an issue with noisy clips where making the background static and coded with 0,0 can have a negative effect, especially at low Q. This is probably because of the loss of smoothing by fractional pixel filters. Change-Id: I7a225613c98067b96f8fc7a7e36f95d465b2b834
2011-10-27Resolve build problemPaul Wilkins
Resolved experimental branch build problem when seg_features not configured. Change-Id: Ia0f9b460a26dc3eac9844ee595a7b196e9faf6a5
2011-10-25Change to prevent encoding of effect-less 2nd order coefficientsYaowu Xu
similar logic to http://gerrit.chromium.org/gerrit/#change,10359 Change-Id: Ia12acd2f2b3b92ef2a601da43c2497034ef62174
2011-10-25Merge "added a last stage rounding for 8x8 inverse dct" into experimentalYaowu Xu
2011-10-24added a last stage rounding for 8x8 inverse dctYaowu Xu
Prior to the added rounding, tests on randomly generated data showed that forward-inverse transform round trip errors are about 3.02/block for input range [-10,10] and 2.68/block for input range [-256, 255]. The added rounding reduced the errors to 0.031/block for input range [-10,10] and 0.037/block for input range [-256, 255]. Maximum round trip error on for any pixel position is 1. The average errors are calculated based on 100,000 blocks of randomly with the specified ranges. Paul mentioned in discussion that the change was not clear on why we need change the rounding, so Patch 2 intends to make the rationale obvious in code, it merged the two separate shifts into one, and the two separate rounding factors into one. Patch 1 and 2 have same numerical test results. Change-Id: Ic5e2f5463de17253084d8b2398c4a210194b20de
2011-10-24Segmentation Features;Paul Wilkins
Only encode sign bit for feature data that can have a sign. Tweaks to the test segmentation rules so that it now actually gives a net benefit on the derf set of about 0.4% though much higher on some clips at the low end. Change-Id: I8e61f1aebf41c9037db7e67e2f8975aa18a0c986
2011-10-24Further segment feature extensions.Paul Wilkins
This quite large check in includes the following: Merge in some code from Ronald (mbgraph.c) that scans a Gf/arf group. This is used as a basis for a simple segmentation for the normal frames in a gf/arf group. This code also uses satd functions from Yaowu. Adds functionality for coding the latest possible position of an EOB for blocks in the segment. (Currently 0-15 only, hence just for 4x4 dct). Where the EOB position is 0 this acts like "skip" and the normal coding of skip at the per mb level is disabled. Added functions (seg_common.c) for setting and reading segment feature elements. These may want to be optimized away at some point but while the mecahnism is in a state of flux they provide a single location for making changes and keep things a bit cleaner. This is still proof of concept code. Currently the tested feature set:- Quantizer, Loop Filter level, Reference frame, Prediction Mode, EOB end stop. TBD:- Add functions for setting and reading the feature data with range and validity checking. Handling of signed and unsigned feature data. At the moment all is assumed to be signed and a sign bit is coded but many cannot be negative. Correct handling of EOB feature with intra coded blocks. Testing/trapping of legal/illegal ref frame and mode combinations. Transform size switch plus merge and test with 8c8 DCT work Merge and test with Sumans Segmenation coding optimizations Change-Id: Iee12e83661c7abbd1e0ce6810915eb4ec35e2d8e
2011-10-14fixed the wrong rounding in inverse haar transformYaowu Xu
Given the current forward haar transform: f0 = I0 + I1 + I2 + I3 f1 = I0 + I1 - I2 - I3 f2 = I0 - I1 + I2 - I3 f3 = I0 - I1 - I2 + I3 the output of the inverse haar prior rounding: i0 = f0 + f1 + f2 + f3 = I0 * 4; i1 = f0 + f1 - f2 - f3 = I1 * 4; i2 = f0 - f1 + f2 - f3 = I2 * 4; i3 = f0 - f1 - f2 + f3 = I3 * 4; As all the numbers are 4 multiples, simply >>2 always produces prefect results in term of forward-inverse transform round trip error. Change-Id: Id6658b00ea819ee61cfeef8c5985d4cd3e77f44e
2011-10-08fixed a decoder bugYaowu Xu
When 8x8 transform is enabled, the decoder does an extra reconstruct on MBs that are coded using 8x8. This commit fixed the logic around the decoding of mb encoded with 8x8 transform. Change-Id: I6926557c9ef00eecb375f62946f7e140c660bf6f
2011-09-30Segment coding of mode and reference frame.Paul Wilkins
Proof of concept test code that encodes mode and reference frame data at the segment level. Decode-able bit stream but some issues not yet resolved. As it this helps a little on a couple of clips but hurts on most as the basis for segmentation is unsound. To build and test, configure with --enable-experimental --enable-segfeatures Change-Id: I22a60774f69273523fb152db8c31f4b10b07c7f4
2011-09-30Experimental: segfeature added.Paul Wilkins
New setting added to configure script
2011-09-21Merge remote branch 'internal/upstream' into HEADJohn Koleszar
2011-09-20Move neon only arm functions under arm/neon.Fritz Koenig
These files don't contain generic arm code, so should only be compiled by neon. Change-Id: Ie712823aa04d4235e7cfe7a3b725e73ee4c3e564