summaryrefslogtreecommitdiff
path: root/vp8
AgeCommit message (Collapse)Author
2011-12-05removed leftover code from a couple merge problems.Yaowu Xu
Change-Id: I17d9c1246d69e102297ec1c3efb359691b3da313
2011-12-02added separate entropy context for alt_refYaowu Xu
This commit added code to keep track of separate entropy contexts for normal frames and alt ref frames. The underly assumption was that the two type of frames have different entropy characteristics given they typically have quite different quantization levels. By keeping entropy contexts separate, it helps the entropy context distribution to be more closely adapted to each frame type. Tests on derf set showed a good and very consistent gain on all clips on all metrics, avg psnr: 0.89%, overall psnr: 0.84% and ssim 0.93%. http://www.corp.google.com/~yaowu/no_crawl/mulcontext.html Change-Id: I15bc9697f6ff7829042911fe0c62930585d7e65d
2011-12-02enabled 8x8 intra prediction modes on inter framesYaowu Xu
This commit enabled the usage of 8x8 intra prediction modes on inter frames. There are a few TODO items related to this: 1)baseline entropy need be calibrated; 2)cost of UV need to be done more properly rather than using decision only relying on Y; 3)Threshold for allowing picking 8x8 intra prediction should be lowered to lower than the B_PRED. Even with all the TODOs, tests showed consistent gain on derf set ~0.1% (PSNR:0.08% and SSIM:0.14%). It is assumed that 8x8 intra prediction will help more on large resolution clips, especially with above TODOs addressed. Change-Id: I398ada49dfc32575cfab962a569c2885111ae3ba
2011-12-02Further work on extended Q range.Paul Wilkins
Fixed some further QIndex related issues and replaced some tables (eg zbin and rounding) Also Added function (currently disabled by default) to populate the main AC and DC quantizer tables. Using the original AC range the resulting computed DC values give behavior broadly comparable on the DERF set. That is not to say that the equations will hold good over a more extended range. The purpose of this code is to make it easier to experiment with further alterations to the Q range and distribution of Q values plus the relative weights given to AC and DC. The function find_fp_qindex() ensures that changes to the Q tables are reflected in the value passed in to the first pass code. Slight experimental adjustment to static segment Q offset. Change-Id: I36186267d55dfc2a3d565d0cff7218ef300d1cd5
2011-12-02CR/LF issue.Paul Wilkins
Change-Id: I95fab6f51967008acf1bc9e98fdb7bb56974807f
2011-12-01added transform type to MB_MODE_INFOYaowu Xu
this commit is to add an variable in the macroblock level mode info structure to track the transform size used in each MB, so the information can be used later in the loop filter to change how loop filter works on MBs with different transform sizes. Change-Id: Id0eeaba6cc854c6d1be00ed8d237b3d9e250e447
2011-11-29MinQ equations.Paul Wilkins
Slight tweaks to the new minq equations to bring results more into line with original lookup tables. Change-Id: I969fc87d95912df549b6775e83ee2345e84d4da0
2011-11-29Extended Q Range:Paul Wilkins
Addressed a couple of other QIndex dependencies. Change-Id: I15b224bffd0210d3c7065cb6905156f2ca8e9ea9
2011-11-29Further work on extended Q range.Paul Wilkins
Fixed bug in firspass.c call to vp8_initialize_rd_consts() This was passing in vp8_dc_quant(cm->base_qindex, cm->y1dc_delta_q) instead of (cm->base_qindex + cm->y1dc_delta_q). It just so happens that for the value 26 used for cm->base_qindex in the unextended Q case, the two give similar results. However, when using the extended Q range the two are very different. Also added more stats output and partly disabled another broken feature. Change-Id: Iddf6cf5ea8467c44b7c133f38e629f6ba6f2581e
2011-11-28mbgraph: fix invalid memory access if motion vectors are too big.Ronald S. Bultje
2011-11-28changed find_near_mvs search to include a mb from last frameYaowu Xu
This is an experiment to include a mv contribution from last frame to nearest and near mv definition. Initial test showed some small though consistent gain. latest patch slightly better result ~.13%-~.18%. TODO: the entropy used to encode the mode choice, i.e. the mv counts based conditional distribution of modes should be re-collected to reflect this change, it is expected that there is some further gain from that. Change-Id: Ief1e284a36d8aa56b49ae5b360c91419ec494fa4
2011-11-23Two pass rate control code changes.Paul Wilkins
This comitt brings accross changes from the public branch commit number Icf74d13af77437c08602571dc7a97e747cce5066. The main puurpose of this comit relates to CQ mode but it also includes some refactoring of the two pass code which I hope will make tuning the experimental branch for the new quantizer range a little less painfull. Change-Id: I278e989436a928fc1fe7761068960048f9d7a376
2011-11-23Further resolution of QIndex LUTS;Paul Wilkins
This commit resolves further QIndex look up tables to facilitate experimentation with the quantizer range. In some cases rather than remove the look up tables completely I have created functions that are called once to populate them using a formulaic approach base on the actual quantizer. The use of these functions based on best fit of data from the original tables does affect the results on some clips but across the derf test set the effect was broadly neutral. Change-Id: I8baa61c97ce87dc09a6340d56fdeb681b9345793
2011-11-22Remove redundant code (lf_or_gf and frame_lf_or_gf)Adrian Grange
Removed unused variables lf_or_gf and frame_lf_or_gf. Change-Id: I88692cd7d53e532d303c4525ee4667c1ecea3026
2011-11-22Removal of Qindex LUTS.Paul Wilkins
One of the problems arising when tweaking or adjusting the quantizer tables is that there are a lot of look up tables that depend on the QINDEX. Any adjustment to the link between QINDEX and real quantizer therefore tends to break aspects of for example the rate control. In this check in I have replaced several of the look up tables with functions that approximate the same results as the old Q luts but use a formulaic approach based on real Q values rather than QIndex. This should hopefully make it easier to experiment with changes to the Q tables without always having to go through and hand optimize a set of look up tables. Once things stabilize we may choose to re-instate luts for the sake of performance. Patch 2: Addressed Ronald's comments. vp8_init_me_luts() Added so luts only initialized once. Change-Id: Ic80db2212d2fd01e08e8cb5c7dca1fda1102be57
2011-11-21Extended Q range Experiment.Paul Wilkins
Corrected dc lookup table to maintain ac/dc balance close to what it was previously. Firstpass not being passed the adjusted Q index for the extended range. Change-Id: Ic0200dabda445fea03bf81067999cb2670e99b77
2011-11-21Cosmetic clean up.Paul Wilkins
Clean up of vp8_kfread_modes(). Remove unnecessary indentation and enforce line length. Change-Id: I0864d1aff55368126db01bb23efa815786b5245d
2011-11-21Decoder segmentation bug.Paul Wilkins
Fix decoder segmentation bug for temporal coding where the segment map was first initialized on a key frame. in vp8_kfread_modes() after reading the segment id it must be written to the pbi->segmentation_map[] for use in temporal coding on subsequent frames. Change-Id: I1489305efc376564e734a216f69c2844646ee3d3
2011-11-21CONFIG_T8X8 experiment.:Paul Wilkins
Block the selection of 4x4 modes in key frames if 8x8 is selected. Change-Id: Ie5729ec22a999d9a1996f020bd4b941e29514992
2011-11-17Clip buffer level to the maximum buffer size in CBRAdrian Grange
The buffer level was able to increase indefinitely rather than being clipped to the maximum buffer size specified by the user. This change checks the buffrer level and prevents it from going beyond the upper limit of the buffer. Change-Id: Ifff55f79d3c018e4d3d77e554b11ada543cc1654
2011-11-17Merge "fixed the scaling in 8x8 trellis quant" into experimentalYaowu Xu
2011-11-16fixed the scaling in 8x8 trellis quantYaowu Xu
This commit has a few minor fixes to the 8x8 trellis quant, so to make it work regardless if extend_qrange is enabled or not. It also borrowed adaptive RDMULT constants from 4x4 trellis that was missed in the 8x8 trellis quant. Change-Id: I60d7769071f102c699b5084597e62bca87a1f759
2011-11-16Header inclusion for Unix buildPaul Wilkins
Explicit inclusion of limits.h to satisfy unix build for definition of INT_MAX. Some commented out code removed. Change-Id: I5b5980dfaa9b4d2d12bfd729cfd35bd982106908
2011-11-15Merge CONFIGURE_SEGMENTATION experiment.Paul Wilkins
Removal of CONFIGURE_SEGMENTATION ifdefs. Removal of legacy support code fo the old coding mechanism. Use local reference "xd" for MACROBLOCKD structure in encode_frame_to_data_rate() Moved call to choose_segmap_coding_method() out of encode loop as the cost of segmentation is not properly accounted in the loop anyway. If this is desirable in the future it can be moved back. The use of this function to do all the analysis and set the probabilities also removes the need to track segment useage in threading code. Change-Id: I85bc8fd63440e7176c73d26cb742698f9b70cade
2011-11-15Further clean up of Segmentation experiment codePaul Wilkins
Changed name and sense of segment_flag to "seg_id_predicted" Added some additional comments and retested. I also did some experimentation with a spatial prediction option using a similar strategy to the temporal mode implemented. This helps in some cases where temporal prediction is bad but I suspect there is more overlap here with work on a larger scale block structure and spatial correlation will likely be better handled through that mechanism. Next check in will remove #ifdefs and legacy mode code. Change-Id: I3b382b65ed2a57bd7775ac0f3a01a9508a209cbc
2011-11-15Further work on Segmentation Experiment:Paul Wilkins
This check in includes quite a lot of clean up and refactoring. Most of the analysis and set up for the different coding options for the segment map (currently simple distribution based coding or temporaly predicted coding), has been moved to one location (the function choose_segmap_coding_method() in segmenation.c). This code was previously scattered around in various locations making integration with other experiments and modification / debug more difficult. Currently the functionality is as it was with the exception that the prediction probabilities are now only transmitted when the temporal prediction mode is selected. There is still quite a bit more clean up work that will be possible when the #ifdef is removed. Also at that time I may rename and alter the sense of macroblock based variable "segment_flag" which indicates (1 that the segmnet id is not predicted vs 0 that it is predicted). I also intend to experiment with a spatial prediction mode that can be used when coding a key frame segment map or in cases where temporal prediction does not work well but there is spatial correlation. In a later check in when the ifdefs have gone I may also move the call to choose_segmap_coding_method() to just before where the bitsream is packed (currently it is in vp8_encode_frame()) to further reduce the possibility of clashes with other experiments and prevent it being called on each itteration of the recode loop. Change-Id: I3d4aba2a2826ec21f367678d5b07c1d1c36db168
2011-11-11Segmentation experiment:Paul Wilkins
Added last_segmentation_map[] structure to keep track of what we had before when doing temporal prediction. With this change the existing code does once again appear to be giving a decodable bitstream for both temporal and standard prediction modes. However, it is still somewhat messy and confused and there is no option to take advantage of spatial prediction so it could do with further work. Some housekeeping / clean out. Change-Id: I368258243f82127b81d8dffa7ada615208513b47
2011-11-11SEGMENTATION experiment:Paul Wilkins
Some initial cleanup to aid testing and debug. Pull code to choose temporal or spatial encoding out of encodeframe.c into a dedicated function in segmentation.c. For now disable broken temporal mode. Move the coding of "temporal_update" flag and only transmit if segment map update is indicated. Rename the functions read_mb_features() and write_mb_features() to read_mb_segid() and read_mb_segid() as they only read and write the macroblock segment id not any of the features. Change-Id: Ib75118520b1144c24d35fdfc6ce46106803cabcf
2011-11-11changed function name for clarityYaowu Xu
The dequantizer functions for 2nd order haar block had confusing 8x8 in their names. this commit fixed their name to avoid confusion. Change-Id: I6ae4e7888330865f831436313637d4395b1fc273
2011-11-11Merge "fixed the decoder when using 8x8 transform" into experimentalYaowu Xu
2011-11-11Make 8x8 and extend_qrange to work togetherYaowu Xu
This commit added scaling factors to 8x8 transform, quant, dequant and inverse transform pipeline to make 8x8 transform to work when configed with enable-extend_qrange. This commit also disabled the trellis-quant when extend_qrange is configured. Change-Id: Icfb3192e4746f70a4bb35ad18b7b47705b657e52
2011-11-10fixed the decoder when using 8x8 transformYaowu Xu
updated the decode_macroblock logic to reflect that 8x8 transform is not used for "SPLITMV". Also fixed an issue where 2nd order haar block has wrong dequant/idct process. Change-Id: I1e373f6535c009dfec503b6362c8a5cfc196e1da
2011-11-10Merge "scaled the threshold for 2nd order coefficient reset" into experimentalYaowu Xu
2011-11-10scaled the threshold for 2nd order coefficient resetYaowu Xu
extend_qrange introduces a different scaling factor, this commit takes the scaling difference into account for reset 2nd order coefficients. Change-Id: Ie58bca9f52698fa759e3f88da2aa4d82630fa91a
2011-11-10Merge "T8x8 experiment merge." into experimentalPaul Wilkins
2011-11-10Merge "fixed an encoder bug" into experimentalYaowu Xu
2011-11-09fixed an encoder bugYaowu Xu
the bug caused the encoder to produce invalid bitstream when configured with enable_extend_qrange. Change-Id: I1e81c48b13359d0043cbbd480e679380a2da117c
2011-11-09T8x8 experiment merge.Paul Wilkins
For ease of testing and merging experiments I have removed in line code in encode_frame() that assigns MBs to be t8x8 or t4x4 coded segments and have moved the decision point and segment setup to the init_seg_features0 test function. Keeping everything in one place helps make sure for now that experiments using segmentation are not fighting each other. Also made sure mode selection code can't choose 4x4 modes if t8x8 is selected. Patch2: In init_seg_features() add checks for SEG_LVL_TRANSFORM active. Change-Id: Ia1767edd99b78510011d4251539f9bc325842e3a
2011-11-09Merging and testing of SEGMENTATION experiment.Paul Wilkins
Removed code in #if CONFIG_SEGMENTATION that enables segmentation and creates a test segmentation map, to avoid conflicts with the other segmentation test code, Change-Id: I7a21a44ed188b814cd80b30dd628c62474eba730
2011-11-09Segmentation feature logic fix.Paul Wilkins
Bug fix to logic in vp8_pick_inter_mode() and vp8_rd_pick_inter_mode(). The block on the use of segment features for the cm->refresh_alt_ref_frame case was just for testing and is not correct. The special case code for alt ref can be re-enabled as an else clause. Change-Id: Ic9b57cdb5f04ea7737032b8fb953d84d7717b3ce
2011-11-08make debug match release build on win32 with 8x8 transform enabledYaowu Xu
The 8x8 forward transform makes use of floating operations, therefore requires emms call to reset mmx registers to correct state. Without the resets, the 8x8 forward transform results are indefinite on win32 platform. Change-Id: Ib5b71c3213e10b8a04fe776adf885f3714e7deb1
2011-11-08Attempt to fix an issue related to 8x8 transform and segfeatureYaowu Xu
logically this commit should NOT change anything, but seems to help revert the 3DB loss on bowing in the following commit: https://on2-git.corp.google.com/g/#change,6193 This is still debugging in progress. Need further investigation to understand the root cause of the issue. Change-Id: I0b49d1ef3a311dfff58c6acd3eaebdb3bda6257c
2011-11-08Segment signaling of TX sizePaul Wilkins
Initial attempt at using new segment feature signaling to indicate 4x4 or 8x8 transform. needs --enable-experimental --enable-t8x8 Note this is work in progress. Change-Id: Ib160d46a5d810307bfcbc79853ce1a65b5b870b7
2011-11-04Merge "Added context reset when 2nd order coefficients are cleared" into ↵Yaowu Xu
experimental
2011-11-04Segment Features with 8x8DCT.Paul Wilkins
Temporary check in to turn off other segment features tests when #if CONFIG_T8X8 is set as the assignment of MBs to differnt segments in each case will conflict. The 8x8 code will be modified to use the new segment feature method properly in a later check in. Increase bits allowed for EOB end stop marker to 6 ready for 8x8. Change-Id: I4835bc8d3bf98e1775c3d247d778639c90b01f7f
2011-11-04Segment Feature Data AccessPaul Wilkins
No change to functionality or output. Updates to the segment feature data structure now all done through functions such as set_segdata() and get_segdata() in seg_common.c. The reason for this is to make changing the structures (if needed) and debug easier. In addition it provides a single location for subsequent addition of range and validity checks. For example valid combination of mode and reference frame. Change-Id: I2e866505562db4e4cb6f17a472b25b4465f01add
2011-11-03make uv intra mode coding adaptive to Y modeYaowu Xu
This commit tries to do UV intra mode coding adaptive to Y intra mode. Entropy context is defined as conditional PDF of uv intra mode given the Y mode. All constants are normalized with 256 to be fit in 8 bits. This provides further coding efficiency beyond the quantizer adaptive y intra mode coding. Consistent gains were observed on all clips and all bit rates for HD all key encoding tests. To test, configure with --enable-experimental --enable-uvintra Change-Id: I2d78d73f143127f063e19bd0bac3b68c418d756a
2011-11-03Added context reset when 2nd order coefficients are clearedYaowu Xu
As discovered in path 10 of Change Ia12acd2f, reset 2nd order coeffs without reset of above and left coding context may have introduced problem that causes encoder/decoder mismatching. This commit added update to coding context when the 2nd coefficients are cleared. In addition, this commit also introduced early breakout in the checks to speed up when coefficients are too significant to be cleared. Change-Id: I85322a432b11e8af85001525d1e9dc218f9a0bd6
2011-11-03Segment Features. Removal of #ifdefsPaul Wilkins
Removal of configure #ifdefs so that segment features always available. Removal of code supporting old segment feature method. Still a good deal of tidying up to do. Change-Id: I397855f086f8c09ab1fae0a5f65d9e06d2e3e39f
2011-11-03Merge "Segmentation: Reference frames" into experimentalPaul Wilkins