Consistently use get_prob(), clip_prob() and newly added clip_pixel().

Add a function clip_pixel() to clip a pixel value to the [0,255] range
of allowed values, and use this where-ever appropriate (e.g. prediction,
reconstruction). Likewise, consistently use the recently added function
clip_prob(), which calculates a binary probability in the [1,255] range.
If possible, try to use get_prob() or its sister get_binary_prob() to
calculate binary probabilities, for consistency.

Since in some places, this means that binary probability calculations
are changed (we use {255,256}*count0/(total) in a range of places,
and all of these are now changed to use 256*count0+(total>>1)/total),
this changes the encoding result, so this patch warrants some extensive
testing.

Change-Id: Ibeeff8d886496839b8e0c0ace9ccc552351f7628

1 files changed, 2 insertions, 8 deletions

diff --git a/vp9/encoder/vp9_rdopt.c b/vp9/encoder/vp9_rdopt.c
index 9b87713f9..9265a0237 100644
--- a/vp9/encoder/vp9_rdopt.c
+++ b/vp9/encoder/vp9_rdopt.c
@@ -3045,7 +3045,6 @@ static void estimate_ref_frame_costs(VP9_COMP *cpi, int segment_id, unsigned int
   int pred_flag;
   int pred_ctx;
   int i;
-  int tot_count;
 
   vp9_prob pred_prob, new_pred_prob;
   int seg_ref_active;
@@ -3069,13 +3068,8 @@ static void estimate_ref_frame_costs(VP9_COMP *cpi, int segment_id, unsigned int
 
   // Predict probability for current frame based on stats so far
   pred_ctx = vp9_get_pred_context(cm, xd, PRED_REF);
-  tot_count = cpi->ref_pred_count[pred_ctx][0] + cpi->ref_pred_count[pred_ctx][1];
-  if (tot_count) {
-    new_pred_prob =
-      (cpi->ref_pred_count[pred_ctx][0] * 255 + (tot_count >> 1)) / tot_count;
-    new_pred_prob += !new_pred_prob;
-  } else
-    new_pred_prob = 128;
+  new_pred_prob = get_binary_prob(cpi->ref_pred_count[pred_ctx][0],
+                                  cpi->ref_pred_count[pred_ctx][1]);
 
   // Get the set of probabilities to use if prediction fails
   mod_refprobs = cm->mod_refprobs[pred_ref];