provide a flag to control the minimum number of iterations

Summary:
Add a --bm_min_iters flag to control the minimum number of iterations
that the benchmark code starts with on each epoch.

This can be used on benchmarks that test very cheap operations, but take
a long time to set up.  Otherwise the benchmark code may have to retry
many times before it hits a large enough number of iterations to get a
meaningful result, and each time it still pays the fixed setup cost.

This also helps with benchmarks when some of the setup cost cannot be
hidden with BenchmarkSuspender for some reason.  --bm_min_iters can be
set to a large enough value so that the extra startup cost will not
affect the measurements too much.

Test Plan:
Used this with the thread local stats benchmark.  During setup/cleanup,
this benchmark starts and synchronizes with many threads.  The entire
setup time cannot be reliably hidden with BenchmarkSuspender; the
synchronization between the threads takes a relatively long time
compared to the cost of the operation being benchmarked.  --bm_min_iters
allows a relatively high number of iterations to be used, masking this
cost.

Reviewed By: rajat@fb.com

FB internal diff: D723304

diff --git a/folly/Benchmark.cpp b/folly/Benchmark.cpp
index 545cefccb574a08cc065b132a97980eaf7876647..6fc442000cc4707f35ce65889426b7167769d8b7 100644 (file)
--- a/folly/Benchmark.cpp
+++ b/folly/Benchmark.cpp
@@ -1,5 +1,5 @@
 /*
- * Copyright 2012 Facebook, Inc.
+ * Copyright 2013 Facebook, Inc.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -40,6 +40,9 @@ DEFINE_string(bm_regex, "",
 DEFINE_int64(bm_min_usec, 100,
              "Minimum # of microseconds we'll accept for each benchmark.");
 
+DEFINE_int64(bm_min_iters, 1,
+             "Minimum # of iterations we'll try for each benchmark.");
+
 DEFINE_int32(bm_max_secs, 1,
              "Maximum # of seconds we'll spend on each benchmark.");
 
@@ -222,7 +225,7 @@ static double runBenchmarkGetNSPerIteration(const BenchmarkFun& fun,
   size_t actualEpochs = 0;
 
   for (; actualEpochs < epochs; ++actualEpochs) {
-    for (unsigned int n = 1; n < (1UL << 30); n *= 2) {
+    for (unsigned int n = FLAGS_bm_min_iters; n < (1UL << 30); n *= 2) {
       auto const nsecs = fun(n);
       if (nsecs < minNanoseconds) {
         continue;