From: Philip Pronin <philipp@fb.com>
Date: Thu, 17 Oct 2013 03:13:33 +0000 (-0700)
Subject: exponential capacity growth for io::TypedIOBuf
X-Git-Tag: v0.22.0~812
X-Git-Url: http://demsky.eecs.uci.edu/git/?a=commitdiff_plain;h=94fce358e890eb92072c8083cf15b4852feedacf;p=folly.git

exponential capacity growth for io::TypedIOBuf

Summary:
Amortized cost of `TypedIOBuf::push()` is `O(N)` at this moment.
Unless we're using jemalloc where we can rely on `realloc()`.  In this
diff we switch to exponential growth otherwise.

Test Plan:
built unicorn/test with ASan (and thus without jemalloc), ran
end-to-end tests and verified we no longer spend minutes to build di4

@override-unit-failures

Reviewed By: tudorb@fb.com

FB internal diff: D1015233
---

diff --git a/folly/io/TypedIOBuf.h b/folly/io/TypedIOBuf.h
index 3e8d2297..90e17126 100644
--- a/folly/io/TypedIOBuf.h
+++ b/folly/io/TypedIOBuf.h
@@ -20,6 +20,8 @@
 #include <algorithm>
 #include <iterator>
 #include <type_traits>
+
+#include "folly/Malloc.h"
 #include "folly/io/IOBuf.h"
 
 namespace folly {
@@ -164,8 +166,14 @@ class TypedIOBuf {
    */
   template <class IT>
   void push(IT begin, IT end) {
-    auto n = std::distance(begin, end);
-    reserve(headroom(), n);
+    uint32_t n = std::distance(begin, end);
+    if (usingJEMalloc()) {
+      // Rely on rallocm() and avoid exponential growth to limit
+      // amount of memory wasted.
+      reserve(headroom(), n);
+    } else if (tailroom() < n) {
+      reserve(headroom(), std::max(n, 3 + size() / 2));
+    }
     std::copy(begin, end, writableTail());
     append(n);
   }