folly.git
12 years agoVerbosify exception tracer dumps
Tudor Bosman [Mon, 10 Sep 2012 20:45:25 +0000 (13:45 -0700)]
Verbosify exception tracer dumps

Test Plan: by hand

Reviewed By: philipp@fb.com

FB internal diff: D570233

12 years agosupport -nan in conversion
Fan Guo [Mon, 10 Sep 2012 17:08:46 +0000 (10:08 -0700)]
support -nan in conversion

Summary:
Folly supports -inf and nan but not -nan, instead it raised unexpected exceptions not handled in the upstream.

This diff is separated out from D569816 -- Diff1.

Test Plan: std::isnan(folly::to<double>(-nan))

Reviewed By: tudorb@fb.com

FB internal diff: D569939

12 years agoFix InpuByteStreamSplitter in debug mode.
Tudor Bosman [Sat, 8 Sep 2012 04:46:58 +0000 (21:46 -0700)]
Fix InpuByteStreamSplitter in debug mode.

Test Plan: stream_test, exception_tracer_test

Reviewed By: soren@fb.com

FB internal diff: D569476

12 years agoHistogram function to write tab-separated values.
Philip Pronin [Sat, 1 Sep 2012 08:03:07 +0000 (01:03 -0700)]
Histogram function to write tab-separated values.

Summary: Export histograms in a tab-separated format.

Test Plan: Used it.

Reviewed By: lucian@fb.com

12 years agoMinor huge pages library changes
Tudor Bosman [Sat, 1 Sep 2012 02:39:12 +0000 (19:39 -0700)]
Minor huge pages library changes

Summary:
Moved to separate library instead of :io
Got rid of the mode argument when creating files -- umask is your friend
Separated getSize interface from create
Added FsUtil.h for some fs::path operations
Switched some interfaces to fs::path instead of string / StringPiece
Canonicalize paths so we don't care whether paths are absolute or
relative

Test Plan: fs_util_test, by hand

Reviewed By: philipp@fb.com

FB internal diff: D564465

12 years agoHuge page library.
Tudor Bosman [Wed, 22 Aug 2012 00:16:02 +0000 (17:16 -0700)]
Huge page library.

Test Plan: by hand

Reviewed By: lucian@fb.com

FB internal diff: D555164

12 years agofix build on: define own version of ALLOCM_LG_ALIGN
Lucian Grijincu [Thu, 30 Aug 2012 03:19:19 +0000 (20:19 -0700)]
fix build on: define own version of ALLOCM_LG_ALIGN

Summary:
On --platform=gcc-4.6.2-glibc-2.13-fb the build would fail because

fbcode/third-party/gcc-4.6.2-glibc-2.13-fb/libgcc/libgcc-4.6.2/b4e37bb/include/c++/4.6.2/bits/basic_fbstring_malloc.h

defines it's own version of ALLOCM_SUCCESS & co., but not ALLOCM_LG_ALIGN :(

The '#ifdef ALLOCM_SUCCESS' check in 'folly/Malloc.h' would pass even
though we didn't include jemalloc.

Test Plan: build it

Reviewed By: philipp@fb.com

FB internal diff: D562273

12 years agoUnbreak build.
Tudor Bosman [Thu, 30 Aug 2012 02:20:04 +0000 (19:20 -0700)]
Unbreak build.

Summary: I'll pretend you reviewed this.

Test Plan: build

Reviewed By: simpkins@fb.com

FB internal diff: D562255

12 years agoMove exception tracer library to folly/experimental
Tudor Bosman [Thu, 30 Aug 2012 01:20:27 +0000 (18:20 -0700)]
Move exception tracer library to folly/experimental

Summary:
This change is mostly mechanical (moving files, changing include paths,
etc).  I made some changes to TARGETS to make it easier for the library
to be linked in (instead of LD_PRELOADed)

Test Plan: by hand

Reviewed By: simpkins@fb.com

FB internal diff: D562196

12 years agoAdd ALLOCM_LG_ALIGN to Malloc.h
Lucian Grijincu [Wed, 29 Aug 2012 23:59:36 +0000 (16:59 -0700)]
Add ALLOCM_LG_ALIGN to Malloc.h

Test Plan: .

Reviewed By: soren@fb.com

12 years agoStream operations: file access, iteration, splitting.
Tudor Bosman [Wed, 2 May 2012 23:42:35 +0000 (16:42 -0700)]
Stream operations: file access, iteration, splitting.

Summary: Intended to complement and replace strings::byLine.

Test Plan: stream_test

Reviewed By: delong.j@fb.com

FB internal diff: D463341

12 years agoFix fedora / ubuntu build
Tudor Bosman [Sun, 26 Aug 2012 18:04:37 +0000 (11:04 -0700)]
Fix fedora / ubuntu build

Test Plan: yes

Reviewed By: andrewjcg@fb.com

FB internal diff: D558628

12 years agofolly: simplify the stats avgHelper() function
Adam Simpkins [Wed, 22 Aug 2012 03:56:09 +0000 (20:56 -0700)]
folly: simplify the stats avgHelper() function

Summary:
When the input type is a long double, perform division using long
double.  In all other cases, divide using double precision.

Also fix the EXPECT_EQ() usage in the test case.

Test Plan: fbconfig -r common/stats folly && fbmake runtests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D555433

12 years agoBucketedTimeSeries: fix type converison issues computing avg()
Adam Simpkins [Mon, 20 Aug 2012 21:58:25 +0000 (14:58 -0700)]
BucketedTimeSeries: fix type converison issues computing avg()

Summary:
D527040 had a bug in the code to compute the average: it incorrectly
performed unsigned division when ValueType was a signed integer type.
As a result, the average was reported incorrectly for stats with
negative values.

This makes the average code more intelligent when handling type
conversions: if the caller wants a floating point value, or if the input
type is floating point, floating point division is always returned.
Otherwise, if the input is a signed type, signed integer division is
performed.  Otherwise, unsigned integer division is performed.

Test Plan: beholdunittests

Reviewed By: lars@fb.com

FB internal diff: D553583

12 years agofolly: add a simple benchmark for Histogram::addValue()
Adam Simpkins [Sun, 19 Aug 2012 19:29:15 +0000 (12:29 -0700)]
folly: add a simple benchmark for Histogram::addValue()

Summary:
I wrote this an an example for the BENCHMARK_NAMED_PARAM() comments in
D527040, and figured I might as well check it in as a real benchmark.

Test Plan:
Here's the output:

======================================================================
folly/test/HistogramBenchmark.cpp         relative  time/iter  iters/s
======================================================================
addValue(0_to_100)                                    15.43ns   64.81M
addValue(0_to_1000)                                   15.45ns   64.70M
addValue(5k_to_20k)                                   15.46ns   64.67M
======================================================================

Reviewed By: rajat@fb.com

FB internal diff: D552875

12 years agochange shared_ptr<>* to raw pointer
Shuai Ding [Thu, 16 Aug 2012 22:00:15 +0000 (15:00 -0700)]
change shared_ptr<>* to raw pointer

Summary:
change shared_ptr<>* to raw pointer in
forwardIndex in RTIndex. This saves 16 bytes per
forward list (decrease from 88 to 72) and translates
to about 4-5% memory saving for RTIndex in ff tier.

Test Plan: unit test

Reviewed By: xliux@fb.com

FB internal diff: D551191

12 years agoAdd BucketedTimeSeries to folly
Adam Simpkins [Sun, 22 Jul 2012 05:40:46 +0000 (22:40 -0700)]
Add BucketedTimeSeries to folly

Summary:
Add the BucketedTimeSeries class to folly.  This tracks time series
counter data, using a circular buffer of buckets to expire old data
points as time progresses.

It supports querying the overall sum, count, average, and rate for the
duration of the time series, as well as estimating these values for
portions of the overall duration.

Test Plan: Unit tests included.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D527040

12 years agoone more simple folly::join optimization
Philip Pronin [Fri, 17 Aug 2012 23:21:26 +0000 (16:21 -0700)]
one more simple folly::join optimization

Summary:
Before:

_bin/folly/test/string_test --benchmark --bm_regex=join --bm_min_usec=1000000
============================================================================
folly/test/StringTest.cpp                       relative  time/iter  iters/s
============================================================================
joinCharStr                                                  3.59us  278.29K
joinStrStr                                                   4.44us  225.09K
joinInt                                                     10.55us   94.76K
============================================================================

And after:

_bin/folly/test/string_test --benchmark --bm_regex=join --bm_min_usec=1000000
============================================================================
folly/test/StringTest.cpp                       relative  time/iter  iters/s
============================================================================
joinCharStr                                                  3.61us  277.01K
joinStrStr                                                   3.77us  264.97K
joinInt                                                      9.92us  100.81K
============================================================================

Test Plan: unittests, benchmark

Reviewed By: tudorb@fb.com

FB internal diff: D552364

12 years agoAdd Bob Jenkins's SpookyHash to folly.
Tudor Bosman [Wed, 15 Aug 2012 21:26:31 +0000 (14:26 -0700)]
Add Bob Jenkins's SpookyHash to folly.

Summary:
http://burtleburtle.net/bob/hash/spooky.html
The code is in the public domain, so republishing it as such is fine.
The code needed modifications (the test was designed to run on Windows
only; removed unhygienic typedefs; moved to folly::hash namespace)

Also updated Makefile.am files.

Test Plan: test added

Reviewed By: soren@fb.com

FB internal diff: D549572

12 years agoisPowTwo
Michael Curtiss [Fri, 17 Aug 2012 05:31:16 +0000 (22:31 -0700)]
isPowTwo

Summary:
a la Hacker's Delight.

Also, fix nonsensical nextPowTwo benchmark.

Test Plan: Added test and benchmark.

Reviewed By: tudorb@fb.com

FB internal diff: D551510

12 years agoone more folly::join overload
Philip Pronin [Wed, 15 Aug 2012 20:52:40 +0000 (13:52 -0700)]
one more folly::join overload

Summary: To allow codemod changes.

Test Plan: ran test

Reviewed By: tudorb@fb.com

FB internal diff: D549516

12 years agoTask #906853: Implement ThreadLocalServiceData and reduce lock contentions during...
Lee Hwa [Mon, 6 Aug 2012 20:42:49 +0000 (13:42 -0700)]
Task #906853: Implement ThreadLocalServiceData and reduce lock contentions during channel counter update

Summary:
Cache the counter updates on thread local data structures and aggregate/publish
the data from a separate thread periodically. The ThreadLocalServiceData is
implemented on top of the TLStats data structures.

I am plannig to deploy a private build to production tomorrow to test this
change in production, but figured it might be better to send the diff out first
and get code review feedbacks at the same time.

Test Plan: Run and pass unit tests. Run end-to-end testings on sandbox. (TBD)
Deploy a private build to a production server and make sure the thrift call
latency is reduced.

Reviewed By: xin@fb.com

FB internal diff: D540239

12 years agofolly: randomNumberSeed should be random wrt multiple threads
Sergey Doroshenko [Tue, 14 Aug 2012 19:54:49 +0000 (12:54 -0700)]
folly: randomNumberSeed should be random wrt multiple threads

Summary:
Right now, it relies on process id rather than on thread id. Thus, it can
return the same seed when called from multiple threads over really short
period of time, and these unlucky threads will end up with identical "random"
sequences.

Test Plan: newly added test fails against trunk, passes against changes in the diff

Reviewed By: tudorb@fb.com

FB internal diff: D548129

12 years agostrings join
Philip Pronin [Wed, 15 Aug 2012 01:55:35 +0000 (18:55 -0700)]
strings join

Summary: The same interface as ##facebook::strings::join##, but few times faster.

Test Plan: folly/test/StringTest.cpp

Reviewed By: tudorb@fb.com

FB internal diff: D548863

12 years agoAdded dynamic::convertTo<Type> method
Nicholas Ormrod [Tue, 14 Aug 2012 23:11:03 +0000 (16:11 -0700)]
Added dynamic::convertTo<Type> method

Summary: convert a dynamic to a well-typed object

Test Plan: run test file

Reviewed By: delong.j@fb.com

FB internal diff: D517021

12 years agoAdd exception_tracer
Nitay Joffe [Tue, 14 Aug 2012 00:36:55 +0000 (17:36 -0700)]
Add exception_tracer

Summary: script to wrap gateway to run it with exception tracer.

Test Plan: deploy

Reviewed By: rajat@fb.com

FB internal diff: D548147

12 years agofolly: print nice time suffixes in benchmark output
Adam Simpkins [Sat, 11 Aug 2012 02:15:08 +0000 (19:15 -0700)]
folly: print nice time suffixes in benchmark output

Summary: Print the times using "ms" for milliseconds, "us" for microseconds, etc.

Test Plan:
$ ./_bin/folly/test/benchmark_test
======================================================================
folly/test/BenchmarkTest.cpp              relative  time/iter  iters/s
======================================================================
bmFun                                                  3.36ns  298.01M
bmRepeatedFun                                          1.76ns  569.38M
----------------------------------------------------------------------
gun                                                    1.70ns  589.82M
----------------------------------------------------------------------
baselinevector                                         3.08us  324.87K
bmVector                                    13.64%    22.56us   44.32K
----------------------------------------------------------------------
superslow                                               1.00s  999.82m
======================================================================

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D546209

12 years agoFix debug build.
Andrei Alexandrescu [Sat, 11 Aug 2012 19:08:51 +0000 (12:08 -0700)]
Fix debug build.

Summary: Only tested in release mode, sigh. Fixed a failing asserd.

Test Plan: tested in dbg mode

Reviewed By: tudorb@fb.com

FB internal diff: D546318

12 years agoFaster repeated append (particularly for short strings)
Andrei Alexandrescu [Fri, 10 Aug 2012 16:50:15 +0000 (09:50 -0700)]
Faster repeated append (particularly for short strings)

Summary:
https://phabricator.fb.com/D544159 reveals a large performance gap between
fbstring and std::string for repeated appends of short strings, which I
consider a relatively urgent matter (as much of our code uses such patterns).

This diff attempts to fix the issue in a principled manner by first appending
the first character with exponential reallocation, after which the rest of the
characters are appended normally.

With the proposed fix the benchmarks are much faster than the previous fbstring
and also than std::string (numbers to follow in comments).

Test Plan: unittested and benchmarked

Reviewed By: soren@fb.com

FB internal diff: D545416

12 years agosizeof works just as well as numeric_limits<T>::digits
Tudor Bosman [Fri, 10 Aug 2012 21:22:05 +0000 (14:22 -0700)]
sizeof works just as well as numeric_limits<T>::digits

Summary: ... and is shorter and more readable

Test Plan: folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D545774

12 years agoIncreased std::vector compatibility of fbvector
Nicholas Ormrod [Fri, 10 Aug 2012 21:04:43 +0000 (14:04 -0700)]
Increased std::vector compatibility of fbvector

Summary:
fbvector was not accepting move_iterators for
InputIterator-templated construct, assign, and insert. The root cause
was the check '(b_ <= &*first && &*first < e_)', used to check if the
assignment was from internal data. This addressof check does not compile
with rvalue-references, and causes the first iterator to be dereferenced
more than once; both against the contract of std::vector. The standard
allows for undefined behaviour in the self-iterator case, so there are
no contractual barriers to removing this check.

Test Plan:
run fbvector test and benchmark. An fbgs for 'assign' turns
up few matches; the seem to be normal use-case. Test fbvector assign
with self-iterators in order (i.e. fbv.assign(fbv.begin(), fbv.end()));
this seems to work fine.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D535012

12 years agoRemoving broken specialization of std::hash<std::tuple>
Tom Jackson [Wed, 8 Aug 2012 23:47:01 +0000 (16:47 -0700)]
Removing broken specialization of std::hash<std::tuple>

Summary: ^

Test Plan: Unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D543586

12 years agoAdd "short append" benchmark to fbstring benchmark suite.
Jeremy Lilley [Thu, 9 Aug 2012 16:57:22 +0000 (09:57 -0700)]
Add "short append" benchmark to fbstring benchmark suite.

Summary:
Appending many small strings to fbstring appears much slower (~3x) vs.
std::string. Adding benchmark case to illustrate.

[If this isn't wanted, I can revert]

Test Plan: This is a benchmark.

Reviewed By: soren@fb.com

FB internal diff: D544159

12 years agofolly::dynamic: comment about std::vector const-iter erase issue
Andrew Gallagher [Thu, 9 Aug 2012 01:11:40 +0000 (18:11 -0700)]
folly::dynamic: comment about std::vector const-iter erase issue

Test Plan: none

Reviewed By: delong.j@fb.com

FB internal diff: D543196

12 years agoUnbreak folly build on fedora / ubuntu
Tudor Bosman [Wed, 8 Aug 2012 21:13:44 +0000 (14:13 -0700)]
Unbreak folly build on fedora / ubuntu

Test Plan: built

Reviewed By: delong.j@fb.com

FB internal diff: D543320

12 years agoUse gcc builtins instead of library functions for ffs*
Tudor Bosman [Wed, 8 Aug 2012 21:32:27 +0000 (14:32 -0700)]
Use gcc builtins instead of library functions for ffs*

Summary:
because, shockingly, glibc doesn't divert to the gcc builtins which are
inlined and fast

Test Plan: folly/test

Reviewed By: sding@fb.com

FB internal diff: D543363

12 years agoRemove 1 instruction from popcount
Tudor Bosman [Wed, 8 Aug 2012 21:08:03 +0000 (14:08 -0700)]
Remove 1 instruction from popcount

Summary:
by removing the unnecessary constraint that the input and output be in
the same register (copypasta from https://phabricator.fb.com/D542718)

Test Plan: folly/test

Reviewed By: soren@fb.com

FB internal diff: D543310

12 years agoDetect popcnt instruction at runtime, use it if available.
Tudor Bosman [Wed, 8 Aug 2012 17:39:47 +0000 (10:39 -0700)]
Detect popcnt instruction at runtime, use it if available.

Summary:
If compiled for a popcnt-supporting target (-march=corei7, for example),
use __builtin_popcount, as it's presumably inlined.  Otherwise, detect
on startup (in the same way as glibc dispatches to one of the many
flavors of memcpy): GCC allows us to add a resolver function which the
dynamic loader will call on startup to resolve a function to one of
various alternatives; we check (using the cpuid instruction) whether
popcnt is supported, and use it if available.

Test Plan: tests added

Reviewed By: soren@fb.com

FB internal diff: D542977

12 years agofolly::dynamic: use std::vector instead of folly::fbvector
Andrew Gallagher [Wed, 8 Aug 2012 19:05:18 +0000 (12:05 -0700)]
folly::dynamic: use std::vector instead of folly::fbvector

Summary:
It appears that std::unordered_map is no longer relocatable in
gcc-4.7.  Use std::vector instead, until fbvector supports non-
relocatable types.

Test Plan: Built and ran facebar tests under gcc-4.7.  Also ran folly unittests.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D543099

12 years agoAdd operator== and operator!= to folly::StlAllocator
Xiaoyang Gao [Tue, 7 Aug 2012 01:53:02 +0000 (18:53 -0700)]
Add operator== and operator!= to folly::StlAllocator

Summary: Add operator== and operator!= to folly::StlAllocator, which are required by some other libraries.

Test Plan: Tested by running with recursive_wrapper in boost::variant, and unit tests in folly/test.

Reviewed By: delong.j@fb.com

FB internal diff: D541237

12 years agoRemove unused logging header
James Sharpe [Mon, 6 Aug 2012 15:29:27 +0000 (16:29 +0100)]
Remove unused logging header

Test Plan: Compiled.

Reviewed By: andrewjcg@fb.com

FB internal diff: D541226

12 years agoMinor typo in comments
Daniel Wu [Wed, 4 Jul 2012 15:34:49 +0000 (08:34 -0700)]
Minor typo in comments

Summary: github pull #11

Test Plan: None.

Reviewed By: andrewjcg@fb.com

FB internal diff: D541230

12 years agofbstring: mute uninitialized warnings in fbstring code
Andrew Gallagher [Wed, 8 Aug 2012 05:49:39 +0000 (22:49 -0700)]
fbstring: mute uninitialized warnings in fbstring code

Summary:
fbstring generates some apparently false positive uninitialized warnings
under gcc-4.7.1.  This diff disables the '-Wuninitialized' flag for
fbstring.

Test Plan: build

Reviewed By: ldemailly@fb.com

FB internal diff: D542722

12 years agoAdd inverses for jenkins_rev_mix32, twang_mix64
Tudor Bosman [Fri, 3 Aug 2012 01:23:23 +0000 (18:23 -0700)]
Add inverses for jenkins_rev_mix32, twang_mix64

Test Plan: test added

Reviewed By: soren@fb.com

FB internal diff: D538703

12 years agoAllow capacity and length to be different for user buffers
Abhijeet Joglekar [Wed, 1 Aug 2012 17:36:14 +0000 (10:36 -0700)]
Allow capacity and length to be different for user buffers

Summary:
Currently, takeOwnership lets a user pass in a data buffer and wraps it in an
IOBuffer. However, the capacity and length of the user buffer are required to
be same.

Added a new API to allow buffers where the capacity and length can be different.

Users of existing API should not be affected.

Test Plan:
1) Added new test case to IOBufTest that checks buffers with length
different from capacity
2) Run existing IOBuf test cases to verify no regressions

Reviewed By: brianp@fb.com

FB internal diff: D536575

12 years agoMake AtomicHashMap support move constructible types
Wei Wu [Mon, 23 Jul 2012 16:20:41 +0000 (09:20 -0700)]
Make AtomicHashMap support move constructible types

Summary: modified AtomicHashArray and AtomicHashMap to support move constructible types

Test Plan: tested with fbcode/folly/test/AtomicHashArrayTest.cpp and fbcode/folly/test/AtomicHashMapTest.cpp

Reviewed By: philipp@fb.com

FB internal diff: D527270

12 years agoMoving some more benchmarks to fbcode benchmark system
Rajat Goel [Thu, 26 Jul 2012 20:24:32 +0000 (13:24 -0700)]
Moving some more benchmarks to fbcode benchmark system

Summary: The format benchmark that was ported some time back has few data points to show - http://fburl.com/3545876

Test Plan: fbmake --log-to-rfe runbenchmarks_opt

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D524306

12 years agoAdd IEC binary unit prefixes
Tudor Bosman [Sun, 22 Jul 2012 02:13:21 +0000 (19:13 -0700)]
Add IEC binary unit prefixes

Summary: KiB, MiB, GiB, TiB, etc

Test Plan: by hand in a program using them

Reviewed By: philipp@fb.com

FB internal diff: D527001

12 years agoadd sched_yield in RWTicketSpinLock
Nathan Bronson [Thu, 19 Jul 2012 20:33:29 +0000 (13:33 -0700)]
add sched_yield in RWTicketSpinLock

Summary:
This diff makes RWTicketSpinLock eventually start calling
sched_yield() during shared and aggressive exclusive lock access, to
avoid pathologies that can arise when the number of threads far
exceeds the number of actual cores.

Test Plan:
1. unit tests
2. benchmark w/o + w/ diff

Reviewed By: nathan@fb.com

FB internal diff: D524897

12 years agoAdd folly::applyTuple utility
Jordan DeLong [Sat, 14 Jul 2012 22:57:32 +0000 (15:57 -0700)]
Add folly::applyTuple utility

Summary: Applies a function to a std::tuple of arguments.

Test Plan: Unit test.

Reviewed By: tjackson@fb.com

FB internal diff: D520190

12 years agoFix an issue in IOBufQueue::move
Tudor Bosman [Sat, 14 Jul 2012 05:56:13 +0000 (22:56 -0700)]
Fix an issue in IOBufQueue::move

Summary: That's all.

Test Plan: .

Reviewed By: philipp@fb.com

FB internal diff: D520116

12 years agoAdd wrapper macros for "final" and "override"
Tudor Bosman [Mon, 16 Jul 2012 03:13:19 +0000 (20:13 -0700)]
Add wrapper macros for "final" and "override"

Summary:
... which are supported in gcc 4.7, but not 4.6, and they enable useful
optimizations.  As long as we're planning to support both 4.6 and 4.7 in
our code base, we should write code using these macros.

Test Plan:
test added, compiled with both 4.6 and 4.7, verified the
assertion in the test comment by looking at dissassembly output in the
4.7 version

Reviewed By: delong.j@fb.com

FB internal diff: D520343

12 years agoAdd missing include
Tudor Bosman [Fri, 13 Jul 2012 23:40:30 +0000 (16:40 -0700)]
Add missing include

Test Plan: yes

Reviewed By: delong.j@fb.com

FB internal diff: D519897

12 years agorallocm(ALLOCM_ZERO) only zeroes *new* memory
Tudor Bosman [Fri, 13 Jul 2012 22:58:47 +0000 (15:58 -0700)]
rallocm(ALLOCM_ZERO) only zeroes *new* memory

Summary:
If you allocate N bytes and then try to grow that in place using rallocm
(ALLOCM_ZERO | ALLOCM_NO_MOVE) to M > N, our code assumed that all
memory from N to M would be zeroed.  In fact, rallocm only zeroes *new*
memory, so if jemalloc actually allocated N1 > N bytes initially,
rallocm will only zero memory from N1 to M.

Fixed by using calloc.

Test Plan: thread_local_test, tested in production

Reviewed By: delong.j@fb.com

FB internal diff: D519781

12 years agoA sample diff thats ports FormatTests to benchmarks
Rajat Goel [Fri, 13 Jul 2012 18:00:51 +0000 (11:00 -0700)]
A sample diff thats ports FormatTests to benchmarks

Summary:
Is there any better solution? Maybe a generic flag in Benchmarks to say
please don't run benchmarks?

Test Plan: (not in folly)

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D503619

12 years agoUnbreak build.
Tudor Bosman [Fri, 13 Jul 2012 03:18:29 +0000 (20:18 -0700)]
Unbreak build.

Summary: I could have sworn I undid that un-templatification of the constructor.

Test Plan: build folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D518981

12 years agoRange<const char*> -> Range<const unsigned char*> implicit conversion
Tudor Bosman [Thu, 12 Jul 2012 23:33:38 +0000 (16:33 -0700)]
Range<const char*> -> Range<const unsigned char*> implicit conversion

Summary: As they can both be used to represent ranges of bytes.

Test Plan: test added

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D518666

12 years ago(minor changes, part of unrelated diff)
Tudor Bosman [Tue, 10 Jul 2012 20:40:31 +0000 (13:40 -0700)]
(minor changes, part of unrelated diff)

Test Plan: No

Reviewed By: lucian@fb.com

FB internal diff: D515713

12 years agoMake Bits<T> work with T = Unaligned<X> (X is unsigned integral type)
Tudor Bosman [Wed, 11 Jul 2012 21:28:00 +0000 (14:28 -0700)]
Make Bits<T> work with T = Unaligned<X> (X is unsigned integral type)

Test Plan: all folly tests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D517118

12 years agofolly/Malloc.h: use libstdc++-safe exception wrappers
Andrew Gallagher [Wed, 11 Jul 2012 21:03:19 +0000 (14:03 -0700)]
folly/Malloc.h: use libstdc++-safe exception wrappers

Summary:
gcc errors out when code that throws is compiled with
'-fno-exceptions'.  Since we add Malloc.h into our custom libstdc++
use the exception wrappers from functexcept.h so that they can build
with third-party projects that use '-fno-exceptions'.

Test Plan:
Built llvm in the gcc-4.7.1-glibc-2.14.1-fb platform with these
changes.

Reviewed By: tudorb@fb.com

FB internal diff: D516577

12 years agofix memory order in AtomicHashMap<>::findInternal()
Philip Pronin [Tue, 10 Jul 2012 08:27:50 +0000 (01:27 -0700)]
fix memory order in AtomicHashMap<>::findInternal()

Summary:
Looks like a typo, it should obviously be std::memory_order_relaxed (all
necessary synchronization is done on numMapsAllocated_).

Test Plan: compiled it, ran tests

Reviewed By: sahrens@fb.com

FB internal diff: D515322

12 years agofolly: fix build on ubuntu and fedora platforms
Andrew Gallagher [Mon, 9 Jul 2012 03:11:11 +0000 (20:11 -0700)]
folly: fix build on ubuntu and fedora platforms

Summary:
A boost regex dependency wasn't added to the Makefile.am file.  Also
the it appears that boost thread lib dependencies had a typo in the
makefile variable name.

Test Plan: ran builds on ubunutu and fedora VMs

Reviewed By: tudorb@fb.com

FB internal diff: D514039

12 years ago(minor changes, part of unrelated diff)
Tudor Bosman [Fri, 6 Jul 2012 02:25:17 +0000 (19:25 -0700)]
(minor changes, part of unrelated diff)

Test Plan: no

Reviewed By: philipp@fb.com

FB internal diff: D512868

12 years agoAdd IOBufQueue::prepend, fix bug in IOBuf::prepend.
Tudor Bosman [Sat, 7 Jul 2012 00:12:07 +0000 (17:12 -0700)]
Add IOBufQueue::prepend, fix bug in IOBuf::prepend.

Summary:
IOBuf::prepend needs to increment length_.

Added IOBufQueue::prepend, which uses the headroom in the first buffer
instead of growing the queue at the head.

Test Plan: tests added

Reviewed By: simpkins@fb.com

FB internal diff: D513676

12 years agoImprove digits_to() performance
Eitan Frachtenberg [Wed, 13 Jun 2012 00:07:02 +0000 (17:07 -0700)]
Improve digits_to() performance

Summary:
This optimization eliminates about 75% of the multiplications and bounds
checking in the original code by using constant lookup tables to convert
from decical digit to (shifted) binary number, instead of arithmetic.
The total cost of the lookup tables is 2KB of static memory, but the
average throughput gain across the range from 1 to 19 characters is 27%.
The throughput distributes as follows: (in million iters/sec)
Bytes   Orig    New     Speedup
1       101.31  110.12  109%
2       97.42   110.12  113%
3       97.42   105.53  108%
4       79.13   101.31  128%
5       74.48   97.42   131%
6       74.48   93.81   126%
7       70.35   93.81   133%
8       61.77   79.14   128%
9       58.9    72.35   123%
10      58.9    74.48   126%
11      56.28   74.48   132%
12      50.22   64.93   129%
13      48.52   60.3    124%
14      47.74   61.77   129%
15      46.88   61.77   132%
16      42.54   55.06   129%
17      41.51   51.69   125%
18      40.97   52.76   129%
19      39.57   52.76   133%

Test Plan: Passes all unit tests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D493245

12 years agoAdd multi-bit operations to folly/experimental/Bits.h
Tudor Bosman [Wed, 4 Jul 2012 00:02:05 +0000 (17:02 -0700)]
Add multi-bit operations to folly/experimental/Bits.h

Summary: Add ability to set and get N consecutive bits from the bit sequence.

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D511438

12 years agoadding StringPiece constructor that takes a piece of another StringPiece
Rafael Sagula [Thu, 28 Jun 2012 23:05:21 +0000 (16:05 -0700)]
adding StringPiece constructor that takes a piece of another StringPiece

Summary:
adding what seems to be a missing constructor to StringPiece --
I need to be able to take a piece of another StringPiece. (It's possible
to do that with all sorts of strings already, except StringPiece...)

Test Plan: na -- tested as part of other (dependent) diffs

Reviewed By: delong.j@fb.com

FB internal diff: D508545

12 years agoRemove unnecessary member variable.
Tudor Bosman [Mon, 2 Jul 2012 00:40:06 +0000 (17:40 -0700)]
Remove unnecessary member variable.

Test Plan: No

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D509324

12 years agoadd bm_max/min_iters and bm_regex
Spencer Ahrens [Sat, 30 Jun 2012 18:01:06 +0000 (11:01 -0700)]
add bm_max/min_iters and bm_regex

Summary: These were handy in the old framework and easy enough to add.

Test Plan:
ran ThreadCachedIntTest - got better (but not perfect due to
subtraction of single-threaded baseline from multi-threaded results).

Reviewed By: sherman.ye@fb.com

FB internal diff: D490324

12 years agofolly: get building with gcc-4.7.1
Andrew Gallagher [Fri, 29 Jun 2012 23:47:49 +0000 (16:47 -0700)]
folly: get building with gcc-4.7.1

Summary:
A lot of libstdc++-4.7 headers dropped includes of unistdc.h, so
we'll probably see more of these.

Test Plan: built folly

Reviewed By: simpkins@fb.com

FB internal diff: D508871

12 years agoMinor changes to folly/experimental/io
Tudor Bosman [Mon, 11 Jun 2012 23:24:30 +0000 (16:24 -0700)]
Minor changes to folly/experimental/io

Reviewed By: philipp@fb.com

FB internal diff: D491952

12 years agoAdding support to output benchmarks result in JSON
Rajat Goel [Mon, 25 Jun 2012 20:41:49 +0000 (13:41 -0700)]
Adding support to output benchmarks result in JSON

Summary: The main reason being so that they can be fed to some automated systems, if required.

Test Plan: Just ran few benchmarks locally

Reviewed By: andrewjcg@fb.com

FB internal diff: D501025

12 years agoAdd TypedIOBuf.
Tudor Bosman [Fri, 15 Jun 2012 00:07:00 +0000 (17:07 -0700)]
Add TypedIOBuf.

Summary: Simple class to handle IOBuf as an array of objects of fixed size.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D497287

12 years agoAdd sizeGuess() member to ProducerConsumerQueue
Mike Curtiss [Thu, 7 Jun 2012 07:32:14 +0000 (00:32 -0700)]
Add sizeGuess() member to ProducerConsumerQueue

Summary:
This is part 1 of a change to add hysteretic behavior to a blocking queue based on ProducerConsumerQueue.

Knowing the size is useful for monitoring and possibly for objects that contain a ProducerConsumerQueue.

Test Plan:
Added tiny test-case.

Tests pass

Reviewed By: delong.j@fb.com

FB internal diff: D496787

12 years agoAdd an allocate() member function on IOBufQueue
Tudor Bosman [Wed, 13 Jun 2012 23:52:38 +0000 (16:52 -0700)]
Add an allocate() member function on IOBufQueue

Summary: Function is documented in the header.

Reviewed By: soren@fb.com

FB internal diff: D494315

12 years agoFix a typo in benchmark comments
Sherman Ye [Wed, 9 May 2012 16:26:13 +0000 (09:26 -0700)]
Fix a typo in benchmark comments

Summary: As title

Test Plan: Deployed to wormhole_apps.dragon.dev

Reviewed By: asharma@fb.com

FB internal diff: D493253

12 years agoUnbreak Ubuntu build.
Tudor Bosman [Thu, 14 Jun 2012 19:14:04 +0000 (12:14 -0700)]
Unbreak Ubuntu build.

Summary: follybenchmark and follyfingerprint need folly to work.

Test Plan: built on ubuntu

Reviewed By: andrewjcg@fb.com

FB internal diff: D494955

12 years agoFix clang narrowing error in GroupVarintTables
Andrew Gallagher [Thu, 14 Jun 2012 19:24:31 +0000 (12:24 -0700)]
Fix clang narrowing error in GroupVarintTables

Summary:
The __m128i type is a pair of 64-bit signed ints and the values
generated in generate_varint_tables.py overflow this range, which
causes clang to complain.

Test Plan:
Built and ran unittests.  Also verified that the GroupVarintTable.o
data section was identical with and w/o this change.

Reviewed By: tudorb@fb.com

FB internal diff: D494645

12 years agoClarify split / splitTo comments.
Tudor Bosman [Thu, 14 Jun 2012 01:57:34 +0000 (18:57 -0700)]
Clarify split / splitTo comments.

Test Plan: No

Reviewed By: delong.j@fb.com

FB internal diff: D494525

12 years agoMove and refactor code
Chip Turner [Fri, 8 Jun 2012 21:53:51 +0000 (14:53 -0700)]
Move and refactor code

Summary:

Moves some string manipulation code like humanify into folly

Moves hash-related functions into folly and removes some old includes
to clean up some code

Adds generic hashing for tuples, similar to pairs

Updates all of the build breakages from the above

Test Plan: run unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D490478

12 years agoFix narrowing warning (if enabled).
Tudor Bosman [Wed, 13 Jun 2012 00:37:21 +0000 (17:37 -0700)]
Fix narrowing warning (if enabled).

Summary: They're (signed) long long.

Test Plan: built, folly/test

Reviewed By: andrewjcg@fb.com

FB internal diff: D493270

12 years agoFix intermittent failure in concurrent_skip_list_test
Tudor Bosman [Wed, 13 Jun 2012 00:08:04 +0000 (17:08 -0700)]
Fix intermittent failure in concurrent_skip_list_test

Summary: It creates too many threads, stop when we run out of resources.

Test Plan: by hand, in a loop, in a memory-constrained environment

Reviewed By: delong.j@fb.com

FB internal diff: D493239

12 years agoFix build and test.
Tudor Bosman [Tue, 12 Jun 2012 22:44:21 +0000 (15:44 -0700)]
Fix build and test.

Summary:
Fix bug in FBStringTest (don't reuse the same file name)

Don't use VLOG, as that requires that glog is compiled with gflags
support, which it isn't always.

Add other targets and tests, make all tests pass.

Test Plan: tests

Reviewed By: delong.j@fb.com

FB internal diff: D493018

12 years agoFix build and test.
Tudor Bosman [Tue, 12 Jun 2012 22:44:21 +0000 (15:44 -0700)]
Fix build and test.

Summary:
Fix bug in FBStringTest (don't reuse the same file name)

Don't use VLOG, as that requires that glog is compiled with gflags
support, which it isn't always.

Add other targets and tests, make all tests pass.

Test Plan: tests

Reviewed By: delong.j@fb.com

FB internal diff: D493018

12 years agoMove Rabin fingerprinting code to folly.
Tudor Bosman [Tue, 12 Jun 2012 16:20:03 +0000 (09:20 -0700)]
Move Rabin fingerprinting code to folly.

Summary:
Also generate fingerprint tables every time, so the code doesn't rot.

TODO(tudorb): move benchmark to folly
TODO(tudorb): Include the program used to generate the polynomials
(can't build as it requires NTL from http://www.shoup.net/ntl/)

Test Plan: folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D492455

12 years agoInternalBuf doesn't need capacity
Tudor Bosman [Tue, 12 Jun 2012 00:09:43 +0000 (17:09 -0700)]
InternalBuf doesn't need capacity

Summary: as it's always kMaxInternalDataSize.

Test Plan: all tests in folly/experimental/io/test, both opt and dbg

Reviewed By: brianp@fb.com

FB internal diff: D492008

12 years agoProperly align internal buffer in IOBuf.
Tudor Bosman [Mon, 11 Jun 2012 23:18:22 +0000 (16:18 -0700)]
Properly align internal buffer in IOBuf.

Summary:
Yes, it's gcc specific, because alignas and max_align_t aren't in gcc
4.6.2.

Test Plan: iobuf_test, test added

Reviewed By: brianp@fb.com

FB internal diff: D491939

12 years agoAdd IOBuf::takeOwnership(unique_ptr<T>&&).
Tudor Bosman [Mon, 11 Jun 2012 20:58:15 +0000 (13:58 -0700)]
Add IOBuf::takeOwnership(unique_ptr<T>&&).

Summary:
Useful for in-place serialization.  Dangerous, obviously, but anything
involving IOBuf usually is.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D491663

12 years agoTypos.
Tudor Bosman [Sat, 9 Jun 2012 22:32:10 +0000 (15:32 -0700)]
Typos.

Test Plan: No

Reviewed By: delong.j@fb.com

FB internal diff: D491038

12 years agoD'oh: fix size computation in ThreadLocalDetail.h
Tudor Bosman [Sat, 9 Jun 2012 04:30:50 +0000 (21:30 -0700)]
D'oh: fix size computation in ThreadLocalDetail.h

Test Plan: thread_local_test

Reviewed By: ldemailly@fb.com

FB internal diff: D490912

12 years agofolly::detail::futex should be inline
Philip Pronin [Fri, 8 Jun 2012 08:37:24 +0000 (01:37 -0700)]
folly::detail::futex should be inline

Summary: It breaks build.

Test Plan: will check *_try builds

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D489935

12 years agoAdd IOBufQueue::wrapBuffer, which handles buffers > 4GB.
Tudor Bosman [Thu, 7 Jun 2012 20:59:38 +0000 (13:59 -0700)]
Add IOBufQueue::wrapBuffer, which handles buffers > 4GB.

Test Plan: test added

Reviewed By: brianp@fb.com

FB internal diff: D489158

12 years agoAdd links to README.
Tudor Bosman [Thu, 7 Jun 2012 21:26:44 +0000 (14:26 -0700)]
Add links to README.

Test Plan: No

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D489199

12 years agoDeuglify unaligned accesses in GroupVarint.
Tudor Bosman [Thu, 7 Jun 2012 02:56:43 +0000 (19:56 -0700)]
Deuglify unaligned accesses in GroupVarint.

Test Plan: folly/test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D488631

12 years agoEventCount: "condition variable" for lock-free algorithms
Tudor Bosman [Thu, 17 May 2012 21:31:04 +0000 (14:31 -0700)]
EventCount: "condition variable" for lock-free algorithms

Summary:
This allows you to convert non-blocking lock-free algorithms into
blocking versions.

See
http://www.1024cores.net/home/lock-free-algorithms/eventcounts,
http://software.intel.com/en-us/forums/showthread.php?t=62364,
http://www.akkadia.org/drepper/futex.pdf

Test Plan: new test

Reviewed By: delong.j@fb.com

FB internal diff: D474570

12 years agofbstring: remove reference to /home/engshare/ paths
Andrew Gallagher [Tue, 5 Jun 2012 23:13:31 +0000 (16:13 -0700)]
fbstring: remove reference to /home/engshare/ paths

Summary: Reported by https://github.com/facebook/folly/issues/8

Test Plan: no

Reviewed By: tudorb@fb.com

FB internal diff: D487572

12 years agoFix some memory_order parameters in AtomicHashMap
Jordan DeLong [Sun, 3 Jun 2012 21:58:50 +0000 (14:58 -0700)]
Fix some memory_order parameters in AtomicHashMap

Summary: Herb Sutter noticed an error in the AHM find pseudo-code at
the C++ conference, and the error is in the real implementation too.
We should be using memory_order_acquire, because we need the load of
the key to happen before any subsequent loads of the value itself.
Our implementation is conservative for a memory_order_relaxed load
(we generate the same code as if it were memory_order_acquire---i.e. a
compiler barrier but no memory barriers), so this should not change the
generated code.  Correctness is a good idea, though (when we upgrade
to gcc 4.7's atomics it might not be as conservative).  Also, erase was
doing three loads when one is enough.  ... also fixes an exception safety
bug in insert.  (Phew.)

Test Plan: Ran the AHM unit tests, debug and opt.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D485744

12 years agoSome typos corrects, and collapsed the two descriptions of Bits.h into one.
Christopher C. Aycock [Sun, 3 Jun 2012 03:59:02 +0000 (00:59 -0300)]
Some typos corrects, and collapsed the two descriptions of Bits.h into one.

Summary: https://github.com/facebook/folly/pull/1

Test Plan: N/A.

Reviewed By: andrewjcg@fb.com

FB internal diff: D485738

12 years agoAdd #include <new> to folly/Malloc.h.
Tudor Bosman [Tue, 5 Jun 2012 07:04:29 +0000 (00:04 -0700)]
Add #include <new> to folly/Malloc.h.

Test Plan: folly tests

Reviewed By: philipp@fb.com, andrei.alexandrescu@fb.com

FB internal diff: D486884D487376

12 years agoCheck the return value from malloc / realloc.
Tudor Bosman [Tue, 5 Jun 2012 05:24:21 +0000 (22:24 -0700)]
Check the return value from malloc / realloc.

Summary:
https://github.com/facebook/folly/issues/7

Wrappers: checkedMalloc / checkedRealloc / checkedCalloc

Test Plan: all folly tests

FB internal diff: D486841