Default accept function in Acceptor

Summary: Since Acceptor can be used for TCP or UDP, make an empty default TCP accept function so subclasses don't have to.

Test Plan:
built D1942242, no build errors

contbuild

Reviewed By: yfeldblum@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948940

Signature: t1:1948940:1427498977:26b7bb23e3367ff6695dad86ebf0e60637815a7b

EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

Summary:
[Folly] EventBase::runImediatelyOrRunInEventBaseThreadAndWait.

When you have code that you need to run in the event loop and wait for it to finish, and you yourself might be in the event loop but might be in another thread instead.

Test Plan:
Unit tests:
* `folly/io/async/test/EventBaseTest.cpp`.

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, agallagher, njormrod, ldbrandy, brettp, dougw

FB internal diff: D1953110

Signature: t1:1953110:1427751076:ad2c9f4d5b65ef952cba94b06be887672ee44583

Fixing predicate inlining

Summary:
as title.

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, ridge@

FB internal diff: D1946589

Extend IO pool with external optional event_base_manager

Summary: as title

Test Plan: run unit tests

Reviewed By: yfeldblum@fb.com

Subscribers: ming, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1950610

asyncudpserversocket reuse port

Summary:
AsyncUDPSocket supports reuse port, also make it available in AsyncUDPServerSocket.
Has to happen before bind

Test Plan:
?? it builds?

Mutliple binds already work with reuseaddr, but load balancing doesn't work as well.  See http://lwn.net/Articles/542629/ for details

Reviewed By: naizhi@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D1948932

Signature: t1:1948932:1427473013:8c114458a0746bb2b6957db33c1834179d860c00

Future::filter(A => bool)

Test Plan: new unit tests

Reviewed By: davejwatson@fb.com

Subscribers: chalfant, bmatheny, trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1828251

Tasks: 6166893

Signature: t1:1828251:1427475565:512463d5728482a40c1da2548a5bed91e7f92d60

Add CaptureFD for log testing (and some glog patterns)

Summary:
Without a gadget like this, it's really hard to test logging output. With it, it's really easy. I found about 50 callsites to the various functions across several projects, so folly seems appropriate.

PS The patterns are functions for two reasons:
- Static variables are a pain.
- This leaves the option of adding an optional argument, so you can grep for a particular kind of error string.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933439

Signature: t1:1933439:1427345479:5b3d1c6566a026fdbccb16b382211688e327ea1a

Add EXPECT_{NO_,}_PCRE_MATCH macros

Summary: There are about 40 callsites to these across a couple of projects, and they seem generally useful.

Test Plan: ```fbconfig folly/experimental/test && fbmake runtests```

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1933415

Signature: t1:1933415:1427345307:6587eea3dac74c5d841950ace3c3501d6e3dbe4a

TemporaryFile and TemporaryDirectory

Summary:
TemporaryFile, TemporaryDirectory, and ChangeToTempDir should all be moveable
objects, but not copiable.  Define default move constructors and move
assignment operators for these classes.  This will prevent copy constructor and
copy assignment operators from being implicitly defined.

Test Plan:
Used this in a new test to write a helper function which created and returned
a new TemporaryFile object using the move constructor.

Reviewed By: yfeldblum@fb.com

Subscribers: doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1945134

Signature: t1:1945134:1427342944:3428327e797ce4b3d362f9a2d2276de6d8b96137

(Wangle) Reduce

Summary:
1] The lambda should ble able to return a `Try<T>`. Maybe? Can a `then`
return a `Try<T>` actually? Can fix this with `resultOf`. (Doubling the number of functions to 4.)

2] `initial` and `func` have to be copyable.

Test Plan: Added tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1870996

Tasks: 6025252

Signature: t1:1870996:1427318511:2ae5894b79022da88990835b26d35c4520fdbd29

Bump version to 32:0

Add a pretty macro for deprecation

Summary: Make it easy for people to mark code as deprecated

Test Plan: Chose a random file and added the FOLLY_DEPRECATED macro to it

Reviewed By: yfeldblum@fb.com

Subscribers: ps, folly-diffs@, yfeldblum

FB internal diff: D1943248

Signature: t1:1943248:1427316689:b3305d134f374697e93d36bafbf0961eec1ef68a

Relax CHECK condition in stringAppendfImpl

Summary:
vsnprintf can return fewer bytes and bytes_used if
we print a string containing '\0' using a width
specifier.

Test Plan: -- ran all tests

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1915035

Signature: t1:1915035:1426799341:4aaea928c4bdde1998bf66cf9e2732a53572c6e3

httpserver on serverbootstrap (2)

Summary:
Original diff D1800100, reverted in D1895181.  Issue seems to have been caused by a copy constructor issue.

Fixes: Remove copy constructor, move contructor only. Separate stop() and join(), since join() can happen only after all stop()s have completed when reusing the same IOThreadPoolExecutors.
Test Plan: build and unit tests of relevant product

Reviewed By: praveenr@fb.com

Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1901718

Tasks: 6431975

Signature: t1:1901718:1426885577:46b63bfe117df207be3d273b953c100249c1a7da

Blame Revision: D1895181

Add ChangeToTempDir to TestUtil

Summary: It's convenient to be able to write tempfiles in the current directory. This provides a one-liner way of doing that in tests.

Test Plan: unit test

Reviewed By: yfeldblum@fb.com

Subscribers: simpkins, folly-diffs@, yfeldblum

FB internal diff: D1931918

Signature: t1:1931918:1426913634:f988fb3c5061f5909e309dcaf42742d9b2ed18c6

Make folly::Singleton DFATAL as late as possible

Summary: We want to print all destructor stack-traces if Singleton destruction happens eventually (even after destroyInstances() was done).

Test Plan:
unit test

Reviewed By: chip@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1929881

Signature: t1:1929881:1426878091:5993d544c0f84bdbf8bcdd420ef01fb7bf88ac3f

Udp Acceptor

Summary:
major changes:

1) ServerSocketFactory and AsyncSocketBase to abstract the differences between UDP and TCP async socket.  Could possibly push some of this to the sockets themselves eventually
2) pipeline() is a pipeline between accept/receive of a UDP message, and before sending it to workers.  Default impl for TCP is to fan out to worker threads.  This is the same as Netty.  Since we don't know if the data is a TCP socket or a UDP message, it's a void*, which sucks (netty uses Object msg, so it isn't any different).

Test Plan: Added lots of new tests.  Doesn't test any data passing yet though, just connects/simple receipt of UDP message.

Reviewed By: hans@fb.com

Subscribers: alandau, bmatheny, mshneer, jsedgwick, yfeldblum, trunkagent, doug, fugalh, folly-diffs@

FB internal diff: D1736670

Tasks: 5788116

Signature: t1:1736670:1424372992:e109450604ed905004bd40dfbb508b5808332c15

fix service memory leak

Summary:
This code is only used in a test so far.

Basically it looks like Pipeline has some dependency issues - any addBack() pipelines have to be deleted *after* the main pipeline is destroyed.  Ideally if the pipeline is already closed, the destruction order wouldn't matter.  There is currently no removeBack() call either.   This is a quick fix for the test to just delete stuff in the right order, we can discuss a better solution when @jsedgwick returns

Test Plan: fbconfig --sanitize=address --with-project-version=clang:3.5 --clang folly/wangle/service && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, fugalh, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1903200

Tasks: 6415578

Signature: t1:1903200:1426528298:e109dcc0ec586a505a26cd95a6f20434d22cbd37

Add comment about djb2 hash

Summary:
In my tests, using djb2 was much worse for any sizable workload. Just use SpookyHashV2.

Test Plan: comment only

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1906439

Signature: t1:1906439:1426117575:f88b7b28b578092c1716433e21150d423d4e94a7

SocketAddress::anyAddress should be a method

Summary:
Calling getaddrinfo from a static constructor is not the most
reliable thing to do. Stop doing it.

Test Plan: existing tests

Reviewed By: davejwatson@fb.com

Subscribers: dancol, mzlee, shikong, kmdent, fma, bmatheny, benyluo, ranjeeth, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1924578

Tasks: 6534662

Signature: t1:1924578:1426718586:d73bc2f001095e66fa77fde9c027af050cc26d2a

(Wangle) Clean up move constructors

Summary: I thought this might fix #6120972. But it doesn't. Still a bit of a cleanup in my opinion.

Test Plan: Run all the tests?

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907259

Signature: t1:1907259:1426613567:9e33fe7e9e8a36ba006d4aee604086a56f128893

Support 'min ms to log'

Summary:
Sometimes we only want to log if enough time has passed, e.g. when logging
inside of a loop doing work to detect if things are slow. Add an option for
that.

Test Plan:
build and unit tests of folly/experimental

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1909055

Signature: t1:1909055:1426627731:6d3ad94fd71967ec89ffebc90cd2907dd1631d36

Fix logFormat

Summary: Ooops during port ;)

Test Plan: unit test

Reviewed By: agoder@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905639

Signature: t1:1905639:1426171227:a6e415046a85f098ac090f069476204fb0367c42

Bump version to 31:0

Fixed README download link to get the correct GTest version

Summary: Signed-off-by: Nicholas Ormrod <njormrod@fb.com>

Test Plan: Clicked link, gtest started downloading

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1901849

Tasks: 6255850

Signature: t1:1901849:1426097071:e887ed8749c70d5e15877082ac0175ce3b56647e

BackoffTimeout disallow copy, move, default ctor

Summary:
BackoffTimeout disallow copy construction, copy assignment, move
construction, move assignment, and default construction.

Closes https://github.com/facebook/folly/pull/146

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912521

Signature: t1:1912521:1426268413:6edb2d8203d1efd1e825816f03be115ac46e55ca

We might avoid some temporaries

Summary:
It seems we might avoid temporaries.  To do so we pass arguments
directly to constructors with the help of emplace_back member.

Test Plan:
all folly/tests, make check for 37 tests, passed.

Closes https://github.com/facebook/folly/pull/150

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912505

Signature: t1:1912505:1426268158:086882bb53f3d79c3f3b7b2c362318f09b5ee3be

AsyncServerSocket disallow copy, move, and default ctor

Summary:
AsyncServerSocket disallow copy construction, copy assignment,
move construction, move assignment, and default
construction.

Closes https://github.com/facebook/folly/pull/148

Test Plan: all folly/tests, make check for 37 tests, passed.

Reviewed By: ldbrandy@fb.com

Subscribers: lins, anca, folly-diffs@, yfeldblum

FB internal diff: D1912488

Signature: t1:1912488:1426267982:a9690ce9429016f89630f0ded64b7be9ff80c39d

fix typo: add backslash to Makefile.am

Summary: backslash

Test Plan: tests

Reviewed By: aap@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1920872

Signature: t1:1920872:1426624366:c984fecb5bd0c292d5a8309fb320a48872fff6db

Blame Revision: D1890511

Add StringKeyedUnorderedMap to Makefile

Test Plan:
unit tests

Reviewed By: bwatling@fb.com

Subscribers: stepan, folly-diffs@, yfeldblum, trunkagent, vikas, alikhtarov

FB internal diff: D1890511

Tasks: 6311387

Signature: t1:1890511:1425924980:71af7c12a09a86b280dc3c9a9ef0756bf56bf639

(folly osx) test for the zip, not the dir

Summary: Apparently `configure` creates this subdirectory and then the curl and unzip commands don't run, and then `make check` fails. Instead, do it if the zip file doesn't exist.

Test Plan:
git clone https://github.com/facebook/folly.git
cd folly/folly
/path/to/this/bootstrap-osx-homebrew.sh
make
make check

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, davejwatson

FB internal diff: D1882472

Signature: t1:1882472:1425920470:bdab25967d3b8e52c690d5627168077c65829b92

exceptionStr(exception_wrapper)

Summary: for consistency with exception and exception_ptr functions provided in String.h

Test Plan: unit

Reviewed By: vloh@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1905087

Tasks: 5961362

Signature: t1:1905087:1426101740:670ceab5140250bbecdd1247025d3d70b7774690

extend future api with Future<T>::poll

Summary:
Added folly::Future<T>::poll() function with the following semantics:
poll() checks if the promise has been fulfilled, if so it will extract the value of the resulting Try<T> to construct a new Try<T> that is wrapped in an Optional<Try<T>>. If the promise has not been fulfilled an empty Optional is returned.

Test Plan: Added two tests to SugarTest.cpp: one where the promise has been fulfilled and one where it hasn't.

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, arturg, yfeldblum

FB internal diff: D1909480

Tasks: 6166875

Signature: t1:1909480:1426280247:6b239c0f1a15c25ce7664afb4bfb3899993c5775

Add ctor to CPUThreadPoolExecutor to enable custom queue sizes

Summary:
Seems useful to do without passing in a totally custom queue. Makes
the client code a bit simpler.

Test Plan: used it, saw low memory usage and no crash

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, iaroslav, fugalh, folly-diffs@, jsedgwick, yfeldblum, slarsen

FB internal diff: D1907874

Signature: t1:1907874:1426268010:3159ae339b51f5dbc7fe034644bbc968b92b072c

Add support for iovec for UDP send in folly::AsyncUDPSocket

Summary:
I will be using multi-buf UDP send soon, so adding the
support in folly::AsyncUDPSocket

Test Plan: Unit tests and turn server

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1907189

Signature: t1:1907189:1426266951:046198e0a009fef085ac7eb44f054c67dfb16ba3

add Cursor::isAtEnd()

Summary:
Add a helper method to efficiently check if the cursor is at the end of the
IOBuf chain.  This is equivalent to (cursor.totalLength() > 0), but it doesn't
need to walk the entire chain just to tell if it is at the end or not.

Test Plan: Updated the unit tests to contain some checks for isAtEnd().

Reviewed By: jasmeetbagga@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@, yfeldblum

FB internal diff: D1875345

Signature: t1:1875345:1425006512:49ac246fd0ac7937fdcd6cf1359a841f048c444e

Protect unprotected

Summary: This used to be protected before.

Test Plan: folly/futures tests

Reviewed By: andrii@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1907063

Signature: t1:1907063:1426116657:d7430304d5f72a048396e83f8dc6b5a92198fb29

CPUThreadPoolExecutor's default queue is quite big

Summary:
By default, CPUThreadPoolExecutor uses 100 MB of rss for the CPU queue. This is
substantially higher than expected.

Reduce the default queue size so that it uses much less memory upfront, down
to about 6 MB going by the heap profile.

Reviewed By: davejwatson@fb.com

Subscribers: mwa, ott, iaroslav, jrsherwood, aflock, slarsen, davejwatson, trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1900539

Tasks: 6453427

Signature: t1:1900539:1426103805:34fb7b081058fbde89370b5d9ec24ea572c4b0e2

fix Makefile.am formatting

Summary: D1798929 broke the automake folly build

Test Plan:
1. careful visual inspection
2. build

Reviewed By: alikhtarov@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1906281

Signature: t1:1906281:1426106340:621f1e7d2fb18ae2ec5dc604dea0d8ad4353486e

SharedMutex - a small fast scalable reader-writer lock

Summary:
SharedMutex is a reader-writer lock.  It is small, very fast, scalable
on multi-core, and suitable for use when readers or writers may block.
Unlike most other reader-writer locks, its throughput with concurrent
readers scales linearly; it is able to acquire and release the lock
in shared mode without cache line ping-ponging.  It is suitable for a
wide range of lock hold times because it starts with spinning, proceeds
to using sched_yield with a preemption heuristic, and then waits using
futex and precise wakeups.

SharedMutex provides all of the methods of folly::RWSpinLock,
boost::shared_mutex, boost::upgrade_mutex, and C++14's
std::shared_timed_mutex.  All operations that can block are available in
try, try-for, and try-until (system_clock or steady_clock) versions.
Both reader-priority and writer-priority versions are provided.
Writer-priority is the default at the moment.

In my tests SharedMutex is as good or better than the other reader-writer
locks in use at Facebook for almost all use cases, sometimes by a wide
margin.  (If it is rare that there are actually concurrent readers
then RWSpinLock can be a few nanoseconds faster.)  I compared it to
folly::RWSpinLock, folly::RWTicketSpinLock64, boost::shared_mutex,
pthread_rwlock_t, and an internal RWLock that uses spinlocks to guard
its state and pthread_mutex_t+pthread_cont_t to perform blocking.
(The other ReadWriteMutex-s I found use pthread_rwlock_t underneath.)
It is generally as good or better than the rest when evaluating size,
speed, scalability, or latency outliers.  In the corner cases where
it is not the fastest (such as single-threaded use or heavy write
contention) it is never very much worse than the best.  See the bottom
of SharedMutexTest.cpp for lots of microbenchmark results.

Test Plan:
1. new unit tests
2. new microbenchmarks included here
3. uncommitted microbenchmark from bmaurer's RWSleepLock
4. replace admarket's RWSpinLock and RWLock with SharedMutex, observe
neutral adindexer perf in canary
5. replace multifeed's thrift ReadWriteLock with SharedMutex, observe
neutral perf in canary

Reviewed By: hans@fb.com

Subscribers: fbcode-common-diffs@, tnovak, march, davejwatson, trunkagent, philipp, folly-diffs@, yfeldblum, bwatling, bmaurer, bol, marccelani, adri, strager, macsyz, dzhulgakov, zamsden

FB internal diff: D1798929

Signature: t1:1798929:1425575976:1c9221317eaa47628a2b8c374f90c7a2d4e3f0f9

Add null check to avoid crash in unit tests that use mock singletons.

Summary: Some unit tests owned by my team are failing due to segmentation fault in SingletonHolder<T>::registerSingletonMock -> SingletonHolder<T>::destroyInstance.

Test Plan: Run folly unit tests and my unit tests.

Reviewed By: henryf@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1873889

Signature: t1:1873889:1425925156:29d54092939d7e9debea3fd55f7105fd320e987e

Blame Revision: 91f4942e

Add JSON Schema Validator

Summary:
This is a validator for  JSON schema (http://json-schema.org/) that works on
folly::dynamic. Apparently there are no good open source ones for C++,
especially not if you want to use folly::dynamic. I am going to use this to
validate JSON configs.

It supports basically everything from the standard, except for fetching schemas
via http, and using id refs. It supports enough to check schemas against the
metaschema.

Currently you can define a schema that will crash on validation, if it's
infinitely self-recursive. See the unit test case that reproduces this.
Fixing this seems hard though, so I didn't bother. It would also probably
be slower for normal usage.

Test Plan: unit test

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1847657

Signature: t1:1847657:1425605163:635dc523aeda1b588c3634d0dc1a48d50a53db79

Bump version to 30:0

Remove unnecessary includes from FBVector

Summary:
Remove unnecessary includes from FBVector

Gonna see just how cleaned-up sandcastle really is :P

Test Plan: Fix errors shown in contbuild

Reviewed By: njormrod@fb.com

Subscribers: cold-storage-diffs@, jcoens, ldbrandy, trunkagent, oleksandr, netego-diffs@, mmandal, weiyan, ws_dev_diffs, jmkaldor, jhunt, fbcode-common-diffs@, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1873145

Signature: t1:1873145:1425505591:9cb345053833afa70f543e2f7acd310ab80b8676

Bump version to 29:0

httpserver on serverbootstrap

Summary:
Cleans up the httpserver startup code nicely.  The only major change to ServerBootstrap was a check if bind failed to throw an exception.

(depends on D1732895)

Test Plan:
fbconfig -r folly/wangle/bootstrap proxygen/httpserver; fbmake runtests

fbconfig -r dfsrouter; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: yfeldblum, cgheorghe, trunkagent, doug, fugalh, bmatheny, folly-diffs@, jsedgwick

FB internal diff: D1800100

Signature: t1:1800100:1424733970:67a61a22d2affadea16d2fd725003915326077b2

Bump version to 28:0

Bugfix uriEscapeTable generate.

Summary:
generate_escape_tables.py uses python range function which does not include the end of the range.

Closes #131

Test Plan: See attached test case.

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1871692

Signature: t1:1871692:1425076132:2438ab7554fe87bdef17c82ff27713811a270d7c

Add FunctionScheduler to Makefile.am

Summary: D1845525 was missing some makefile changes.

Test Plan: Examine similar change in D1732895

Reviewed By: markisaa@fb.com

Subscribers: sdwilsh, folly-diffs@, yfeldblum

FB internal diff: D1890197

Signature: t1:1890197:1425526255:e9a23fd96a89d4463cbdfda1385d4fd5f5781e8e

MoveWrapper::move()

Summary: sugar providing `foo.move()` instead of `std::move(*foo)`.

Test Plan: Inspection. Using it in another diff.

Reviewed By: yfeldblum@fb.com, hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum

FB internal diff: D1882208

Signature: t1:1882208:1425331389:a9c09ad2739838c829e2afdad64e985810154226

Bump version to 27:0

Add test for running after terminate

Summary:
Add a test to check what will happen if
someone tries to run a job on evb after
termination.

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: seanc, folly-diffs@, yfeldblum

FB internal diff: D1882308

Signature: t1:1882308:1425333248:7ab6692eb0b866fcc9685048eb385bacd90023d1

makeMoveWrapper(lvalue)

Summary: Because why not?

Test Plan: new unit tests

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum, marccelani

FB internal diff: D1866123

Signature: t1:1866123:1425331395:742369597757456e5925af55c5d4198b607126f1

MacOS doesn't have malloc_usable_size()

Summary:
Use malloc_size() instead

Test Plan: HHVM build on MacOS

Reviewed By: joelm@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1875364

Signature: t1:1875364:1425061479:7d8cec757c186fa8c30b6500fca6b2fe4c1ed137

Creating a flatten() method to get the inner Future<T> value out of a Future<Future<T>>.

Summary: Method will only be applicable for types of the form Future<Future<T>>, calling flatten() on a Future<T> where <T> != Future<R> for any <R> will result with a compile time error.

Test Plan: Added tests in FutureTest.cpp.

Reviewed By: hans@fb.com

Subscribers: ldbrandy, trunkagent, folly-diffs@, jsedgwick, yfeldblum, icanadi

FB internal diff: D1843581

Tasks: 6166903

Signature: t1:1843581:1424399517:7146924db3c238b77dd309f1d5916e44a7e81968

Add const to the variable: needle

Summary:
Make it clear that the variable is const

Closes #139

Test Plan:
authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878361

Signature: t1:1878361:1425071448:6871577ddb34f4a8dc30ea37842f9950c3825dde

Update UnitTest of MapUtilTest.cpp

Summary:
1. Split the original simple test
2. Add unit test of function: get_ref_default

Closes #138

Test Plan: authored tested, contbuild

Reviewed By: markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1878603

Signature: t1:1878603:1425073330:914a2365fa648142d11b41b387423d78e52f2e08

Fix comment of Benchmark.h

Summary:
Remove one of the "one" in the comment of BENCHMARK

Closes: #140
Test Plan: Inspection.

Reviewed By: markisaa@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1877997

Signature: t1:1877997:1425063475:c53c09d105809de9a9070f464fb0571a4dd1e5d1

attempt at putting thrift2 on ChannelPipeline

Summary:
Basically a ripoff of Dave's D1483148 but with ChannelPipeline instead of rx.
There's going to be a long tail of unit tests here, this is still pretty broken...

Not convinced that this is the way to go but I wanted to get a feel for what it would be like to plug it into an IRL system.
I think the simplicity/clarity of the handler implementations and to some extent the pipeline spec is an improvement over the rx way
but I'm less sure about everything else.

Test Plan: compiles and kind of works some of the time but not really

Reviewed By: alandau@fb.com

Subscribers: wormhole-diffs@, mcduff, hitesh, doug, yfeldblum, folly-diffs@, jsedgwick, subodh, andrewcox, njormrod, trunkagent, fugalh, alandau, bmatheny, mshneer

FB internal diff: D1618704

Tasks: 5981284

Signature: t1:1618704:1424980854:83c6193b1156be2ec65cf79f9f2ad1af670da7f8

Add willEqual to C++ Futures Code

Summary: Added willEqual function and wrote unit tests for it.

Test Plan: fbconfig --clang folly/futures && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: hannesr, trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1859840

Tasks: 6166911

Signature: t1:1859840:1424967149:865ee96ab4d3f5dbf17eb371b2ac3ccb5066ac87

support IndexedMemPool for types without default constructor

Summary:
This diff gives IndexedMemPool<T> emplace-like semantics when T
is not trivial.

Test Plan:
1. new unit tests
2. LifoSem benchmark

Reviewed By: march@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1874941

Signature: t1:1874941:1424987308:61bbe7b7e5e6df625a6208cd873c65e523a79fa0

Copyright 2014->2015

Summary:
Update copyright notices in folly to 2015

find folly/ -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'find . -type f | xargs sed -i 's/Copyright 2014 Facebook, Inc./Copyright 2015 Facebook, Inc./g'

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: ldbrandy, shilin, sdwilsh, fugalh, njormrod, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1869183

Signature: t1:1869183:1424823514:a2dd33741a86bf099d5c9e3d8702e6cca6dd672f

EventBase::runAfterDelay to throw an exception

Summary:
EventBase::runAfterDelay to throw an exception in case it fails to
schedule a proper execution of the given callback
(instead of silently returning false)

it appears to be the right thing to do, since:
@davejwatson: Digging through the layers of code, it appears this can only fail if
epoll_ctl() with EPOLL_CTL_ADD fails. Ignoring libevent bugs, it looks
like the only relevant errors could be ENOMEM or ENOSPC. So
nonrecoverable

Test Plan: Unit tests

Reviewed By: anca@fb.com

Subscribers: yzhan, haijunz, simpkins, net-systems@, varunk, zeus-diffs@, nli, dfechete, fugalh, atlas2-eng@, alandau, bmatheny, everstore-dev@, zhuohuang, wormhole-diffs@, anca, mwa, jgehring, oujin, alikhtarov, fuegen, mshneer, wch, bil, sanketh, zippydb, maxwellsayles, jsedgwick, trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, andrewcox, unicorn-diffs@, tw-eng@, xie, kennyyu, yfeldblum, folly-diffs@, davejwatson

FB internal diff: D1805125

Signature: t1:1805125:1424927912:8bebb4c3b9f1fa189c0ce97b12cdb8f95dba97ae

Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'

Summary:
Updating Folly Formatting's use of separators for Decimal Integers 'd' and Numbers 'n'.
Updated Decimal Integers 'd' use of separators to directly use commas ',' and grouping of 3 digits.
Updated Numbers use of separators and grouping to follow locale settings.

Test Plan:
Wrote unit tests to validate the insertion of thousandsSeparators works.
Note that the numbers unit test does not modify the locale since the test cases are not thread safe.

Reviewed By: lesha@fb.com

Subscribers: tudorb, trunkagent, folly-diffs@, yfeldblum, aric

FB internal diff: D1821156

Tasks: 6087521

Signature: t1:1821156:1424923837:d0fb383a07fd733375b72b1905e6112afa141265

Reduce AsyncSocket code size

Summary:
Manual inspection of AsyncSocket.o shows the constructors are generating ugly
(and large) code. Call the smaller constructors instead of duplicating the code. This
reduces the debug .o size from ~1.9MB to ~1.6MB.

Test Plan: existing tests

Reviewed By: benyluo@fb.com

Subscribers: mzlee, folly-diffs@, yfeldblum, mathieubaudet, subodh, pgriess, benyluo

FB internal diff: D1870896

Signature: t1:1870896:1424885638:d37fc79c0f88d04109c8bb73e632dc506b9f773b

Future::ensure

Summary: Unconditionally execute the action, passthrough semantics.

Test Plan: new unit test

Reviewed By: bmatheny@fb.com

Subscribers: trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1868837

Tasks: 6166860

Signature: t1:1868837:1424820181:0e83f54b59d7091dac60ab65feb387992e8ae89c

folly/FileUtil.h: fix compiler warning signed/unsigned comparison

Summary:
writeFull() returns ssize_t and without proper casting, comparing
it with data.size() triggers a compiler warning (which is
treated as an error) in the gcc-4.9-glibc-2.20 toolchain.

Test Plan: fbmake runtests

Reviewed By: mhx@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1870710

Signature: t1:1870710:1424874592:f51026c35196d763ad4b192d43c8ccee0255b41d

Improvement fbvector.computePushBackCapacity

Summary:
See the github pull request at https://github.com/facebook/folly/issues/135

Important optimizations: remove one of the branches and change ##empty()## to ##capacity()==0##

Test Plan:
Ran extended fbvector test suite (re-enabled in experimental/njormrod)

Also

fbconfig -r folly && fbmake runtests

Reviewed By: markisaa@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1869112

Tasks: 6338531

Signature: t1:1869112:1424823901:d2d7331aef82edad1e8c159005cc1c7185550d0c

Removing call to google::setThreadName from FunctionScheduler

Summary: Funnily, google::setThreadName is Facebook specific: it's introduced only in the FB build of glog. This diff removes the call because folly should not depend on FB's glog.

Test Plan: made sure the line was deleted

Reviewed By: ldbrandy@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum

FB internal diff: D1868249

Tasks: 6345935

Signature: t1:1868249:1424812595:2a084754d614ab9b7b6168454f33638882e3f6db

EmulatedFutexAtomic doesn't copy or move

Summary:
EmulatedFutexAtomic concisely says here that it doesn't copy construct,
move construct, copy assign, or move assign.

Closes #130

Test Plan:
Inspection and all unit tests.

```
Summary (total time 66.97s):
PASS: 2029
FAIL: 0
SKIP: 0
FATAL: 0
TIMEOUT: 0
```

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1868033

Signature: t1:1868033:1424808823:d9dd618981b29b5949c00190d69d670d96e60e07

add preprocessor to hide unused functions

Summary:
encodeVarintToIOBuf and decodeVarintFromCursor are not used when
FOLLY_HAVE_LIBLZMA and FOLLY_HAVE_LIBLZ4 are false, which will trigger
-Werror,-Wunused-function on compile on mobile.

Test Plan: unit test

Reviewed By: seanc@fb.com

Subscribers: meisner, trunkagent, seanc, folly-diffs@, yfeldblum

FB internal diff: D1864016

Signature: t1:1864016:1424702813:75535ea92ac16385ce5c272d93bba507ccbb67a1

Fix folly::Singleton DFATAL

Summary:
Previously leak-fatal could never happen because we were incorrectly checking singleton state.
Sending this diff to see how many tests will actually fail and potentially fix worst offenders.

Test Plan: unit tests

Reviewed By: stepan@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1838405

Signature: t1:1838405:1424478983:94cda86ed57f38b0cf626b74804fbc168d182c66

Optimize perf of EventBase with new option.

Summary:
Add option to disable time measurement in EventBase::loopBody()
to improve performances when time measurement is not used.

Test Plan:
Unit tests, benchmarking and manual tests.

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1846020

Tasks: 6086197

Signature: t1:1846020:1424459703:9686a84027e1137ee43eb0458b0baa4fee3a8abf

Add AutoTimer to folly

Summary:
This 'moves' bistro::AutoTimer into folly. I actually changed it a bit, so I'm
not migrating any existing uses in this diff, but I can do that as a followup.
We're using it in various places now, so it doesn't make sense to have it in
bistro.

Basically this class lets you conveniently time a series of things while
printing log messages, e.g.:
{
AutoTimer t("Done");
processA();
t.log("processA() complete");
}

would print
"processA() complete in XXX seconds"
"Done in YYY seconds"

I changed some things from bistro::AutoTimer:
- Use std::chrono::time_point<Clock> to store the start time.
- I removed the gflags. It seems weird to rely on those to control the timer,
since they would affect all uses of this. If a specific user of the timer
wants to actually control timing at runtime, I think it's better to use a
custom logger.
- I removed the 'min log duration'. As implemented it actually made the
constructor conflict: AutoTimer("foo", 1.2) is ambiguous. Also, it should
probably be set on the logger, so this likewise feels like a place where you
could make a custom logger.

I'm not sure that the way I use the default logger as a template argument is
best, so please suggest improvements to that.

Test Plan: unit test

Reviewed By: lesha@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1860794

Signature: t1:1860794:1424458219:a2dd0d428aedf063a888b2937b1443c7142554e8

(folly/futures) Fix get() bug

Summary: I thought this was a race, but I think now it was something to do with using a value that had been moved out or something. Anyway, this refactor is cleaner and consolidates a few methods so it's all kinds of fuzzy feelings.

Test Plan: unit tests

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, jsedgwick

FB internal diff: D1861376

Tasks: 6298004

Signature: t1:1861376:1424465861:736353ab3174656fec9e036e0ebd964970da38d0

revert D1471958. ParseError is not intended to be caught

Summary:
a couple of files were catching this exception and included the .cpp file :O. So we moved
to .h before realizing this was not intended to be caught. reverting the move to .h. codemod will follow.

Test Plan: fbmake -r folly && fbmake && fbmake runtests

Reviewed By: delong.j@fb.com

Subscribers: trunkagent, doug, seanc, folly-diffs@, yfeldblum

FB internal diff: D1854301

Tasks: 6245781

Signature: t1:1854301:1424375745:7e226196315fac99835752896cc5e549e91df2da

(futures) Make executors sticky

Summary:
Instead of returning a deactivated future, have `via` just set the executor. Propagate the executor from `then`. This fixes the `via().get()` problem, and has semantics similar to before for `via().then().then()`.

However, the semantics are now slightly different - each `then` goes back through the executor. This adds some overhead and tweaks the semantics (e.g. if the executor is a threadpool it might execute subsequent `then`s in another thread). However, with `futures::chain` recently introduced, and any other convenience methods that you can dream up and make a case for, we can reasonably get the old once-through-the-executor behavior when performance or other concerns demand it. e.g. `via().then(futures::chain(a, b, c))`.

Test Plan: unit tests

Reviewed By: hannesr@fb.com

Subscribers: zeus-diffs@, mmandal, steveo, rituraj, trunkagent, exa, folly-diffs@, yfeldblum, jsedgwick, davejwatson

FB internal diff: D1839691

Tasks: 6048744

Signature: t1:1839691:1424397180:ca0b0ea7b3867769ab8abd254a510059df67011e

Revert: (Wangle) Clean up move constructors

Summary:
This reverts commit 1e407b48d379a41f32e7a980285dbdf4dadb2e33.

Test Plan: revert-hammer

Reviewed By: yitingli@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1858994

Signature: t1:1858994:1424386588:5c4608ecbe1f9ab1108ac12e506b157b54d0078b

(Wangle) Clean up move constructors

Summary:
These were doing more assignments than necessary.
1) We don't need to set `core_` to `nullptr` and then immediately
overwrite it with `other.core_`.
2) Don't need to swap the booleans (this was actually an unitialized
memory access).

Test Plan: Ran all the tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1852386

Signature: t1:1852386:1424375154:25997209e76ccd16169125597e868932a5143ffb

Remove @/folly dependency from folly/io:compression

Summary: This diff should let us include folly/io:compression into iOS and Android projects without pulling in a lot of dependencies and should allow the use of zlib.

Test Plan: fbconfig -r folly/io/tests && fbmake runtests

Reviewed By: seanc@fb.com

Subscribers: trunkagent, bmatheny, folly-diffs@, yfeldblum, seanc, benyluo

FB internal diff: D1853058

Tasks: 6245912

Signature: t1:1853058:1424370881:2ea243d93e6041502e5e356fa430c2483f652b40

use ServerBootstrap

Summary:
Use server bootstrap.

* code reuse
* Allows multiple accept threads easily.

Accept now doesn't happen in the server's eventBase, but a new thread.

I think I hit all the corner cases involving existingSocket, duplex, stopListening(), etc.

There are a lot of minor changes here to support all the cases, let me know if I should split anything to separate diffs

Test Plan: fbconfig -r thrift/lib/cpp2; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: haijunz, yfeldblum, ruibalp, fbcode-common-diffs@, davejwatson, jsedgwick, trunkagent, doug, alandau, bmatheny, mshneer, folly-diffs@, mcduff, andrewcox, fugalh

FB internal diff: D1732895

Tasks: 5788102

Signature: t1:1732895:1423087631:379bbb131c35ce2221801bc7cec226f87ba0b1d9

some moar unittests

Summary: from discussion

Test Plan: unit tests

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1829282

Signature: t1:1829282:1423180907:3630dac1378750b05f316c672fbbd71138d2bc0a

Use folly's async udp socket

Summary: They are functionaly equivalent, no need to have more than one

Test Plan: fbconfig -r realtime/voip; fbmake runtests

Reviewed By: naizhi@fb.com

Subscribers: trunkagent, doug, ps, bmatheny, folly-diffs@, yfeldblum

FB internal diff: D1828044

Tasks: 6154007

Signature: t1:1828044:1423165354:f71d2fd28ca76a8f67a597c747f8578d2909823c

Kill get_fast/get_weak_fast Singletonn API

Summary: After D1827390 regular get and get_weak is comparable to Meyers and static singletons, so there's no need to keey _fast APIs.

Test Plan: benchmark && fbmake

Reviewed By: mshneer@fb.com

Subscribers: trunkagent, configerator-diffs@, fbcode-common-diffs@, mcduff, hitesh, mshneer, fugalh, acampi, alikhtarov, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1843219

Signature: t1:1843219:1424216566:f2f182a1c86bb5f0fb1f978d8c6b7a4388198f5f

Migrate FunctionScheduler from common/concurrency/ to folly/experimental/

Summary: This are the open-source-related additions. A separate diff will replace the current implementation in common/concurrency/ to use this one.

Test Plan: fbmake runtests

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, net-systems@, folly-diffs@, yfeldblum

FB internal diff: D1845525

Signature: t1:1845525:1424207291:d30e3c5e85222527c2aff39c1250aa1e41b9a2cf

Reformat comment for readability, 80-col rule, and gcc cleanliness

Summary:
The continuation at the end of the line with the while
was causing gcc to complain about it because it's a
single line comment.

Reformat it into a multiline comment and wrap it to 80 chars.

Test Plan: It's just a comment...

Reviewed By: paulbiss@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1850876

Signature: t1:1850876:1424124973:b2afbc2a8177ac6d11a21bf5a52093bcfd9cbb3e

(Wangle) Shrink Core to 128 bytes

Summary: I think this makes sense, especially if `interrupt_` is really rare.

Test Plan:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             101.25ns    9.88M
promiseAndFuture                                  92.86%   109.03ns    9.17M
withThen                                          28.96%   349.65ns    2.86M
----------------------------------------------------------------------------
oneThen                                                    337.33ns    2.96M
twoThens                                          58.46%   577.06ns    1.73M
fourThens                                         32.42%     1.04us  961.23K
hundredThens                                       1.45%    23.24us   43.04K
----------------------------------------------------------------------------
no_contention                                                4.45ms   224.48
contention                                        72.89%     6.11ms   163.61
----------------------------------------------------------------------------
throwAndCatch                                                6.34us  157.77K
throwAndCatchWrapped                              95.65%     6.63us  150.91K
throwWrappedAndCatch                             142.58%     4.45us  224.95K
throwWrappedAndCatchWrapped                      488.86%     1.30us  771.27K
----------------------------------------------------------------------------
throwAndCatchContended                                        1.01s  987.01m
throwAndCatchWrappedContended                    100.03%      1.01s  987.27m
throwWrappedAndCatchContended                    148.37%   682.86ms     1.46
throwWrappedAndCatchWrappedContended            18765.19%     5.40ms   185.21
============================================================================

Compare to D1843129#31 (seems to be an improvement?)

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1847843

Signature: t1:1847843:1423855151:3602d49df245eda3f85325ab5efd63514213ca44

ExceptionWrapperTest.cpp vs clang's new warning about side effects in typeid argument list

Summary:
Disable the -Wunevaluated-expression warning for this test:

EXPECT_EQ(typeid(ie), typeid(IntException));

Otherwise, clang warns about the unevaluated expression because
the expansion of EXPECT_EQ applies sizeof to an expression
with side effects:

folly/test/ExceptionWrapperTest.cpp:179:122: error: expression with side effects has no effect in an unevaluated context [-Werror,-Wunevaluated-expression]
switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal:: EqHelper<(sizeof(::testing::internal::IsNullLiteralHelper(typeid(ie))) == 1)>::Compare("typeid(ie)", "typeid(IntException)", typeid(ie), typeid(IntException)))) ; else ::testing::internal::AssertHelper(::testing::TestPartResult::kNonFatalFailure, "folly/test/ExceptionWrapperTest.cpp", 179, gtest_ar.failure_message()) = ::testing::Message();

Test Plan:
Ensure this compiles with clang-3.4, 3.5 and clang:dev.
I.e., ensure that this prints PASS at the end.

for i in '' 3.5 dev; do
test -n "$i" && i=--with-project-version=clang:$i
fbconfig -r --clang $i folly/test:exception_wrapper_test && fbmake dbgo
done && echo PASS

Reviewed By: mpawlowski@fb.com

Subscribers: mathieubaudet, folly-diffs@, yfeldblum

FB internal diff: D1850778

Tasks: 6244745

Signature: t1:1850778:1424111029:136478e9a3cc3a219047547d501de4c579a1a181

wangle: make new pragma use portable to older clang

Summary:
* folly/wangle/channel/test/MockChannelHandler.h:
Avoiding a new warning that rendered this file uncompilable
by clang-3.5 and older.  Add cpp conditionals to protect
against that.

Test Plan:
Ensure it works with clang-3.4, 3.5 and clang:dev.
I.e., ensure that this prints PASS at the end.

for i in '' 3.5 dev; do
test -n "$i" && i=--with-project-version=clang:$i
fbconfig -r --clang $i folly/wangle/channel/test && fbmake runtests_dbgo
done && echo PASS

Reviewed By: meyering@fb.com

Subscribers: mpawlowski, fugalh, mathieubaudet, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1849474

Tasks: 6244745

Blame Revision: D1848330
@build-break

folly: OptionalTest.cpp: suppress a -Wself-move warning

Summary:
Clang's new -Wself-move warning dings this code:

b = std::move(b);

But in this case, that is a deliberate test of
this capability, so we suppress that particular
warning for the duration of this small test function.

Test Plan:
Run this and note that it now compiles: (it passes the tests, too)
fbconfig --clang --with-project-version=clang:dev -r folly/test:optional_test && fbmake dbgo

Reviewed By: watashi@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1849689

Tasks: 6244745

Signature: t1:1849689:1423877932:4f479d682bd8ba97dcabed7fc29ce0a22c013969

@build-break (using this to work around bogus push failure: https://phabricator.fb.com/P19730606)

folly/ExceptionWrapper.h: avoid warning about typeid operand side effects

Summary:
* folly/ExceptionWrapper.h (class_name): Clang is not yet smart enough to
see that there is no harm in dereferencing an "item_" that we've just
verified is non-NULL.  Accommodate it, to avoid this:
./folly/ExceptionWrapper.h:199:30: error: expression with side effects will be evaluated despite being used as an operand to 'typeid' [-Werror,-Wpotentially-evaluated-expression]
return demangle(typeid(*item_));
^

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: mhorowitz@fb.com

Subscribers: trunkagent, yfeldblum, folly-diffs@

FB internal diff: D1848327

Tasks: 6244745

Signature: t1:1848327:1423869597:a58c9e9e3671befb78ae07fbd274a13d08ffb2a7

Blame Revision:

clang:dev rejects vector<atomic<T>> in EventBaseTest.cpp.

Summary: [Folly] clang:dev rejects vector<atomic<T>> in EventBaseTest.cpp.

Test Plan: Build and run `folly/io/async/test/EventBaseTest.cpp` using the current `clang`; try building with `clang:dev` and verifying that at least the related failures are gone.

Reviewed By: meyering@fb.com

Subscribers: mathieubaudet, folly-diffs@, yfeldblum, dougw, brettp

FB internal diff: D1848749

Tasks: 6244720

Signature: t1:1848749:1423866420:7b7354d4568b6a6d6a824236ae8d271f6855d90b

folly: move side effects out of EXPECT_EQ args

Summary:
This avoids warnings about e.g., sizeof(k++) that result
from macro expansion.
* folly/futures/test/ExecutorTest.cpp: Move increment out
of macro argument list.
* folly/test/FBVectorTestBenchmarks.cpp.h: Likewise.
* folly/test/LazyTest.cpp: Likewise.
Otherwise, we'd get errors like this:

folly/test/LazyTest.cpp:49:118: error: expression with side effects has no effect in an unevaluated context [-Werror,-Wunevaluated-expression]
switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal:: EqHelper<(sizeof(::testing::internal::IsNullLiteralHelper(++globalCount())) == 1)>::Compare("++globalCount()", "1", ++globalCount(), 1))) ; else ::testing::internal::AssertHelper(::testing::TestPartResult::kNonFatalFailure, "folly/test/LazyTest.cpp", 49, gtest_ar.failure_message()) = ::testing::Message();
^

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: njormrod@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1848324

Tasks: 6244745

Signature: t1:1848324:1423860890:bce44c5e0895804a21957893ae6b78e76dfbc4d3

folly: avoid new warnings from -Winconsistent-missing-override

Summary:
Upgrading to clang:dev (clang >3.6) brought in some new warnings.
This change address all of the issues exposed by the new
-Winconsistent-missing-override, usually by simply adding the
missing "override" keyword.  However, in
folly/wangle/channel/test/MockChannelHandler.h, I chose
to ignore those warnings for the mocked functions.
* folly/futures/test/ViaTest.cpp: Add missing "override"(s).
* folly/io/async/AsyncSSLSocket.h: Likewise.
* folly/io/async/AsyncSocket.h: Likewise.
* folly/io/async/EventBase.h: Likewise.
* folly/test/ExceptionWrapperTest.cpp: Likewise.
* folly/wangle/channel/AsyncSocketHandler.h: Likewise.
* folly/wangle/channel/test/MockChannelHandler.h: Ignore
the new warning for these functions.

Test Plan:
Run these commands and note there are fewer errors than before:
fbconfig --clang --with-project-version=clang:dev -r folly && fbmake dbgo

Reviewed By: hans@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1848330

Tasks: 6244745

Signature: t1:1848330:1423858909:b167ca220d6c9fe036d3adca08cf3053a7a9de16

(Wangle) Have Core own an FSM instead of inheriting

Summary:
This seems cleaner. I also moved a Core member around and changed state to be a smaller size to save us 16 bytes. Might be neat to try to get this down to 128?

Fixed compilation with GCC 4.9, too.

Test Plan: Ran all the tests. Will also benchmark.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1843129

Signature: t1:1843129:1423789502:60e56d1af553994991195fcc350d37d644a556ca

Be able to access proxy client IP (including Lua)

Summary:
Pass TransportInfo object through connection setup.  This allows us to
pass along additional info from the setup if necessary.

Test Plan:
Run proxygen on my devserver listening on ipv4 and ipv6 for ports 80 and 443.
Configure it to run a lua request rule with the following lines:

io.stderr:write("jonlog: proxy " .. conn:getClientAddrOriginal() .. "\n")
io.stderr:write("jonlog: client " .. request:getClientIPStr() .. "\n")

Run haproxy also on my devserver, listening on ports 8555-8558, with each
forwarding to one of the four pairs (ipv4-localhost/ipv6-localhost, 80/443)

From my laptop, run curls against each of these four endpoints and make
sure that we print out "127.0.0.1" as the proxy IP, and my laptop's IP
as the client IP.

Reviewed By: cgheorghe@fb.com

Subscribers: jsedgwick, yfeldblum, trunkagent, ruibalp, bmatheny, folly-diffs@, dsp, nbm

FB internal diff: D1746590

Tasks: 5688127

Signature: t1:1746590:1423695880:f647964d95636a69a00304e144aef71ee0213d28

Fix EliasFanoReader position() when past-the-end

Summary:
`EliasFanoReader::position()` used to return `size() - 1` both when
the reader is positioned on the last element, and after `next()` is
called after that (and it return `false`). Now in the latter case
`position()` returns `size()` (consistently with the usual behaviour
of past-the-end iterators).

Also fix the return type of `jumpTo`.

Test Plan:
fbconfig folly/experimental/test:eliasfano_test && fbmake runtests_opt

Reviewed By: philipp@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum

FB internal diff: D1846275

Signature: t1:1846275:1423790264:151f5d2e1e09d4e24dfb758473dfb9cd52c070bd

std::atomicize MicroSpinLock.

Summary:
A colleague at another company started making fun of
MicroSpinLock for its x86 assembly and ad hoc compiler memory
barriers. Use C++11 (which wasn't really a thing at the time I wrote
this).

Test Plan: folly's runtests. What else would we like?

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: folly-diffs@, yfeldblum

FB internal diff: D1841563

Signature: t1:1841563:1423780458:a447c081fbd72e3420b23e95dcf26575c9a06798

Elias-Fano micro-optimizations

Summary:
Short skips have been optimized by adding special cases that use simple iteration when it is convenient. Large skips have been optimized by using the broadword selection algorithm by Vigna (improved with ideas by Gog&Petri) instead of iterating on the zeros/ones of the upper bits.

The benchmarks had to be made more granular to measure the differences, in particular they used to test skipping with a fixed skip length for each test, while now we average over a range of skips to better simulate a random distribution.

The improvements are very significant for `skipTo()` on short skips, about 2-3x for skips at distance 1 or 2, which can occur when intersecting dense lists. On large skips the gain is about 17%.

`skip()` exhibits slightly smaller improvements.

before               after
============================================================================  ==================
folly/experimental/test/EliasFanoCodingTest.cpp relative  time/iter  iters/s  time/iter  iters/s
============================================================================  ==================
Next                                                         2.52ns  396.26M     2.52ns  397.28M
Skip_ForwardQ128(1)                                          8.66ns  115.52M     3.92ns  255.28M
Skip_ForwardQ128(2)                                          8.37ns  119.42M     5.08ns  197.04M
Skip_ForwardQ128(4_pm_1)                                     9.67ns  103.41M     7.04ns  142.02M
Skip_ForwardQ128(16_pm_4)                                   21.44ns   46.65M    19.68ns   50.82M
Skip_ForwardQ128(64_pm_16)                                  30.86ns   32.40M    27.58ns   36.26M
Skip_ForwardQ128(256_pm_64)                                 37.80ns   26.45M    32.49ns   30.78M
Skip_ForwardQ128(1024_pm_256)                               38.99ns   25.65M    33.39ns   29.95M
Jump_ForwardQ128                                            37.91ns   26.37M    34.05ns   29.37M
----------------------------------------------------------------------------  ------------------
SkipTo_SkipQ128(1)                                          13.87ns   72.10M     4.42ns  226.49M
SkipTo_SkipQ128(2)                                          18.80ns   53.20M     8.58ns  116.55M
SkipTo_SkipQ128(4_pm_1)                                     23.59ns   42.38M    11.43ns   87.50M
SkipTo_SkipQ128(16_pm_4)                                    36.04ns   27.74M    31.19ns   32.06M
SkipTo_SkipQ128(64_pm_16)                                   53.34ns   18.75M    43.88ns   22.79M
SkipTo_SkipQ128(256_pm_64)                                  62.27ns   16.06M    49.08ns   20.37M
SkipTo_SkipQ128(1024_pm_256)                                65.63ns   15.24M    52.24ns   19.14M
JumpTo_SkipQ128                                             65.89ns   15.18M    54.61ns   18.31M
----------------------------------------------------------------------------  ------------------
Encode_10                                                  111.94ns    8.93M   117.24ns    8.53M
Encode                                                       5.35ms   187.02     5.64ms   177.15
----------------------------------------------------------------------------  ------------------
Select64                                                     8.07ns  123.96M     8.04ns  124.35M
============================================================================  ==================

Test Plan:
fbconfig folly/experimental/test:eliasfano_test && fbmake runtests_opt

Reviewed By: philipp@fb.com

Subscribers: yfeldblum, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, unicorn-diffs@, trunkagent, folly-diffs@

FB internal diff: D1793554

Signature: t1:1793554:1423619344:1b078c0789639f317342ebcc77b11fe91859cd7b

(Wangle) Catch all exceptions in `then() -> Future`

Summary: Saw this while debugging some stuff. Just an oversight, right?

Test Plan: Ran all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum

FB internal diff: D1843016

Signature: t1:1843016:1423704557:d9fdb4b879ab4fcfda54a5fcb16a639df15872b5