Futex: ignore errors on wake

Summary: See the comment for details

Reviewed By: @nbronson

Differential Revision: D2181231

rearrange Pipeline to have more functionality in PipelineBase

Summary: This way, handlers can carry out more complex manipulations of the pipeline via ctx->getPipeline() without knowing the R/W types

Reviewed By: @djwatson

Differential Revision: D2158736

Fix for clang (llvm 3.7.0+)

Summary: Roughly described in https://github.com/facebook/hhvm/issues/5344.

Closes #206

Reviewed By: @yfeldblum

Differential Revision: D2170803

Pulled By: @sgolemon

Remove -Wno-inconsistent-missing-override suppression in folly

Summary: It is fixed in gmock. No need to suppress it in sources.

Reviewed By: @meyering

Differential Revision: D2181564

instance Eq Unit

Summary: [Folly] instance Eq Unit

    Unit::operator==(const Unit&)
    Unit::operator!=(const Unit&)

Reviewed By: @Gownta

Differential Revision: D2179384

Add missing override and remove redundant virtual in folly

Summary: Running clang-tidy to fix this problem, using script D2171334.

I noticed that even without this change clang dev build succeeds for folly and opened a separate task to investigate why.

Reviewed By: @markisaa

Differential Revision: D2179299

Add ./configure check for vsnprintf returning negative on error

Summary: This is slightly more correct than assuming that
__APPLE__ is the only place this assumption breaks.

Pulled from relevant section of folly-PR#95, other sections
are either already applied by other fixes, or have been code-moved.

Closes #95

Reviewed By: @jwatzman

Differential Revision: D2178439

Building instructions for MAC OS X (MacPorts)

Summary: Tested on Mac OS X 10.10.3 and MacPorts 2.3.3.
Closes #227

Reviewed By: @yfeldblum

Differential Revision: D2176700

Pulled By: @sgolemon

Bump version to 47:0

new small_vector tests that fail on trunk and uncover a bug in emplace_back on references to memory inside the vector + a fix + perf improvement to const lvalue push_back

Summary: emplace_back() on a small_vector applied on data inside the vector doesn't work properly.
In standard vectors, this usage is required to work properly, but I'm not sure whether it should in small_vector. Consider fixing / adding a lint rule.

Reviewed By: @yfeldblum

Differential Revision: D2122931

ARM64 assembler fixes for Folly.

Summary: Wrap asm("pause") in an inline so that it becomes
asm("wfe") on aarch64.

Closes #187
Closes #190

Reviewed By: @yfeldblum

Differential Revision: D2152868

Pulled By: @sgolemon

then() ropagates exceptions properly

Summary: fixed then() exceptions propagation and added unit-test

Reviewed By: @fugalh

Differential Revision: D2159075

Move folly::compression::Instructions to a separate file

Summary: Moved abstraction layer for platform-specific bit-manipulation instructions from the EliasFanoCoding.h to Instructions.h since we will need to use it also for BitVectors:

Reviewed By: @ot

Differential Revision: D2165314

move SharedMutex from folly/experimental to folly

Summary: SharedMutex has been in heavy production use for a while with no
bugs reported and no API changes requested, it is no longer experimental.

Reviewed By: @yfeldblum

Differential Revision: D2165275

Check array get for < 0

Summary: dynamic's integer type is signed, so make sure array indices
are not negative.

(See https://our.intern.facebook.com/intern/tasks/?t=7445055)

Reviewed By: @Gownta

Differential Revision: D2145689

use Unit in Pipeline

Summary: Instead of the one-off Nothing struct

Reviewed By: @yfeldblum

Differential Revision: D2158621

various perf improvements

Summary: Three strategies
1. Optimistic locking
2. Acquire-release memory ordering instead of full sequential consistency
3. Some low-hanging branch miss optimizations

Please review carefully; the dogscience is strong with this one

```
Before:

============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             127.99ns    7.81M
promiseAndFuture                                  94.89%   134.89ns    7.41M
withThen                                          28.40%   450.63ns    2.22M
----------------------------------------------------------------------------
oneThen                                                    446.68ns    2.24M
twoThens                                          58.35%   765.55ns    1.31M
fourThens                                         31.87%     1.40us  713.41K
hundredThens                                       1.61%    27.78us   35.99K
----------------------------------------------------------------------------
no_contention                                                4.63ms   216.00
contention                                        80.79%     5.73ms   174.52
----------------------------------------------------------------------------
throwAndCatch                                               10.91us   91.64K
throwAndCatchWrapped                             127.14%     8.58us  116.51K
throwWrappedAndCatch                             178.22%     6.12us  163.32K
throwWrappedAndCatchWrapped                      793.75%     1.37us  727.38K
----------------------------------------------------------------------------
throwAndCatchContended                                        1.35s  741.33m
throwAndCatchWrappedContended                    139.18%   969.23ms     1.03
throwWrappedAndCatchContended                    169.51%   795.76ms     1.26
throwWrappedAndCatchWrappedContended            17742.23%     7.60ms   131.53
----------------------------------------------------------------------------
complexUnit                                                127.50us    7.84K
complexBlob4                                     100.14%   127.32us    7.85K
complexBlob8                                     100.16%   127.30us    7.86K
complexBlob64                                     96.45%   132.19us    7.57K
complexBlob128                                    92.83%   137.35us    7.28K
complexBlob256                                    87.79%   145.23us    6.89K
complexBlob512                                    81.64%   156.18us    6.40K
complexBlob1024                                   72.54%   175.76us    5.69K
complexBlob2048                                   58.52%   217.89us    4.59K
complexBlob4096                                   32.54%   391.78us    2.55K
============================================================================

After:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                              85.28ns   11.73M
promiseAndFuture                                  88.63%    96.22ns   10.39M
withThen                                          30.46%   279.99ns    3.57M
----------------------------------------------------------------------------
oneThen                                                    231.18ns    4.33M
twoThens                                          60.57%   381.70ns    2.62M
fourThens                                         33.52%   689.71ns    1.45M
hundredThens                                       1.49%    15.48us   64.58K
----------------------------------------------------------------------------
no_contention                                                3.84ms   260.19
contention                                        88.29%     4.35ms   229.73
----------------------------------------------------------------------------
throwAndCatch                                               10.63us   94.06K
throwAndCatchWrapped                             127.17%     8.36us  119.61K
throwWrappedAndCatch                             179.83%     5.91us  169.15K
throwWrappedAndCatchWrapped                     1014.48%     1.05us  954.19K
----------------------------------------------------------------------------
throwAndCatchContended                                        1.34s  749.03m
throwAndCatchWrappedContended                    140.66%   949.16ms     1.05
throwWrappedAndCatchContended                    164.87%   809.77ms     1.23
throwWrappedAndCatchWrappedContended            49406.39%     2.70ms   370.07
----------------------------------------------------------------------------
complexUnit                                                 86.83us   11.52K
complexBlob4                                      97.42%    89.12us   11.22K
complexBlob8                                      96.63%    89.85us   11.13K
complexBlob64                                     92.53%    93.84us   10.66K
complexBlob128                                    90.85%    95.57us   10.46K
complexBlob256                                    82.56%   105.17us    9.51K
complexBlob512                                    74.13%   117.12us    8.54K
complexBlob1024                                   63.67%   136.37us    7.33K
complexBlob2048                                   50.25%   172.79us    5.79K
complexBlob4096                                   26.63%   326.05us    3.07K
============================================================================
```

Reviewed By: @djwatson

Differential Revision: D2139822

Using emplace_back to avoid temporary

Summary: Directly pass the arguments to respective constructors.
Instead of first making temporary and then pass that.

Closes #218

Reviewed By: @yfeldblum

Differential Revision: D2156978

Pulled By: @sgolemon

Bump version to 46:0

Fix leaks in tests

Summary: Fix leaks in unit tests

Reviewed By: @djwatson

Differential Revision: D2153681

Implement previousValue on EliasFanoReader

Summary: It is often useful to retrieve the value preceding the current value
in an Elias-Fano iterator, for example when the list represents
adjacent ranges. This diff implements a new method `previousValue` in
`EliasFanoReader`.

It also adds a new `kUnchecked` boolean template argument to
`EliasFanoReader` which skips bounds checking.

Reviewed By: @philippv

Differential Revision: D2155049

pipeline handler removal, fix service test

Summary: add remove<T>, remove(Handler*), removeFront(), removeBack() to Pipeline
employ these to fix up reusing client pipelines with client dispatchers, which in turn fixes the broken ServiceTest

Reviewed By: @djwatson

Differential Revision: D2152636

Move AtomicUnorderedInsertMap to folly.

Summary: AtomicUnorderedInsertMap is a concurrent hash table that firmly
at the performance end of the generality <-> performance spectrum.
If you don't need updates (or can use your own concurrency control when
overwriting values), you never need to delete, and you can predict your
capacity perfectly, then you will get wait-free reads, lock-free inserts,
safe concurrent iteration, and excellent cache and performance outlier
behavior.  Arbitrary key and value types are supported.

Reviewed By: @yfeldblum

Differential Revision: D2145281

do not suppurt compress and expand

Summary: the openssl improvement does not support compression and decompression.

Reviewed By: @Invalid auth token., @afrind

Differential Revision: D2151003

Revert "Using emplace_back to avoid temporary"

Summary: This reverts commit 0968dcc2f0e02998fa53007853e0c0aad0a423b7.

Reverts D2151582

Reviewed By: @​iainb

Differential Revision: D2153840

Check readCallback before calling handleRead

Summary: Since readCallback_ could be uninstalled in any of callbacks,
we need to ensure that readCallback_ != nullptr before calling
handleRead.

Reviewed By: @djwatson

Differential Revision: D2140054

ExceptionWrapper: use std::decay in implicit constructor

Summary: Because Ex resolves to SomeException&, and then all the tests in optimize<>
fail.

Reviewed By: @yfeldblum

Differential Revision: D2152893

Merge pull request #225 from wb14123/dup

Remove duplicate header file
Somehow this was already correct in the internal repo, but not the external one.
Thanks!

Remove duplicate header file

Using emplace_back to avoid temporary

Summary: Directly pass the arguments to respective constructors.
Instead of first making temporary and then pass that.

Closes #218

Reviewed By: @JoelMarcey

Differential Revision: D2151582

Pulled By: @sgolemon

Get *=default*ed default constructors

Summary: Defaulted (*=default*ed) default constructors are better
because they can be constexpr and/or noexcept when C++ Standard
thinks it is right to do so. And they remain user-declared rather
than user-provided. Regarding *=default*ed default constructor,
benifit is that the work might be done during compilation and we
might not need to worry about exception paths. And for destructors,
apart from that that =defaulted definition is in sync with defaulted
default constructor we might discover that in some cases "() {}" might
be ill-formed when "= default;" compiltion indicates so. If =defaulted
definition for destructor doesn't do any harm then why not go for it.

Closes #216

Reviewed By: @yfeldblum

Differential Revision: D2145322

Pulled By: @sgolemon

Needless temporary gone

Summary: We might be doing:
1) Create a temporary
2) Copy/Move out of it
3) Destroy that temporary.
Which isn't needed in many places.
And copy/move elision doesn't work for a temporary
bound to a reference.

We can forward arguments, directly.
To get the work done three constructors were added.

Closes #222

Reviewed By: @JoelMarcey, @yfeldblum

Differential Revision: D2151731

Pulled By: @sgolemon

Added fiber supported mutex to folly

Summary: To be able to use fiber lock in thrift

Reviewed By: @brianwatling, @yfeldblum

Differential Revision: D2149936

Add EOS checking to 'parseLeadingNumber' function.

Summary: parseLeadingNumber does not contain EOS validation and raise an exception when '/sys/devices/system/cpu/cpu0/index0/shared_cpu_list' contains only one number like '0'.
Closes #207

Reviewed By: @yfeldblum

Differential Revision: D2148015

Pulled By: @sgolemon

apple-gate ThreadLocal in fibers code

Summary: Some internal malloc shims broke on using folly::ThreadLocal, so only do that for `__APPLE__`.

Reviewed By: @brianwatling

Differential Revision: D2151318

Suppress clang memcpy warnings

Summary: Clang warns when types with vtables are memcpy'd. If the type
has declared itself to be relocatable, then this is the desired
behavior. If the type has not declared itself to be relocatable, then
the memcpy codepath is dead. However, the dead codepath is still
instantiated (it's inside an if block with a static check, but c++
doesn't have static-if), so the compiler spits out a nasty warning
anyways.

Each memcpy reference inside of fbvector has been void-ified. I have
looked at all the codepaths leading to the memcpys, and see that they
have isRelocatable or isTriviallyCopyable checks.

Reviewed By: @markisaa

Differential Revision: D2148286

Temporarily disable performWriteIovec

Summary: As in title

Reviewed By: @djwatson

Differential Revision: D2151092

Recycle heap on assignment

Summary: For standard containers, assignment tries to reuse heap space.
Dynamic assignment does not - it calls destroy(), and then reconstructs
a new dynamic. In the case that the old and new types are the same (eg
assigning a dynamic-vector to a dynamic-vector) we can call through to
the underlying type's assignment operator.

Reviewed By: @jdelong

Differential Revision: D2148093

opt proxygen with new SSL_write_iovec function

Summary: this taks advantage of the SSL_write_iovec function to improve CPU.  According our testing, it saves 10+% CPU.

Reviewed By: @djwatson

Differential Revision: D2119223

fix for Boost >= 1.56

Summary: Folly still continue to build and work on earlier boost versions(boost < 1.56).

**boost::context::make_fcontext() change history:**
* __1.55:__    fcontext_t * make_fcontext( void * sp, std::size_t size, void(* fn)(intptr_t) );
* __1.56:__    fcontext_t make_fcontext(void* sp,std::size_t size,void(*fn)(intptr_t));

**boost doc:**
http://www.boost.org/doc/libs/1_55_0/libs/context/doc/html/context/context/boost_fcontext.html
http://www.boost.org/doc/libs/1_56_0/libs/context/doc/html/context/context/boost_fcontext.html
Closes #205
Closes #210

Reviewed By: @jwatzman

Differential Revision: D2148069

Pulled By: @sgolemon

Check for python in configure

Summary: Some OSs have python named according to a version ID
e.g. `python2.7` without a symlink from plain `python`

Use autotool's AM_PATH_PYTHON to search for a suitable interpreter
and use that to generate the *Tables.cpp files.

Closes #109

Reviewed By: @yfeldblum

Differential Revision: D2147977

Keep Makefile.am from double-installing detail/Clock.h

Summary: On non-Linux systems, the autotools-based build system conditionally appends `detail/Clock.h` to `libfollybase_la_SOURCES` – even though that file is in there statically no matter what (as per [line 46](https://github.com/facebook/folly/blob/master/folly/Makefile.am#L46)), which causes the `install` command to error out trying to install the file twice, during `make install`… this patch takes out the conditional append.
Closes #201

Reviewed By: @yfeldblum

Differential Revision: D2148004

Pulled By: @sgolemon

Replace thread_local with FOLLY_TLS

Summary: OSX doesn't support C++11's `thread_local`. :-/ (yet? http://stackoverflow.com/questions/28094794/why-does-apple-clang-disallow-c11-thread-local-when-official-clang-supports/29929949#29929949)
We have `FOLLY_TLS` in `Portability.h` https://github.com/facebook/folly/blob/master/folly/Portability.h#L163-L175

So replace the usages of `thread_local` with `FOLLY_TLS`. But `__thread` is not drop-in compatible with `thread_local`, specifically the former requires global storage and needs a constexpr initializer, and a trivial destructor.

`futures/QueuedImmediateExecutor` now uses `folly::ThreadLocal` (because `std::queue` has a non-trivial destructor). It might end up in one more alloc, per thread. I also rearranged the code to clarify the fact that although there may be many `QueuedImmediateExecutor`s, they all share the same queue per thread.

I didn't use `folly::ThreadLocal` for fibers, because I thought it might be too expensive. But now I'm not so sure. I had to do the "check for default and then initialize" thing because of the requirement for constexpr initialization for `__thread` (and therefore `FOLLY_TLS`).

Reviewed By: @jsedgwick

Differential Revision: D2140369

disable ThreadExecutor in FutureTest.cpp, on OSX

Summary: Something's broken here, will track fixing in #7372389

Reviewed By: @jsedgwick

Differential Revision: D2144077

Build and run futures tests on OSX

Summary: They don't pass, but they build!

Reviewed By: @jsedgwick

Differential Revision: D2141717

Future::unit()

Summary: Discard a result but propagate an exception. cf https://www.facebook.com/groups/715931878455430/permalink/879624552086161/

Reviewed By: @​sammerat

Differential Revision: D2144567

refactor the interpretation of SSL errors into a function

Summary: This fucntion will be re-used for different SSL write functions in future.

Reviewed By: @djwatson

Differential Revision: D2117529

use LifoSem instead of sem_t for ManualExecutor

Summary: OSX apparently knows better than all of us plebes, and got rid of anonymous semaphores. Rather than try to work around that directly, I just switched to folly::LifoSem, which seems like a not-terrible idea anyway.

Reviewed By: @jsedgwick

Differential Revision: D2143566

folly::via(Executor*, Func)

Summary: ((not yet) more performant) sugar for `via(&x).then(func)`

Reviewed By: @​hannesr

Differential Revision: D2131246

Make IOBuf copyable

Summary: Give IOBuf a copy constructor, which clones the whole chain. (IOBufs have
shared pointer semantics). You could argue for cloning one single IOBuf,
but cloning the whole chain has the nice side effect of making Thrift
structures with IOBufs in them copyable, so there's that.

Reviewed By: @djwatson

Differential Revision: D2127209

RequestEventBaseCache -> IOObjectCache

Summary: Next step towards killing RequestEventBase hackery.
Use EBs from global IO executor instead

Reviewed By: @fugalh

Differential Revision: D2123142

wangle::getEventBase()

Summary: this is going to be a really common pattern, so I'm providing a shortcut

Reviewed By: @fugalh

Differential Revision: D2146008

optimize makeFuture and Future<T>::Future()

Summary: No reason to go through the whole Promise rigamarole. Add an appropriate Core ctor and use that to make a completed Future with just the core alloc

Note the big win in the `constantFuture` benchmark.

```
Before:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             120.50ns    8.30M
promiseAndFuture                                  91.99%   130.98ns    7.63M
withThen                                          28.17%   427.77ns    2.34M
----------------------------------------------------------------------------
oneThen                                                    430.48ns    2.32M
twoThens                                          58.03%   741.86ns    1.35M
fourThens                                         31.85%     1.35us  739.97K
hundredThens                                       1.61%    26.80us   37.32K
----------------------------------------------------------------------------
no_contention                                                4.58ms   218.48
contention                                        83.70%     5.47ms   182.86
----------------------------------------------------------------------------
throwAndCatch                                                8.09us  123.55K
throwAndCatchWrapped                              94.43%     8.57us  116.67K
throwWrappedAndCatch                             154.69%     5.23us  191.12K
throwWrappedAndCatchWrapped                      614.06%     1.32us  758.70K
----------------------------------------------------------------------------
throwAndCatchContended                                     967.54ms     1.03
throwAndCatchWrappedContended                    103.48%   935.04ms     1.07
throwWrappedAndCatchContended                    148.24%   652.70ms     1.53
throwWrappedAndCatchWrappedContended            14313.28%     6.76ms   147.94
============================================================================

After:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                              69.11ns   14.47M
promiseAndFuture                                  55.12%   125.37ns    7.98M
withThen                                          16.49%   419.18ns    2.39M
----------------------------------------------------------------------------
oneThen                                                    370.39ns    2.70M
twoThens                                          55.11%   672.05ns    1.49M
fourThens                                         29.00%     1.28us  782.89K
hundredThens                                       1.23%    30.22us   33.09K
----------------------------------------------------------------------------
no_contention                                                4.56ms   219.46
contention                                        82.82%     5.50ms   181.77
----------------------------------------------------------------------------
throwAndCatch                                                8.30us  120.42K
throwAndCatchWrapped                              96.40%     8.61us  116.08K
throwWrappedAndCatch                             162.66%     5.11us  195.89K
throwWrappedAndCatchWrapped                      680.39%     1.22us  819.36K
----------------------------------------------------------------------------
throwAndCatchContended                                     979.17ms     1.02
throwAndCatchWrappedContended                    103.09%   949.84ms     1.05
throwWrappedAndCatchContended                    153.55%   637.71ms     1.57
throwWrappedAndCatchWrappedContended            10468.47%     9.35ms   106.91
============================================================================
```

Reviewed By: @fugalh, @​hannesr

Differential Revision: D2144664

Fix libc++ use

Summary: Use of std::max from libc++ resulted in error.
std::max found type of its arguments diffrent
whereas it expected them to be same.
This diff fix the error.
Closes #223

Reviewed By: @yfeldblum

Differential Revision: D2144639

Pulled By: @sgolemon

Fix capitalization error in README.md

Summary: Closes #211

Reviewed By: @Gownta, @lbrandy

Differential Revision: D2144702

Pulled By: @sgolemon

Fixing typos in README.md

Summary: Closes #202

Reviewed By: @paulbiss

Differential Revision: D2144850

Pulled By: @sgolemon

Update FBVector.md

Summary: Minor typo when referencing `std::allocator`
Closes #84

Reviewed By: @yfeldblum

Differential Revision: D2144830

Pulled By: @sgolemon

Update github.com/floitsch/double-conversion to new location

Summary: Closes #212

Reviewed By: @JoelMarcey, @paulbiss

Differential Revision: D2144749

Pulled By: @sgolemon

fix Baton.h typo

Summary: checked the sense
Closes #224

Reviewed By: @paulbiss

Differential Revision: D2144508

Pulled By: @sgolemon

(Wangle) Re-add race test

Summary: I figured this test would be useless since it tests a check added before
everything was converted to FSM. But since it appears to fail on Mac OS
X, it might be useful to have after all.

Reviewed By: @fugalh

Differential Revision: D2143500

Bump version to 45:0

fix typo in SharedPromise comment

Summary: ^

Test Plan: OK

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2140212

Signature: t1:2140212:1433877717:86b7bf372091b455cdb25a377451e1ed4e21f50d

Bump version to 44:0

AsyncSSLSocket StartTLS

Summary:
Adds a StartTLS mode to AsyncSSLSocket.  Previously I could only find anyone doing something like this by using AsyncSocket, calling detachFd, then creating a new AsyncSSLSocket, and calling sslConn/sslAccept.

That had a couple downsides: 1) All pointers to the previous AsyncSocket become invalid and similarly 2) have to be super careful reads/writes happen on the correct socket, are flushed before changing socket types, etc.

This makes it super easy to just use the same AsyncSSLSocket for everything:
a) Create AsyncSSLSocket in StartTLS mode
b) send/recv anything
c) Call sslAccept/sslConn.  Existing writes are still flushed in the correct order, any additional writes are buffered until handshake completes
d) Start receiving encrypted data.

I made it a new mode (vs. the default), since it seems bad to unintentionally send unencrypted data.

Use case is easy secure thrift upgrade (similar to how current kerberos does it)

Test Plan: New unittest

Reviewed By: afrind@fb.com

Subscribers: doug, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, haijunz, andrewcox, alandau, alikhtarov, jsedgwick, simpkins

FB internal diff: D2120114

Signature: t1:2120114:1433798448:caeddc8feb6cc10fb34200ba97ea323bcaf09f7a

fix osx bootstrap for new double-conversion directory structure

Summary:
Because https://github.com/google/double-conversion/commit/64702b40a2499702f9157ef015f0da6e006114bf

If this breaks an existing checkout, it is because you have an old version of double-conversion now. Go ahead and `rm -rf` it, or start with a clean folly clone.

Test Plan: Clean repo, `./bootstr<tab>`.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2136836

Signature: t1:2136836:1433874889:6f6c6a7bd337e052c088aabf89955e9b21b1d8e1

Add a guard page to a limited number of stacks

Summary:
GuardedPageAllocator now maintains a process-wide cache
of preallocated stacks with guard pages.  We can't create too many of these, since
each stack has the overhead of two memory mappings.  Once we run out of preallocated stacks
we simply fallback on the default allocator.

Test Plan:
unit tests

perflab
TAO unit tests

Reviewed By: pavlo@fb.com

Subscribers: zhuohuang, trunkagent, sameen, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129510

Tasks: 7319041

Signature: t1:2129510:1433542031:324942af3c9813344c1b331ee2c9b66c4bfa3f03

(Wangle) variadic collect

Summary:
For D2099047 (matthieu) and also for symmetry. Can re-use most of the
code, also refactored it a bit (using an empty base case).

Test Plan:
Run all the tests.

Will add some more before committing.

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, matthieu

FB internal diff: D2131515

Signature: t1:2131515:1433776852:544166fbfdfabf6760fd78f87821290e17e6e4a3

remove Singleton from io/async deps

Summary: unused, caused circular dep issues

Test Plan: compiles

Reviewed By: mzlee@fb.com

Subscribers: subodh, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2135780

Signature: t1:2135780:1433786607:015cfa9bb14472f55f80467f2cfeb7426be9f7ea

fix the compling error caused by tcpinfo of low version

Summary: my D2097198 break the project relying on old glibc

Test Plan: compling

Reviewed By: wez@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2135297

Tasks: 7283522

Signature: t1:2135297:1433791259:8b82b8d9b16da32be54c0dff1214fa20c94840e2

Add internal mode for HHWheelTimer

Summary:
This allows an EventBase to exit loop() even if there are pending timeouts in the timer (make sure to cancelAll before deleting!)

Test Plan: Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2111942

Signature: t1:2111942:1432934212:7ac1973bdbbfdbda714699ab373618399f29ef76

Fix haystack threading crashes

Summary:
proxygen/facebook/httpserver/filter/StatsFilter expects the threadlocal stats_ variable to be initialized before any requests, and destroyed on the same thread.

There were two issues:
1) addObserver was being called after bind(), so it was possible to get some requests before the stats_ variable was initialized.  Moved up the observer, should fix it (although I don't have a repro for this)

2) on server stop(), sometimes the observer onServerStop() callback was called, but the eventBase had already stopped.  The IOThreadPoolExecutor eventually deleted the eventBase, causing consumeUntilDrained to be called, but in the main thread, and not in the eventBase's thread.  I'm not as sure about this fix, hopefully @jsedgwick can chime in: I explicitly deleted the eventBase on the eventBase thread before join.  Here's the stack for this issue:

P19866015

Reviewed By: shiva@fb.com

Subscribers: doug, fugalh, bmatheny, folly-diffs@, yfeldblum, chalfant, shiva, jsedgwick

FB internal diff: D2132208

Signature: t1:2132208:1433537787:4379ba50a908ca6f19924c2339876afaf663364c

FileServer

Summary: Based on new FileRegion interface

Test Plan: ran it, streamed lots of files. I have found an obscure way to break it but I don't think it's a FileRegion issue, rather a pipeline/bootstrap issue. Still digging

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2088094

Signature: t1:2088094:1432758904:0f1e5765dbedea5ddbe047db50e7b632cfb0c053

AsyncSocket::writeRequest() and its first user wangle::FileRegion

Summary: similar to D2050808, but move the functionality into AsyncSocket itself so that you have a consistent interface and contiguous writes for a single file

Test Plan: added unit, will hook this up to a file server example next

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, net-systems@, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2084452

Signature: t1:2084452:1433181933:175158618966706db00bf6620cc86ae145d04ecf

Conv.h: fix a type error caught by clang

Summary: Need to be more explicit about types; it was trying to add types deduced to be `unsigned long` + `long` which didn't agree and could be an overflow.  Be explicit anyway about the types involved.  Appease Clang, and also make the code more obvious to the reader.

Test Plan: Tried with gcc 4.9, clang 3.5, clang 3.6.

Reviewed By: yfeldblum@fb.com

Subscribers: mathieubaudet, maoy, folly-diffs@, yzhan, yfeldblum, chalfant

FB internal diff: D2134814

Tasks: 7337462

Signature: t1:2134814:1433726862:5dd80b198187c610f793e26160919922863a22a2

Blame Revision: D1934777

decimal conversion: digits10 using bit-counting instructions on x86-64

Summary: To estimate length of a number's base-10 length in digits, use insn `bsrq` (Bit Scan Reverse) to count significant bits.  From that, approximate base-10 length.  Tries to avoid branchiness, expensive math, and loops.

Test Plan:
1) Tested correctness by comparing results with `snprintf` and ensuring same string lengths.  Tested at each boundary condition (2^k)-1, 2^k, (2^k+1); and similar for base 10.
2) Benchmarked with gcc 4.9 and clang 3.5.

Before/after values are millions operations / sec
GCC 4.9                             Clang 3.5
1       111.09  111.7   1.005       1       115.36  393.81  3.414
2       115.36  111.7   0.968       2       115.36  393.89  3.414
3       114.91  111.34  0.969       3       111.09  393.56  3.543
4       114.91  111.34  0.969       4       111.09  393.86  3.545
5       115.36  111.36  0.965       5       111.09  392.18  3.530
6        99.99  104.32  1.043       6       103.43  393.74  3.807
7        83.31   84.71  1.017       7        81.06  268.39  3.311
8        76.9    78.23  1.017       8        76.91  268.26  3.488
9        62.48   68.26  1.093       9        65.56  190     2.898
10        59.98   63.65  1.061      10        61.17  190.54  3.115
11        50.6    55.87  1.104      11        54.54  148.03  2.714
12        47.19   51.7   1.096      12        50.84  148.57  2.922
13        40.53   46.99  1.159      13        43.33  115.91  2.675
14        40.48   43.42  1.073      14        41.5   115.97  2.794
15        34.92   40.21  1.151      15        37.27   94.89  2.546
16        33.49   37.51  1.120      16        35.77   94.88  2.653
17        29.89   35.02  1.172      17        31.7    80.67  2.545
18        29.11   32.98  1.133      18        30.76   80.63  2.621
19        26.58   31.05  1.168      19        28.22   70.15  2.486
20        25.64   29.38  1.146      20        27.96   70.16  2.509

Reviewed By: ldbrandy@fb.com

Subscribers: dancol, trunkagent, marcelo, chalfant, maoy, folly-diffs@, yzhan, yfeldblum

FB internal diff: D1934777

Signature: t1:1934777:1433523486:3acbe7ed9c9560c44194f854754529041eaa289d

(Wangle) Clean up tests

Summary:
This always bothered me. Now:

1) Separate tests for each feature
2) `XTest.cpp` contains `TEST(X, lowerCaseCamelCase)`

Also cleaned up some header files and using statements.

Shamelessly trying to increase the number of lines that blame to me.

Test Plan:
Run all the tests.

`git grep 'TEST' -- 'folly/futures/test/*Test.cpp' | cut -d ',' -f 1 | cut -d '/' -f 4 | sort -u`

`diff <(git ls-files -- 'folly/futures/test/*Test.cpp' | cut -d '/' -f 4 | sort) <(git grep 'Test.cpp' -- folly/futures/TARGETS | cut -d '/' -f 4 | cut -d "'" -f 1 | sort)`

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2127359

Tasks: 6019442

Signature: t1:2127359:1433459532:54a91ae83d7fb2d0b3f3769f673fefea20f35435

Bump version to 43:0

disable stack madvise optimization on non-Linux platforms

Summary:
This diff strengthens the preconditions for attempting to use
optimize by madvising away idle stacks, because it seemed to be causing
problems on OS X (https://github.com/facebook/proxygen/issues/3).

Test Plan: unit tests

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129100

Signature: t1:2129100:1433458268:0b6b3696dde6c2bd13b89bc7ec58f0b1898be458

Add missing include

Summary:
`std::runtime_error` is defined in this header. Most build
systems seem to end up including it transitively anyways, but one version
of Clang on my Mac does not.

Test Plan:
`fbconfig -r folly && fbmake dbg` still works. Builds on my
Mac with MacPorts clang 3.6.

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2127268

Signature: t1:2127268:1433437985:546d0de71d8973d10f8ae3fdb6c61efae59444b9

merge dex and readme

Summary:
This is still manual as in the previous DEX.md revision, but I incorporated README.md into dex and redid the manual export.

Test Plan: reading_intensifies

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2124555

Tasks: 6047407

Signature: t1:2124555:1433370669:91f8bb55c1e6de4dbeccbb85556f54ae49464c7a

Reduce footprint of EliasFanoReader

Summary:
`EliasFanoReader` has a copy of `EliasFanoCompressedList` as member, but it only needs few of its members. With this diff, it only copies the members it needs.
Also, `progress_` is a duplicate of `upper_.position()`, so it was removed.
Microbenchmarks do not indicate any significant change in performance.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2125956

Signature: t1:2125956:1433381848:2a333ce7a741bec5d059e9e771309463d6018ea2

Add cloning constructor to folly::IOBuf

Summary:
``` lang=cpp
folly::IOBuf buf(folly::IOBuf::CLONE, other);
```

is more convenient (and expression-friendly) than

``` lang=cpp
folly::IOBuf buf;
other.cloneInto(buf);
```

Test Plan: used it

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2124605

Signature: t1:2124605:1433372767:c167a36ffdc6d20c740aa57895532e9445766fce

remove useless self-assignment in folly::iobuf

Summary:
I'm pretty sure this doesn't do anything.

Based on a cursory review, I also don't think this is a bug (ie, it should have been assigned to something else).

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: tudorb@fb.com

Subscribers: net-systems@, enis, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2124497

Tasks: 7306976

Signature: t1:2124497:1433368083:1097dc22734dacad59b30d4b721be111d021bb43

Bump version to 42:0

(Futures) manually-generated and manually-sanitized DEX.md

Summary:

Test Plan: I copied this to a test github project and see that it renders well enough.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2120950

Tasks: 7295194

Signature: t1:2120950:1433289846:8fbf5123e71d5594742305009fadb516206e2ba6

More flexible constructors for Elias-Fano lists

Summary:
Implement constructors for EliasFanoCompressedList to read a list from a contiguous byte range given either size and upper bound, or size, lower bits width, and upper bits size.

Refactor the rest accordingly.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: trunkagent, chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2105658

Tasks: 5474196

Signature: t1:2105658:1433270469:9948b159504e08c1b00eeb4cbe327752364ec300

fix mem leak

Summary:
- use folly::ThreadLocal to work around GCC bug 57914 (with the benefit of accessAllThreads)
- clean up corresponding thread-local and global cache entries before eventbase gets destructed since there was a possible memory leak for short-term living eventbase.

Test Plan:
tests

Reviewed By: andrii@fb.com

Subscribers: smarlow, rushix, ilyam, trunkagent, folly-diffs@, yfeldblum, chalfant, jinfu

FB internal diff: D2116216

Tasks: 7291028, 7279391

Signature: t1:2116216:1433212893:e57a7df90b15b89ccd9471469e669c6e7dc477bf

Blame Revision: D1941662

(Wangle) Don't use std::bind in Core

Summary:
Seems like an unnecessary indirection that might make it harder to for
the compiler to inline everything (if that's even possible with
`std::bind`)?

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2115032

Signature: t1:2115032:1433182233:3938d0498cad50df32c8b6ee4536de14ae5fd006

Minor fix

Summary:
`LockFreeRingBuffer` was using `std::unique_ptr` but not importing `memory`.
Also, Android NDK's libstdc++ doesn't provide `std::llround`, use straight
`llround`, we don't need the constexpr-ness anyway.

Test Plan: `fbmake runtests`

Reviewed By: jmkaldor@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116923

Signature: t1:2116923:1433210832:9ec0ed7f558c271a51d528b8f88fbce9b466ba3a

AsyncServerSocket: more tries for binding to a port

Summary: User reports of intermittent failures at 5.  Setting up a server is expensive.  Binding to a socket is pretty cheap.  Why not 25 tries?  Why not 100?

Test Plan: hope

Reviewed By: alandau@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116187

Signature: t1:2116187:1433199889:09d6916eddf8521f120894ce361c3e1a96905c90

folly: adding folly::ltrimWhitespace / folly::rtrimWhitespace

Summary:
folly: adding folly::trimWhitespace

extending the idea of folly::skipWhitespace.
the reason for adding it, we want to be able to do:

auto s = skipWhitespace(trimWhitespace(" aaaa "))

very similar to python's strip()

Test Plan:
adding unit tests:

# make ; _bin/folly/test/string_test --gtest_filter="String.whitespace"
# fbmake --ccache off --distcc off dbg -j 16
fbmake dbg -j 16
Fbmake run id: G62i4cDP42U
Note: Google Test filter = String.whitespace
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from String
[ RUN      ] String.whitespace
[       OK ] String.whitespace (0 ms)
[----------] 1 test from String (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[  PASSED  ] 1 test.

Reviewed By: ldbrandy@fb.com

Subscribers: ldbrandy, folly-diffs@, yfeldblum, chalfant, azhavnerchik

FB internal diff: D2109364

Signature: t1:2109364:1433192444:862e237bba1928fcb94be1f95c57a68d457939e9

enrich network-quality HTTPHeader field with retransmission rate

Summary:
estimating tcp retransmission rate of the socket

Test Plan: unit tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, njormrod, bmatheny, trunkagent, chalfant, yfeldblum, jsedgwick

FB internal diff: D2097198

Tasks: 4888253

Signature: t1:2097198:1433196365:16db26dfd721514481497eddfc7820a453618d33

folly: exception_tracer: add throw() to __cxa_begin_catch

Summary:
http://libcxxabi.llvm.org/spec.html

void* __cxa_begin_catch(void* exceptionObject) throw();

```
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:145:7: error: '__cxa_begin_catch' is missing exception specification 'throw()' [-Werror]
void* __cxa_begin_catch(void *excObj) {
^
throw()
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:35:7: note: previous declaration is here
void* __cxa_begin_catch(void* excObj);
^
1 error generated.
11.44s error: folly/experimental/exception_tracer/ExceptionTracerLib.cpp
fbmake: *** [_build/opt/folly/experimental/exception_tracer/b187621/ExceptionTracerLib.o] Error 1
```

Test Plan: fbmake

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2115305

Tasks: 5908365

Signature: t1:2115305:1433186210:3652d81ab10effe1be6da86705322ce090fb6cd4

Blame Revision: https://phabricator.fb.com/D2114085

folly: -Wsign-compare

Test Plan: runtests

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, tudort, chalfant

FB internal diff: D2114063

Signature: t1:2114063:1433111051:6686508d78de39ea4ea5dc283189c472e23f8f42

Comparing against tlsExthostname_

Summary:
Instead of comparing tlsExthostname_ with NULL, we are comparing it
against tlsExthostname_.

Test Plan: Automated tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, moa

FB internal diff: D2106257

Tasks: 4751985

Signature: t1:2106257:1432932687:78c6244392d7842e844a3c1e654ffc4dc0e760a2

update wangle/rx/README.md

Summary:
https://github.com/facebook/wangle/pull/1

Test Plan: reads better

Reviewed By: jsedgwick@fb.com

Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2111498

Signature: t1:2111498:1432927058:1f14b38faa324d0bdef9421ae5592244a6b78928

Fix asan test crashes

Summary: We need to prevent the socket from being deleted while in the loopCallback.

Test Plan: --  fbconfig '--sanitize' 'address' 'folly/io/async/test' && fbmake runtests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2109031

Tasks: 7250307

Signature: t1:2109031:1432918801:77f0352254925d3f33f1e004c8bba89cde63e2cf

Blame Revision: D2102601

Chain executor in timeout functions

Summary: within, onTimeout, etc use the executor of the future if it is set.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2104770

Tasks: 6958727

Signature: t1:2104770:1432773599:a71c00827071bef46e5700de04c7125142e4eb17

Make TurnSequencer non-x86-safe

Summary:
`TurnSequencer` uses inline assembly, which obviously won't work on ARM.
Guard said assembly to x86{,_64} targets.

Test Plan:
Added an always-false `static_assert` to the `ifdef` block, confirmed it gets
triggered (i.e., we're not removing the `pause` on x86_64).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069320

Signature: t1:2069320:1431663264:3b3fb4282cf2b755f9b26c76b608b91ff665e226

Make detail/Futex.cpp bionic-safe

Summary:
The headers that the Android NDK exports are all sorts of screwed up.
In particular, they expose a subset of the `futex.h`, so, even though the
kernel supports newer options, we can't reference them from the NDK headers.

This diff ensures Futex.cpp redefines all the things it uses, if the headers
are lacking.

It also transitions away from `SYS_` to `__NR_` which is apparently the newer
convention.

Lastly, the duration usage is made explicitly `time_t`-safe for use on 32-bit
platforms where `time_t` is `long int` but `std::chrono:seconds` is stored as
`long long int`.

Test Plan:
Existing tests, sample app compiled and ran cleanly on Android as well
(tests not verified on Android due to folly largely being unported to Android yet).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069306

Signature: t1:2069306:1431721711:36c77b1afe8dd9259c1050f11a87511dcf7dd25f