Zonr Chang [Wed, 12 Oct 2016 20:16:27 +0000 (13:16 -0700)]
Access SSL cipher info with a const reference.
Summary:
BoringSSL returns `const SSL_CIPHER*` from sk_SSL_CIPHER_value().
Closes https://github.com/facebook/folly/pull/492
Reviewed By: yfeldblum
Differential Revision:
D4009712
Pulled By: Orvid
fbshipit-source-id:
cca2e8cb6aab0a459dca55b8f23ef57fa5c8251f
Zonr Chang [Wed, 12 Oct 2016 20:14:31 +0000 (13:14 -0700)]
Fix call to SSL_SESSION_up_ref().
Summary:
`SSL_SESSION_up_ref()` in both BoringSSL and OpenSSL 1.1.0 takes a
`SSL_SESSION*`.
Closes https://github.com/facebook/folly/pull/493
Reviewed By: yfeldblum
Differential Revision:
D4009706
Pulled By: Orvid
fbshipit-source-id:
68ea201821e7a6d5835a79b34187344ba251e239
Philip Pronin [Wed, 12 Oct 2016 17:57:13 +0000 (10:57 -0700)]
ZSTDCodec should properly handle non-shortcut levels
Summary:
Currently only shortcut levels are properly handled, others result in
using level 1.
Reviewed By: yfeldblum
Differential Revision:
D4008123
fbshipit-source-id:
37845eeec139007738f99e72ecfb969c6a2e5652
Phil Willoughby [Wed, 12 Oct 2016 09:44:45 +0000 (02:44 -0700)]
Print why we called std::terminate
Summary:
When terminating because we were told to rethrow a nonexistent exception, we
now print that to `std::cerr` for the benefit of users on platforms which do not dump a stack trace on
`std::terminate`
Reviewed By: yfeldblum
Differential Revision:
D4001087
fbshipit-source-id:
5b834cfc75bf42bec06a49ac9db55789a83df180
Giuseppe Ottaviano [Wed, 12 Oct 2016 04:12:59 +0000 (21:12 -0700)]
Add support in enumerate() for iterators with exotic pointers
Summary: I thought we wouldn't need this but it turns out Thrift Frozen2 iterators return proxies (it's proxies all the way down). Thanks to ericniebler for trick.
Reviewed By: yfeldblum
Differential Revision:
D4005700
fbshipit-source-id:
1911996afa075c1d819a3aaea2ee924bc2ae2f20
Christopher Dykes [Wed, 12 Oct 2016 00:40:01 +0000 (17:40 -0700)]
Fix build on Mac OSX Sierra
Summary:
There are two changes here.
The first is to eliminate `detail::DEFAULT_CLOCK_ID` from `Benchmark.[cpp|h]` as Sierra defines `clockid_t` as an enum type, which means that calling `clock_gettime(detail::DEFAULT_CLOCK_ID` would fail, because the enums are incompatible. As this was being used as a default, but is not actually changable anywhere, I just got rid of `detail::DEFAULT_CLOCK_ID` entirely.
The second is to move `portability/BitsFunctexcept.cpp` into `libfollybase_la_SOURCES`, because it's needed for generating the fingerprint tables.
Reviewed By: yfeldblum
Differential Revision:
D4004843
fbshipit-source-id:
b2a9c33f8e516d8eb3cdc5ab093f4946ac9ed37e
Lucian Grijincu [Wed, 12 Oct 2016 00:03:55 +0000 (17:03 -0700)]
folly: signal-handler: dynamic cache size, based on number of dynamic-loaded ELF files
Summary:
I added kFatalSignalHandlerCacheSize in {
D3984649}, and chose a big
number to satisfy most use-cases. But the size increase is non-trivial.
Memory usage:
- before: 5.1 MB in folly::symbolizer::SignalSafeElfCache::SignalSafeElfCache
- after: 80.0 MB in folly::symbolizer::SignalSafeElfCache::SignalSafeElfCache
Switch to a dynamic approach and everyone pays for what they use.
https://github.com/bminor/glibc/blob/
ee19f1de0d0da24114be554fdf94243c0ec6b86c/elf/rtld-debugger-interface.txt#L10-L20
```
The r_debug structure contains (amongst others) the following fields:
struct link_map *r_map:
A linked list of loaded objects.
enum { RT_CONSISTENT, RT_ADD, RT_DELETE } r_state:
The current state of the r_map list. RT_CONSISTENT means that r_map
is not currently being modified and may safely be inspected. RT_ADD
means that an object is being added to r_map, and that the list is
not guaranteed to be consistent. Likewise RT_DELETE means that an
object is being removed from the list.
```
https://github.com/bminor/glibc/blob/
ee19f1de0d0da24114be554fdf94243c0ec6b86c/elf/rtld.c#L303-L307
```
/* Call the OS-dependent function to set up life so we can do things like
file access. It will call `dl_main' (below) to do all the real work
of the dynamic linker, and then unwind our frame and run the user
entry point on the same stack we entered on. */
start_addr = _dl_sysdep_start (arg, &dl_main);
```
dl_main: https://github.com/bminor/glibc/blob/
ee19f1de0d0da24114be554fdf94243c0ec6b86c/elf/rtld.c#L1192-L1199
```
/* Initialize the data structures for the search paths for shared
objects. */
_dl_init_paths (library_path);
/* Initialize _r_debug. */
struct r_debug *r = _dl_debug_initialize (GL(dl_rtld_map).l_addr,
LM_ID_BASE);
r->r_state = RT_CONSISTENT;
...
/* We start adding objects. */
r->r_state = RT_ADD;
_dl_debug_state ();
LIBC_PROBE (init_start, 2, LM_ID_BASE, r);
...
/* Notify the debugger all new objects are now ready to go. We must re-get
the address since by now the variable might be in another object. */
r = _dl_debug_initialize (0, LM_ID_BASE);
r->r_state = RT_CONSISTENT;
_dl_debug_state ();
LIBC_PROBE (init_complete, 2, LM_ID_BASE, r);
```
Reviewed By: bixue2010
Differential Revision:
D3996974
fbshipit-source-id:
e24d72e3cc0339e4cf1acdd2f4c9a7ebfcfdf739
Jim Meyering [Tue, 11 Oct 2016 18:25:41 +0000 (11:25 -0700)]
folly/ConcurrentSkipList.h: avoid shadowing warnings (trivial)
Summary:
Compiling with gcc's proposed -Wshadow-compatible-local option
exposed some fix-worthy warnings. Rename to avoid the shadowing.
Reviewed By: evilmucedin
Differential Revision:
D3999026
fbshipit-source-id:
26cb3033ba8c5538cc9217993f2fda6aef954a8f
Shubhanshu Agrawal [Tue, 11 Oct 2016 15:02:40 +0000 (08:02 -0700)]
BatchDispatcher add() interface
Summary: converting add(value&&) to add(value) to make it easier for callsites to use.
Reviewed By: A5he
Differential Revision:
D4000416
fbshipit-source-id:
c66f4d93d1223286bbc786d17f65ed02a63cc794
Louis Boval [Tue, 11 Oct 2016 05:11:03 +0000 (22:11 -0700)]
Disabling failing test for fbandroid
Summary: this particular test fails reliably for fbandroid. Modifying the code in fbcode, assuming it will be synced promptly.
Reviewed By: edelron
Differential Revision:
D4000722
fbshipit-source-id:
fca5e691ea5c972ddd245acee1c17720730539d4
Nicholas Ormrod [Mon, 10 Oct 2016 22:45:25 +0000 (15:45 -0700)]
Remove memcpy-UB in fbstring
Summary:
The bug report in t13764686 shows that `folly::StringPiece().str()` causes undefined behavior by passing nullptr into memcpy: the default-initialized StringPiece has two null pointers, which will call `string(char*, size)` with nullptr and 0. This gets forwarded to `podCopy` in fbstring, which calls memcpy and has undefined behavior.
The call pattern `string(nullptr, 0)` is not illegal (note: that syntax is ambiguous, as the literal `0` can be converted to a pointer). Prevent this legal call pattern from causing undefined behavior in fbstring.
This diff puts asserts in podCopy to catch errors, and ensures that current podCopy callsites ensure that the pointer is non-null.
Most podCopy callsites do not need to guarded, since the size is guaranteed to be greater than 0 (which will hard crash instead). The interesting callsite is in `initMedium`, because there is a mode (disableSSO) which will allocate empty strings on the heap, so we need to add a guard there.
Reviewed By: luciang
Differential Revision:
D3996440
fbshipit-source-id:
b311a311973d1d969542245c72035c5b38da58e3
Maged Michael [Mon, 10 Oct 2016 13:43:52 +0000 (06:43 -0700)]
Update hazard pointers prototype
Summary:
Interface:
- Got rid of std::function reclamation functions and added a Deleter template parameter.
- Got rid of the flush() member functions of hazptr_domain
- Added a lock-free non-bool member function to get a protected pointer.
Implementation:
- Implemented the interface changes.
- Changed the order of accesses in reading the shared list of objects vs reading the hazard pointers. I think the previous order would have allowed recently protected objects to be reclaimed incorrectly.
Updated the examples and tests accordingly.
Reviewed By: davidtgoldblatt
Differential Revision:
D3981284
fbshipit-source-id:
35ff60da3aea1f67c58d82437dda58f6d8b07bf5
Phil Willoughby [Mon, 10 Oct 2016 10:55:14 +0000 (03:55 -0700)]
Make exception_wrapper::throwException() strict
Summary:
The definition of `throwException()` is changed to add the `[[noreturn]]`
attribute for consistency with `std::rethrow_exception`. If the
`exception_wrapper` contains an exception, that is rethrown; if not, then
`std::terminate` is called.
An alternative design which threw an exception if the `exception_wrapper` did not contain an exception was rejected because of the ambiguity it introduced as to whether the exception thrown was wrapped or not.
Benchmarks before:
```
============================================================================
folly/test/ExceptionWrapperBenchmark.cpp relative time/iter iters/s
============================================================================
exception_ptr_create_and_test 1.45us 689.01K
exception_wrapper_create_and_test 4337.80% 33.46ns 29.89M
----------------------------------------------------------------------------
exception_ptr_create_and_test_concurrent 342.99us 2.92K
exception_wrapper_create_and_test_concurrent 101.41% 338.21us 2.96K
----------------------------------------------------------------------------
exception_ptr_create_and_throw 3.05us 327.89K
exception_wrapper_create_and_throw 140.22% 2.17us 459.77K
exception_wrapper_create_and_cast 8956.80% 34.05ns 29.37M
----------------------------------------------------------------------------
exception_ptr_create_and_throw_concurrent 372.68us 2.68K
exception_wrapper_create_and_throw_concurrent 102.54% 363.44us 2.75K
exception_wrapper_create_and_cast_concurrent 110.93% 335.97us 2.98K
============================================================================
```
and after:
```
============================================================================
folly/test/ExceptionWrapperBenchmark.cpp relative time/iter iters/s
============================================================================
exception_ptr_create_and_test 1.46us 684.09K
exception_wrapper_create_and_test 4368.84% 33.46ns 29.89M
----------------------------------------------------------------------------
exception_ptr_create_and_test_concurrent 341.20us 2.93K
exception_wrapper_create_and_test_concurrent 99.88% 341.60us 2.93K
----------------------------------------------------------------------------
exception_ptr_create_and_throw 3.08us 324.93K
exception_wrapper_create_and_throw 106.93% 2.88us 347.46K
exception_wrapper_create_and_cast 9532.05% 32.29ns 30.97M
----------------------------------------------------------------------------
exception_ptr_create_and_throw_concurrent 363.73us 2.75K
exception_wrapper_create_and_throw_concurrent 101.04% 360.00us 2.78K
exception_wrapper_create_and_cast_concurrent 104.50% 348.07us 2.87K
============================================================================
```
Reviewed By: yfeldblum
Differential Revision:
D3923939
fbshipit-source-id:
6fc58ac571e59e4122f1fbd194c678e3a3841057
Jim Meyering [Sun, 9 Oct 2016 21:53:30 +0000 (14:53 -0700)]
folly/Foreach.h: allow FOR_EACH to be nested with no shadowing
Summary:
Prior to this change, any nested use of FOR_EACH would
induce a shadowed declaration for each of the two local
state variables it declares.
This makes the names of those variables __LINE__-dependent,
so that there is no shadowing, as long as each nested use
is on a different line.
This also adds a new test that (prior to this change) would fail to
compile with an option like -Werror=shadow-compatible-local.
Since this change relies on cpp token concatenation, I have included
The fix defines a new helper macro, _FE_ANON, to derive each new variable
name. I wondered whether to do this for every other FOR_* macro here, but
since so far, I have encountered more than 10 cases of nested FOR_EACH
uses in a large corpus, but no nesting of any other FOR_* macro, I am
content to do it only for this one.
Reviewed By: yfeldblum
Differential Revision:
D3992956
fbshipit-source-id:
f26fba89bc661bb9d22747dec0acdcf8c648fb83
Jim Meyering [Sat, 8 Oct 2016 15:39:18 +0000 (08:39 -0700)]
folly/io/async/HHWheelTimer.cpp: remove decl of unused variable (trivial)
Summary:
In HHWheelTimer::scheduleTimeout, remove the declaration of the
unused variable, "prev".
Reviewed By: yfeldblum
Differential Revision:
D3987409
fbshipit-source-id:
3a9291bf104e6865d608adfb6118609faa0aea96
Yinghai Lu [Sat, 8 Oct 2016 01:38:14 +0000 (18:38 -0700)]
Limit maximum number of iterations in folly::Benchmark with bm_max_iters
Summary:
We are using `bm_max_iters` in folly and other part of the code in fbcode but it didn't really exist in Benchmark.cpp. For example: https://github.com/facebook/folly/blob/master/folly/test/function_benchmark/main.cpp#L29
This patch adds back this functionality.
This diff depends on
D3990600.
Reviewed By: yfeldblum
Differential Revision:
D3989499
fbshipit-source-id:
d2303b2bebb196e84a592d54a72af68171971491
Yinghai Lu [Sat, 8 Oct 2016 01:38:12 +0000 (18:38 -0700)]
Clang-format the flag declarations in Benchmark.cpp
Summary: This patch prepares the later diff which will add a new flag. Just make things nice.
Reviewed By: igorsugak
Differential Revision:
D3990600
fbshipit-source-id:
d4f53a5b44708ca025e4f6615d161b24de685931
Nicholas Ormrod [Fri, 7 Oct 2016 20:05:44 +0000 (13:05 -0700)]
Disallow COW in FBString
Summary:
Remove all code that create large strings.
This diff does not remove code that deals with large strings, so that the transition to libgcc will be smoother. Note that the FBSTRING_ASSERT inside of setCapacity will be disabled in the libgcc build.
Reviewed By: luciang, ot
Differential Revision:
D3977459
fbshipit-source-id:
09c1712854b31509138c5cc8baf19caf3c715bb6
Lucian Grijincu [Fri, 7 Oct 2016 19:58:50 +0000 (12:58 -0700)]
folly: symbolizer: increase default signal-handler symbolizer elf file cache size (make it work with large number of .so)
Summary:
Bump the default size so we can symbolize symbols in dynamically linked
binaries with lots and lots of shared libraries :)
folly/experimental/symbolizer/ElfCache.h
```
* Will not grow; once the capacity is reached, lookups for files that
* aren't already in the cache will fail (return nullptr).
class SignalSafeElfCache
```
Reviewed By: ot, philippv
Differential Revision:
D3984649
fbshipit-source-id:
acdca9b9adf6c16f3a1556bc3f57b28844cdd9cc
Mirek Klimos [Fri, 7 Oct 2016 18:18:57 +0000 (11:18 -0700)]
FunctionScheduler - return whether shutdown was successful
Summary: We want to fail fast if we attempt to stop a scheduler that was not running
Reviewed By: yfeldblum
Differential Revision:
D3976919
fbshipit-source-id:
d8aa8b35aa9e22758e6a7ef64f48a7dd6d990b1c
Alex Yarmula [Thu, 6 Oct 2016 15:51:33 +0000 (08:51 -0700)]
Reverted commit
D3979179
Summary:
When `delayed` is called on the Future, the underlying `futures::sleep` call runs on a timer thread, and the resulting callback is called on the same thread. Therefore, in the following sequence:
f.via(&someExecutor).within(one_ms).then([&]() { /* [1] */ })
The code in [1] is not running in someExecutor. This can cause confusion by users of the library who expect the initial `via` to be sticky.
This change returns to the prior `Executor` after `delayed` is finished.
Reviewed By: yfeldblum
Differential Revision:
D3979179
fbshipit-source-id:
e1448f5603f0c9440490ae3bf0e670687f4179f3
Jim Meyering [Thu, 6 Oct 2016 15:07:29 +0000 (08:07 -0700)]
folly/io: avoid shadowing warnings
Summary: Fix shadowing warnings.
Reviewed By: russoue
Differential Revision:
D3979165
fbshipit-source-id:
19a3940e210e4d5a698dbc0e6d74b317a74a94fa
Alex Yarmula [Thu, 6 Oct 2016 14:21:22 +0000 (07:21 -0700)]
use previous Executor after delayed
Summary:
When `delayed` is called on the Future, the underlying `futures::sleep` call runs on a timer thread, and the resulting callback is called on the same thread. Therefore, in the following sequence:
f.via(&someExecutor).within(one_ms).then([&]() { /* [1] */ })
The code in [1] is not running in someExecutor. This can cause confusion by users of the library who expect the initial `via` to be sticky.
This change returns to the prior `Executor` after `delayed` is finished.
Reviewed By: yfeldblum
Differential Revision:
D3979179
fbshipit-source-id:
936ff5626e8ac377ffb15babf573349466984e3a
Michael Lee [Wed, 5 Oct 2016 23:15:02 +0000 (16:15 -0700)]
Gate `__returns_nonnull__` attribute to gcc 4.9 or later
Summary: The attribute `__returns_nonnull__` is only applicable for gcc 4.9 or later (https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Function-Attributes.html#Function-Attributes)
Reviewed By: ot
Differential Revision:
D3976598
fbshipit-source-id:
3c678a11c5046e5a8b7686327dbb3ebc9b598616
Lucian Grijincu [Wed, 5 Oct 2016 20:49:26 +0000 (13:49 -0700)]
folly: deprecate StringPiece::hash
Summary: It's bad mkey. Says so on the tin can. Alert/annoy its users.
Reviewed By: yfeldblum, ot
Differential Revision:
D3973625
fbshipit-source-id:
23f4e16182749d016084fcb378a1170ff402c259
Lucian Grijincu [Wed, 5 Oct 2016 20:46:42 +0000 (13:46 -0700)]
folly: kill StringPieceHash
Summary: no uses remain in folly
Reviewed By: yfeldblum
Differential Revision:
D3972507
fbshipit-source-id:
356ddbcf24c217b0f589989ab6cc3910d0f7baf3
Maged Michael [Wed, 5 Oct 2016 12:18:09 +0000 (05:18 -0700)]
Updated example and methodology for using DeterministicSchedule support for auxiliary data and global invariants
Summary:
Depends on
D3792669
Updating the test and methodology based on the experience with fine-grained testing dynamic MPMCQueue using DeterministicSchedule's support for auxiliary data and global invariants.
Updates
D3675447
Reviewed By: djwatson
Differential Revision:
D3794217
fbshipit-source-id:
d2862895cb8dea120e758beeb24d6ae15191b013
Matthieu Martin [Wed, 5 Oct 2016 06:28:35 +0000 (23:28 -0700)]
Fix optional default initialization
Summary: Initialize Optional internal memory, so that "un-initialized memory" tools don't flag its callsites.
Reviewed By: yfeldblum
Differential Revision:
D3960462
fbshipit-source-id:
3bd0109959fb93e040fa2e874f586b3508e46dd2
Lucian Grijincu [Wed, 5 Oct 2016 04:28:20 +0000 (21:28 -0700)]
folly: experimental: ABI-break: replace deprecated folly::StringPieceHash with folly::Hash
Summary:
folly::StringPieceHash has a large comment explaining why not to use it, but it's not marked as deprecated.
It's also conveniently named StringPieceHash and not StringPieceHashDEPRECATED so new uses appear from time to time.
Let's nuke the default before it does more damage.
Reviewed By: yfeldblum
Differential Revision:
D3967239
fbshipit-source-id:
04b0564fdf2a70f1195ef2afdff092bc63cde975
Lucian Grijincu [Wed, 5 Oct 2016 00:56:57 +0000 (17:56 -0700)]
folly: elfcache: nuke StringPieceHash
Summary: #accept2ship
Reviewed By: bixue2010
Differential Revision:
D3971138
fbshipit-source-id:
25e0aeddebf001dd8e27a4939842915ccf1000ad
Qinfan Wu [Tue, 4 Oct 2016 20:58:10 +0000 (13:58 -0700)]
Implicit construct sorted_vector_set from std::initializer_list
Summary:
The following thrift file doesn't work:
typedef set<i32> (cpp.template = "folly::sorted_vector_set") IntSet
struct Test {
1: map<i32, IntSet> m = {7: [1, 2]},
}
The generated file looks like this:
Test()
: m(std::initializer_list<
std::pair<const int32_t, folly::sorted_vector_set<int32_t>>>{
{7, std::initializer_list<int32_t>{1, 2}}}) {}
Given that `std::set` allows implicit construction from `std::initializer_list`, it's probably okay to make `folly::sorted_vector_set` the same.
Reviewed By: ot
Differential Revision:
D3968054
fbshipit-source-id:
978d59a7e545e44d603eeb9671815ac7f2c57342
Jim Meyering [Tue, 4 Oct 2016 19:41:07 +0000 (12:41 -0700)]
folly/io/async/AsyncSocket.cpp: avoid shadowing warning for "rv"
Summary:
Enabling the proposed-for-upstream gcc -Wshadow-local option exposes this shadowing:
```
folly/io/async/AsyncSocket.cpp: In member function 'virtual void folly::AsyncSocket::connect(folly::AsyncSocket::ConnectCallback*, const folly::SocketAddress&, int, const OptionMap&, const folly::SocketAddress&)':
folly/io/async/AsyncSocket.cpp:421:11: error: declaration of 'rv' shadows a previous local [-Werror=shadow-compatible-local]
folly/io/async/AsyncSocket.cpp:364:9: error: shadowed declaration is here [-Werror=shadow-compatible-local]
```
Reviewed By: ngoyal, knekritz
Differential Revision:
D3963873
fbshipit-source-id:
85b3ab1fc3b200b81615a00d66a183cc9c792a90
Arthur O'Dwyer [Tue, 4 Oct 2016 17:24:45 +0000 (10:24 -0700)]
hazptr: Replace friend "swap" with a member function and a non-friend
Summary:
This matches what the STL does with e.g. std::vector::swap() and std::shared_ptr::swap().
http://en.cppreference.com/w/cpp/container/vector/swap
http://en.cppreference.com/w/cpp/memory/shared_ptr/swap
Should be relatively uncontroversial, I would think.
Closes https://github.com/facebook/folly/pull/489
Reviewed By: magedm
Differential Revision:
D3963285
Pulled By: yfeldblum
fbshipit-source-id:
3fa6bf77e66fa9a673cb648b0bf87b1db3caa6c8
Jim Meyering [Tue, 4 Oct 2016 16:11:08 +0000 (09:11 -0700)]
folly/io/Compression.cpp: avoid shadowing warnings for "rc"
Summary:
gcc's -Wshadow (and the under-review -Wshadow-local) would evoke this:
```
folly/io/Compression.cpp:650:9: error: declaration of 'rc' shadows a previous local [-Werror=shadow-compatible-local]
folly/io/Compression.cpp:637:7: error: shadowed declaration is here [-Werror=shadow-compatible-local]
folly/io/Compression.cpp:566:9: error: declaration of 'rc' shadows a previous local [-Werror=shadow-compatible-local]
folly/io/Compression.cpp:548:7: error: shadowed declaration is here [-Werror=shadow-compatible-local]
```
In each case, I removed the "int" from the latter declaration.
Reviewed By: philippv
Differential Revision:
D3966308
fbshipit-source-id:
584cb9ffe8ba0e56914223c440efabe9e0de6b17
Giuseppe Ottaviano [Tue, 4 Oct 2016 04:39:06 +0000 (21:39 -0700)]
Outline several fbstring/malloc functions
Summary: Outline all the non-trivial constructor and the destructor, `assign()`, and `append()` to reduce code size. The fast path of copy, and moves, are instead still inlined.
Reviewed By: philippv
Differential Revision:
D3896612
fbshipit-source-id:
25050d4ba28d25da226a7bc49d5b542947d0c512
Giuseppe Ottaviano [Tue, 4 Oct 2016 03:01:55 +0000 (20:01 -0700)]
Fix a typo
Reviewed By: lbrandy
Differential Revision:
D3964328
fbshipit-source-id:
0b1dc2d4b2a1565019770668445a624625b8257e
Shubhanshu Agrawal [Mon, 3 Oct 2016 19:13:32 +0000 (12:13 -0700)]
add BatchDispatcher
Summary:
This diff adds BatchDispatcher, which can be used to batch values while performing IO.
This would be useful in writing single id code in node, and would be use to batch ids while doing IO at storage adapter.
Differential Revision:
D3900404
fbshipit-source-id:
f53aa352344ff55674c7544302b6a1b4726214b6
Giuseppe Ottaviano [Mon, 3 Oct 2016 18:58:07 +0000 (11:58 -0700)]
Optimize frequently inlined FBString methods
Summary:
Almost every method of `fbstring` needs to perform category
dispatching. The category constants are `size_t`, which become 8-byte
immediate values in the dispatching code, so even a simple `if
(category() == Category::isSmall)` is quite large. When inlined
hundreds of thousands of time, it adds up.
This diff redefines the category type to be 1 byte (without changing
the ABI). It also optimizes `size()` and `c_str()` and makes them
branch-free, which probably is not going to have any perf impact but
it saves a few bytes.
Generated code for some small functions:
- `reset()`
Before:
```
48 ba 00 00 00 00 00 movabs $0x1700000000000000,%rdx
00 00 17
48 89 f8 mov %rdi,%rax
c6 07 00 movb $0x0,(%rdi)
48 89 57 10 mov %rdx,0x10(%rdi)
```
20 bytes
After:
```
48 89 f8 mov %rdi,%rax
c6 47 17 17 movb $0x17,0x17(%rdi)
c6 07 00 movb $0x0,(%rdi)
```
10 bytes
- `c_str()`
Before:
```
48 b8 00 00 00 00 00 movabs $0xc000000000000000,%rax
00 00 c0
48 85 47 10 test %rax,0x10(%rdi)
74 08 je 401fd8
48 8b 07 mov (%rdi),%rax
c3 retq
0f 1f 40 00 nopl 0x0(%rax)
48 89 f8 mov %rdi,%rax
```
26 bytes (without the `retq`)
After:
```
f6 47 17 c0 testb $0xc0,0x17(%rdi)
48 89 f8 mov %rdi,%rax
48 0f 45 07 cmovne (%rdi),%rax
```
11 bytes
- `size()`
Before:
```
48 b8 00 00 00 00 00 movabs $0xc000000000000000,%rax
00 00 c0
48 85 47 10 test %rax,0x10(%rdi)
74 08 je 401fa8
48 8b 47 08 mov 0x8(%rdi),%rax
c3 retq
0f 1f 00 nopl (%rax)
48 0f be 57 17 movsbq 0x17(%rdi),%rdx
b8 17 00 00 00 mov $0x17,%eax
48 29 d0 sub %rdx,%rax
```
36 bytes (without the `retq`)
After:
```
0f b6 57 17 movzbl 0x17(%rdi),%edx
b8 17 00 00 00 mov $0x17,%eax
48 29 d0 sub %rdx,%rax
48 0f 48 47 08 cmovs 0x8(%rdi),%rax
```
17 bytes
Reviewed By: philippv
Differential Revision:
D3957276
fbshipit-source-id:
ef40d82bbbb0456b1044421cd02133c268abe39b
Philip Pronin [Mon, 3 Oct 2016 04:25:03 +0000 (21:25 -0700)]
switch back to inline assembly in compression::instructions
Summary:
D3278901 never worked, GCC (at least 4.9) failed to propagate
`FOLLY_TARGET_ATTRIBUTE("arch=haswell")` down, `blsr` and `select64`
weren't inlined in EF coding on Haswell, showing up in CPU profile.
This diff switches back to inline assembly, removing second dispatching
mechanism (IFUNC; in additional to caller-side `Default` / `Nehalem` / `Haswell`
dispatching) for `compression::instructions`, and disables functionality on
compilers not supporting AT&T syntax (MSVC++) for now.
Added `FOLLY_ALWAYS_INLINE` to make sure failure to inline results in
compilation failure.
Reviewed By: ot
Differential Revision:
D3959438
fbshipit-source-id:
e40573fbfbf38991caa2cd70293aeaeeec3afad7
Sven Over [Sun, 2 Oct 2016 17:22:13 +0000 (10:22 -0700)]
Fix folly::Partial copy/move construction
Summary:
Any attempt to copy or move an object returned by folly::partial
yields a compiler error because it is invoking the constructor
for wrapping callable objects, which triggers a type mismatch.
This diff fixes that by explicitly naming the default implementations
of copy and move constructors.
This diff also adds additional tests that fail to compile without this fix.
And then this diff also moves the Partial class into folly::detail,
because it is not meant to be named in user code, but only returned
by the folly::partial function.
Reviewed By: mhx
Differential Revision:
D3923809
fbshipit-source-id:
a8883951afd2a1999acbfffc51296393b058f860
Lucian Grijincu [Fri, 30 Sep 2016 19:12:32 +0000 (12:12 -0700)]
folly: use std::__throw_out_of_range
Summary:
A ton of these functions are tiny and should be inlined. `throw std::out_of_range("bla")` generates quite a lot of code.
See godbolt from {
D3940968}: https://godbolt.org/g/9K36Km
advanceNoInline(S):
movq %rdi, %rax
sarq $32, %rax
cmpl %eax, %edi
jg .L20
leal 1(%rdi), %eax
ret
.L20:
subq $8, %rsp
call S::outOfRange() [clone .isra.0]
vs previous implementation
advance(S):
movq %rdi, %rdx
sarq $32, %rdx
cmpl %edx, %edi
jg .L14
leal 1(%rdi), %eax
ret
.L14:
pushq %rbp
pushq %rbx
movl $16, %edi
subq $8, %rsp
call __cxa_allocate_exception
movl $.LC0, %esi
movq %rax, %rdi
movq %rax, %rbx
call std::out_of_range::out_of_range(char const*)
movl std::out_of_range::~out_of_range(), %edx
movl typeinfo for std::out_of_range, %esi
movq %rbx, %rdi
call __cxa_throw
movq %rax, %rbp
movq %rbx, %rdi
call __cxa_free_exception
movq %rbp, %rdi
call _Unwind_Resume
This pattern is in the standard library as well:
```
reference
at(size_type __n)
{
if (__n >= _Nm)
std::__throw_out_of_range(__N("array::at"));
return _M_instance[__n];
}
```
https://github.com/gcc-mirror/gcc/blob/
edd716b6b1caa1a5cb320a8cd7f626f30198e098/libstdc%2B%2B-v3/include/tr1/array#L138
Reviewed By: yfeldblum
Differential Revision:
D3947075
fbshipit-source-id:
6e174c725791762a533a534c8482ea9576460b86
Ted Percival [Fri, 30 Sep 2016 17:40:00 +0000 (10:40 -0700)]
Find dwarf.h in libdwarf subdirectory
Summary:
libdwarf headers moved from `/usr/include` to `/usr/include/libdwarf` in [dwarfutils
20160613-2](http://metadata.ftp-master.debian.org/changelogs/main/d/dwarfutils/unstable_changelog).
Adds a check for the library as well, and fixes a tiny typo in the `libunwind` check.
No longer checks for `libdwarf.h`; only `dwarf.h` is used.
Closes https://github.com/facebook/folly/pull/480
Reviewed By: yfeldblum
Differential Revision:
D3878961
Pulled By: Orvid
fbshipit-source-id:
7970d33511bf6d351cec2c8077cb07b85f2f8d33
Lucian Grijincu [Thu, 29 Sep 2016 21:40:13 +0000 (14:40 -0700)]
folly: Range: detail::throwOutOfRange -> std::__throw_out_of_range
Summary:
No need to define our own.
Generated code is similar: https://godbolt.org/g/5xWrNx (one extra move instruction).
```
.L20:
subq $8, %rsp
call S::outOfRange() [clone .isra.0]
```
```
.LC1:
.string "out of range"
.L26:
subq $8, %rsp
movl $.LC1, %edi
call std::__throw_out_of_range(char const*)
```
Reviewed By: Orvid
Differential Revision:
D3945578
fbshipit-source-id:
c65e9dea55e8f01f51766b2695af68d2bc92c266
Dan Schatzberg [Thu, 29 Sep 2016 15:22:50 +0000 (08:22 -0700)]
Fix ThreadCachedInt race condition
Summary:
Acquire a SharedMutex at ThreadExit to ensure that after unlinking the ThreadEntry from
the list, future accessAllThreads() won't miss a destroying thread.
This is quite a dangerous fix as it changes some lock ordering semantics. ThreadLocal
elements are now destroyed while holding a lock, so if the destruction function
acquires a different lock, ordering must be consistent with other
uses of accessAllThreads().
I've made accessAllThreads() an opt-in feature via a template parameter and changed
all existing uses. I've also fixed a few lock ordering issues that arose due to this
change.
Reviewed By: andriigrynenko
Differential Revision:
D3931072
fbshipit-source-id:
4d464408713184080079698df453b95873bb1a6c
Subodh Iyengar [Thu, 29 Sep 2016 13:42:11 +0000 (06:42 -0700)]
Fix apple bug around TFO writes
Summary:
When using connectx to do TFO, apple has a bug
where the second write after a TFO write will cause
the socket to throw an ENOTCONN error instead of an
EAGAIN. Linux handles this case fine and returns an
EAGAIN, however apple returns ENOTCONN.
We solve this by treating ENOTCONN as an EAGAIN temporarily.
Reviewed By: yfeldblum
Differential Revision:
D3942681
fbshipit-source-id:
ab4f0b5fd6cdcfe9c584ea00849705a2d739d65f
Subodh Iyengar [Thu, 29 Sep 2016 13:42:10 +0000 (06:42 -0700)]
Add tfo functions for apple
Summary:
Adds TFO functions for apple devices
Also allows android as well by removing that
restriction. Newer versions of android support
TFO, so this allows us to experiment with them
Reviewed By: yfeldblum
Differential Revision:
D3942664
fbshipit-source-id:
faf439783b018cf7c987a2e3ade5ea6c0c02bf48
Lucian Grijincu [Thu, 29 Sep 2016 05:33:39 +0000 (22:33 -0700)]
folly: Range: outline exception throwing
Summary:
Here's some godbolt: https://godbolt.org/g/9K36Km
advanceNoInline(S):
movq %rdi, %rax
sarq $32, %rax
cmpl %eax, %edi
jg .L20
leal 1(%rdi), %eax
ret
.L20:
subq $8, %rsp
call S::outOfRange() [clone .isra.0]
vs previous implementation
advance(S):
movq %rdi, %rdx
sarq $32, %rdx
cmpl %edx, %edi
jg .L14
leal 1(%rdi), %eax
ret
.L14:
pushq %rbp
pushq %rbx
movl $16, %edi
subq $8, %rsp
call __cxa_allocate_exception
movl $.LC0, %esi
movq %rax, %rdi
movq %rax, %rbx
call std::out_of_range::out_of_range(char const*)
movl std::out_of_range::~out_of_range(), %edx
movl typeinfo for std::out_of_range, %esi
movq %rbx, %rdi
call __cxa_throw
movq %rax, %rbp
movq %rbx, %rdi
call __cxa_free_exception
movq %rbp, %rdi
call _Unwind_Resume
Reviewed By: ot
Differential Revision:
D3940968
fbshipit-source-id:
b47a41e7cdd863fcef099ff3c21860b2979ee6e8
Giuseppe Ottaviano [Wed, 28 Sep 2016 22:37:33 +0000 (15:37 -0700)]
Add unchecked versions of advance(), subtract(), and subpiece() in Range
Summary:
`Range` has a somewhat inconsistent API: most methods assume
argument validity as a precondition (mirroring their STL
counterparts), others check the arguments and throw for invalid ones.
Since `Range` is intended as a zero-cost abstraction on top of
pointers/iterators, unchecked methods should be preferred. At this
point however we cannot change the semantics of `advance()` and other
methods. This diff adds new unchecked versions of these methods.
Reviewed By: luciang
Differential Revision:
D3938480
fbshipit-source-id:
6952683ee0716aa1584e79584158fbf3e083b52e
Witchakorn Kamolpornwijit [Wed, 28 Sep 2016 20:00:04 +0000 (13:00 -0700)]
Fix SUPPLY_MISSING_INT128_TRAITS
Summary:
The original code `AC_DEFINE([FOLLY_SUPPLY_MISSING_INT128_TRAITS...` resulted in defining the constant FOLLY_FOLLY_SUPPLY_MISSING_INT128_TRAITS in folly-config.h. This patch fix the name of the constant so that the name match with what appear in Traits.h.
Closes https://github.com/facebook/folly/pull/485
Reviewed By: yfeldblum
Differential Revision:
D3931349
Pulled By: Orvid
fbshipit-source-id:
bd7e7d3580d02134d36706bd5836822135232d26
Maged Michael [Wed, 28 Sep 2016 08:36:57 +0000 (01:36 -0700)]
folly/experimental/hazptr: fix gcc 5 build
Summary: Fix includes in memory_resource.h
Reviewed By: yfeldblum
Differential Revision:
D3935848
fbshipit-source-id:
713ce85ad17072779dfa1c667459e367c5e1d4b7
Michael Lee [Wed, 28 Sep 2016 00:31:22 +0000 (17:31 -0700)]
Set a default value for slot in SharedMutex.h
Summary:
`SharedMutexImpl::lockSharedImpl` has a potentially uninitialized access:
Assume state = 0
canAlreadyDefer = (state & kMayDefer) != 0 ==> false
aboveDeferThreshold = (state & kHasS) >= (kNumSharedToStartDeferring - 1) * kIncrHasS ==> false
if (canAlreadyDefer || (aboveDeferThreshold && !drainInProgress)) ==> false
line:1452: gotSlot(slot)->compare_exchange_strong(...) uses slot uninitialized
Reviewed By: Orvid
Differential Revision:
D3933638
fbshipit-source-id:
0fbce5c00b8b1f34e50c302cb88def97853c5afe
Sven Over [Tue, 27 Sep 2016 14:44:32 +0000 (07:44 -0700)]
Introducing folly::FunctionRef
Summary:
This commit introduces a simple function reference type, similar to
std::reference_wrapper, but the template parameter is the function
signature type rather than the type of the referenced object.
A folly::FunctionRef is cheap to construct as it contains only a
pointer to the referenced callable and a pointer to a function which
invokes the callable.
The user of FunctionRef must be aware of the reference semantics:
storing a copy of a FunctionRef is potentially dangerous and should
be avoided unless the referenced object definitely outlives the
FunctionRef object. Thus any function that accepts a FunctionRef
parameter should only use it to invoke the referenced function and
not store a copy of it. Knowing that FunctionRef itself has reference
semantics, it is generally okay to use it to reference lambdas that
capture by reference.
Reviewed By: ericniebler
Differential Revision:
D3277364
fbshipit-source-id:
0a7676919cd240da5b6e1f94cadba4289e0aca28
Anirudh Ramachandran [Mon, 26 Sep 2016 21:55:14 +0000 (14:55 -0700)]
SSL_SESSION wrapper
Summary:
This is a start to wrapping various SSL objects going forward so different
binaries can choose different version of OpenSSL (i.e., BoringSSL, OpenSSL
1.1.0, OpenSSL 1.0.2, etc.). There's no change to the caller - everyone just
uses 'SSLSession', but the implementation details vary
Reviewed By: siyengar
Differential Revision:
D3707791
fbshipit-source-id:
f895334a768cb7d43b41af40c9bc06be5307cc7f
Anirudh Ramachandran [Mon, 26 Sep 2016 20:14:55 +0000 (13:14 -0700)]
AsyncSSLSocket::getSSLClientCiphers using static map
Summary:
OpenSSL SSL_METHOD->get_cipher_by_char is not present in either OpenSSL
1.1.0 or BoringSSL. In addition, knekritz reports that time's being spent in
binary searching for cipher names.
Since the ciphercodes and names are (fairly) static, we store these in a static
hash map.
Reviewed By: siyengar
Differential Revision:
D3275185
fbshipit-source-id:
08b36f3e73239b415b74c6ecc30ed65832d9ebd0
Phil Willoughby [Mon, 26 Sep 2016 11:59:05 +0000 (04:59 -0700)]
Make a SharedPromise from a Future
Summary: Makes it easy to split a Future into multiple Futures
Reviewed By: rongrong
Differential Revision:
D3885897
fbshipit-source-id:
6ac9fb22444dd828fbdebb44b06bf3d93d0f7583
Eli Pozniansky [Fri, 23 Sep 2016 20:08:23 +0000 (13:08 -0700)]
Improve documentation of MPMCQueue size and sizeGuess methods
Summary: See title.
Reviewed By: nbronson
Differential Revision:
D3914188
fbshipit-source-id:
dd9ccd0c48911632d229ae675cc40d835ea14724
Alisson Gusatti Azzolini [Fri, 23 Sep 2016 01:49:16 +0000 (18:49 -0700)]
Avoid external linkage call inside __ifunc__
Summary:
In debug mode, CpuId() ends up triggering external linkage from inside this __ifunc__.
However, in PIC, the relocation of external symbols are not ready yet, causing a segfault.
This only reproes in dynamic linking (PIC / so) and dbg mode.
Reviewed By: luciang
Differential Revision:
D3895239
fbshipit-source-id:
2b7856c10abb5cfe24736d5bfac28e7e9d0e8272
Michael Lee [Thu, 22 Sep 2016 18:09:57 +0000 (11:09 -0700)]
Switch SocketAddressTest to use TemporaryFile instead of mkstemp
Summary: Switch from mkstemp to the slightly more portable `folly::test::TemporaryFile`
Reviewed By: jsedgwick
Differential Revision:
D3890411
fbshipit-source-id:
e98d1e3a5adae92af1bb36f6213b194f633fab0f
Giuseppe Ottaviano [Thu, 22 Sep 2016 18:06:31 +0000 (11:06 -0700)]
Fix a couple comments
Reviewed By: nbronson
Differential Revision:
D3905865
fbshipit-source-id:
2743af4ae1b34adb0f8e611e672f9b6068430ec9
Tianjiao Yin [Thu, 22 Sep 2016 08:06:15 +0000 (01:06 -0700)]
fix flaky TimeKeeper unit-test
Summary:
Sometime we have such unit-test failure
```
folly/futures/test/TimekeeperTest.cpp:134: Failure
Value of: flag
Actual: false
Expected: true
```
This diff should make it less flaky, though I am not sure what's the best way to fix this flaky unit-test.
Reviewed By: yfeldblum
Differential Revision:
D3889630
fbshipit-source-id:
e7486d75fbcb3081d06724d213d4a2cf8942955d
Subodh Iyengar [Thu, 22 Sep 2016 08:01:42 +0000 (01:01 -0700)]
Add logs for TFO succeded
Summary:
Add logs for TFO to AsyncSocket
and wangle
Reviewed By: knekritz
Differential Revision:
D3906104
fbshipit-source-id:
9a79b6f91273f8017a5e0d72fe5bdc8eff645ebc
Michael Lee [Thu, 22 Sep 2016 06:29:55 +0000 (23:29 -0700)]
experimental/observer/detail/ObserverManager.cpp missing gflags
Summary: Fix the build due to a missing gflags header
Reviewed By: yfeldblum
Differential Revision:
D3905525
fbshipit-source-id:
cc09189196a04741162c2d1561545d3fec417e5e
Shoaib Meenai [Wed, 21 Sep 2016 15:07:33 +0000 (08:07 -0700)]
Fix compilation for libc++ 3.9
Summary:
libc++ 3.9 and above define the __throw* functions inside stdexcept, so
defining them ourselves leads to compilation errors when compiling
against libc++ 3.9. Add appropriate preprocessor guards to avoid this.
Reviewed By: meyering
Differential Revision:
D3898284
fbshipit-source-id:
435a28c2b3a83ee4d8f5af0df0343c524469011e
Subodh Iyengar [Tue, 20 Sep 2016 23:25:32 +0000 (16:25 -0700)]
Fix detaching from evb
Summary:
When we attach and detach from the
event base threads during connections
in the case of SSL it could result in
a crash while using TFO.
When TFO was enabled connectTimeout_
was set, which was not detached.
In the case when we would
fall back from TFO in case TFO did not
succeed, we would try and schedule a
connect timeout again. However because
we are now scheduled in a new evb thread,
this would cause the scheduleTimeout to
assert that it was not in the correct evb
thread.
This fixes the issue by attaching and detaching
connect timeouts as well.
Reviewed By: ngoyal
Differential Revision:
D3892490
fbshipit-source-id:
278c0b8029022144cd59366ceb0ce83f0a60a307
Sven Over [Tue, 20 Sep 2016 22:04:07 +0000 (15:04 -0700)]
use folly::Function<void()> in folly::Executor interface
Summary:
This diff changes the definition of `folly::Func` from `std::function<void()>`
over to `folly::Function<void()>`. This mostly affects the interface of
`folly::Executor` and derived and related classes. By using
`folly::Function<void()>`, we allow to have lambdas capturing move-only types
passed to executors. This continues an effort to get rid of the potentially
dangerous `folly::MoveWrapper` by allowing to capture move-only types in lambdas
when using `folly::Future`, `folly::EventBase` and now `folly::Executor`.
Reviewed By: yfeldblum
Differential Revision:
D3706449
fbshipit-source-id:
11c2cee32cb9f0298c39b7b1695a477777eeb3aa
Maged Michael [Tue, 20 Sep 2016 20:47:17 +0000 (13:47 -0700)]
Draft prototype of hazard pointers C++ template library
Summary: Make draft of hazard pointers prototype public
Reviewed By: djwatson
Differential Revision:
D3870280
fbshipit-source-id:
e029efa336585055f67687059e10ae11766f8d7f
Andrey Ignatov [Tue, 20 Sep 2016 19:14:33 +0000 (12:14 -0700)]
Remove boost::barrier from AsyncUDPSocketTest.
Summary:
`EventBase` has its own methods to wait till it's ready so we can avoid using
barriers and remove boost dependency.
Reviewed By: mzlee
Differential Revision:
D3894408
fbshipit-source-id:
f050a982c98c4f672cf295845115686c95fc7919
Igor Sugak [Tue, 20 Sep 2016 07:51:26 +0000 (00:51 -0700)]
Fix fibers asan integration for new asan APIs
Summary:
Updating fibers code to use https://github.com/llvm-mirror/compiler-rt/commit/
b0477747dfa8a9706f2c902e877e616aca51e06f
Patch by andriigrynenko!
Reviewed By: andrewcox
Differential Revision:
D3500482
fbshipit-source-id:
f51f4bb4ebf0d95a898eb1d4098459aa691acd61
Philip Pronin [Tue, 20 Sep 2016 03:52:08 +0000 (20:52 -0700)]
fix ZSTD support
Summary:
Existing logic is broken (unable to correctly handle chained `IOBuf`
in case of both `compress` and `uncompress`) and has unnecessarly strict
`needsUncompressedLength() == true` requirement.
This diff switches `ZSTDCodec` to use streaming to handle chained `IOBuf`,
drops `needsUncompressedLength() == true`.
Reviewed By: luciang
Differential Revision:
D3827579
fbshipit-source-id:
0ef6a9ea664ef585d0e181bff6ca17166b28efc2
Qi Wang [Mon, 19 Sep 2016 23:48:17 +0000 (16:48 -0700)]
Try using the last Deferred reader slot first
Summary:
When trying to find an empty deferred reader slot, getting the current CPU can
take quite a few cycles, e.g. >1% CPU on SMC (https://fburl.com/
434646643).
Let's track the last slot used by this thread and try that slot first before reading
CPU id and doing the search.
u-benchmark results seem to be improving generally (though a bit noisy and not
sure how much to trust). Results w/ this diff on left side:
P56648675
Reviewed By: nbronson
Differential Revision:
D3857793
fbshipit-source-id:
8b1c005362c82e748a663100f889b0b99dc257fe
Dominik Gabi [Fri, 16 Sep 2016 22:00:45 +0000 (15:00 -0700)]
move `shellQuote` to implementation file
Summary: Fixing ODR violations...
Reviewed By: simpkins, yfeldblum
Differential Revision:
D3875667
fbshipit-source-id:
0d8ec0b48e14fffb7e3e60c0e68e2576b2f58d1e
Dominik Gabi [Fri, 16 Sep 2016 22:00:43 +0000 (15:00 -0700)]
escape `{` in cmd for deprecated `Substring(std::string...)`
Summary:
Since `shellify` interprets the command as a format string we need to escape
`{` and `}`.
Reviewed By: ldemailly, simpkins
Differential Revision:
D3874605
fbshipit-source-id:
f47db387c3a44a3ba1c0c1d4726b7212fcb5ef3e
Adam Simpkins [Fri, 16 Sep 2016 19:23:41 +0000 (12:23 -0700)]
update stats APIs to use TimePoint vs Duration correctly
Summary:
Update the stats APIs to correcly distinguish between TimePoint and Duration
types.
This does leave addValue() and update() APIs in place that accept Duration
values, for backwards compatibility. These should eventually be removed once
all code has been converted to call the new APIs.
Reviewed By: yfeldblum
Differential Revision:
D3808805
fbshipit-source-id:
36d6574ba4a09db7eb9f1a35e47addd3e07f8461
Tom Jackson [Fri, 16 Sep 2016 17:09:32 +0000 (10:09 -0700)]
Return rvalue references from &&-qualified members of dynamic
Summary: Let the caller do a move, don't force one.
Reviewed By: yfeldblum
Differential Revision:
D3873129
fbshipit-source-id:
40c6bf564bcbf794830c99ea1248a9c1bb30e9b0
Vladimir Slaykovskiy [Fri, 16 Sep 2016 09:53:58 +0000 (02:53 -0700)]
Reduce footprint of ScribeClient
Summary:
I've found all code paths that contributed to the number of threads in new ScribeClient. Here's a list of flags that reduce the number of thread:
--default_tls_thread_count=1 # was 32
--sr2_event_base_pool_size=1
--has_thrift_dispatcher_reporter=0 # creates a thrift server with a few extra threads
--observer_manager_pool_size=1 # default is 4
Optimizations in this diff:
- Don't initialize OBCClient object if it is not used
- Added FLAG_observer_manager_pool_size to control size of the pool in ObserverManager
Currently OBC counters are switched off globally in ScribeCliean, it means that new code path that creates SR instances is disabled, but eventually we plan to switch it on. Clients that don't use SR and want to keep their RSS/threads profile low, should pass flags listed above. I'll announce this in ScribeUsers group as well.
Differential Revision:
D3870704
fbshipit-source-id:
0efad6b3dc43c072ab11cac7e9461c09532ea11c
Eric Niebler [Fri, 16 Sep 2016 00:00:30 +0000 (17:00 -0700)]
work around LLVM#30305 in folly::Expected, use unified initialization syntax for extra goodness
Summary: The behavior of inheriting constructors changed in clang 3.9 (see https://llvm.org/bugs/show_bug.cgi?id=30305), and folly::Expected needs to change accordingly. As a drive-by improvement, change all invocations of default constructors to use unified initialization syntax.
Reviewed By: igorsugak
Differential Revision:
D3872994
fbshipit-source-id:
fdaea8b35980df21b8522e2c3d5a8c3be1d84efa
Jim Meyering [Thu, 15 Sep 2016 23:26:34 +0000 (16:26 -0700)]
folly/.../ExceptionTracerLib.cpp: provide less DOF fodder (avoid heap use-after-free)
Summary:
Before this change, an application would fail consistently with a shutdown-time heap use-after-free (i.e., destructor-order-fiasco when one of the following symbols was used after its static-destruction-triggered free):
DECLARE_CALLBACK(CxaThrow);
DECLARE_CALLBACK(CxaBeginCatch);
DECLARE_CALLBACK(CxaRethrow);
DECLARE_CALLBACK(CxaEndCatch);
DECLARE_CALLBACK(RethrowException);
Each of those would define a classic meyers singleton.
Since each destructor is trivial, we can fix this by making each a pseudo-leaky (indestructible) meyers singleton instead. Since each static value is never destroyed,
it can never cause a DOF error again.
Differential Revision:
D3870740
fbshipit-source-id:
625f5d5268768ca0e4125bed72bc66c53618be29
Christopher Dykes [Thu, 15 Sep 2016 20:00:46 +0000 (13:00 -0700)]
Kill unneeded dependency on portability/Memory.h
Summary: This dependency was added but isn't actually needed, so get rid of it.
Reviewed By: mzlee
Differential Revision:
D3863923
fbshipit-source-id:
407d2b3db759e30c37d876dde133fc962daeaeae
Neel Goyal [Thu, 15 Sep 2016 19:00:17 +0000 (12:00 -0700)]
Add flag to indicate a session resumption attempt
Summary: Track when session resumption was attempted during connect
Reviewed By: shamdor-fb
Differential Revision:
D3861028
fbshipit-source-id:
26ca41084faeeb64666b6833896b9510ef2b4c25
Adam Simpkins [Thu, 15 Sep 2016 18:02:47 +0000 (11:02 -0700)]
a simple first step towards using clocks properly in the stats code
Summary:
Update the timeseries and histogram classes to accept a clock parameter as a
template parameter, instead of a time duration type.
This is a first step towards transitioning the code to correctly distinguishing
between time_point and duration types. This defines TimePoint and Duration
type aliases, but does not start using them yet.
In upcoming diffs I will start converting more APIs to correctly use TimePoint
instead of just Duration.
For now the default clock type is folly::LegacyStatsClock, which still uses
std::chrono::seconds as the default duration. At the moment the stats code is
optimized for second granularity--the addValue() code has a fast path when
called in the same second as the last update. When using finer granularity
durations this fast path can't be used as often. I will also send out
subsequent diffs to make the code optimized for updates within the same bucket,
rather than just updates with the exact same time value.
Reviewed By: yfeldblum
Differential Revision:
D3807715
fbshipit-source-id:
77696c4f44a8d85e4d6ff84d7656fe7a9709797c
Dominik Gabi [Thu, 15 Sep 2016 14:57:15 +0000 (07:57 -0700)]
ignore `$SHELL` in `shellify`
Summary:
Why `$SHELL` is a bad idea:
- getenv() is not thread safe. (In practice it should work if other threads aren't calling setenv(), but still seems undesirable if we can avoid it.)
- It seems confusing for the program to have different behavior for different developers.
- Other shells besides /bin/sh may have different quoting behaviors. For instance, I don't think csh allows unescaped newlines inside single quotes.
- SHELL might be set to other non-shell-like programs in some cases. (Say, if this gets run from inside a git or mercurial hook it might end up being set to the restricted git or hg shell that only lets you run specific commands.)
Anyway, this isn't related to your diff, so nothing needs to be done for now,
but I would vote for changing this to always use /bin/sh in a future diff.
Both the C system() call and python's subprocess module appear to
always use /bin/sh and ignore $SHELL.
Reviewed By: simpkins
Differential Revision:
D3867047
fbshipit-source-id:
dab0e6afbe1c20ff13d9a52f212df95af425dd77
Dominik Gabi [Thu, 15 Sep 2016 14:57:14 +0000 (07:57 -0700)]
formatting support for `Subprocess::shellify`
Summary:
Adding formatting support with automatic escaping of parameters. Will
subsequentyl try and make it harder for people to use non-statically determined
format strings for this.
Reviewed By: simpkins
Differential Revision:
D3866174
fbshipit-source-id:
80bb5886386692643876116fbf338ca0d6986e6a
Dominik Gabi [Thu, 15 Sep 2016 14:57:13 +0000 (07:57 -0700)]
deprecate `folly::Subprocess(std::string, ...)`
Summary:
introducing `Subprocess::shellify` to get around this. Still thinking
about how to make sure people escape arguments passing into that function. I'd
like to have something along the lines of Phabricator's `csprintf` here.
Reviewed By: yfeldblum
Differential Revision:
D3857827
fbshipit-source-id:
8afbc9f1c62c62e0fc91782e11b808145b370933
Mark Williams [Wed, 14 Sep 2016 22:48:52 +0000 (15:48 -0700)]
Make ElfCache follow .gnu_debuglink
Summary:
If you split out debug info into a separate file, folly::Symbolizer
didn't find it, and failed to include file and line number info in
backtraces.
This adds a new open mode which follows the .gnu_debuginfo link, and
uses it from ElfCache and SignalSafeElfCache.
Reviewed By: meyering, luciang
Differential Revision:
D3852311
fbshipit-source-id:
fec9e57378ae59fa1b82d41a163bb9cfcf9ca23c
Christopher Dykes [Wed, 14 Sep 2016 21:12:21 +0000 (14:12 -0700)]
Swap a newly added include of gtest.h with portability/GTest.h
Summary: Because it's needed with the way the portability layer works.
Reviewed By: yfeldblum
Differential Revision:
D3863954
fbshipit-source-id:
3b682dcc2d2799f17c61fe16a4391fc9f0c2b127
Phil Willoughby [Wed, 14 Sep 2016 08:53:09 +0000 (01:53 -0700)]
Test for folly::SingletonThreadLocal
Summary: Validates that we create a distinct singleton for each thread and that each such singleton is destroyed at its owning-thread's exit.
Reviewed By: yfeldblum
Differential Revision:
D3849146
fbshipit-source-id:
af878b32ecfc6c82b866d7a805e1385d74a8a5f5
Michael Lee [Tue, 13 Sep 2016 23:02:19 +0000 (16:02 -0700)]
Remove Benchmark.h dependency in SpookyHash tests
Summary: It would seem the folly/Benchmark.h dependency was to pull in glog/logging.h. Include what you use.
Reviewed By: yfeldblum
Differential Revision:
D3857112
fbshipit-source-id:
8d5bd105e10eee8f5b6e5d377d58ae84d908ed57
Milo Yip [Tue, 13 Sep 2016 18:24:35 +0000 (11:24 -0700)]
Optimize toJson() Performance
Summary:
Removing string::reserve() which causes O(n^2) penalty.
Fixes #477
Closes https://github.com/facebook/folly/pull/478
Reviewed By: yfeldblum
Differential Revision:
D3850509
Pulled By: Orvid
fbshipit-source-id:
ecf44c35b2aedadc5385d23c325cacab1abfd02d
Stepan Palamarchuk [Tue, 13 Sep 2016 02:49:22 +0000 (19:49 -0700)]
Fix recordStack feature in FiberManager
Summary:
It appears that a feature of recording exact stack usage on fibers is broken for a long time.
This diff brings it back.
Reviewed By: andriigrynenko
Differential Revision:
D3847035
fbshipit-source-id:
d9cf532774204b6ac2f40e1b39fd98ea06c08f55
Andrii Grynenko [Tue, 13 Sep 2016 00:53:52 +0000 (17:53 -0700)]
Fix a bug in ObserverManager destructor
Summary: facepaw
Reviewed By: ckwalsh
Differential Revision:
D3853363
fbshipit-source-id:
f863a3e3ae830256f5cff6c56f23254ff6fdee2a
Andrii Grynenko [Tue, 13 Sep 2016 00:53:09 +0000 (17:53 -0700)]
Fix TLObserver copy constructor
Reviewed By: yfeldblum
Differential Revision:
D3853494
fbshipit-source-id:
23f5cadcb55a9307fb2a825fe4c53a03c5978d12
Giuseppe Ottaviano [Mon, 12 Sep 2016 21:00:40 +0000 (14:00 -0700)]
Do not mess with NDEBUG in fbstring
Summary: Temporarily overriding `NDEBUG` can have unexpected side-effects depending on the implementation of `<assert.h>`. Better use an explicit macro for `assert()` that we can explicitly disable.
Reviewed By: Gownta
Differential Revision:
D3850717
fbshipit-source-id:
b1e7fce89ab4120b0224e9c6bf668d983b3d2d31
Christopher Dykes [Sat, 10 Sep 2016 00:44:01 +0000 (17:44 -0700)]
Use the GTest portability headers
Summary:
Switch all of the uses of `<gtest/gtest.h>` to `<folly/portability/GTest.h>`.
This is painful but necessary to get the tests to compile nicely under MSVC.
Reviewed By: yfeldblum
Differential Revision:
D3837300
fbshipit-source-id:
7ae43a5d5ba94c8c24fa23a485f18546416e7191
Christopher Dykes [Sat, 10 Sep 2016 00:44:00 +0000 (17:44 -0700)]
Use the GMock portability header
Summary:
Switch all of the uses of <gmock/gmock.h> to <folly/portability/GMock.h>.
This is painful but necessary to get the tests to compile nicely under MSVC.
Reviewed By: yfeldblum
Differential Revision:
D3837529
fbshipit-source-id:
7221dfea8f2a880919690b5b0601ca91642991ae
Christopher Dykes [Sat, 10 Sep 2016 00:43:59 +0000 (17:43 -0700)]
Add portability headers for GTest and GMock
Summary:
Both the gtest and gmock headers include `<io.h>` on Windows, which conflicts with how the portability headers include it, so a specific include order is required before either of those headers can be included. As that's brittle and messy, create a pair of portability headers for them instead, so that those can be included instead.
This only adds the headers. The switch to using them will be done in a later pair of diffs.
Reviewed By: mzlee
Differential Revision:
D3837221
fbshipit-source-id:
82a273485cdd4536f1153d958c171bfe3ec32e0b
Eric Niebler [Fri, 9 Sep 2016 23:19:15 +0000 (16:19 -0700)]
Refactor basic_fbstring
Summary: Move fbstring_core and basic_fbstring member functions from in-situ to out-of-class inlines and refactor for readability. Remove superfluous dependence on scope_guard.
Reviewed By: ot, luciang
Differential Revision:
D3795250
fbshipit-source-id:
f6edca25d4771181faff9e0a4339bbaffd71a370
Yedidya Feldblum [Fri, 9 Sep 2016 21:44:11 +0000 (14:44 -0700)]
Nomenclature in Synchronized - prefer read to shared
Summary:
[Folly] Nomenclature in `Synchronized` - prefer `read` to `shared`.
`folly::Synchronized` is a higher-level abstraction, so it will use higher-level nomenclature than will lower-level tools like `std::mutex` and `folly::LockTraits`. `shared` describes the lock state, and is used in the C++ libraries. `read` describes the class of operations that calling code is permitted to perform.
Reviewed By: simpkins
Differential Revision:
D3840060
fbshipit-source-id:
4b23eaa391cb59d1eca2bfacf72db89d3c7c591e
Daniel Sommermann [Fri, 9 Sep 2016 20:50:08 +0000 (13:50 -0700)]
Add forwarding gather() function to IOBufQueue
Summary:
I'm working with a parser that requires a certain number of
contiguous bytes to be able to make forward progress. I'm also using
IOBufQueue to receive data from an AsyncReader::ReadCallback with
pre/postallocate. So, I need to call gather() to ensure that the queue's
front IOBuf has the right number of contiguous bytes available.
Reviewed By: djwatson
Differential Revision:
D3838079
fbshipit-source-id:
9f1ec5c86895eb1b2b109f9f145ca42d2dbba4c6
Anirudh Ramachandran [Fri, 9 Sep 2016 17:23:15 +0000 (10:23 -0700)]
Make folly::PasswordCollector::getPassword const
Summary: As in title
Reviewed By: siyengar
Differential Revision:
D3794648
fbshipit-source-id:
f0b7052f34ecce65cf4e21d546d08c7a6b0a8ee3
Michael Lee [Fri, 9 Sep 2016 00:02:37 +0000 (17:02 -0700)]
#if FOLLY_HAVE_INT128_T rather than #ifdef
Summary: Switch from #ifdef to #if so the configuration can define the macro to 0
Reviewed By: yfeldblum, Orvid
Differential Revision:
D3838748
fbshipit-source-id:
e287b07f0fdfdc86c882538e96f2078795b85bfd