Maged Michael [Thu, 7 Dec 2017 22:41:53 +0000 (14:41 -0800)]
FlatCombining: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton
Summary: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton, in preparation for restoring Baton to being single-poster and with blocking wait().
Reviewed By: davidtgoldblatt
Differential Revision:
D6510619
fbshipit-source-id:
eed87beabc1b698080b8e41c746d570d30804ce7
Yedidya Feldblum [Thu, 7 Dec 2017 20:38:59 +0000 (12:38 -0800)]
Fix crash in exception_wrapper::get_exception<>
Summary:
[Folly] Fix crash in `exception_wrapper::get_exception<>`.
When the contained object is unthrown and does not inherit `std::exception`, `get_exception` templated over a type that does not inherit `std::exception` must throw and catch internally and may then return a pointer to the internally thrown object, which has since been deallocated.
Attempting to dereference that pointer is undefined behavior is correctly caught by ASAN as heap-use-after-free.
Fix it by storing objects not inheriting `std::exception` using only the `std::exception_ptr` representation.
As a downside, we no longer get the small-object optimization or the non-throwing optimization for objects which do not inherit `std::exception`. But this is not likely to be terribly concerning in practice.
Reviewed By: ericniebler
Differential Revision:
D6504911
fbshipit-source-id:
0065de911733b5cab87be55e7e4e47f0a9c09140
Yedidya Feldblum [Thu, 7 Dec 2017 20:38:57 +0000 (12:38 -0800)]
Use simpler tags for ctor dispatch in exception_wrapper
Summary:
[Folly] Use simpler tags for ctor dispatch in `exception_wrapper`.
The tags do not need to be `std::integral_constant`, but can just be arbitrary empty types.
Also format the usage sites of the tags.
Reviewed By: ericniebler
Differential Revision:
D6506417
fbshipit-source-id:
377359086d94fc1a8bf8eebf6f8b058ff544829f
Stepan Palamarchuk [Thu, 7 Dec 2017 15:43:57 +0000 (07:43 -0800)]
Fix DCHECKs in IOBufQueue
Summary: glog actually tries to pull srtings from these pointers when the checks fail, which obfuscates the error
Reviewed By: yfeldblum
Differential Revision:
D6506886
fbshipit-source-id:
0283e02aecaa248b84fca1433d3f29f92c37b2e6
Mehdi Bouaziz [Thu, 7 Dec 2017 10:57:26 +0000 (02:57 -0800)]
Removed dead IsSomeVector
Summary: deadcode
Reviewed By: ddrcoder
Differential Revision:
D6498754
fbshipit-source-id:
9dda0ade6928d648b8938a06b185e430753f2318
Andrii Grynenko [Thu, 7 Dec 2017 01:50:03 +0000 (17:50 -0800)]
Fix SignalHandlerTest with ASAN
Reviewed By: yfeldblum
Differential Revision:
D6503762
fbshipit-source-id:
f82b88c401979761053004eadd2e4c6a74f05f9c
Adam Simpkins [Thu, 7 Dec 2017 01:29:32 +0000 (17:29 -0800)]
logging: allow partial updates to log handler settings
Summary:
This updates the LogHandlerConfig code to allow changing the settings on an
existing log handler without listing all of its existing options from scratch.
This also changes the syntax of the basic log handler configuration string to
use a colon to separate the log handler name+type from the options list. In
other words, rather than specifying `default=stream,stream=stderr,async=true`
you now say `default=stream:stream=stderr,async=true`.
The primary motivation for this change is to make it easy for users to switch
the async setting for the default log handler on or off. Callers can now
specify `default:async=true` to easily enable async mode on the default log
handler without having to completely re-list the full settings for the default
handler.
Reviewed By: yfeldblum
Differential Revision:
D6494228
fbshipit-source-id:
52a296f800a5456f0c3aa10546298139c8db52fc
Adam Simpkins [Thu, 7 Dec 2017 01:29:30 +0000 (17:29 -0800)]
logging: split FileHandlerFactory into two classes
Summary:
Split FileHandlerFactory into separate FileHandlerFactory and
StreamHandlerFactory classes, where FileHandlerFactory only handles logging to
files by path name, and StreamHandlerFactory only supports logging to stdout or
stderr.
The primary motivation for this is to allow logging to stdout or stderr in some
cases without allowing arbitrary files to be opened by FileHandlerFactory.
This can be achieved now by registering StreamHandlerFactory but not
FileHandlerFactory. This makes it safer to allow controlling logging
configuration via command line flags even in setuid binaries.
Reviewed By: yfeldblum
Differential Revision:
D6494226
fbshipit-source-id:
a3ec371ca4266424d07dff20be18e6e13c057593
Adam Simpkins [Thu, 7 Dec 2017 01:29:27 +0000 (17:29 -0800)]
logging: add a StandardLogHandlerFactory helper class
Summary:
This moves some of the FileHandlerFactory code out into a new
StandardLogHandlerFactory helper class. This will make it easier in the future
to add new LogHandlerFactory implementations that create StandardLogHandler
objects.
In particular, I plan to use this soon to split FileHandlerFactory into two
separate classes: one for writing to files on disk and a separate class for
writing to stdout or stderr.
Reviewed By: yfeldblum
Differential Revision:
D6494227
fbshipit-source-id:
52e24250d020d21a5395d2a68fa5bd40bb32fbd4
Louis Brandy [Thu, 7 Dec 2017 01:10:31 +0000 (17:10 -0800)]
fix ubsan-detected UMR in TemporaryFile
Summary:
Move constructors are just like any other constructors. `this` is not in a valid state. You aren't supposed to destroy `this` before move-constructing it.
This was detected by ubsan stress runs.
UBSAN report:
```
[ RUN ] TemporaryFile.moveCtor
folly/experimental/TestUtil.cpp:91:20: runtime error: load of value 3, which is not a valid value for type 'bool'
[ failed: 1 ]
````
Reviewed By: yfeldblum
Differential Revision:
D6501500
fbshipit-source-id:
83bd0525ac2c1f5951f70f5596034bb6cb0ca985
Yedidya Feldblum [Wed, 6 Dec 2017 19:36:23 +0000 (11:36 -0800)]
Rename Baton::timed_wait to try_wait_for and try_wait_until
Summary:
[Folly] Rename `Baton::timed_wait` to `try_wait_for` and `try_wait_until`.
For consistency with `std::timed_mutex`'s suite of `lock` member functions, as well as `SaturatingSemaphore` and others.
Reviewed By: davidtgoldblatt
Differential Revision:
D6493714
fbshipit-source-id:
566c6d5dc0f6db4a0308cbd1972d53794f898383
Yedidya Feldblum [Wed, 6 Dec 2017 06:03:40 +0000 (22:03 -0800)]
Move folly/Baton.h to folly/synchronization/
Summary: [Folly] Move `folly/Baton.h` to `folly/synchronization/`.
Reviewed By: phoad, Orvid
Differential Revision:
D6490282
fbshipit-source-id:
66e2d25ffe3275d576b97b81c1987709000f6649
Yedidya Feldblum [Wed, 6 Dec 2017 05:00:02 +0000 (21:00 -0800)]
Use boost::filesystem::current_path in ChangeToTempDir
Summary:
[Folly] Use `boost::filesystem::current_path` in `ChangeToTempDir`.
It will report failures noisily.
Reviewed By: pixelb
Differential Revision:
D6493243
fbshipit-source-id:
423dc0e3a46781e9af42fee69060d31085f1a7c6
Liang Zhu [Wed, 6 Dec 2017 00:45:14 +0000 (16:45 -0800)]
exclude Unix Domain Socket from enableTTLBANotifications
Reviewed By: yfeldblum
Differential Revision:
D6482348
fbshipit-source-id:
5b2d3c984b761a54d5a9eba48953c556ed22cda6
Pingjia Shan [Tue, 5 Dec 2017 21:03:44 +0000 (13:03 -0800)]
Fix RequestContext held too long issue in EventBase
Summary:
Upon debugging for the attached task, it appears to me the problem are in two places:
. After a callback has been run, context_ wasn't reset
. After a callback has been canceled, context_ wasn't reset
In this diff:
. Fix these two places.
. Updating unit tests to cover these two cases.
Reviewed By: yfeldblum
Differential Revision:
D6465788
fbshipit-source-id:
85b3b29dc80c9f3971c85f302385d41ded44fa0e
Aaryaman Sagar [Tue, 5 Dec 2017 18:54:55 +0000 (10:54 -0800)]
Split get_default() into two for deferred default construction and added forwarding to speed up default element construction with rvalues
Summary:
As it stood currently folly::get_default() would unnecessarily
construct a value into the third parameter, which was unnecessary in the code
path where the element was found in the map. Also the default value can be
forwarded to the return type in the code path where the element is not found
and an rvalue parameter is passed as the default value
Reviewed By: yfeldblum
Differential Revision:
D6390315
fbshipit-source-id:
ef692b827d5a36751b4eb1e12042869e8fbba2e5
Dave Watson [Tue, 5 Dec 2017 17:14:34 +0000 (09:14 -0800)]
bitreverse
Summary:
Adds a bitreverse function to Bits.h
AFAIK there is no great way to do this on x86, takes ~4-5ns.
arm has a single instruction we can drop in in the future.
Reviewed By: yfeldblum
Differential Revision:
D6459283
fbshipit-source-id:
129db196b2fac95386f601ae57843aa87523b915
Liang Zhu [Tue, 5 Dec 2017 11:49:11 +0000 (03:49 -0800)]
exclude Unix Domain Socket from setErrMessageCB
Summary:
In the latest stable kernel 4.14.3 as of 2017-12-04, unix domain socket does not support MSG_ERRQUEUE. So recvmsg(MSG_ERRQUEUE) will read application data from unix doamin socket as error message, which breaks the message flow in application. This diff disable setErrMessageCB for Unix Domain Socket.
Both [[ https://github.com/torvalds/linux/blob/master/net/ipv4/tcp.c#L1782 | tcp_recvmsg ]] and [[ https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L1571 | udp_recvmsg ]] will check flag MSG_ERRQUEUE and read from error queue. However, there is nothing about MSG_ERRQUEUE in [[ https://github.com/torvalds/linux/blob/master/net/unix/af_unix.c#L2249 | af_unix ]].
Reviewed By: yfeldblum
Differential Revision:
D6479465
fbshipit-source-id:
eba2c8650e96466f2b361a42ddf90053d65f19bd
Yedidya Feldblum [Tue, 5 Dec 2017 04:57:02 +0000 (20:57 -0800)]
Add missing check against SemiFuture in FutureBase ctor
Summary: [Folly] Add missing check against `SemiFuture` in `FutureBase` ctor.
Reviewed By: LeeHowes
Differential Revision:
D6469021
fbshipit-source-id:
0ca31461afee4bf64725e065a8cb4d5e933d2013
Nathan Bronson [Tue, 5 Dec 2017 00:39:00 +0000 (16:39 -0800)]
better error handling in MemoryIdler for inside jails
Summary:
/proc may be unavailable in restricted environments, which can
cause pthread_getattr_np to fail. This diff adds checking to the return
code from that function.
Reviewed By: yfeldblum
Differential Revision:
D6473789
fbshipit-source-id:
de016d32f29edf8410808dfb491987692f39e768
Stepan Palamarchuk [Mon, 4 Dec 2017 23:08:24 +0000 (15:08 -0800)]
Improve QueueAppender/IOBufQueue performance
Summary:
Currently QueueAppender needs to follow a chain of 4 indirections (QueueAppender->IOBufQueue->IOBuf(head)->IOBuf(tail)->data).
This diff adds a cache of writable tail range in IOBufQueue and allows it to be placed externally.
Before this diff on hot path QueueAppender::write<signed char> was ~167 bytes of code (with majority being actually executed), after this diff it's down to ~30 bytes:
0x0000000000419d10 <+0>: mov (%rdi),%rax
0x0000000000419d13 <+3>: cmp %rax,0x8(%rdi)
0x0000000000419d17 <+7>: je 0x419d28 <folly::io::QueueAppender::write<signed char>(signed char)+24>
0x0000000000419d19 <+9>: mov %sil,(%rax)
0x0000000000419d1c <+12>: addq $0x1,(%rdi)
0x0000000000419d20 <+16>: retq
0x0000000000419d21 <+17>: nopl 0x0(%rax)
0x0000000000419d28 <+24>: movsbl %sil,%esi
0x0000000000419d2c <+28>: jmpq 0x419ca0 <folly::io::QueueAppender::writeSlow<signed char>(signed char)>
With this diff, Thrift serialization performance is improved up to 2x with production workloads (2x for compact, 3x for binary).
Thrift benchmark output:
Before:
============================================================================
thrift/lib/cpp2/test/ProtocolBench.cpp relative time/iter iters/s
============================================================================
BinaryProtocol_write_Empty 58.05ns 17.23M
BinaryProtocol_write_SmallInt 75.17ns 13.30M
BinaryProtocol_write_BigInt 74.60ns 13.41M
BinaryProtocol_write_SmallString 85.12ns 11.75M
BinaryProtocol_write_BigString 802.96ns 1.25M
BinaryProtocol_write_BigBinary 174.69ns 5.72M
BinaryProtocol_write_LargeBinary 171.81ns 5.82M
BinaryProtocol_write_Mixed 130.97ns 7.64M
BinaryProtocol_write_SmallListInt 123.99ns 8.06M
BinaryProtocol_write_BigListInt 40.72us 24.56K
BinaryProtocol_write_BigListMixed 784.78us 1.27K
BinaryProtocol_write_LargeListMixed 98.84ms 10.12
CompactProtocol_write_Empty 64.38ns 15.53M
CompactProtocol_write_SmallInt 76.74ns 13.03M
CompactProtocol_write_BigInt 83.62ns 11.96M
CompactProtocol_write_SmallString 86.05ns 11.62M
CompactProtocol_write_BigString 786.18ns 1.27M
CompactProtocol_write_BigBinary 184.91ns 5.41M
CompactProtocol_write_LargeBinary 182.12ns 5.49M
CompactProtocol_write_Mixed 120.89ns 8.27M
CompactProtocol_write_SmallListInt 119.74ns 8.35M
CompactProtocol_write_BigListInt 43.76us 22.85K
CompactProtocol_write_BigListMixed 595.90us 1.68K
CompactProtocol_write_LargeListMixed 72.80ms 13.74
============================================================================
After:
============================================================================
thrift/lib/cpp2/test/ProtocolBench.cpp relative time/iter iters/s
============================================================================
BinaryProtocol_write_Empty 65.97ns 15.16M
BinaryProtocol_write_SmallInt 72.31ns 13.83M
BinaryProtocol_write_BigInt 72.67ns 13.76M
BinaryProtocol_write_SmallString 77.56ns 12.89M
BinaryProtocol_write_BigString 782.07ns 1.28M
BinaryProtocol_write_BigBinary 179.69ns 5.57M
BinaryProtocol_write_LargeBinary 182.62ns 5.48M
BinaryProtocol_write_Mixed 91.62ns 10.92M
BinaryProtocol_write_SmallListInt 96.22ns 10.39M
BinaryProtocol_write_BigListInt 19.65us 50.90K
BinaryProtocol_write_BigListMixed 245.69us 4.07K
BinaryProtocol_write_LargeListMixed 46.56ms 21.48
CompactProtocol_write_Empty 74.44ns 13.43M
CompactProtocol_write_SmallInt 80.35ns 12.45M
CompactProtocol_write_BigInt 85.30ns 11.72M
CompactProtocol_write_SmallString 82.61ns 12.10M
CompactProtocol_write_BigString 784.77ns 1.27M
CompactProtocol_write_BigBinary 193.20ns 5.18M
CompactProtocol_write_LargeBinary 192.53ns 5.19M
CompactProtocol_write_Mixed 99.78ns 10.02M
CompactProtocol_write_SmallListInt 104.77ns 9.54M
CompactProtocol_write_BigListInt 25.62us 39.03K
CompactProtocol_write_BigListMixed 272.42us 3.67K
CompactProtocol_write_LargeListMixed 38.32ms 26.09
============================================================================
QueueAppender Benchmark output (although not very representative due to a tight loop):
Before:
============================================================================
folly/io/test/QueueAppenderBenchmark.cpp relative time/iter iters/s
============================================================================
write_uint8 10.50us 95.20K
write_uint16 5.48us 182.49K
write_uint32 2.73us 366.22K
push_64b 9.77us 102.36K
push_1024b 112.87us 8.86K
append 64.21us 15.57K
preallocate_postallocate_1b 16.34us 61.19K
preallocate_postallocate_4b 15.56us 64.26K
preallocate_postallocate_32b 22.17us 45.11K
preallocate_postallocate_256b 149.55us 6.69K
============================================================================
After:
============================================================================
folly/io/test/QueueAppenderBenchmark.cpp relative time/iter iters/s
============================================================================
write_uint8 8.86us 112.81K
write_uint16 3.91us 255.68K
write_uint32 2.08us 481.78K
push_64b 8.24us 121.39K
push_1024b 115.50us 8.66K
append 67.52us 14.81K
preallocate_postallocate_1b 13.86us 72.17K
preallocate_postallocate_4b 11.67us 85.71K
preallocate_postallocate_32b 20.35us 49.14K
preallocate_postallocate_256b 148.57us 6.73K
============================================================================
Reviewed By: yfeldblum
Differential Revision:
D6427749
fbshipit-source-id:
8495cc74b6106b15d201e37533ae4c0a1abc9d74
Dave Watson [Mon, 4 Dec 2017 20:57:07 +0000 (12:57 -0800)]
add AtFork::unregisterHandler
Summary: Adds an AdFork::unregisterHandler command, such that we can register and unregister fork handlers.
Reviewed By: yfeldblum
Differential Revision:
D6439796
fbshipit-source-id:
b710152f8cf98371c330b484cd5fe42de44e74ac
Maged Michael [Mon, 4 Dec 2017 19:07:30 +0000 (11:07 -0800)]
Unbounded queue
Summary:
UnboundedQueue supports:
- SPSC, MPSC, SCMP, MPMC
- Non-waiting, waiting, and timed consumer operations.
- Producers never wait or fail (unless out-of-memory).
- Memory usage grows and shrinks dynamically
```
/// UnboundedQueue supports a variety of options for unbounded
/// dynamically expanding an shrinking queues, including variations of:
/// - Single vs. multiple producers
/// - Single vs. multiple consumers
/// - Blocking vs. spin-waiting
/// - Non-waiting, timed, and waiting consumer operations.
/// Producer operations never wait or fail (unless out-of-memory).
///
/// Template parameters:
/// - T: element type
/// - SingleProducer: true if there can be only one producer at a
/// time.
/// - SingleConsumer: true if there can be only one consumer at a
/// time.
/// - MayBlock: true if consumers may block, false if they only
/// spins. A performance tuning parameter.
/// - LgSegmentSize (default 8): Log base 2 of number of elements per
/// segment. A performance tuning parameter. See below.
///
/// When to use UnboundedQueue:
/// - If a small bound may lead to deadlock or performance degradation
/// under bursty patterns.
/// - If there is no risk of the queue growing too much.
///
/// When not to use UnboundedQueue:
/// - If there is risk of the queue growing too much and a large bound
/// is acceptable, then use DynamicBoundedQueue.
/// - If the queue must not allocate on enqueue or it must have a
/// small bound, then use fixed-size MPMCQueue or (if non-blocking
/// SPSC) ProducerConsumerQueue.
///
/// Template Aliases:
/// USPSCQueue<T, MayBlock, LgSegmentSize>
/// UMPSCQueue<T, MayBlock, LgSegmentSize>
/// USPMCQueue<T, MayBlock, LgSegmentSize>
/// UMPMCQueue<T, MayBlock, LgSegmentSize>
///
/// Functions:
/// Producer operations never wait or fail (unless OOM)
/// void enqueue(const T&);
/// void enqueue(T&&);
/// Adds an element to the end of the queue.
///
/// Consumer operations:
/// void dequeue(T&);
/// Extracts an element from the front of the queue. Waits
/// until an element is available if needed.
/// bool try_dequeue(T&);
/// Tries to extracts an element from the front of the queue
/// if available. Returns true if successful, false otherwise.
/// bool try_dequeue_until(T&, time_point& deadline);
/// Tries to extracts an element from the front of the queue
/// if available until the specified deadline. Returns true
/// if successful, false otherwise.
/// bool try_dequeue_for(T&, duration&);
/// Tries to extracts an element from the front of the queue
/// if available for for the specified duration. Returns true
/// if successful, false otherwise.
///
/// Secondary functions:
/// size_t size();
/// Returns an estimate of the size of the queue.
/// bool empty();
/// Returns true only if the queue was empty during the call.
/// Note: size() and empty() are guaranteed to be accurate only if
```
Reviewed By: djwatson
Differential Revision:
D6157613
fbshipit-source-id:
db423f86d1d0604d22f6b9c71ea0ed08be32e2a1
Anirudh Ramachandran [Mon, 4 Dec 2017 07:56:21 +0000 (23:56 -0800)]
Add wrapper for getting X509_digest from a cert
Summary:
X509_digest, or the "fingerprint" of the cert, is useful to identify
certificates with the same CN/SANs. This adds an API to get the SHA1 or SHA256
digest of a cert.
Reviewed By: yfeldblum
Differential Revision:
D6457917
fbshipit-source-id:
148a5a2270e938b39065e00d7947c2fc57cd7f75
Alastair Daivis [Sun, 3 Dec 2017 01:35:33 +0000 (17:35 -0800)]
Statically allocate futex array
Summary:
AFAICT this is currently the only thing preventing the non-dynamic version of Folly's MPMCQueue from being allocation-free on enqueue and dequeue operations. Allocating this with a constant expression should move the allocation from runtime to link (possibly compile?) time, which will let us use it in allocation sensitive contexts.
Feel free to suggest other reviewers, I couldn't find an existing folly reviewer group.
Reviewed By: yfeldblum, agola11, nbronson
Differential Revision:
D6447848
fbshipit-source-id:
86b84b19d62f1e1bcecdb9e757a6dfa90597b084
David Goode [Sat, 2 Dec 2017 03:56:14 +0000 (19:56 -0800)]
allow fiber executor to override fiber options
Summary: As in title, for eg larger stack size
Reviewed By: yfeldblum
Differential Revision:
D6466762
fbshipit-source-id:
303e8732c8bfb64fbd8619163779942fe5f8b5a8
Adam Simpkins [Sat, 2 Dec 2017 02:05:54 +0000 (18:05 -0800)]
add makeSystemError*() helper functions
Summary:
Add makeSystemError() helper functions that are similar to the existing
throwSystemError() functions but just return the exception rather than throwing
it. This is helpful for callers using folly::Expected or folly::Future, where
they need to return an exception type rather than throwing it.
This also includes comments about the fact that this code is using the wrong
error category. The C++ standard indicates that std::generic_category() for
POSIX errno values. I am not fixing this as part of this diff, since this
change has the potential to break existing users, and would need a fair amount
of testing first.
Reviewed By: yfeldblum
Differential Revision:
D6456771
fbshipit-source-id:
4724b51b8d4a7f513ae70ea1b0c4f0516cfc205f
Adam Simpkins [Sat, 2 Dec 2017 02:01:59 +0000 (18:01 -0800)]
logging: fix compilation error on older C++ compilers
Summary:
Add an explicit `std::move()` around the return value of
`logConfigToDynamic()`. This explicit move is required pre-C++14,
but is not required after DR 1579:
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579
Newer versions of g++ and clang no longer require the move. However gcc
versions earlier than 5.0 do require it.
Reviewed By: yfeldblum
Differential Revision:
D6466447
fbshipit-source-id:
1b15934490d4966e9b3e5612e0e2ecbb43c979ca
Adam Simpkins [Sat, 2 Dec 2017 00:53:51 +0000 (16:53 -0800)]
logging: update initialization code to use the new LogConfig logic
Summary:
Replace the initLoggingGlogStyle() function with a more generic initLogging()
function that accepts a log config string to be parsed with parseLogConfig().
Reviewed By: bolinfest, yfeldblum
Differential Revision:
D6342086
fbshipit-source-id:
fb1bffd11f190b70e03e2ccbf2b30be08d655242
Igor Sugak [Fri, 1 Dec 2017 19:19:44 +0000 (11:19 -0800)]
unify FOLLY_SANITIZE existing uses and add a few new
Reviewed By: yfeldblum
Differential Revision:
D6453972
fbshipit-source-id:
fa083c0b782188f5f45a2a354045aee03e528829
Adam Simpkins [Fri, 1 Dec 2017 19:18:17 +0000 (11:18 -0800)]
implement chrono conversions for unusual duration types
Summary:
Implement conversions between std::chrono::duration and POSIX-style time
structures even when neither the numerator nor the denominator of the duration
ratio are 1.
Both of these are done by first converting to an intermediate type where the
numerator is 1, and then using the conversion routines for that case.
Reviewed By: yfeldblum
Differential Revision:
D6366647
fbshipit-source-id:
8f9495fb4101cac6d8b4cf0353a679107007b298
Dave Watson [Fri, 1 Dec 2017 15:09:43 +0000 (07:09 -0800)]
Move threadlocal_detail::Atfork to its own file
Summary: As title
Reviewed By: yfeldblum
Differential Revision:
D6440723
fbshipit-source-id:
3168d7bb616ae0ff3fe42f7584c5a255c4953875
Bin Liu [Thu, 30 Nov 2017 23:52:54 +0000 (15:52 -0800)]
fix ASAN build
Summary:
Fix bug in non-jemalloc build.
(Note: this ignores all push blocking failures!)
Reviewed By: igorsugak
Differential Revision:
D6451342
fbshipit-source-id:
44725d4e0c685a59325fa3d08877d0f62120c5c2
Yedidya Feldblum [Thu, 30 Nov 2017 22:56:07 +0000 (14:56 -0800)]
Let SaturatingSemaphore::try_wait be non-const and add ready
Summary:
[Folly] Let `SaturatingSemaphore::try_wait` be non-`const` and add `ready`.
For internal API consistency.
Reviewed By: magedm
Differential Revision:
D6450089
fbshipit-source-id:
65b9b723672521710a69719b192eb2922a27b778
Igor Sugak [Thu, 30 Nov 2017 22:48:24 +0000 (14:48 -0800)]
fix build when sanitizers are enabled and jemalloc is disabled
Summary: Add preprocessor checks to guard jemalloc headers if any sanitizer is enabled.
Reviewed By: yfeldblum
Differential Revision:
D6441578
fbshipit-source-id:
85364fca5af33c95cd05f232d1e3f9cbac9a4120
Yedidya Feldblum [Thu, 30 Nov 2017 06:13:39 +0000 (22:13 -0800)]
Some fixes for custom conversions of enums
Summary:
[Folly] Some fixes for custom conversions of enums.
Of note, `to` was defined for enum -> all conversions, including enum -> string conversions, but we actually want enum -> string conversions to be done via ADL-discovered toAppend.
Reviewed By: ot
Differential Revision:
D6411250
fbshipit-source-id:
852b64309e6adf1c68e5153635cb29632e2d86d4
Maged Michael [Thu, 30 Nov 2017 05:12:16 +0000 (21:12 -0800)]
Saturating semaphore
Summary:
SaturatingSemaphore is a flag that allows:
- multiple concurrent posters
- multiple concurrent waiters
- idempotent posting
- non-destructive waiting
- blocking and spinning
- pre-block spin time control
```
/// SaturatingSemaphore is a flag that allows concurrent posting by
/// multiple posters and concurrent non-destructive waiting by
/// multiple waiters.
///
/// A SaturatingSemaphore allows one or more waiter threads to check,
/// spin, or block, indefinitely or with timeout, for a flag to be set
/// by one or more poster threads. By setting the flag, posters
/// announce to waiters (that may be already waiting or will check
/// the flag in the future) that some condition is true. Posts to an
/// already set flag are idempotent.
```
Reviewed By: djwatson
Differential Revision:
D6379704
fbshipit-source-id:
59aed76caa2d159639e75425a778a9c63f18f375
Igor Sugak [Thu, 30 Nov 2017 02:14:04 +0000 (18:14 -0800)]
add FOLLY_SANITIZE macro
Summary: Most of the time we want to guard some code when any of the sanitizers are enabled not just one.
Reviewed By: yfeldblum
Differential Revision:
D6429720
fbshipit-source-id:
7157a0a13f2b2acabd36a19c61b9fdbf38ba96b8
Adam Simpkins [Thu, 30 Nov 2017 01:35:21 +0000 (17:35 -0800)]
logging: add a LogConfig::update() method
Summary:
Add a method for merging the settings from two LogConfig objects.
This allows LogConfig objects to be merged before applying them to the
LoggerDB. The effects are the same as two sequential LoggerDB::updateConfig()
calls, but without having to apply the intermediate state to the LoggerDB.
Reviewed By: bolinfest
Differential Revision:
D6342085
fbshipit-source-id:
0f8a1b7d8d195a80bc74342444dd3152d331fcb6
Adam Simpkins [Thu, 30 Nov 2017 01:35:19 +0000 (17:35 -0800)]
logging: add an assert for the FLAG_INHERIT bit
Summary:
Add a static_assert() to check that the FLAG_INHERIT bit does not conflict with
valid LogLevel values.
Reviewed By: bolinfest
Differential Revision:
D6341242
fbshipit-source-id:
cec284623c8a612a3c54f1b593d169310790616e
Adam Simpkins [Thu, 30 Nov 2017 01:35:17 +0000 (17:35 -0800)]
logging: remove comment about LogLevel::ERROR
Summary:
The logging code used to support a LogLevel::ERROR value on non-Windows
platforms. I removed it in
D5288600 to make all platforms consistent, but I
forgot to remove this comment describing it.
Reviewed By: bolinfest
Differential Revision:
D6341244
fbshipit-source-id:
054427d342066f354c859b12611b907dc2d4fa35
Adam Simpkins [Thu, 30 Nov 2017 01:35:16 +0000 (17:35 -0800)]
logging: add LoggerDB::updateConfig() and resetConfig()
Summary: Add methods for applying config changes from a LogConfig object to the LoggerDB.
Reviewed By: bolinfest
Differential Revision:
D6200564
fbshipit-source-id:
a25eb99e84b2885bf6853e2222db0d7432a6c37b
Adam Simpkins [Thu, 30 Nov 2017 01:35:13 +0000 (17:35 -0800)]
logging: add a LoggerDB::getConfig() method
Summary: Add a method to get the current LogConfig state from the LoggerDB.
Reviewed By: bolinfest
Differential Revision:
D6200596
fbshipit-source-id:
3bc57d498a5d25d19099d861376d71ea9f7e4039
Adam Simpkins [Thu, 30 Nov 2017 01:35:11 +0000 (17:35 -0800)]
logging: don't clamp the log level to DFATAL in debug builds
Summary:
Remove the logic that clamps the log level to DFATAL in debug builds.
This behavior logically makes some sense, but results in subtle and potentially
confusing behavior changes. It seems confusing that after calling
`setLevel(LogLevel::MAX_LEVEL)` on a log category, calling `getLevel()` on that
object would return `DFATAL` rather than `MAX_LEVEL`. We also weren't clamping
the level to DFATAL consistently: `setLevel()` would clamp the value, but on
construction the initial level was still set to MAX_LEVEL rather than DFATAL.
This resulted in some issues when implementing `LoggerDB::getConfig()` since we
could not consistently detect if a log category was using the default level
settings. Rather than fix this inconsistency it seems better to simply remove
this clamping behavior.
This means that it is possible for users to disable DFATAL log messages even in
debug builds if they really want to. Previously this was only allowed in
release builds. This protection doesn't really seem all that
valuable--presumably most developers won't really want to do this, and if they
really do request this configuration it doesn't seem all that bad to honor it.
Reviewed By: bolinfest, yfeldblum
Differential Revision:
D6200569
fbshipit-source-id:
83ef8e2e4d3b61bc5b105038cbe3132979e9ac67
Adam Simpkins [Thu, 30 Nov 2017 01:35:09 +0000 (17:35 -0800)]
logging: add a LogHandler::getConfig() method
Summary:
Add a method to LogHandler to return its current configuration. This will
make it possible to query the LoggerDB for its current configuration state.
Reviewed By: bolinfest
Differential Revision:
D6200563
fbshipit-source-id:
2b8b9752bbeb26c8aac28d1a73b7e2312fd198c8
Adam Simpkins [Thu, 30 Nov 2017 01:35:08 +0000 (17:35 -0800)]
logging: add a LogHandler registry to LoggerDB
Summary:
Update the LoggerDB to track a list of LogHandlers by name, and
LogHandlerFactories by handler type.
This will be needed to support updating the LoggerDB configuration from a
LogConfig object.
Reviewed By: bolinfest
Differential Revision:
D6200562
fbshipit-source-id:
e365b4e0df65aa5aaa34e118eb3cee9c9c45cb05
Yedidya Feldblum [Wed, 29 Nov 2017 20:12:01 +0000 (12:12 -0800)]
Synchronize coupled caches in folly::threadlocal_detail::StaticMeta
Summary:
[Folly] Synchronize coupled caches in `folly::threadlocal_detail::StaticMeta`.
The caches should be set together, and only together, because they are coupled. This prevents bugs where one function that sets one cache but not the other cache is inlined into the caller in one module, and another function that reads both caches is inlined into the caller in another module.
Reviewed By: djwatson
Differential Revision:
D6435175
fbshipit-source-id:
846c4972b40e525f2c04da6e6609c2ad54f674c0
Adam Simpkins [Wed, 29 Nov 2017 01:44:55 +0000 (17:44 -0800)]
logging: add a FileHandlerFactory class
Summary:
Add a new LogHandlerFactory interface for creating LogHandler objects from a
LogHandlerConfig.
Also add an initial FileHandlerFactory implementation capable of creating
LogHandler objects that write to a file descriptor.
Reviewed By: bolinfest
Differential Revision:
D6200567
fbshipit-source-id:
14b86fc14ad475223aa4b57d45c40638b48c7594
Andrii Nakryiko [Wed, 29 Nov 2017 00:48:14 +0000 (16:48 -0800)]
Add unit tests for opening non-ELF files.
Summary:
Just testing that ElfFile returns expected error code and message for
non-ELF files.
Depends on
D6410210
Reviewed By: yfeldblum
Differential Revision:
D6418365
fbshipit-source-id:
aaab3b9f3ca1a12d384ae98a1772b7f640115192
Sarang Masti [Tue, 28 Nov 2017 23:51:10 +0000 (15:51 -0800)]
Fix incorrect usages of folly::Synchronized
Summary:
We need to keep LockedPtr returned by rlock/wlock alive for the entire duration
of the loop. Else we are working on a snapshot of the data structure, which
might not be what we want.
Reviewed By: aary, ot, luciang
Differential Revision:
D6426613
fbshipit-source-id:
294c0a7b9d756f3116d5cdb3a7b3678f693e0754
Adam Simpkins [Tue, 28 Nov 2017 20:02:02 +0000 (12:02 -0800)]
logging: convert assert() checks to FOLLY_SAFE_DCHECK()
Summary:
Replace all assert() checks in the folly logging code with
FOLLY_SAFE_DCHECK().
Reviewed By: yfeldblum
Differential Revision:
D6422885
fbshipit-source-id:
5cb12dd59f2fe5d346f15b9d317abede8654a879
Alexander Kabaev [Tue, 28 Nov 2017 19:42:20 +0000 (11:42 -0800)]
Unbreak folly on x86-32
Summary:
_mm_cvtsi64_si128 and _mm_cvtsi128_si64 intrinsics are available on
x86_64 only, so do not enable SSE instructions when compiling for 32 bits.
Reviewed By: yfeldblum
Differential Revision:
D6422271
fbshipit-source-id:
a3c5cf3f2ee4b47d5a1edcf93cfff3d1f3fe907e
Yedidya Feldblum [Tue, 28 Nov 2017 06:18:54 +0000 (22:18 -0800)]
Remove some custom test main()'s in folly/experimental/symbolizer/
Summary:
[Folly] Remove some custom test `main()`'s in `folly/experimental/symbolizer/`.
In one of the two cases, we need to do some extra work to give `ElfFile` knowledge of the program path.
Reviewed By: ot, Orvid, philippv, luciang
Differential Revision:
D6420517
fbshipit-source-id:
11c694a8342abb6fff05c368f3cda2e6287ccb4b
Pingjia Shan [Tue, 28 Nov 2017 06:10:59 +0000 (22:10 -0800)]
Reset context shared_ptr in AsyncTimeout::cancelTimeout()
Summary:
This seems to fix issue in the attached task.
`context_` is set in `scheduleTimeout()` and never gets reset. So when an AsyncTimeout object is held across requests by objects like `IOThreadPool0` in I/O Thread Pool used by wangle/acceptor/ConnectionManager, RequestContext object created for the last request is leaked until `IOThreadPool0` handles another request and overrides `context_`. In the issue described in attached task, unit test has single request, next request never comes in, RequestContext doesn't end until test service stops, logging feature relying on RequestContext dtor doesn't get called in time, and thus unit test fails.
Reviewed By: yfeldblum
Differential Revision:
D6402268
fbshipit-source-id:
200c6d358dfa6d7d9aa68ab05f6f1c7f4117b0ec
Adam Simpkins [Tue, 28 Nov 2017 05:30:58 +0000 (21:30 -0800)]
logging: fix unused variable warning in non-debug builds
Summary:
Use FOLLY_SAFE_DCHECK() rather than assert() to avoid compiler errors about
this variable being unused in builds where assert() statements are disabled.
Reviewed By: bolinfest, yfeldblum
Differential Revision:
D6421978
fbshipit-source-id:
ab1019fa1fa420856e1bf3bc39c441662c4efd75
Tom Jackson [Tue, 28 Nov 2017 01:13:37 +0000 (17:13 -0800)]
Use "auto" with GENERATOR instead of std::function
Summary: Now that we're using C++14 more broadly, we should use it to improve generator performance. This speeds up a microbenchmark >5x by removing type erasure.
Reviewed By: philippv
Differential Revision:
D6398730
fbshipit-source-id:
5809058a3b5ff0e66fd4b1e8954698944e1a7d09
Xiangyu Bu [Mon, 27 Nov 2017 23:37:45 +0000 (15:37 -0800)]
Revise API to load cert/key in SSLContext.
Summary:
When loading cert/key pair, order matters:
(a) Wrong key will fail to load if a cert is loaded;
(b) Wrong cert will succeed to load even if a private key is loaded.
So this diff adds:
(1) SSLContext::checkPrivateKey() -- must call for case (b).
(2) SSLContext::loadCertKeyPairFromBufferPEM() -- use this if one loads both cert and key. Guaranteed to throw if cert/key mismatch.
Reviewed By: yfeldblum
Differential Revision:
D6416280
fbshipit-source-id:
8ae370883d46e9b5afb69c506c09fbf7ba82b1b9
Andrii Nakryiko [Mon, 27 Nov 2017 19:31:18 +0000 (11:31 -0800)]
Fix ElfFile crashing when opening short (<64 bytes) files.
Summary:
ElfFile in init() assumes file is long enough to contain complete
ElfHeader, which is 64 bytes long. This is not true for valid non-ELF files like
short scripts. They shouldn't cause crash.
Reviewed By: myreg
Differential Revision:
D6410210
fbshipit-source-id:
28fd017d8de17c431d7d006a1655ade8a95994bd
Dave Watson [Mon, 27 Nov 2017 18:50:26 +0000 (10:50 -0800)]
Use hazptr_local and hazptr_array
Summary: Use newest hazptr hotness in concurrenthashmap. Shaves ~10% off of the single-thread find performance.
Reviewed By: magedm
Differential Revision:
D6259947
fbshipit-source-id:
7ecf99d38fdf8e311fca3313137e0fca5af3f165
Bin Liu [Mon, 27 Nov 2017 06:07:41 +0000 (22:07 -0800)]
make `usingJEMalloc()` a constexpr when USE_JEMALLOC is defined
Summary: This would save some function calls and reduce code size if we know we will use jemalloc.
Reviewed By: yfeldblum
Differential Revision:
D6411810
fbshipit-source-id:
9db471678e473945f5f00354c0a1285ecdb91de6
Giuseppe Ottaviano [Sun, 26 Nov 2017 22:43:06 +0000 (14:43 -0800)]
Add folly::hasher support for floating point types
Summary:
Move `folly::hasher` closer to feature parity with `std::hash`.
This is in order to replace some instances of `folly::hash::hash_combine(...)` with `folly::Hash()(...)` (`std::hash` is the identity for integers, which makes it an unsafe default for more sophisticated hash data structures, including open-addressing hash tables).
The implementation is similar to `libstdc++`'s implementation, in that we handle separately the `0` case, because `0` and `-0` have different binary representations but are equal according to `operator==`, and hash the bytes otherwise. It is probably a little faster than `libstdc++`'s implementation, that delegates a out-of-line Murmur hash routine for arbitrary buffers, while this uses a minimal inlineable machine word hashing routine.
Reviewed By: yfeldblum
Differential Revision:
D6410713
fbshipit-source-id:
86d9e4ed8da04fffe283949825852e539ec7d5cf
Yedidya Feldblum [Thu, 23 Nov 2017 19:59:32 +0000 (11:59 -0800)]
QueuedImmediateExecutor tweaks
Summary:
[Folly] `QueuedImmediateExecutor` tweaks.
* Add a leaky meyers singleton instance.
* Make the thread-local queue an instance variable. Callers which want the singleton thread-local queue can use the singleton executor instance instance, for the same effect.
* Simplify the body of `add`, and perform the thread-local lookup only once per invocation.
Reviewed By: djwatson
Differential Revision:
D6399067
fbshipit-source-id:
03904885a70c4b943141bd83868414d27232fd6a
Fuat Geleri [Wed, 22 Nov 2017 22:20:42 +0000 (14:20 -0800)]
Enable EventBase switching for AsyncSocket even if it has registered events
Summary:
AsyncSocket will return isDetachable() -> false if there is a read callback set.
So,
- ignore event registration status
- unregister the events when detaching and
- register them back when attaching again.
Reviewed By: afrind, eduardo-elizondo
Differential Revision:
D6331787
fbshipit-source-id:
fca4e6101f2d5666edbafa09116aa50f34cb084d
Guo Xiao [Wed, 22 Nov 2017 19:19:03 +0000 (11:19 -0800)]
Link to unwind when using symbolizer
Summary:
Fix #673
Closes https://github.com/facebook/folly/pull/715
Reviewed By: meyering
Differential Revision:
D6394731
Pulled By: yfeldblum
fbshipit-source-id:
fb92828d0c633278601125dbc64cadf0cdbaa7fe
Dave Watson [Wed, 22 Nov 2017 16:16:04 +0000 (08:16 -0800)]
Add a fast path to folly::ThreadLocal
Summary:
Currently folly::ThreadLocal[Ptr] is pretty heavy-weight for a get():
1) call instance(), take a static init guard, branch
2) call getThreadEntry, check if thread_local is not null, branch
3) check if id < threadEntry->capacity, branch
4) Finally, return threadEntry->elements[id]
If we have real thread_locals, we can do better by caching the capacity directly,
combining all three checks:
1) checkif id < threadLocalCapacityCheck, branch. If not, do slow path.
2) return threadEntry->elements[id]. Threadentry is never null if capacity > 0, and
instance() setup work is called during the first getThreadEntry call when threadlocalcapacity == 0.
Reviewed By: yfeldblum
Differential Revision:
D6379878
fbshipit-source-id:
4fc7564bbb2f319d65875124026aef28d910ef06
Andrii Grynenko [Wed, 22 Nov 2017 04:58:05 +0000 (20:58 -0800)]
Fix ASAN failure in FutureDAG test
Summary: dogscience
Reviewed By: yfeldblum
Differential Revision:
D6387604
fbshipit-source-id:
3e33da3e3c6801d6e2186637c8fb6adaf1eef6f6
Aaron Orenstein [Tue, 21 Nov 2017 22:02:18 +0000 (14:02 -0800)]
Add -Werror to -Wshadow-local and -Wshadow-compatible-local warning check
Summary:
Without -Werror Apple LLVM will print a warning about -Wshadow-local
and -Wshadow-compatible-local but won't fail. This causes problems for
downstream code which uses -Werror.
Reviewed By: meyering
Differential Revision:
D6375771
fbshipit-source-id:
4ac3a4ff46bdb76f1f07b0c8ba479a7d404c8c9b
Adam Simpkins [Tue, 21 Nov 2017 18:44:02 +0000 (10:44 -0800)]
logging: add a LogConfig class and parsing code
Summary:
Add a new LogConfig class to represent the current configuration state of the
LoggerDB. This also includes code to parse config strings in both JSON and a
simpler more human-writable format (intended primarily for use in command line
arguments).
I generally expect the human-writable format to be used mainly to configure log
levels. It also supports configuring log handler details as well, but the
format for this data is slightly more cumbersome and will probably be harder
for people to easily remember.
The parsing code is intentionally kept as part of the 'init' library rather
than the core 'logging' library so that other libraries that simply wish to log
messages do not need to depend on it. For instance, this would allow the folly
JSON library to use the logging library without causing a circular dependency.
Reviewed By: bolinfest
Differential Revision:
D6200560
fbshipit-source-id:
e4e3c7f941808251b6c7bcbbdac0210118675fb0
Dave Watson [Tue, 21 Nov 2017 15:34:15 +0000 (07:34 -0800)]
improve ThreadLocalBenchmark
Summary:
Benchmark appears to be memory bound, and is affected by different cpus.
Add a new benchmark that does more work.
Reviewed By: yfeldblum
Differential Revision:
D6380904
fbshipit-source-id:
5cfbaab70379aa4a2923d957e1147d8486deeff7
Dave Watson [Tue, 21 Nov 2017 15:33:09 +0000 (07:33 -0800)]
fix hazptr_array move
Summary: Broken, found via build failures in
D6259947
Reviewed By: yfeldblum
Differential Revision:
D6374215
fbshipit-source-id:
3a438b5416683c602966c2f029c6b7787acdccaa
Giuseppe Ottaviano [Tue, 21 Nov 2017 07:10:34 +0000 (23:10 -0800)]
Revert
D6366352: [folly] Split get_default() into two for deferred default construction
Summary:
This reverts commit
db55b944ca63e565997094c11b90c4ebe98531ce
bypass-lint
Differential Revision:
D6366352
fbshipit-source-id:
e25906409186b077ef9117aa524cc7c86314ae12
Aaryaman Sagar [Mon, 20 Nov 2017 23:59:34 +0000 (15:59 -0800)]
Split get_default() into two for deferred default construction
Summary:
As it stood currently folly::get_default() would unnecessarily
construct a value into the third parameter, which was unnecessary in the fast
path where the element was found in the map
Reviewed By: yfeldblum
Differential Revision:
D6366352
fbshipit-source-id:
db55b944ca63e565997094c11b90c4ebe98531ce
Dylan Yudaken [Mon, 20 Nov 2017 23:59:02 +0000 (15:59 -0800)]
correct usage of namespace std for coroutines_trait specialization
Summary: When specializing traits you should not use the libcpp inline std namespace, but rather the plain one.
Reviewed By: yfeldblum
Differential Revision:
D6373787
fbshipit-source-id:
d62ef9ccbf90c6f952db5f93f2377c920e68a809
Christopher Dykes [Mon, 20 Nov 2017 21:44:02 +0000 (13:44 -0800)]
Work around a bug in MSVC name lookup within templated friend contexts
Summary: MSVC has a bug that causes it to think that `once_flag` in this context resolves to the `using` statement further up, rather than `detail::once_flag`.
Reviewed By: yfeldblum
Differential Revision:
D6376501
fbshipit-source-id:
481dbd75ed21f3d519bd920258fa743f314668ad
Yedidya Feldblum [Mon, 20 Nov 2017 21:29:48 +0000 (13:29 -0800)]
No need for a wrapping structure for posixTimeToDuration
Summary:
[Folly] No need for a wrapping structure for `posixTimeToDuration`.
We can just use a variant of tag dispatch. In this variant, we pass to `posixTimeToDuration` a default-initialized value of the desired return type and we write overload templates for each possible variant. The argument is used purely for overload resolution and return type deduction, not for its runtime value. It is slightly different from tag dispatch because we do not use separate types which are purely tag types.
Reviewed By: simpkins
Differential Revision:
D6371572
fbshipit-source-id:
1987dee31fceec8733caa61495e96489dbf1ca39
Dave Watson [Mon, 20 Nov 2017 16:18:27 +0000 (08:18 -0800)]
Add free-function retire
Summary:
Adds a hazptr_retire and domain::retire methods to hazptr. They both allocate memory instead of inheriting.
This will make implementation of google's cell proposal trivial, vs.
D4754972 which felt overwraught.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0561r0.html
Reviewed By: magedm
Differential Revision:
D6361162
fbshipit-source-id:
9f259f434139f960483b2ab7c5190d3807edcf52
Tianjiao Yin [Mon, 20 Nov 2017 02:51:19 +0000 (18:51 -0800)]
fix SingletonTest
Summary: There is no guarantee that this code will finish in 6 seconds (nor other threads will release singleton in 5 seconds), especially in ASAN mode. Though I don't have better idea, this diff will relax the condition in ASAN mode to make the unit-test less flaky.
Reviewed By: yfeldblum
Differential Revision:
D6371692
fbshipit-source-id:
58dd15cc0b3273719314c8b323ba88ee47e8ff61
Adam Simpkins [Sun, 19 Nov 2017 23:14:20 +0000 (15:14 -0800)]
implement to() conversions for std::chrono to timespec/timeval
Summary:
Add folly::to() conversions to convert between std::chrono::duration or
std::chrono::time_point types and struct timespec or struct timeval types.
To conform to the behavior of the existing arithmetic-to-arithmetic
conversions, this code performs proper overflow checking and throws a
`ConversionError` on overflow. This unfortunately does make the code rather
complicated compared to a non-checking implementation.
Conversions between some unusual duration types is not implemented yet, and
will fail at compile time if someone tries to use it. This happens for
durations where neither the numerator nor the denominator of the ratio is 1.
For instance, 7/13ths of a second.
Reviewed By: yfeldblum
Differential Revision:
D6356700
fbshipit-source-id:
9dce8ab8f32d8c18089f32c7176a8abf3c3f11f7
Pingjia Shan [Sun, 19 Nov 2017 05:55:44 +0000 (21:55 -0800)]
get rid of redundant calls to RequestContext::saveContext()
Summary:
In the past, these calls were required.
Used to solve static destruction ordering issue. Any static object that uses RequestContext must call this function in its constructor.
That is when we were using `static folly::ThreadLocal<std::shared_ptr<RequestContext>>`, which was non-leaky.
The problem being addressed is when we have some code of the form:
```lang=c++
void doWork() {
static EventBase eb;
}
```
But now we are using `SingletonThreadLocal<std::shared_ptr<RequestContext>>`, which is leaky.
So the issue that these calls were there to address seems to have been resolved.
Reviewed By: yfeldblum
Differential Revision:
D6332597
fbshipit-source-id:
c6aba6620ef2fb3a344ea20f56c8b9c0cdf42c70
Pádraig Brady [Sat, 18 Nov 2017 22:25:54 +0000 (14:25 -0800)]
folly: support FOLLY_FALLTHROUGH on GCC
Summary:
This is required to avoid new GCC 7 -Wimplict-fallthrough warnings.
We also update to use the C++17 [[fallthrough]] attribute if supported.
Reviewed By: yfeldblum
Differential Revision:
D6367140
fbshipit-source-id:
d5380983cb300f944df9c2885d0faa0155994be7
Tianjiao Yin [Sat, 18 Nov 2017 14:39:09 +0000 (06:39 -0800)]
reduce the number of iteration when test SharedMutexTest in ASAN mode
Summary: This unit-test takes too long to finish in ASAN mode from heavily loaded system (more than 10 minutes).
Reviewed By: yfeldblum
Differential Revision:
D6362111
fbshipit-source-id:
b097eff60f88ace4fb869132598806700804e267
Pingjia Shan [Fri, 17 Nov 2017 17:40:14 +0000 (09:40 -0800)]
Remove unnecessary white line
Summary:
Seems to be causing rendering error:
{
F114885894}
Created from Diffusion's 'Open in Editor' feature.
Reviewed By: lskuff
Differential Revision:
D6358188
fbshipit-source-id:
3c437007b425bcadb79a71807ad92d520a70e8cb
Jim Meyering [Fri, 17 Nov 2017 04:52:38 +0000 (20:52 -0800)]
folly/fibers/test/FibersTest.cpp: accommodate ASAN's detect_stack_use_after_return=1
Summary:
With ASAN enabled (actually, only with ASAN *and* its detect_stack_use_after_return=1 option),
the addTaskFinally test would fail. This adapts to accommodate the larger stack offsets.
Also, use EXPECT_GT and EXPECT_LT rather than EXPECT_TRUE.
Reviewed By: yfeldblum
Differential Revision:
D6353666
fbshipit-source-id:
39e04caffa7b24cde97c749686c7e651a071dcec
Jody Ho [Thu, 16 Nov 2017 07:23:02 +0000 (23:23 -0800)]
Expose the time remaining in HHWheelTimer::Callback
Summary:
We would like to know the time remaining for a scheduled timeout to decide
whether a new event should override the scheduled timeout.
Reviewed By: yfeldblum
Differential Revision:
D6334067
fbshipit-source-id:
f172d5cd7fc804db5fd53a42d06cadfddf857e22
Subodh Iyengar [Thu, 16 Nov 2017 01:59:20 +0000 (17:59 -0800)]
Add unit test for timeout=0
Summary:
Add unit test for immediate timeout since
we're using it in a few places.
Reviewed By: yfeldblum
Differential Revision:
D6327012
fbshipit-source-id:
ea80763d080b06e2a43277118d3147dc8016f348
Yedidya Feldblum [Wed, 15 Nov 2017 23:52:02 +0000 (15:52 -0800)]
Fix wrong source name in tests Makefile
Summary:
[Folly] Fix wrong source name in tests `Makefile.am`.
Closes #714.
Reviewed By: Orvid
Differential Revision:
D6340209
fbshipit-source-id:
88da57fb4d663071eb578efec50c7e162b0c03d1
Benny Chen [Wed, 15 Nov 2017 22:03:27 +0000 (14:03 -0800)]
allow small vector to be storage for sorted_vector_map
Summary:
this is to allow small_vector to be a storage option for sorted_vector_map. Reas
on I want to do this is because in ads there are a lot of small maps where we wo
uld have to allocate separately.
Reviewed By: yfeldblum
Differential Revision:
D6318811
fbshipit-source-id:
b145d1bef2cbbeb946995aa66b55aaadeb6c54f5
Adam Simpkins [Wed, 15 Nov 2017 20:48:56 +0000 (12:48 -0800)]
add EXPECT_THROW_RE() and EXPECT_THROW_ERRNO() test macros
Summary:
Add EXPECT_THROW_RE() and EXPECT_THROW_ERRNO() macros to folly/test/TestUtils.h
These allow more precise checks than the basic EXPECT_THROW() macro provided as
part of gtest.
These macros are being moved into folly from Facebook's eden repository
(https://github.com/facebookexperimental/eden)
This will allow us to use them in folly tests and in other projects that depend
on folly.
Reviewed By: yfeldblum
Differential Revision:
D6301760
fbshipit-source-id:
1f434fb5bc9b7859f763171264fb0b2e1b4bda62
Igor Sugak [Wed, 15 Nov 2017 06:42:28 +0000 (22:42 -0800)]
fix asan build
Reviewed By: yfeldblum
Differential Revision:
D6332075
fbshipit-source-id:
26a279b6ee4253a8be8f9f63e6900c2082a5486d
Yedidya Feldblum [Wed, 15 Nov 2017 06:39:18 +0000 (22:39 -0800)]
Extra expectations for ADL toAppend
Summary: [Folly] Extra expectations for ADL `toAppend`.
Reviewed By: Orvid
Differential Revision:
D6330547
fbshipit-source-id:
d1363280f097f860883ba84d6cfe7caa4e4cf9de
Christopher Dykes [Tue, 14 Nov 2017 22:01:08 +0000 (14:01 -0800)]
Minor tweak to the wording of the vcpkg section
Summary:
The commands as they were previously would attempt to install Folly as 32-bit, which would fail.
Also removes an extra `-` in the link's url.
Closes: https://github.com/facebook/folly/issues/713
Reviewed By: yfeldblum
Differential Revision:
D6327981
fbshipit-source-id:
ced8c86948b8be5c1dd88e14d6c3a77fc783aac9
Jason Fried [Mon, 13 Nov 2017 21:32:49 +0000 (13:32 -0800)]
Add ByteRange/StringPiece Conversion for Cython
Summary:
Moving the cython definition to folly/python.
Adding simple conversion helper to_bytes
This is to cut down on duplicate folly::range cython definitions
Reviewed By: yfeldblum
Differential Revision:
D6291125
fbshipit-source-id:
314b732a1516a03fb5c9a57939552bbabd81970b
Andrew Krieger [Sun, 12 Nov 2017 22:31:02 +0000 (14:31 -0800)]
Fix an ICE in MSVC from functional/Invoke.h
Summary:
Similar to https://github.com/facebook/folly/commit/
29ffcc50981fd50dd2ab1a69f8a262c4b7c27ad4,
(
D6254219), this was manifesting as random ICEs that repro consistently in a given file on a given machine,
but in different files for different machines and sometimes not at all.
Reviewed By: Orvid, yfeldblum
Differential Revision:
D6303186
fbshipit-source-id:
d91ad633352fc5c28640e71fd1539f6a4ba8f70f
Yedidya Feldblum [Sun, 12 Nov 2017 00:16:59 +0000 (16:16 -0800)]
Fix some declared tests not being included in Makefile TESTS list
Summary: [Folly] Fix some declared tests not being included in `Makefile` `TESTS` list.
Reviewed By: pixelb
Differential Revision:
D6307755
fbshipit-source-id:
8853d8fce25768ff17f9b1e53b1fbba0aa213691
Yedidya Feldblum [Sat, 11 Nov 2017 00:03:47 +0000 (16:03 -0800)]
Backport C++17 container access functions: size, empty, data
Summary: [Folly] Backport C++17 container access functions: `size`, `empty`, `data`.
Reviewed By: Orvid
Differential Revision:
D6301986
fbshipit-source-id:
65c31df68b7743d5ed4a2cf2419586b862683c48
Sergey Doroshenko [Fri, 10 Nov 2017 23:27:22 +0000 (15:27 -0800)]
fix typo in Subprocess.h comments
Summary: (Note: this ignores all push blocking failures!)
Reviewed By: yfeldblum
Differential Revision:
D6299313
fbshipit-source-id:
95be2339f4845502e3c446698e54643eeac8055f
Christopher Dykes [Fri, 10 Nov 2017 20:28:17 +0000 (12:28 -0800)]
Minor tweaks to the wording of a couple of errors in the CMake build
Summary:
Tweak the wording of a couple error messages to make them clearer.
Also disable the 'digraphs not supported' warning, as none of our supported compilers actually interpret digraphs in any of our supported build configurations.
Closes: https://github.com/facebook/folly/issues/706
Reviewed By: yfeldblum
Differential Revision:
D6299715
fbshipit-source-id:
7c847ac859e082aea711f6751f626b4b43886da4
Pádraig Brady [Fri, 10 Nov 2017 03:55:39 +0000 (19:55 -0800)]
folly: fix sysMembarrier() with newer kernel headers
Summary:
When __NR_membarrier is defined (on newer kernels),
we don't define the constants, but also by default
don't include the membarrier header to define the necessary constants.
Therefore split the definition of the syscall value and
the constants used with it, to ensure both are defined.
Reviewed By: yfeldblum
Differential Revision:
D6292178
fbshipit-source-id:
31ba9d4a698a4f5e14ae34de0acf8f851d75527d
Pádraig Brady [Fri, 10 Nov 2017 03:26:16 +0000 (19:26 -0800)]
folly: avoid compile warning/failure due to lvalue-to-rvalue conversion
Summary:
With gcc 7.2 we get the warning:
folly/io/async/DelayedDestructionBase.h:252:20:
error: parameter ‘right’ set but not used [-Werror=unused-but-set-parameter]
std::nullptr_t right) {
^~~~~
I presume this is due to the implicit conversion, hence the named parameter is
never assigned. Instead we use an explicit nullptr.
Reviewed By: yfeldblum
Differential Revision:
D6279302
fbshipit-source-id:
ed449601b0410c178777f20e82ed09d9097bd024
Shubhanshu Agrawal [Fri, 10 Nov 2017 01:35:30 +0000 (17:35 -0800)]
adding a fibers compatible once flag
Summary:
The current folly::once_flag is not compatible with folly fibers
and when using it with fibers is inefficient and also cause deadlocks.
This diff makes the once flag's mutex be a templatable paramter
and overrides it in fibers library with a fiber compatible mtuex.
Reviewed By: yfeldblum
Differential Revision:
D6288508
fbshipit-source-id:
6f82e1794d1f417f8d267061f1702a26a7b4ff12