folly.git
8 years agofolly: fix AtomicUnorderedMap compilation on macOS
Wez Furlong [Wed, 27 Jul 2016 04:07:55 +0000 (21:07 -0700)]
folly: fix AtomicUnorderedMap compilation on macOS

Summary:
MAP_POPULATE is not defined on this system.  Instead we will `madvise` the kernel
that we will need it so that it will populate the mapping.

Reviewed By: yfeldblum

Differential Revision: D3584325

fbshipit-source-id: ece52f3d55c475bcd41367f4e9744d6f41001cd5

8 years agoSupport PicoSpinLock on MSVC
Christopher Dykes [Tue, 26 Jul 2016 23:17:39 +0000 (16:17 -0700)]
Support PicoSpinLock on MSVC

Summary:
It was using inline assembly in order to get atomic single-bit operations, so add a variant for MSVC that uses intrinsics.
MSVC is also weird in-that it doesn't have a 16-bit variant of these, so use an atomic OR and AND to achieve the required effect.

Reviewed By: yfeldblum

Differential Revision: D3623220

fbshipit-source-id: b4ff985ef2ed7787115f4d20de6f244123410dc8

8 years agoConsistently reference an internal struct as a struct
Christopher Dykes [Tue, 26 Jul 2016 22:54:14 +0000 (15:54 -0700)]
Consistently reference an internal struct as a struct

Summary: That way the mismatch warning can stay as an error.

Reviewed By: yfeldblum

Differential Revision: D3613761

fbshipit-source-id: 6e0ee42c658d95786f578f6500aba113c00d8f77

8 years agoFix the weak linking of JEMalloc under MSVC
Christopher Dykes [Tue, 26 Jul 2016 22:53:34 +0000 (15:53 -0700)]
Fix the weak linking of JEMalloc under MSVC

Summary: While the previous version did compile, it wouldn't actually work at link time.

Reviewed By: meyering

Differential Revision: D3614424

fbshipit-source-id: 578677b65edca720330319594e1163d214e1400d

8 years agoDon't run the thread local tests requiring dlopen on Windows
Christopher Dykes [Tue, 26 Jul 2016 22:53:04 +0000 (15:53 -0700)]
Don't run the thread local tests requiring dlopen on Windows

Summary: When I initially added the surrounding `#ifdef` I failed at where I put my `#endif`, as I left the shared library test enabled. This disables it for Windows.

Reviewed By: yfeldblum

Differential Revision: D3622769

fbshipit-source-id: f263201239095bbef076300d5f40649ebee1173c

8 years agoGet MSVC able to construct an Indestructible
Christopher Dykes [Tue, 26 Jul 2016 22:52:25 +0000 (15:52 -0700)]
Get MSVC able to construct an Indestructible

Summary: It didn't like using `decltype(args)`, which wasn't really necessary as we already have it's type.

Reviewed By: yfeldblum

Differential Revision: D3623424

fbshipit-source-id: 994c23cbf486d427c6dd0fe4c6f768e51ea15ad1

8 years agoOnly try to use the membarrier on Linux
Christopher Dykes [Tue, 26 Jul 2016 22:52:13 +0000 (15:52 -0700)]
Only try to use the membarrier on Linux

Summary: It doesn't exist anywhere else, and `syscall` isn't even defined on Windows, so this completely failed to compile before.

Reviewed By: yfeldblum

Differential Revision: D3623382

fbshipit-source-id: 302093ddc96d6465e0dcc1883747229a7fb25a84

8 years agoAdd folly::assume_unreachable
Christopher Dykes [Tue, 26 Jul 2016 22:50:47 +0000 (15:50 -0700)]
Add folly::assume_unreachable

Summary: So that the `[[noreturn]]` attribute can be applied.

Reviewed By: yfeldblum

Differential Revision: D3614122

fbshipit-source-id: 4b95cb553e85c85c277c00b8165671dcc75afac8

8 years agoMake RangeEnumerator C++17 compliant (Generalizing the Range-Based For Loop)
Christopher Dykes [Tue, 26 Jul 2016 22:48:54 +0000 (15:48 -0700)]
Make RangeEnumerator C++17 compliant (Generalizing the Range-Based For Loop)

Summary:
Specifically the loosening of the definition of a range where-by the end of an iterator may be represented by a different type than the beginning of the range.
Oh, and it also fixes compilation on MSVC, which didn't like the decltype being used to determine the iterator type.

Reviewed By: yfeldblum

Differential Revision: D3613993

fbshipit-source-id: 2940a15d0f93c5b6310d0b1896f5d12ca9aec639

8 years agoAdd a dummy FlagSaver class.
Michael Lee [Tue, 26 Jul 2016 22:43:00 +0000 (15:43 -0700)]
Add a dummy FlagSaver class.

Summary: Add a declaration to mock out FlagSaver on platforms that don't support gflags.

Reviewed By: ivmaykov, yfeldblum

Differential Revision: D3620421

fbshipit-source-id: c1047f34f384f34276b18f53e966a65cdd5c8075

8 years agoSupport custom loop runner for EventBaseLoopController
Andrii Grynenko [Tue, 26 Jul 2016 21:45:42 +0000 (14:45 -0700)]
Support custom loop runner for EventBaseLoopController

Summary: This allows having some logic executed before and after FiberManager loop (e.g. grabbing a Python GIL).

Reviewed By: A5he

Differential Revision: D3616752

fbshipit-source-id: 3be35d54ced458328816d583133457a44a863acd

8 years agoA BufferCallback was created but never deleted in AsyncSocketTest2
Yang Chi [Tue, 26 Jul 2016 20:49:40 +0000 (13:49 -0700)]
A BufferCallback was created but never deleted in AsyncSocketTest2

Summary: This crashes unit test with asan, since we never deletes it.

Reviewed By: djwatson

Differential Revision: D3622847

fbshipit-source-id: f7ea7c4f464843353b24a59adefd5d66b95bc169

8 years agoNo need to go through the list when you just want the size
Yang Chi [Tue, 26 Jul 2016 20:45:01 +0000 (13:45 -0700)]
No need to go through the list when you just want the size

Summary: cb is an unused variable, and i got error in android. It seems like we don't need to go through the list to increase count.

Differential Revision: D3622772

fbshipit-source-id: fe1f81a1fcad5bacad052e14b7b0b370beb3f3e5

8 years agoHandle MSVC's preprocessor oddities in the SharedMutex test 2016.07.26
Christopher Dykes [Tue, 26 Jul 2016 15:37:22 +0000 (08:37 -0700)]
Handle MSVC's preprocessor oddities in the SharedMutex test

Summary: MSVC needs a bit more glue to expand the varargs.

Reviewed By: yfeldblum

Differential Revision: D3614681

fbshipit-source-id: 901d8c5138b1d2d28434c51bdff31f6d21f26681

8 years agoInclude sys/types.h portability header in Enumerate
Christopher Dykes [Tue, 26 Jul 2016 15:36:28 +0000 (08:36 -0700)]
Include sys/types.h portability header in Enumerate

Summary: Because it's using `ssize_t`, which MSVC defines weirdly.

Reviewed By: yfeldblum

Differential Revision: D3614050

fbshipit-source-id: 422621ea0dc92b374fa6dfc32a5391c1093d039e

8 years agoRefactor socket peeker
Neel Goyal [Tue, 26 Jul 2016 15:09:18 +0000 (08:09 -0700)]
Refactor socket peeker

Summary:
Refactor the socket peeker into its
own class so that it can be used in
other places as well, for example
for peeking at plaintext traffic.

Reviewed By: knekritz

Differential Revision: D3521899

fbshipit-source-id: fcd5615dd5d7cd1ddece4c624efbdecfbbc88a09

8 years agoFix gcpu regression associated with 1ms tick in HHWheelTimer
Misha Shneerson [Tue, 26 Jul 2016 05:14:37 +0000 (22:14 -0700)]
Fix gcpu regression associated with 1ms tick in HHWheelTimer

Summary:
We should go back to 10ms granularity.
Because regressions -
https://www.facebook.com/groups/wfi/permalink/1107102889338325/

Reviewed By: haijunz

Differential Revision: D3616784

fbshipit-source-id: dc1d7ff5195ae836df4a334a9ac5c41164c0627b

8 years agoFix typo in SharedMutex comment
Giuseppe Ottaviano [Sun, 24 Jul 2016 18:48:49 +0000 (11:48 -0700)]
Fix typo in SharedMutex comment

Reviewed By: yfeldblum

Differential Revision: D3611881

fbshipit-source-id: 3991c18cc2956e9d0b50a20e6daf913ec58ee501

8 years agoFix a namespace issue
Giuseppe Ottaviano [Sun, 24 Jul 2016 17:21:07 +0000 (10:21 -0700)]
Fix a namespace issue

Reviewed By: dkgi

Differential Revision: D3586189

fbshipit-source-id: ce817233a0293e19400b5689c283531be6302c10

8 years agoAdd helper functions for mallctl
Qinfan Wu [Sun, 24 Jul 2016 04:34:49 +0000 (21:34 -0700)]
Add helper functions for mallctl

Summary: As more code is using mallctl, it's worth making these helper functions available everywhere.

Reviewed By: yfeldblum

Differential Revision: D3576190

fbshipit-source-id: 968e80e00f2ed93542e117c24861c21745b63f20

8 years agoMove RequestContext definitions to source files
Yedidya Feldblum [Sat, 23 Jul 2016 22:44:57 +0000 (15:44 -0700)]
Move RequestContext definitions to source files

Summary:
[Folly] Move `RequestContext` definitions to source files.

Keeping headers lightweight can help with build times.

Reviewed By: djwatson

Differential Revision: D3609809

fbshipit-source-id: 20608e3ff764c86c24355a328da1dcca9a08fce4

8 years agoRequestContext changes to facilitate thread tracing
Melanie Subbiah [Fri, 22 Jul 2016 22:01:39 +0000 (15:01 -0700)]
RequestContext changes to facilitate thread tracing

Summary: In order to trigger trace points when a new thread begins/finishes executing, I added functionality to call onSet and onUnset methods on all RequestData objects whenever setContext is called. The main question: will this approach cost us too much overhead?

Reviewed By: djwatson

Differential Revision: D3604948

fbshipit-source-id: 3b704ca0f2b713458427aa49be12f776939057f8

8 years agoFix ASAN exposed heap-use-after-free
Mark Isaacson [Fri, 22 Jul 2016 17:47:38 +0000 (10:47 -0700)]
Fix ASAN exposed heap-use-after-free

Summary: This code very obviously wrote past the end of the buffer when the length was 1. Furthermore, it was just downright broken for all values. The author obviously meant to type * instead of +. I took the time to verify that the algorithm is actually correct, while I was working on this. My proof is in the test plan.

Reviewed By: yfeldblum, meyering

Differential Revision: D3603255

fbshipit-source-id: 5f2a0011ff5401a70ba03993eab6e53e29d87c1c

8 years agoRename GetTickCount to GetClockTickCount
Christopher Dykes [Fri, 22 Jul 2016 17:16:18 +0000 (10:16 -0700)]
Rename GetTickCount to GetClockTickCount

Summary: Because Windows already defines a funtion named `GetTickCount` with a different signature.

Reviewed By: yfeldblum

Differential Revision: D3601877

fbshipit-source-id: fa0fb422156a3dc71e149e02a000ccdf3479eea5

8 years agoUpdate FB_ONE_OR_NONE in Preproc.h to work with MSVC
Christopher Dykes [Fri, 22 Jul 2016 17:15:29 +0000 (10:15 -0700)]
Update FB_ONE_OR_NONE in Preproc.h to work with MSVC

Summary: There was an MSVC section, but it's much simpler to just use the glue to solve the issue instead.

Reviewed By: yfeldblum

Differential Revision: D3601184

fbshipit-source-id: f95bd3dfc4b244e1bce21997f44de2970575ad7f

8 years agoAlways use the 64-bit conversion function in ConvBenchmark.cpp
Christopher Dykes [Fri, 22 Jul 2016 17:15:00 +0000 (10:15 -0700)]
Always use the 64-bit conversion function in ConvBenchmark.cpp

Summary: A `long` on MSVC is only 4 bytes, so the `static_assert` will fail. Remove the static assert and use the `long long` version instead. Do the same for the unsigned version.

Reviewed By: yfeldblum

Differential Revision: D3601064

fbshipit-source-id: 70c5f4bca597ba05c3729f5d15feeea3cc8fde57

8 years agoInclude the Builtins portability header in Bits.h
Christopher Dykes [Fri, 22 Jul 2016 17:14:46 +0000 (10:14 -0700)]
Include the Builtins portability header in Bits.h

Summary: Because we need it for MSVC.

Reviewed By: yfeldblum

Differential Revision: D3601138

fbshipit-source-id: 4d2ba3f9c972717745289686b025d2763f9ef30d

8 years agoInclude the sys/types.h portability header
Christopher Dykes [Fri, 22 Jul 2016 17:14:15 +0000 (10:14 -0700)]
Include the sys/types.h portability header

Summary: As the comment in the file says, this has to be included in a weird order due to issues on Windows.

Reviewed By: yfeldblum

Differential Revision: D3601008

fbshipit-source-id: fca4b6d906eb0d1f001c28197987165cd075752d

8 years agoUse decltype to get the type of a non-static local
Christopher Dykes [Fri, 22 Jul 2016 17:13:42 +0000 (10:13 -0700)]
Use decltype to get the type of a non-static local

Summary: As the `sizeof()` is being evaluated in a static context, MSVC doesn't let you reference non-static locals. Solve the issue by getting the type of the local via `decltype` instead.

Reviewed By: yfeldblum

Differential Revision: D3600845

fbshipit-source-id: 825d93ced8f09d9f4bf0dcf02142f47a0ec32605

8 years agoDon't typedef types to the same name in different namespaces
Christopher Dykes [Fri, 22 Jul 2016 17:13:30 +0000 (10:13 -0700)]
Don't typedef types to the same name in different namespaces

Summary:
This was causing MSVC to complain due to `DSched` already being declared in `BatonTestHelpers.h` as `::folly::DSched`, but it's then typedef'd again in `BatonTest.cpp`, this time as `::DSched`. MSVC complains about the duplicate definitions even though they resolve to the same type, so kill the one in the source file and keep the one in the header.
The same is the case with `GroupVarint32Decoder` and `GroupVarint64Decoder`, which are both declared in `GroupVarint.h` and then again, this time in an anon namespace, in `GroupVarintTest.cpp`.

Reviewed By: yfeldblum

Differential Revision: D3600948

fbshipit-source-id: 401790aa918289fe4a34dd5af683f8c6ca50d847

8 years agofolly: fixup folly::Random FixedSeed test expectations on macOS
Wez Furlong [Fri, 22 Jul 2016 00:51:38 +0000 (17:51 -0700)]
folly: fixup folly::Random FixedSeed test expectations on macOS

Summary:
uniform_int_distribution is implemented differently on this system and
always returns the generated number from our constant random number generator.

Reviewed By: yfeldblum

Differential Revision: D3585418

fbshipit-source-id: 754076599ba77b49c17f03bb0c92264a3bed4ab0

8 years agoDon't use ?:
Christopher Dykes [Thu, 21 Jul 2016 23:56:32 +0000 (16:56 -0700)]
Don't use ?:

Summary:
Because it doesn't exist in the C++ standard.

This switches them to the explicitly expanded form, which is in the spec.
It also removes a few that were doing absolutely nothing. (MSVC still complained about the division by zero)

Reviewed By: yfeldblum

Differential Revision: D3479260

fbshipit-source-id: 5386e27057beeb4b228f5d6be4e1cf9941cf3176

8 years agoSupport StlVectorTest for MSVC
Christopher Dykes [Thu, 21 Jul 2016 23:16:12 +0000 (16:16 -0700)]
Support StlVectorTest for MSVC

Summary:
Sure, it's big, but it compiles.

There are a few different changes to make this happen, the first is to deal with quite a few places where MSVC was complaining about the implicit coersion of template and noexcept parameters to bool.
Next, the gating that disabled this for 4.7 is now gone.
MSVC's runtime expected `difference_type` to be defined on our custom allocator, so it now is.
`ReadTSC()` was changed to use MSVC's intrinsic for this.
And finally, the named variadic parameters to macros were switched to normal variadic macro params, as MSVC doesn't support named ones.

Reviewed By: yfeldblum

Differential Revision: D3479761

fbshipit-source-id: 703a5ef942ffc9b63381d13fc3960ac6f2780aa0

8 years agoMark a couple of local constants as static constexpr
Christopher Dykes [Thu, 21 Jul 2016 23:12:57 +0000 (16:12 -0700)]
Mark a couple of local constants as static constexpr

Summary: Because they aren't captured, and thus aren't accessible, in the callback used further on in the function.

Reviewed By: yfeldblum

Differential Revision: D3600874

fbshipit-source-id: 699b3d4caa0a310b1ccc7810d670671850f4366b

8 years agoDisable a test of ThreadLocal across forks for Windows
Christopher Dykes [Thu, 21 Jul 2016 23:09:26 +0000 (16:09 -0700)]
Disable a test of ThreadLocal across forks for Windows

Summary: We don't have `fork()` to begin with, so disable the test and the includes it required if we're compiling on Windows.

Reviewed By: yfeldblum

Differential Revision: D3600600

fbshipit-source-id: 5399705753b456139105f4ac757190e1ba1765f2

8 years agoallocate_sys_buffer
Yedidya Feldblum [Wed, 20 Jul 2016 22:07:31 +0000 (15:07 -0700)]
allocate_sys_buffer

Summary:
[Folly] `allocate_sys_buffer`.

For when a `malloc`'d buffer is required, with an associated deleter that calls `free`.

Reviewed By: JonCoens

Differential Revision: D3590516

fbshipit-source-id: 644f4b5d5e8f19dbc8f29efe3e93517fba0ad72f

8 years agoRecord whether cached certificate was used
Mingtian Yin [Wed, 20 Jul 2016 18:43:19 +0000 (11:43 -0700)]
Record whether cached certificate was used

Summary: Record whether cached certificate was used

Reviewed By: anirudhvr

Differential Revision: D3582807

fbshipit-source-id: 246107ce383ff31718ee7dcccf8bbea459b559a8

8 years agofolly: allow folly::init to build on systems without the symbolizer (macOS)
Wez Furlong [Wed, 20 Jul 2016 17:58:54 +0000 (10:58 -0700)]
folly: allow folly::init to build on systems without the symbolizer (macOS)

Summary:
This makes folly/init/Init.cpp compile on macOS by leveraging
equivalent functionality in the glog library when the folly symbolizer is not
available.  This is true for macOS and also for Windows.  I haven't done
anything to handle Windows in this diff.

Reviewed By: yfeldblum

Differential Revision: D3585509

fbshipit-source-id: 2e0c29520a53826acbf656a7a02659b4e905802f

8 years agoWrappers folly::chrono::clock_gettime and clock_gettime_ns
Christopher Dykes [Tue, 19 Jul 2016 22:58:01 +0000 (15:58 -0700)]
Wrappers folly::chrono::clock_gettime and clock_gettime_ns

Summary:
On Linux hosts, the fast path to get the current time in
nanoseconds without doing a syscall to the kernel is available via the
VDSO kernel-runtime interface.

In this diff, I:

1. Expose portability wrappers `folly::chrono::clock_gettime()` and
`folly::chrono::clock_gettime_ns()`
2. Implement a VDSO wrapper on Linux hosts to implement those without
a round-trip to the kernel

Depends On D3418054

Reviewed By: bmaurer

Differential Revision: D3418087

fbshipit-source-id: 3fb99f0dd946f19ba29d0d52a1038dad3556bafd

8 years agoGenerate a pkg-config file for Folly
Steve Muir [Tue, 19 Jul 2016 22:48:41 +0000 (15:48 -0700)]
Generate a pkg-config file for Folly

Summary: First cut at autoconf rules for generating a pkg-config file for Folly. The only interesting part is that we want to handle packages that provide their own .pc files as dependencies rather than just stuffing them into the list of libraries required from packages that don't provide .pc.

Reviewed By: simpkins

Differential Revision: D3556611

fbshipit-source-id: d1c5f89416d1d96d2cdf5ccdc5bd7117fb822a82

8 years agoHandle small_vectors with 0 inline capacity correctly
Christopher Dykes [Tue, 19 Jul 2016 22:44:31 +0000 (15:44 -0700)]
Handle small_vectors with 0 inline capacity correctly

Summary: It is an error to attempt to get the size of a zero length array, such as `unsigned char[0]`, which is what you get if `MaxInline` has been passed in as 0. We can work around this by simply defining `InlineStorageType` to be `void*` if the size is exactly 0, which will result in the capacity correctly being stored out of line.

Reviewed By: yfeldblum

Differential Revision: D3572898

fbshipit-source-id: c96bb7cc6a890044bb74b0f6d5238c503552ee25

8 years agoCorrectly use iters_ in ProducerConsumerQueueBenchmark
Elliott Clark [Tue, 19 Jul 2016 22:30:08 +0000 (15:30 -0700)]
Correctly use iters_ in ProducerConsumerQueueBenchmark

Summary: iters was only being used on one side of the producer/consumer.

Reviewed By: yfeldblum

Differential Revision: D3588593

fbshipit-source-id: 5e9a8a02bb9addbd6f936b8cc411d58cdf82b6e2

8 years agoMake the mprotect variant of asymmetricHeavyBarrier work when mlock fails
Yedidya Feldblum [Tue, 19 Jul 2016 21:38:12 +0000 (14:38 -0700)]
Make the mprotect variant of asymmetricHeavyBarrier work when mlock fails

Summary: [Folly] Make the `mprotect` variant of `asymmetricHeavyBarrier` work when `mlock` fails.

Reviewed By: djwatson

Differential Revision: D3585948

fbshipit-source-id: c3a46884434b7f9da9caa9cf203573f9e3ce7444

8 years agoFix macro check in SysMembarrier
Michael Lee [Tue, 19 Jul 2016 20:26:37 +0000 (13:26 -0700)]
Fix macro check in SysMembarrier

Summary:
FOLLY_X64 is usually defined, so check whehter it is defined
to 0 or 1.  In addition, memory barriers are not necessarily widely
available on the mobile platforms. What leads me to believe this is
that, at least for older ndk's, atomics are not guaranteed to work for
multicore platforms.

Reviewed By: andriigrynenko

Differential Revision: D3586806

fbshipit-source-id: 3ac8c4b74cac09e41bc3cb65c8adc2732b8b2256

8 years agofolly: unconditionally include boost mutex definitions on macOS
Wez Furlong [Tue, 19 Jul 2016 16:45:24 +0000 (09:45 -0700)]
folly: unconditionally include boost mutex definitions on macOS

Summary:
macOS doesn't have timedwait functionality and this header file was
gating including the boost headers on that check which caused dependent modules
to fail to compile because `boost::mutex` and `boost::recursive_mutex` were not
known to the compiler.

Reviewed By: yfeldblum

Differential Revision: D3585470

fbshipit-source-id: 7f8d9603e95ce01328103c7c6ac0bc75a35ddf4d

8 years agoMake sure to return from atomicHashArrayInsertRaceThread
Christopher Dykes [Tue, 19 Jul 2016 02:49:24 +0000 (19:49 -0700)]
Make sure to return from atomicHashArrayInsertRaceThread

Summary: Because, unfortunately, `pthread_exit` is not marked as noreturn in the pthread library primarily used on Windows, we need to return something to avoid errors.

Reviewed By: yfeldblum

Differential Revision: D3571465

fbshipit-source-id: 4713b2364d6584ae255fb83cd21406fdc33ee299

8 years agoUse membarrier in TLRefCount
Andrii Grynenko [Tue, 19 Jul 2016 01:17:06 +0000 (18:17 -0700)]
Use membarrier in TLRefCount

Summary: membarrier guarantees that there's at most one update to thread-local counter, which collecting thread may not see.

Reviewed By: djwatson

Differential Revision: D3532952

fbshipit-source-id: 6106bfe87c70c5f864573a424662778e20423bbb

8 years agofolly: improve setThreadName for macOS
Wez Furlong [Mon, 18 Jul 2016 23:33:31 +0000 (16:33 -0700)]
folly: improve setThreadName for macOS

Summary:
Since OS X 10.6 it is possible to set the name of the current thread.
This diff adjusts our setThreadName routine to do this, partially fixing
the associated test case.  Even though this doesn't completely cover
all cases it is still a valid improvement: most callers are
threads setting their own name.

I've amended the tests so that they can accomodate systems that cannot
set the names of other threads.

Reviewed By: yfeldblum

Differential Revision: D3576281

fbshipit-source-id: 13caf0dca6496aa2da897631e8d7327a6ee452bb

8 years agoFix mocking to support multiple overrides
Andrii Grynenko [Mon, 18 Jul 2016 20:26:59 +0000 (13:26 -0700)]
Fix mocking to support multiple overrides

Summary: When installing the mock, we should make sure to remove singleton from the creation_order list.

Differential Revision: D3580725

fbshipit-source-id: dfb489de1be860ab639380644eab0b45a07a1450

8 years agoAlways pack small_vector
Christopher Dykes [Mon, 18 Jul 2016 16:58:00 +0000 (09:58 -0700)]
Always pack small_vector

Summary:
The check was overly restrictive, the conditional packing was originally used to limit it to GCC only (as it used the `__attribute__` directly), but that restriction is no longer needed as `FOLLY_PACK_*` will be defined appropriately for whatever platform we are currently on.

This just switches it to unconditionally use the `FOLLY_PACK_*` macros rather than defining it's own `FB_PACK_*` macros.

See https://github.com/facebook/folly/commit/aafd8ff9151615a8de09682d41d1c29966e4c8fe for where the `FB_PACK_*` macros could originally have been removed.

Reviewed By: yfeldblum

Differential Revision: D3373779

fbshipit-source-id: 9e41b0836f8ba12de4f37aba58c51f841be08b41

8 years agoMake static analysis happy about the length of patternBuf
Christopher Dykes [Mon, 18 Jul 2016 16:56:55 +0000 (09:56 -0700)]
Make static analysis happy about the length of patternBuf

Summary: The static analysis doesn't quite understand that we are assigning a NUL to, at most, `patternBuf[MAX_PATH + 1]` so increase the buffer size slightly to make it happy.

Reviewed By: yfeldblum

Differential Revision: D3571923

fbshipit-source-id: c027e2b253dc3e56d13e9a15a0e960a6aa3a0e6c

8 years agoAdd an Atomic portability header
Christopher Dykes [Mon, 18 Jul 2016 16:48:58 +0000 (09:48 -0700)]
Add an Atomic portability header

Summary: Because there are situations where we need to do an atomic write to a plain pointer, rather than an atomic value. Unfortunately, there is no support in the standard library for this, so different compilers implement the primitives differently. This specifically implements an `int64_t` overload for `__sync_fetch_and_add` for use in the atomic hash map test.

Reviewed By: yfeldblum

Differential Revision: D3571830

fbshipit-source-id: c27d8d2a5238bbc9aba6a9e48e4b3412a199288f

8 years agofolly: fix initialization for CLOCK_PROCESS_CPUTIME_ID emulation on macOS
Wez Furlong [Sun, 17 Jul 2016 06:02:32 +0000 (23:02 -0700)]
folly: fix initialization for CLOCK_PROCESS_CPUTIME_ID emulation on macOS

Summary:
this fixes TimeTest on macOS.

The problem was that `task_info_count` is an in/out parameter and it wasn't correctly initialized due to a typo.  The lack of initialization meant that the `task_info()` call would fail at runtime.

Reviewed By: yfeldblum

Differential Revision: D3576262

fbshipit-source-id: 8388fa3c5020309d64a0fa7e01fd6cea80200219

8 years agofolly: fix constexpr methods in RandomTest on macos
Wez Furlong [Sat, 16 Jul 2016 21:32:38 +0000 (14:32 -0700)]
folly: fix constexpr methods in RandomTest on macos

Summary:
the RNG max and min methods must be marked constexpr in order to
compile.  The macos compiler doesn't know to propagate the constexpr-ness from
numeric_limits::max on its own, so we have to be explicit (this also matches
the annotations in ThreadLocalPRNG in Random.h)

Reviewed By: yfeldblum

Differential Revision: D3576189

fbshipit-source-id: e4eeb3361d1c48f582dad5a52e35cae133f007a1

8 years agofolly: fixup CompressionTest compilation on macos
Wez Furlong [Sat, 16 Jul 2016 21:13:32 +0000 (14:13 -0700)]
folly: fixup CompressionTest compilation on macos

Summary:
The compiler on macos is pedantic about the size of the 9UL not
matching the size of the type returned by `oneBasedMsbPos`.  Cast it to
the appropriate size.

Reviewed By: meyering

Differential Revision: D3576183

fbshipit-source-id: 41e9afc78eed2994e34238da119774e9bf6b7cea

8 years agoFix FBStringTest build failure and hash computation for wchar
Qinfan Wu [Sat, 16 Jul 2016 05:52:10 +0000 (22:52 -0700)]
Fix FBStringTest build failure and hash computation for wchar

Summary:
Getting some build errors when doing `buck test folly/...`. This seems to be fixing it.

  folly/test/FBStringTest.cpp:1290:23: error: unknown type name 'basic_fbstring'; did you mean 'basic_fstream'?
    using u16fbstring = basic_fbstring<char16_t>;
                        ^~~~~~~~~~~~~~
                        basic_fstream

Also the test `testHashChar16` was failing because only part of the string is used when computing hash.

Reviewed By: yfeldblum

Differential Revision: D3575858

fbshipit-source-id: cdd5bdb9653d50beaf0ec82b659d31354b345441

8 years agoget_or_throw(map, key) returns references
Tom Jackson [Sat, 16 Jul 2016 00:07:29 +0000 (17:07 -0700)]
get_or_throw(map, key) returns references

Differential Revision: D3572671

fbshipit-source-id: a80390921b41e47ed2794d48d943a9e4060c7135

8 years agoFix compilation issue related to Fibers WhenN
Matthieu Martin [Fri, 15 Jul 2016 23:49:20 +0000 (16:49 -0700)]
Fix compilation issue related to Fibers WhenN

Summary: In a separate diff, I made changes that switched include order, and suddently this file failed to compile. This fixes it.

Reviewed By: andriigrynenko

Differential Revision: D3574775

fbshipit-source-id: 02422d803708aa8490123d19aab5a16c10fdc36e

8 years agoMake Synchronized tests work with gtest-1.7
Eric Niebler [Fri, 15 Jul 2016 22:42:01 +0000 (15:42 -0700)]
Make Synchronized tests work with gtest-1.7

Summary: Gtest-1.7's ASSERT macros don't work with types with explicit conversions to bool, so do the cast for it.

Reviewed By: yfeldblum

Differential Revision: D3570782

fbshipit-source-id: 885a1022675e63afca00219c5b9de3791742207f

8 years agoout-line the SharedMutexImpl members that directly reference tls_lastTokenlessSlot
Eric Niebler [Fri, 15 Jul 2016 21:43:18 +0000 (14:43 -0700)]
out-line the SharedMutexImpl members that directly reference tls_lastTokenlessSlot

Summary: Recent changes to SharedMutex.h broke mcrouter's open source build. See https://travis-ci.org/facebook/mcrouter/builds/140608809. It looks like we're getting bitten by https://sourceware.org/bugzilla/show_bug.cgi?id=16773. We're hitting it now because the extern template is forcing the instantiation of the thread-local static member to be located in SharedMutex.o, but the inline members that reference that thread-local are *not* located in SharedMutex.o. binutils seems to be stepping on its own feet trying to fix up the references at link time. We can fix it by making sure the code that references the thread-local is colocated with the thread-local.

Reviewed By: yfeldblum

Differential Revision: D3498477

fbshipit-source-id: 86ea86812010ff1ef7351e6f8c106bb4291d0234

8 years agoAssume GFlags isn't present if FOLLY_HAVE_LIBGFLAGS isn't defined
Christopher Dykes [Fri, 15 Jul 2016 21:00:33 +0000 (14:00 -0700)]
Assume GFlags isn't present if FOLLY_HAVE_LIBGFLAGS isn't defined

Summary: Because, if it follows the same pattern as a normal config flag, it simply won't be defined if it isn't available.

Reviewed By: yfeldblum

Differential Revision: D3571862

fbshipit-source-id: 3322ceb0ce349de16b29310b66d1333aa04da516

8 years agoFix the base gen test with GTest 1.7
Christopher Dykes [Fri, 15 Jul 2016 20:16:45 +0000 (13:16 -0700)]
Fix the base gen test with GTest 1.7

Summary: Becuase it has bugs that cause the compiler to error :(

Reviewed By: ericniebler

Differential Revision: D3571263

fbshipit-source-id: 9a17fc2c3c7344c3532df33cd4b84517c97d8efc

8 years agoAdd a check for wchar support
Michael Lee [Fri, 15 Jul 2016 00:37:31 +0000 (17:37 -0700)]
Add a check for wchar support

Summary:
Not all libc's are the same. Some support wchar_t and some
don't do a great job...

Reviewed By: Orvid

Differential Revision: D3565016

fbshipit-source-id: 91da4f1332e30bdb20a93d0a26a0445d5eadd1b7

8 years agoAdding std::unique_ptr definition for the ASN1_TIME type.
Saurabh Singh [Thu, 14 Jul 2016 23:51:10 +0000 (16:51 -0700)]
Adding std::unique_ptr definition for the ASN1_TIME type.

Summary:
The ASN1_TIME is the typically used within OpenSSL for representing
time. For example, it can be used to represent the NotBefore and NotAfter
properties of a certificate to be verified. This diff introduces a
ASN1TimeUniquePtr type to take care of the ownership of the ASN1_TIME.

Reviewed By: yfeldblum

Differential Revision: D3561274

fbshipit-source-id: ca4d1431b33098cccd727253dd301e209d2561a5

8 years agos/PortabilityTimeTest.cpp/TimeTest.cpp
Vikas Kumar [Thu, 14 Jul 2016 18:11:43 +0000 (11:11 -0700)]
s/PortabilityTimeTest.cpp/TimeTest.cpp

Summary:
The test file in `../portability/test` is named `TimeTest.cpp`. Makefile refers to it as `PortabilityTimeTest.cpp`. Possibly a typo. Or the test file should be renamed to `PortabilityTimeTest.cpp`.
Closes https://github.com/facebook/folly/pull/438

Reviewed By: yfeldblum

Differential Revision: D3558452

Pulled By: Orvid

fbshipit-source-id: 0cee874b63e98e671153c507f42a7a3b9d8084c1

8 years agofbstring: Fix std::hash specializations for non-char types
Tavian Barnes [Thu, 14 Jul 2016 18:11:22 +0000 (11:11 -0700)]
fbstring: Fix std::hash specializations for non-char types

Summary: Closes https://github.com/facebook/folly/pull/407

Reviewed By: ot

Differential Revision: D3428571

Pulled By: Orvid

fbshipit-source-id: 0b82afae2df24803250e8d6005e2e59bbc8348c9

8 years agoEnable GroupVarint on Arm64
zhangkehong2brad [Thu, 14 Jul 2016 18:09:26 +0000 (11:09 -0700)]
Enable GroupVarint on Arm64

Summary: Closes https://github.com/facebook/folly/pull/430

Reviewed By: yfeldblum

Differential Revision: D3558454

Pulled By: Orvid

fbshipit-source-id: 8a69b4db31b73806b770fbcf15663411140a04f0

8 years agoSwitch away from things that are removed in C++17
Christopher Dykes [Thu, 14 Jul 2016 18:08:40 +0000 (11:08 -0700)]
Switch away from things that are removed in C++17

Summary:
That currently includes `std::binary_function`, `std::unary_function`, and `std::random_shuffle`.
`std::{unary|binary}_function<T{, T2}, Ret>` changes to `std::function<Ret(T{, T2})>`.
`std::random_shuffle` has no immediate equivalent, but `std::shuffle` while passing a specific RNG achieves the same effect.

Reviewed By: yfeldblum

Differential Revision: D3506405

fbshipit-source-id: cdefc698a841eca762174eddd8ce636e2d8d26ef

8 years agoManualExecutor: add clear method
Sven Over [Thu, 14 Jul 2016 15:13:31 +0000 (08:13 -0700)]
ManualExecutor: add clear method

Summary:
ManualExecutor::clear removes all waiting functions from the
executor.

Reviewed By: yfeldblum

Differential Revision: D3555434

fbshipit-source-id: 604c352f2299b0dada062e5f8287be98e2a5f72c

8 years agoMove DecoratedAsyncTransportWrapper and WriteChainAsyncTransportWrapper to folly.
Kyle Nekritz [Wed, 13 Jul 2016 01:04:37 +0000 (18:04 -0700)]
Move DecoratedAsyncTransportWrapper and WriteChainAsyncTransportWrapper to folly.

Reviewed By: siyengar

Differential Revision: D3550430

fbshipit-source-id: 1489fe502f41e65ce4ce45f26de59db30c9874b8

8 years agoFix Build: sorted_vector_types.h on GCC v4.8
Yedidya Feldblum [Tue, 12 Jul 2016 22:26:39 +0000 (15:26 -0700)]
Fix Build: sorted_vector_types.h on GCC v4.8

Summary:
[Folly] Fix Build: `sorted_vector_types.h` on GCC v4.8.

Problem: that compiler does not yet support `auto`-typed lambda parameters.

Solution: specify the types of the lambda parameters.

Reviewed By: mzlee, Orvid

Differential Revision: D3551262

fbshipit-source-id: 160d3245ec422060175ce59ee653d158954477ed

8 years agoDefine TCPI_OPT_SYN_DATA if it isn't defined
Christopher Dykes [Tue, 12 Jul 2016 21:58:19 +0000 (14:58 -0700)]
Define TCPI_OPT_SYN_DATA if it isn't defined

Summary: Because it doesn't in a lot of our open-source builds.

Differential Revision: D3551445

fbshipit-source-id: 599f45f51d1cbecedd02cb2ca3ef611648986039

8 years agoAdding the ability to check for whether TCP fast open succeeded on a socket
Kevin Lewi [Tue, 12 Jul 2016 20:11:39 +0000 (13:11 -0700)]
Adding the ability to check for whether TCP fast open succeeded on a socket

Summary: Modifies AsyncSocket to check if TFO succeeded. Also added checks to AsyncSocketTest2 to check if TFO succeeded when it should.

Reviewed By: siyengar

Differential Revision: D3540330

fbshipit-source-id: 8a4b64fdb040dea73ba264b8e3dfff4d717fd96f

8 years agoImprove folly/portability/Time.cpp OS X and Windows compatibility
Christopher Dykes [Tue, 12 Jul 2016 18:34:23 +0000 (11:34 -0700)]
Improve folly/portability/Time.cpp OS X and Windows compatibility

Summary:
Folly currently contains wrappers to emulate
`clock_gettime()` on platforms which don't support it.

The OS X and Windows wrappers are missing a bit of functionality we
need. In this diff, I:

1. Bring the OS X and Windows wrappers closer to Linux's functionality
2. Add unit tests

Reviewed By: bmaurer, mzlee

Differential Revision: D3418054

fbshipit-source-id: 47eba871e844b4c3dccd7ab201ec82e1f6edf729

8 years agoadd Synchronized::withLock() methods
Adam Simpkins [Tue, 12 Jul 2016 01:36:01 +0000 (18:36 -0700)]
add Synchronized::withLock() methods

Summary:
Add withLock() and related methods for invoking a lambda function while the
lock is held.  This is sometimes more convenient than opening a new scope and
using lock().

withLock() also retains some of the benefits of the SYNCHRONIZED macro in that
it forces users to put their critical section code in a new scope, making the
critical section more visibly distinct in the code.  This also encourages users
to only put necessary work inside the critical section, and do to other work
once the lock is released.

This also adds a LockedGuardPtr class, which is a slightly cheaper version of
LockedPtr.  The relationship between LockedGuardPtr and LockedPtr is very much
like that between std::lock_guard and std::unique_lock.  It saves a branch in
the destructor, and in the case of std::mutex it also saves a small amount of
storage space (since LockedPtr is specialized for std::mutex to also store a
std::unique_lock).

Reviewed By: yfeldblum, djwatson

Differential Revision: D3530368

fbshipit-source-id: 72a4f457b3f18e8e8f4cc6713218f6882bb89818

8 years agoimprove Synchronized LockedPtr class, and add new lock() APIs
Adam Simpkins [Tue, 12 Jul 2016 01:36:00 +0000 (18:36 -0700)]
improve Synchronized LockedPtr class, and add new lock() APIs

Summary:
This refactors the Synchronized::LockedPtr class, and adds new
lock()/wlock()/rlock() APIs to Synchronized.

The LockedPtr changes include:

- Consolidate code so a single template class can be used for both const
  and non-const operation, rather than requiring separate class definitions.
  A LockPolicy template parameter controls if the lock should be acquired in
  exclusive or shared mode, and a SynchronizedType parameter controls whether
  or not the internal data is const or not.

- Specialize LockedPtr for std::mutex, so it uses std::unique_lock<std::mutex>
  internally.  This requires slightly more storage space internally, but it
  allows Synchronized<T, std::mutex> to be used with std::condition_variable.

- Implement operator*() to dereference the pointer and retrieve a reference to
  the locked data.

- Implement operator!() and provide an isValid() method to check the validity
  of the LockedPtr.  These are needed to tell if a timed acquire operation
  succeeded or timed out.

- Drop the LockedPtr copy constructor.  Previously the copy constructor
  acquired the lock a second time.  If anyone needs the ability to hold a
  shared lock a second time solely via a LockedPtr (and not via the original
  Synchronized object), I think we should add back a much more explicit API to
  do this.

Furthermore, this adds lock(), wlock(), and rlock() methods to Synchronized to
explicitly obtain a LockedPtr.  These APIs behave similar to operator->(), but
are more explicit, and require the caller to make a concious choice about
whether or not an exclusive or shared lock should be acquired.  The lock()
method is present only on Synchronized instantiations using an exclusive mutex,
and the wlock() and rlock() methods are present only on Synchronized
instantiations that use a shared mutex.

I plan to deprecate the existing Synchronized::operator->() method and the
various SYNCHRONIZED macros in upcoming diffs.  For now this adds comments
directing users to the new methods, but does not start any of the technical
deprecation changes yet.

Reviewed By: yfeldblum

Differential Revision: D3526489

fbshipit-source-id: 8a96a09b68656ff9215dcdfdf32ecd2bfbb1727f

8 years agoInline the lower_bound and upper_bound callbacks in sorted_vector_map
Yedidya Feldblum [Tue, 12 Jul 2016 01:27:49 +0000 (18:27 -0700)]
Inline the lower_bound and upper_bound callbacks in sorted_vector_map

Summary:
[Folly] Inline the `lower_bound` and `upper_bound` callbacks in `sorted_vector_map`.

Avoids unnecessary use of legacy `boost::bind`, and allows the compiler to avoid the indirection it introduces. This way, we use templates instead of vtables.

Reviewed By: Orvid

Differential Revision: D3545939

fbshipit-source-id: 277e9e4862beb71e99b94a62308783771071d2bc

8 years agoMove IPAddress definitions to source files
Yedidya Feldblum [Sat, 9 Jul 2016 09:40:45 +0000 (02:40 -0700)]
Move IPAddress definitions to source files

Summary:
[Folly] Move `IPAddress` definitions to source files.

And to internal header files. Keeping headers lightweight can help with build times.

Reviewed By: simpkins

Differential Revision: D3514455

fbshipit-source-id: de78f4ef9e70e7ddd7fb666348ed705c5228531c

8 years agoFolly::FutureDAG <-> Gossit
Shayan Mohanty [Sat, 9 Jul 2016 00:12:44 +0000 (17:12 -0700)]
Folly::FutureDAG <-> Gossit

Summary:
Implements remove(), state_clean(), and reset() functions in order to allow for static FutureDAGS that can be modified in place and executed multiple times.

remove() removes the given handle from the nodes vector and cleans up all dependencies associated with it. Because of the way Handles are implemented, all Handles greater than the one removed are decremented (and therefore must be accounted for in the client-code). Current best-practice would be to remove nodes by most-recently added.

state_clean() removes the sink/source nodes added by go().

reset() removes all nodes but the top-level source node and resets dependency properties.

Reviewed By: tjkswaine

Differential Revision: D3486947

fbshipit-source-id: c8b9db6a139ee5b36aae6e9366c9b338cc49ede1

8 years agoupdate LockTraitsBoost to handle arbitrary duration types
Adam Simpkins [Fri, 8 Jul 2016 18:42:35 +0000 (11:42 -0700)]
update LockTraitsBoost to handle arbitrary duration types

Summary:
Update the boost LockTraits classes to accept arbitrary std::chrono::duration
types, rather than just std::chrono::milliseconds.

Reviewed By: yfeldblum

Differential Revision: D3533556

fbshipit-source-id: e764cdf4dafa0b11cd7558c2d5bfb6f80dce88cf

8 years agoHandle conversion from float in toDynamic()
Marcus Holland-Moritz [Thu, 7 Jul 2016 08:27:05 +0000 (01:27 -0700)]
Handle conversion from float in toDynamic()

Summary:
This adds the necessary ConversionHelper to enable float-to-double
conversion when using `toDynamic` on any type that contains `float`.

Fixes the added test case, which previously failed to compile.

Reviewed By: yfeldblum

Differential Revision: D3525942

fbshipit-source-id: d904dde5585316ea9a15e21430e91ac4e33116b9

8 years agoFix heading capitalization in docs/FBVector.md
Jasmine Chen [Thu, 7 Jul 2016 00:10:38 +0000 (17:10 -0700)]
Fix heading capitalization in docs/FBVector.md

Summary: Closes https://github.com/facebook/folly/pull/431

Reviewed By: yfeldblum

Differential Revision: D3526251

Pulled By: elliottneilclark

fbshipit-source-id: 38ddd275394888cffcf3ad4134d76340be03c97e

8 years agoPython-like enumerate()
Giuseppe Ottaviano [Wed, 6 Jul 2016 23:24:04 +0000 (16:24 -0700)]
Python-like enumerate()

Summary:
Range-based for cannot be used if the element index is needed along
with the element. In these situations, it is often necessary to fall
back to the standard for loop, which is easy to get wrong, or maintain
an extra count variable, which is error-prone when control flow is
nontrivial (for example in the presence of `continue`).

This diff introduces a simple implementation of Python's
`enumerate()`, with the same signature. Since in C++ tuple-unpacking
is verbose, the iteration variable returned is a proxy object `it`
where the iteration index can be retrieved with `it.idx`, and the
value with `*it` or `it->...`, like a normal iterator.

Differential Revision: D3477877

fbshipit-source-id: 376af7f559e8b60f02a3f81f0c026a901e23ddcf

8 years agovarious improvements to the Synchronized tests
Adam Simpkins [Wed, 6 Jul 2016 23:21:37 +0000 (16:21 -0700)]
various improvements to the Synchronized tests

Summary:
Add a runParallel() helper function, to clean up logic that was copy-and-pasted
through most of the test functions.  Additionally, clean up the tests to avoid
unnecessary sleeps.  Also fix backwards arguments to EXPECT_EQ() calls--gtest
assumes the first argument is the expected value, and the second argument is
the value being checked.

Reviewed By: yfeldblum

Differential Revision: D3521565

fbshipit-source-id: e4f007d52c114080cff1fd7a0a407fba39fa8b0e

8 years agoRemove LockTraitsBoost from Synchronized.h
Adam Simpkins [Wed, 6 Jul 2016 23:21:34 +0000 (16:21 -0700)]
Remove LockTraitsBoost from Synchronized.h

Summary:
Update Synchronized.h to no longer include LockTraitsBoost.h
Callers that want to use folly::Synchronized with a boost lock type will now
need to explicitly include LockTraitsBoost.h on their own.

Reviewed By: yfeldblum

Differential Revision: D3521168

fbshipit-source-id: 08f0041f51fe2e9566bde58e9f039a6d187b54e3

8 years agoRemove extra ';' from used macros in Traits.h
Mohamed Amin JABRI [Wed, 6 Jul 2016 21:23:51 +0000 (14:23 -0700)]
Remove extra ';' from used macros in Traits.h

Summary:
Using folly on OSX using gcc-5.3 causes errors due to extra ';' in macro expansion in Traits.h.
Closes https://github.com/facebook/folly/pull/394

Reviewed By: yfeldblum

Differential Revision: D3205265

Pulled By: elliottneilclark

fbshipit-source-id: f49ab04add382660719b3bb83bce3c8dc303db8b

8 years agoClean up int128_t macro gating
Michael Lee [Wed, 6 Jul 2016 20:41:57 +0000 (13:41 -0700)]
Clean up int128_t macro gating

Summary: There is inconsistent use of `#ifdef` and `#if`. Cleaning this up

Reviewed By: yfeldblum

Differential Revision: D3523561

fbshipit-source-id: 23f62d632efeb1b9b137b3e3582c37aae591d901

8 years agoCleanup of how we use BIO/BIO_METHODs
Anirudh Ramachandran [Wed, 6 Jul 2016 16:57:59 +0000 (09:57 -0700)]
Cleanup of how we use BIO/BIO_METHODs

Summary:
AsyncSSLSocket's eorAwareBioWrite does some invasive stuff like
reaching into a BIO and replacing its method (and the 'write' funcptr). This
approach won't work with OpenSSL 1.1.0 or BoringSSL due to API changes and
structs being made opaque. This diff adds a layer of wrappers for some BIO
operations. Note that this is still only tested on 1.0.2

Reviewed By: siyengar

Differential Revision: D3338861

fbshipit-source-id: 2ac9318b0df1709873511bfde0fa85d87c5dd29a

8 years agoRefactor folly::to<>
Marcus Holland-Moritz [Wed, 6 Jul 2016 10:56:58 +0000 (03:56 -0700)]
Refactor folly::to<>

Summary:
This is the main diff of the series. Its main purpose is to make
the internals of folly::to<> propagate error codes instead of
throwing exceptions. Along with this, it makes the following
changes:

- Move most of the string-to-int implementation out of the header file

- Unify error/exception strings across conversion routines

- Introduce a ConversionError class that derives from std::range_error

- Capture an error code in ConversionError in addition to a string

- Optimize tolower() calls in Conv.cpp

- Introduce ConversionResult<>, which is used as the internal result wrapper

- Get rid of all error checking macros

There are quite a few benefits here.

== Faster conversions ==

For a large set of conversions, the performance is unchanged. I've removed
all benchmarks that were unchanged from the table below for simplicity.

A few things stand out:

- `follyAtoiMeasure` is consistently faster, sometimes by quite a large margin

- The cost of throwing exceptions is significantly reduced, as throwing them
  further down on the call stack will reduce the amount of stack unwinding

- String-to-boolean and string-to-float conversions are significantly faster
  when passing in a string representation (e.g. "off" or "infinity") thanks
  to the optimized tolower_ascii() call (column `New+Ascii` in the table)

- Conversions between int and float are significantly faster and almost back
  at the performance of before the undefined behaviour fix

- All string-to-(int|float|bool) conversions are consistently faster

The columns in the table are as follows:

  Original:  Original code before the undefined behaviour fix
  Fix UB:    Code with the undefined behaviour fix; this impacts mostly the
             float <-> int conversions, but appears to have a small effect
             on some other benchmarks
  New:       New code introduced by this diff, but without the tolower_ascii()
             optimization
  New+Ascii: New code, including the tolower_ascii() optimization

  ===========================================================================================
                                                   Original   Fix UB     New        New+Ascii
  folly/test/ConvBenchmark.cpp                     time/iter  time/iter  time/iter  time/iter
  ===========================================================================================
  handwrittenAtoiMeasure(1)                           3.95ns     3.95ns     3.95ns     3.95ns
  follyAtoiMeasure(1)                                 6.08ns     6.08ns     3.95ns     3.95ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(2)                           5.47ns     5.47ns     5.47ns     5.47ns
  follyAtoiMeasure(2)                                 5.77ns     5.77ns     3.95ns     3.95ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(3)                           6.08ns     6.08ns     6.08ns     6.08ns
  follyAtoiMeasure(3)                                 6.08ns     6.08ns     4.25ns     4.25ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(4)                           6.99ns     6.99ns     6.99ns     6.99ns
  follyAtoiMeasure(4)                                 6.99ns     6.99ns     4.56ns     4.56ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(5)                           7.90ns     8.20ns     7.90ns     7.90ns
  follyAtoiMeasure(5)                                 7.29ns     7.29ns     4.86ns     4.86ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(6)                           8.81ns     9.42ns     8.81ns     8.81ns
  follyAtoiMeasure(6)                                 7.29ns     7.29ns     4.86ns     4.86ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(7)                           9.72ns    10.63ns     9.72ns     9.72ns
  follyAtoiMeasure(7)                                 7.60ns     7.60ns     5.16ns     5.16ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(8)                          10.63ns    11.85ns    10.63ns    10.63ns
  follyAtoiMeasure(8)                                 8.51ns     8.51ns     6.08ns     6.08ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(9)                          11.54ns    13.07ns    11.54ns    11.54ns
  follyAtoiMeasure(9)                                 8.81ns     8.81ns     6.08ns     6.08ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(10)                         12.46ns    14.28ns    12.46ns    12.46ns
  follyAtoiMeasure(10)                                8.81ns     8.81ns     6.38ns     6.38ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(11)                         13.37ns    15.50ns    13.37ns    13.37ns
  follyAtoiMeasure(11)                                9.12ns     9.12ns     6.38ns     6.38ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(12)                         14.28ns    16.71ns    14.28ns    14.28ns
  follyAtoiMeasure(12)                               10.03ns    10.03ns     7.29ns     7.29ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(13)                         15.19ns    17.92ns    15.19ns    15.19ns
  follyAtoiMeasure(13)                               10.33ns    10.33ns     7.60ns     7.60ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(14)                         16.10ns    19.14ns    16.10ns    16.10ns
  follyAtoiMeasure(14)                               10.33ns    10.33ns     7.60ns     7.60ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(15)                         17.01ns    20.36ns    17.01ns    17.01ns
  follyAtoiMeasure(15)                               10.63ns    10.63ns     7.90ns     7.90ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(16)                         17.92ns    21.57ns    17.92ns    17.92ns
  follyAtoiMeasure(16)                               11.55ns    11.55ns     8.81ns     8.81ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(17)                         18.84ns    22.79ns    18.84ns    18.84ns
  follyAtoiMeasure(17)                               11.85ns    11.85ns     8.81ns     8.81ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(18)                         19.75ns    24.00ns    19.75ns    19.75ns
  follyAtoiMeasure(18)                               11.85ns    11.85ns     9.12ns     9.12ns
  -------------------------------------------------------------------------------------------
  handwrittenAtoiMeasure(19)                         20.66ns    25.22ns    20.66ns    20.66ns
  follyAtoiMeasure(19)                               12.16ns    12.16ns     9.11ns     9.11ns
  -------------------------------------------------------------------------------------------
  stringToBoolNumClassic                             12.76ns    12.76ns    11.96ns    12.15ns
  stringToBoolNumClassicError                         3.19us     3.18us     1.58us     1.58us
  stringToBoolStrClassic                             17.92ns    17.92ns    15.50ns     7.60ns
  stringToBoolStrClassicError                         3.21us     3.18us     1.57us     1.57us
  -------------------------------------------------------------------------------------------
  stringToFloatNumClassic                            32.96ns    32.81ns    32.10ns    31.12ns
  stringToFloatNumClassicError                        2.73us     2.69us     1.65us     1.66us
  stringToFloatStrClassic                            37.37ns    38.58ns    36.76ns    16.71ns
  stringToFloatStrClassicError                        2.87us     2.87us     1.60us     1.59us
  stringToDoubleNumClassic                           31.30ns    31.82ns    29.77ns    29.17ns
  stringToDoubleNumClassicError                       2.69us     2.66us     1.65us     1.66us
  stringToDoubleStrClassic                           37.67ns    37.67ns    35.84ns    16.71ns
  stringToDoubleStrClassicError                       2.87us     2.86us     1.58us     1.58us
  -------------------------------------------------------------------------------------------
  stringToCharSignedClassic                          16.71ns    18.08ns    15.49ns    14.59ns
  stringToCharSignedClassicError                      3.87us     3.82us     1.61us     1.61us
  stringToCharUnsignedClassic                        15.49ns    15.19ns    12.46ns    12.66ns
  stringToCharUnsignedClassicError                    2.73us     2.70us     1.62us     1.62us
  stringToIntSignedClassic                           21.26ns    19.44ns    17.92ns    18.40ns
  stringToIntSignedClassicError                       3.94us     3.89us     1.64us     1.64us
  stringToIntUnsignedClassic                         17.93ns    18.53ns    15.50ns    15.50ns
  stringToIntUnsignedClassicError                     2.72us     2.71us     1.62us     1.61us
  stringToLongLongSignedClassic                      34.63ns    30.58ns    27.04ns    27.04ns
  stringToLongLongSignedClassicError                  3.94us     3.90us     1.63us     1.63us
  stringToLongLongUnsignedClassic                    51.04ns    47.96ns    46.44ns    46.68ns
  stringToLongLongUnsignedClassicError                2.73us     2.71us     1.61us     1.61us
  -------------------------------------------------------------------------------------------
  ptrPairToCharSignedClassic                          5.16ns     5.16ns     3.34ns     3.65ns
  ptrPairToCharSignedClassicError                     3.56us     3.54us     1.61us     1.61us
  ptrPairToCharUnsignedClassic                        2.43ns     2.43ns     2.13ns     2.13ns
  ptrPairToCharUnsignedClassicError                   2.63us     2.63us     1.61us     1.61us
  ptrPairToIntSignedClassic                           6.99ns     6.99ns     5.16ns     5.16ns
  ptrPairToIntSignedClassicError                      4.08us     4.06us     1.61us     1.61us
  ptrPairToIntUnsignedClassic                         4.25ns     4.56ns     3.34ns     3.34ns
  ptrPairToIntUnsignedClassicError                    2.70us     2.70us     1.60us     1.60us
  ptrPairToLongLongSignedClassic                     12.16ns    12.16ns     9.72ns     9.72ns
  ptrPairToLongLongSignedClassicError                 4.06us     4.06us     1.61us     1.61us
  ptrPairToLongLongUnsignedClassic                   29.13ns    29.13ns    27.61ns    27.61ns
  ptrPairToLongLongUnsignedClassicError               2.71us     2.72us     1.63us     1.64us
  -------------------------------------------------------------------------------------------
  intToCharSignedClassic                            405.02ps   506.35ps   405.02ps   405.02ps
  intToCharSignedClassicError                         2.10us     2.09us     1.63us     1.64us
  intToCharUnsignedClassic                          303.79ps   303.78ps   303.77ps   303.77ps
  intToCharUnsignedClassicError                       2.10us     2.09us     1.63us     1.64us
  intToIntSignedClassic                             405.02ps   405.02ps   405.01ps   405.01ps
  intToIntSignedClassicError                          1.99us     1.98us     1.72us     1.72us
  intToIntUnsignedClassic                           405.03ps   405.03ps   379.71ps   379.71ps
  intToIntUnsignedClassicError                        2.09us     2.09us     1.63us     1.63us
  -------------------------------------------------------------------------------------------
  intToFloatClassic                                 545.11ps     3.34ns     1.23ns     1.23ns
  intToFloatClassicError                              1.67us     2.37us     1.73us     1.72us
  -------------------------------------------------------------------------------------------
  floatToFloatClassic                               759.47ps   759.47ps   759.45ps   759.45ps
  floatToFloatClassicError                            6.45us     6.44us     1.77us     1.77us
  -------------------------------------------------------------------------------------------
  floatToIntClassic                                 637.82ps     2.89ns     1.50ns     1.50ns
  floatToIntClassicError                              1.92us     2.61us     1.82us     1.83us
  ===========================================================================================

== Improved build times ==

I've checked this with gcc 4.9.3, and compile times for both ConvTest and
ConvBenchmark are reduced by roughly 10%:

  ====================================
                    original  new code
  ------------------------------------
  ConvTest.o         14.788s   13.361s
  ConvBenchmark.o    16.148s   14.578s
  ====================================

== Smaller binary size ==

Again, checked with gcc 4.9.3, stripped binaries are slightly smaller with
the new code:

  ====================================
                    original  new code
  ------------------------------------
  conv_test           761704    749384
  conv_benchmark      539632    510928
  ====================================

== Ability to add new non-throwing interfaces ==

This change sticks to the original API that will throw an exception in case
of an error. A subsequent diff will introduce non-throwing interfaces with
a minimum of additional code.

Reviewed By: ericniebler

Differential Revision: D3433856

fbshipit-source-id: 9bc976ebc181fe2f172ae47c78edf407e9ee7bb0

8 years agoupdate Synchronized to use LockTraits
Adam Simpkins [Wed, 6 Jul 2016 01:26:33 +0000 (18:26 -0700)]
update Synchronized to use LockTraits

Summary:
Update the Synchronized code to use the new LockTraits added in D3504625.
This also removes the acquireRead*() and releaseRead*() adapter functions that
had been defined for various other lock types, which are no longer needed.

Reviewed By: yfeldblum

Differential Revision: D3512310

fbshipit-source-id: daedd47c0378aebd479dbfe7aba24978deb9cc05

8 years agoadd LockTraits
Adam Simpkins [Wed, 6 Jul 2016 01:26:31 +0000 (18:26 -0700)]
add LockTraits

Summary:
This adds a new LockTraits template class, for specifying how to use arbitrary
custom mutex types.

The goal of this new class is to replace the acquireRead(), acquireReadWrite(),
releaseRead(), and releaseReadWrite() functions currently defined in
Synchronized.h.  I have not replaced these functions yet in this diff, but will
do so in a follow-up diff.

LockTraits has a few advantages over the existing methods:
* It provides mechanisms for telling if a given mutex supports shared access
  and timed access.
* The default LockTraits implementation automatically figures out the correct
  behavior for classes that define lock(), unlock(), methods.  It automatically
  detects sharing and timed support based on the presence of lock_shared() and
  try_lock_for() methods.

LockTraits can be specialized for custom lock types that do not conform with
the lock method names used by the C++ standard.  This does differ slightly from
the argument dependent lookup mechanism used by the acquireRead() functions.

Reviewed By: yfeldblum

Differential Revision: D3504625

fbshipit-source-id: 40320997e9ae2147baecd10b70e8dc06a35e49e1

8 years agoDisable PthreadKeyUnregister on iOS and MSVC
Andrii Grynenko [Tue, 5 Jul 2016 18:34:29 +0000 (11:34 -0700)]
Disable PthreadKeyUnregister on iOS and MSVC

Summary: ~PthreadKeyUnregister logic is not safe if we can't guarantee that it has the maximum pririty (i.e. is the last to be run on shutdown).

Reviewed By: ericniebler

Differential Revision: D3517589

fbshipit-source-id: 3340e2e19cf52973ee677288bc4ac6105f3f2543

8 years agofix flaky ConnectTFOTimeout and ConnectTFOFallbackTimeout tests
Adam Simpkins [Tue, 5 Jul 2016 18:20:28 +0000 (11:20 -0700)]
fix flaky ConnectTFOTimeout and ConnectTFOFallbackTimeout tests

Summary:
In the ConnectTFOTimeout and ConnectTFOFallbackTimeout tests in
AsyncSSLSocketTest.cpp, the client runs for 1ms before timing out and quitting.
It may end up shutting down the server thread before the server has even
received the TCP connect callback.  If this happened it would cause the test to
fail, since the server code checked to make sure the callback was invoked.

This diff creates a new ConnectTimeoutCallback server-side callback for these
tests to use, which doesn't care if it gets told about a new connection or not.

Reviewed By: siyengar

Differential Revision: D3512809

fbshipit-source-id: ce77fe944fb06a38a84c1458356f161cec7387b3

8 years agoAdd sa_len for sockaddr conversions
Subodh Iyengar [Tue, 5 Jul 2016 05:46:06 +0000 (22:46 -0700)]
Add sa_len for sockaddr conversions

Summary:
Some platforms like Apple add a additional
field to sockaddr called sa_len.

This is not normally required by POSIX, so
all posix methods work correctly when a sockaddr
is passed in without a sa_len set.

However other functions which are not defined
by posix such as connectx require this field to
operate correctly.

Reviewed By: yfeldblum

Differential Revision: D3514266

fbshipit-source-id: f8e2941f337222486c81c911dbd06a2ce35e4f00

8 years agofolly:thread_local_test: avoid test failure due to dlopen of *SAN-enabled binary...
Jim Meyering [Sat, 2 Jul 2016 16:31:28 +0000 (09:31 -0700)]
folly:thread_local_test: avoid test failure due to dlopen of *SAN-enabled binary: skip it

Summary:
The ThreadLocal.SharedLibrary test would fail when compiled with any sanitizer.
Skip that test when any sanitizer is enabled.

Reviewed By: Mizuchi

Differential Revision: D3508099

fbshipit-source-id: 0419269f6454ee4edb93fe00b6f0e79756e609d0

8 years agoInclude the correct headers in various tests
Christopher Dykes [Sat, 2 Jul 2016 04:33:07 +0000 (21:33 -0700)]
Include the correct headers in various tests

Summary: There are two types of includes this adds. The first are standard library headers that weer previously included by transitive dependencies, but aren't included under MSVC, so need to be explicitly included. The second type is a couple of portability headers.

Reviewed By: yfeldblum

Differential Revision: D3513196

fbshipit-source-id: 4f2ac1207aee887ba41c19f5490003e5fe4088f4

8 years agoDon't do #if in the middle of a parameter to a preprocessor macro
Christopher Dykes [Sat, 2 Jul 2016 04:32:07 +0000 (21:32 -0700)]
Don't do #if in the middle of a parameter to a preprocessor macro

Summary: Because MSVC doesn't let you do this. Separate the tests out and disable the entire thing instead.

Reviewed By: yfeldblum

Differential Revision: D3513174

fbshipit-source-id: 7411418204083f172883ca96eff3785c912a9647

8 years agoMark a few constants as constexpr so VLAs aren't required
Christopher Dykes [Sat, 2 Jul 2016 04:31:35 +0000 (21:31 -0700)]
Mark a few constants as constexpr so VLAs aren't required

Summary: Because MSVC doesn't support VLAs.

Reviewed By: yfeldblum

Differential Revision: D3513143

fbshipit-source-id: 256e9e843004a3c37821b3dddc6ecd4c6b5645d9

8 years agoUse a defined form of uniform_int_distribution
Christopher Dykes [Sat, 2 Jul 2016 03:39:26 +0000 (20:39 -0700)]
Use a defined form of uniform_int_distribution

Summary: As per http://en.cppreference.com/w/cpp/numeric/random/uniform_int_distribution the behavior of using `uint8_t` as the template parameter is undefined, and is not supported on MSVC, so use `unsigned short` instead, which is a defined form.

Reviewed By: yfeldblum

Differential Revision: D3507309

fbshipit-source-id: c4c830371d08aee4a3de90bb394d22d92ad9a575

8 years agoDon't use a VLA for the dest buffer when testing wide FBString to multi-byte FBString
Christopher Dykes [Sat, 2 Jul 2016 03:34:53 +0000 (20:34 -0700)]
Don't use a VLA for the dest buffer when testing wide FBString to multi-byte FBString

Summary: Because MSVC doesn't support VLA's.

Reviewed By: yfeldblum

Differential Revision: D3507441

fbshipit-source-id: a50bdbad31674d236e4994903c75232d70f32bc0