folly.git
9 years agoAdd "consume all captured output" callback to CaptureFD
Alexey Spiridonov [Sat, 3 Oct 2015 05:30:41 +0000 (22:30 -0700)]
Add "consume all captured output" callback to CaptureFD

Summary: I noticed myself trying to fake this kind of callback for a log-based test I was writing. It seems much nicer to add the callback to `CaptureFD` than roll ugly wrappers around it to do the same thing.

Reviewed By: @yfeldblum

Differential Revision: D2506106

9 years agoConverge folly::make_unique to C++14's std::make_unique
Yedidya Feldblum [Sat, 3 Oct 2015 04:47:38 +0000 (21:47 -0700)]
Converge folly::make_unique to C++14's std::make_unique

Summary: [Folly] Converge `folly::make_unique` to C++14's `std::make_unique`.

The primary use-case of ther deleter variety of `folly::make_unique` is `DelayedDestruction`. Let the classes matching this use-case define their own factories.

Reviewed By: @fugalh

Differential Revision: D2495718

9 years agoSupport SSE-enabled Range functions under Clang
Yedidya Feldblum [Fri, 2 Oct 2015 08:52:56 +0000 (01:52 -0700)]
Support SSE-enabled Range functions under Clang

Summary: [Folly] Support SSE-enabled `Range` functions under Clang.

Basically, we remove GCC-only -isms. In particular, we use only names from the `<emmintrins.h>` and `<smmintrins.h>` headers, instead of the GCC-specific SSE builtins that the original code uses.

But in order to get this to work, we must split out the SSE-enabled functions into their own sources and compile them with the `-msse4.2` flag. There is a way to get around this in GCC by marking individual functions as SEE-enabled, but that technique is not supported in Clang. This is the new `RangeSse42` module.

And in order to do that, we must split out the code that is called by both the original `Range` and the new SSE-enabled functions into yet a third module, `RangeCommon`.

Ahhhhhh, dependencies.

Note that we always compile the SSE-enabled functions, but we only run them on architectures supporting SSE. The original code tests the CPU's capabilities before running any SSE-enabled functions; this change retains that check.

Reviewed By: @nbronson

Differential Revision: D2428983

9 years agofix compiler warnings from gcc-4.9 + -Wunused-variable
Nathan Bronson [Fri, 2 Oct 2015 01:39:47 +0000 (18:39 -0700)]
fix compiler warnings from gcc-4.9 + -Wunused-variable

Summary: This diff fixes the compiler warnings emitted by gcc-4.9's
stricter -Wsigned-comparison check, as well as those revealed by
-Wunused-variable.

Reviewed By: @meyering, @yfeldblum

Differential Revision: D2494514

9 years agothrift-tls: Expose client common name in connection context
Alex Landau [Thu, 1 Oct 2015 22:35:37 +0000 (15:35 -0700)]
thrift-tls: Expose client common name in connection context

Summary: Server can find out client's common name through
Cpp2ConnContext::getPeerCommonName

Reviewed By: @siyengar

Differential Revision: D2469588

9 years agofix build
Alecs King [Thu, 1 Oct 2015 19:19:18 +0000 (12:19 -0700)]
fix build

Summary: fix typo

jenkins and oss build (e.g. travis) failing.

Reviewed By: @pavlo-fb

Differential Revision: D2499462

9 years agoAPIs to determine which end of the socket has closed it
Mohammad Husain [Wed, 30 Sep 2015 21:32:33 +0000 (14:32 -0700)]
APIs to determine which end of the socket has closed it

Reviewed By: @afrind

Differential Revision: D2466921

9 years agofolly Symbolizer: fix destruction-order fiasco problem
Steve O'Brien [Wed, 30 Sep 2015 21:06:22 +0000 (14:06 -0700)]
folly Symbolizer: fix destruction-order fiasco problem

Summary: On a (broken) unit test there is a bad `CHECK` during destruction time.  This triggers an ABRT, goes through the signal handler, and in the process ends up doing a use-after-free.

Moved some globals inside the functions in which they are used (w/ little duplication, as they are used in one place, mostly).

Reviewed By: @luciang, @meyering

Differential Revision: D2494046

9 years agoUpdateableSharedPtr in folly
Dave Watson [Wed, 30 Sep 2015 14:42:24 +0000 (07:42 -0700)]
UpdateableSharedPtr in folly

Summary: This diff adds UpdateableSharedPtr and FastUpdateableSharedPtr classes to folly. They are efficiet RCU-style holders for a shared_ptr. See the comment at the top of UpdateableSharedPtr.h for a more detailed description and benchmark results.

All the added files are copied from logdevice code with (almost) no changes (`logdevice/common/` on branch `origin/logdevice-latest`). We've been using them in logdevice for a few months. D1821723 introduced it.

Reviewed By: @​bmaurer

Differential Revision: D1919702

9 years agoExtract SparseByteSet into its own module
Yedidya Feldblum [Wed, 30 Sep 2015 08:29:59 +0000 (01:29 -0700)]
Extract SparseByteSet into its own module

Summary: [Folly] Extract `SparseByteSet` into its own module.

`SparseByteSet`, formerly `FastByteSet`, is actually a generic, fully standalone class. It does not need to be embedded in `Range.cpp`.

Reviewed By: @​bmaurer

Differential Revision: D2460180

9 years agoadd callback to specify a client next protocol filter
Ranjeeth Dasineni [Wed, 30 Sep 2015 00:57:06 +0000 (17:57 -0700)]
add callback to specify a client next protocol filter

Summary: From the client perspective, we set the list in order of
preference once and call into openssl to do the selection. This adds
a little more flexibility in that client optionally can customize the
selection for each negotiation. added tests for the no-op case and the
customized case. Feel free to suggest improvements.

Reviewed By: @afrind

Differential Revision: D2489142

9 years agoPre-prune iobufs from clone
Subodh Iyengar [Tue, 29 Sep 2015 22:09:36 +0000 (15:09 -0700)]
Pre-prune iobufs from clone

Summary: It's pretty common during parsing protocols with cursors
that there will be empty iobufs along the way.

This changes it so that we prune empty IOBufs in the cloned
output before returning it to the user.

Reviewed By: @djwatson

Differential Revision: D2464488

9 years agoOpen source AsyncPipe
Andre Pinto [Tue, 29 Sep 2015 21:13:54 +0000 (14:13 -0700)]
Open source AsyncPipe

Summary: AsyncPipeReader and AsyncPipeWriter are classes to asynchronously
read and write to pipes.

Reviewed By: @djwatson

Differential Revision: D2479514

9 years agoCodeMod apache::thrift::async::TEventBase to folly::EventBase (remainder)
Yedidya Feldblum [Tue, 29 Sep 2015 19:04:29 +0000 (12:04 -0700)]
CodeMod apache::thrift::async::TEventBase to folly::EventBase (remainder)

Summary: [Thrift] CodeMod `apache::thrift::async::TEventBase` to `folly::EventBase` (remainder).

A few straggler bits left out of D2349828.

Reviewed By: @Gownta

Differential Revision: D2489063

9 years agoBroke build
Ben Maurer [Mon, 28 Sep 2015 18:46:44 +0000 (11:46 -0700)]
Broke build

Summary: sorry

Reviewed By: @​chiunlin

Differential Revision: D2485925

9 years agoEasy SIOF-proofing
Ben Maurer [Mon, 28 Sep 2015 17:58:44 +0000 (10:58 -0700)]
Easy SIOF-proofing

Summary: These classes are likely to be used as static variables and can
easily be constructed with constexpr.

One that we really ought to fix is SpinLock. Sadly we have a bunch of
implementations of it some of which need initialization.

Reviewed By: @meyering

Differential Revision: D2459355

9 years agoSingleton: refine "eager" initialization
Steve O'Brien [Fri, 25 Sep 2015 18:02:46 +0000 (11:02 -0700)]
Singleton: refine "eager" initialization

Summary: * `registrationComplete` has a slightly different interface (uses enums now)
* `void doEagerInit()` method initializes synchronously; `Future<Unit> doEagerInitVia(Executor*)` now available.

Reviewed By: @luciang, @meyering

Differential Revision: D2463464

9 years agoemplace() support for AtomicHashArray/Map
Justin Gibbs [Thu, 24 Sep 2015 16:44:07 +0000 (09:44 -0700)]
emplace() support for AtomicHashArray/Map

Summary: Allow objects that only support in-place construction (no copy/move
constructor) to be stored in AtomicHashMap and AtomicHashArray via the
emplace() method. This uses variadic template parameters and perfect
forwarding to allow the arguments for any valid constructor for the
object to be used during insertion.

Reviewed By: @nbronson

Differential Revision: D2458152

9 years agoA qfind_first_byte_of using a bitset in folly/Range.cpp
Yedidya Feldblum [Wed, 23 Sep 2015 23:10:09 +0000 (16:10 -0700)]
A qfind_first_byte_of using a bitset in folly/Range.cpp

Summary: [Folly] A `qfind_first_byte_of` using a `bitset` in `folly/Range.cpp`.

With additions to the existing benchmark program.

Reviewed By: @nbronson

Differential Revision: D2465890

9 years agorevert D2379210
James Sedgwick [Wed, 23 Sep 2015 21:17:46 +0000 (14:17 -0700)]
revert D2379210

Summary: This broke EventBase tests, which I managed not to run because they're marked as extended for some reason :( I don't have time to take another stab at this at the moment so I'm reverting for now

Reviewed By: @djwatson

Differential Revision: D2467789

9 years agoCpuIdTest.cpp does not need glog
Yedidya Feldblum [Wed, 23 Sep 2015 08:31:53 +0000 (01:31 -0700)]
CpuIdTest.cpp does not need glog

Summary: [Folly] `CpuIdTest.cpp` does not need `glog`.

Reviewed By: @​@​philippv

Differential Revision: D2470503

9 years agoSearch unpadded strings in Range benchmarks
Yedidya Feldblum [Wed, 23 Sep 2015 05:22:20 +0000 (22:22 -0700)]
Search unpadded strings in Range benchmarks

Summary: [Folly] Search unpadded strings in `Range` benchmarks.

`vstr` has padding to adjust alignment. `vstrp` is a view into `vstr` that skips the padding, so that searches start out unaligned.

Before this diff, haystack is 8-byte aligned: `uintptr_t(haystack.begin()) % 8 == 0`. After this diff, it is not.

Reviewed By: @​@​nbronson

Differential Revision: D2468934

9 years agoUse constexpr for folly::dynamic::TypeInfo static fields
Yedidya Feldblum [Wed, 23 Sep 2015 05:06:25 +0000 (22:06 -0700)]
Use constexpr for folly::dynamic::TypeInfo static fields

Summary: [Folly] Use constexpr for folly::dynamic::TypeInfo static fields

These things are compile-time values. So yeah.

Reviewed By: @​@​JoelMarcey

Differential Revision: D2466270

9 years agoFully clean up before each round of benchmarks in RangeFindBenchmark.cpp
Yedidya Feldblum [Tue, 22 Sep 2015 21:03:09 +0000 (14:03 -0700)]
Fully clean up before each round of benchmarks in RangeFindBenchmark.cpp

Summary: [Folly] Fully clean up before each round of benchmarks in `RangeFindBenchmark.cpp`.

The `vstr` and `vstrp` globals need to be cleaned up before each round of benchmarks.

Reviewed By: @​@​nbronson

Differential Revision: D2467254

9 years agoMove definition of strlcpy to outside an #if
Chip Turner [Mon, 21 Sep 2015 22:58:03 +0000 (15:58 -0700)]
Move definition of strlcpy to outside an #if

Summary: Looks like some platforms fail to build (open source, not internal) due to #if conditional.

Reviewed By: @ldemailly

Differential Revision: D2464555

9 years agoFix potential delete behavior of guard
Subodh Iyengar [Mon, 21 Sep 2015 22:38:25 +0000 (15:38 -0700)]
Fix potential delete behavior of guard

Summary: There is a potential double free in destructor guard
if someone calls a method which takes a DG in the destructor
of the DG.

This is potential in the case when someone is holding onto
a DG while calling destroy() on the object.

Reviewed By: @djwatson

Differential Revision: D2463113

9 years agoMake strlcpy available in folly, second attempt
Chip Turner [Mon, 21 Sep 2015 20:42:50 +0000 (13:42 -0700)]
Make strlcpy available in folly, second attempt

Summary: strncpy is bad.  strlcpy is somewhat less bad.  We already had
this function, but let's move it somewhere more reasonable.

This would make sense to live in String.h, but then we have circular
dependencies.  Since String.h includes Demangle.h, though, you get
strlcpy by including it, so this should be okay enough.

Reviewed By: @lbrandy

Differential Revision: D2097590

9 years agomake to<bool> skip range check
Marc Horowitz [Mon, 21 Sep 2015 16:42:44 +0000 (09:42 -0700)]
make to<bool> skip range check

Summary: to<bool>(42) should return true, not throw an exception.

Reviewed By: @yfeldblum

Differential Revision: D2459766

9 years agoUse string for benchmark names to support computed names
Yedidya Feldblum [Sun, 20 Sep 2015 04:51:15 +0000 (21:51 -0700)]
Use string for benchmark names to support computed names

Summary: [Folly] Use string for benchmark names to support computed names.

A benchmark program might call `folly::addBenchmark` with a computed benchmark name without wishing to maintain the lifetime of the string. Support this case.

Reviewed By: @philippv

Differential Revision: D2460621

9 years agoimplement ThreadCachedArena::totalSize()
Philip Pronin [Sat, 19 Sep 2015 06:38:39 +0000 (23:38 -0700)]
implement ThreadCachedArena::totalSize()

Summary: Semantics matches `Arena<>::totalSize()`.

Reviewed By: @luciang, @ot

Differential Revision: D2456388

9 years agomove ThreadWheelTimekeeper out of detail
James Sedgwick [Fri, 18 Sep 2015 22:24:53 +0000 (15:24 -0700)]
move ThreadWheelTimekeeper out of detail

Summary: This can be useful on its own, will use in twagent

Reviewed By: @fugalh

Differential Revision: D2457778

9 years agoSupport MSVC, which does not have VLAs, in folly/io/async/AsyncSocket.cpp
Yedidya Feldblum [Fri, 18 Sep 2015 08:37:13 +0000 (01:37 -0700)]
Support MSVC, which does not have VLAs, in folly/io/async/AsyncSocket.cpp

Summary: [Folly] Support MSVC, which does not have VLAs, in `folly/io/async/AsyncSocket.cpp`.

We use VLAs in compilers that have them, and fixed-size arrays in compilers that do not.

Reviewed By: @JoelMarcey

Differential Revision: D2450689

9 years agofolly Singleton: "eager" option to initialize upfront
Steve O'Brien [Thu, 17 Sep 2015 13:30:23 +0000 (06:30 -0700)]
folly Singleton: "eager" option to initialize upfront

Summary: Instead of the default lazy-loading behavior (still the default) some singletons might need to get initialized at startup time.  This would be for singletons which take a long time for the instance's constructor to run, e.g. expensive initialization by reading some large dataset, talking to an outside service, and so on.

Provides a way for singletons to opt-in to this, and get populated at the time  `registrationComplete()` is called, instead of lazily.

Some notes about the way I implemented here, mainly, why I did this as a "builder-pattern" kind of thing and not some other way.  I could probably be convinced to do otherwise. :)

* Changing the constructor: the constructor's already slightly fiddly with the two optional -- well, one optional construct function, and another optional-but-only-if-construct-provided, destruct function.  Didn't want to pile more into the ctor.
* New superclass called `EagerLoadedSingleton`; just didn't want to add more classes, esp. if it's just to add one more option.
* Method like `void setEagerLoad()` that makes this eager-load; not sure where one would write the `shouldEagerLoad()` call, probably in some central initialization spot in `main()`, but all the maintenance would have to go there.  I like that it's "attached" to the singleton being defined.  (Though you can still do this.)  Bonus #2; the rule that builds the cpp containing "main" doesn't need to import this dependency and the cpp doesn't have to include Singleton just to do this eager-load call, nor the header for the type itself.
* Omitting this altogether and just saying `folly::Singleton<Foo>::get_weak()` to "ping" the singleton and bring into existence: see last point.  Still might need to have the file containing this initialization decorum include/link against Foo, as well as have one place to maintain the list of things to load up-front.

Reviewed By: @meyering

Differential Revision: D2449081

9 years agofolly: clock: remove static globals, replace with meyers singleton
Lucian Grijincu [Thu, 17 Sep 2015 03:20:29 +0000 (20:20 -0700)]
folly: clock: remove static globals, replace with meyers singleton

Reviewed By: @yfeldblum

Differential Revision: D2446000

9 years agoCompatibility: libevent: accessors for struct event
Yedidya Feldblum [Wed, 16 Sep 2015 21:15:19 +0000 (14:15 -0700)]
Compatibility: libevent: accessors for struct event

Summary: [Folly] Compatibility: libevent: accessors for `struct event`.

Some libevent fields in `struct event` moved between `v1.4` and `v2`. Add some accessors which are defined per-libevent-version.

Reviewed By: @JoelMarcey

Differential Revision: D2447537

9 years agofolly: gen: remove static member
Lucian Grijincu [Wed, 16 Sep 2015 18:37:49 +0000 (11:37 -0700)]
folly: gen: remove static member

Summary: there's no need for this static member, but it does get
generated. We're only curious about it's type. Replace with Meyer's
singleton to get the same effect.

Reviewed By: @ddrcoder, @yfeldblum

Differential Revision: D2446131

9 years agofolly: ProgramOptions: avoid static init fiasco by using meyer's singleton
Lucian Grijincu [Wed, 16 Sep 2015 18:32:54 +0000 (11:32 -0700)]
folly: ProgramOptions: avoid static init fiasco by using meyer's singleton

Reviewed By: @elsteveogrande

Differential Revision: D2446056

9 years agoLimit declaration of GroupVarint tables to SSE >= 2
Sara Golemon [Wed, 16 Sep 2015 18:26:52 +0000 (11:26 -0700)]
Limit declaration of GroupVarint tables to SSE >= 2

Summary: __m128i is only available in SSE 2 and later.
We technically only use it in folly/GroupVarint.h for SSE 3+,
but that's due to various other intrinsics and we could
potentially lower the requirement in certain contexts,
so do the right thing in the generated table.
Revisit the uses in folly/GroupVarint.h later.

Reviewed By: @yfeldblum

Differential Revision: D2445798

9 years agouse HHWheelTimer for EventBase::runAfterDelay
James Sedgwick [Wed, 16 Sep 2015 17:38:01 +0000 (10:38 -0700)]
use HHWheelTimer for EventBase::runAfterDelay

Summary: For the perfs - big improvement on one fd per timeout.

Reviewed By: @djwatson, @fugalh

Differential Revision: D2379210

9 years agofolly: io: SSLContext/AsyncSSLSocket: drop global static mutex (siof)
Lucian Grijincu [Wed, 16 Sep 2015 17:05:52 +0000 (10:05 -0700)]
folly: io: SSLContext/AsyncSSLSocket: drop global static mutex (siof)

Reviewed By: @yfeldblum

Differential Revision: D2446538

9 years agofolly: async: fix SIOF in test
Lucian Grijincu [Tue, 15 Sep 2015 22:54:00 +0000 (15:54 -0700)]
folly: async: fix SIOF in test

Summary: avoid sillyness:

```
$ _build/opt/folly/io/async/test/async_test --gtest_list_tests
ASAN:SIGSEGV
=================================================================
==3245135==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x000000583444 sp 0x7fff17ba0c40 bp 0x7fff17ba0c80 T0)
    #0 0x583443 in std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138
    #1 0x583443 in std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&) third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_map.h:295
    #2 0x583443 in folly::SSLContext::setSSLLockTypes(std::map<int, folly::SSLContext::SSLLockType, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >) folly/io/async/SSLContext.cpp:682
    #3 0x40e028 in Initializer folly/io/async/test/AsyncSSLSocketTest2.cpp:143
    #4 0x40e028 in __static_initialization_and_destruction_0 folly/io/async/test/AsyncSSLSocketTest2.cpp:146
    #5 0x40e028 in _GLOBAL__sub_I__ZN5folly47AsyncSSLSocketTest2_AttachDetachSSLContext_Test10test_info_E folly/io/async/test/AsyncSSLSocketTest2.cpp:147
    #6 0x66cf2e in __libc_csu_init /home/engshare/third-party2/glibc/2.20/src/glibc-2.20/csu/elf-init.c:88
    #7 0x7f7145600084 in __libc_start_main (/usr/local/fbcode/gcc-4.9-glibc-2.20-fb/lib/libc.so.6+0x20084)
    #8 0x410be5 (/data/users/lucian/fbcode2/_build/opt/folly/io/async/test/async_test+0x410be5)

AddressSanitizer can not provide additional info.
AAAAAAASUMMARY: AddressSanitizer: SEGV third-party2/libgcc/4.9.x/gcc-4.9-glibc-2.20-fb/024dbc3/include/c++/4.9.x/bits/stl_tree.h:1138 std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > >::operator=(std::_Rb_tree<int, std::pair<int const, folly::SSLContext::SSLLockType>, std::_Select1st<std::pair<int const, folly::SSLContext::SSLLockType> >, std::less<int>, std::allocator<std::pair<int const, folly::SSLContext::SSLLockType> > > const&)
==3245135==ABORTING
```

Reviewed By: @philippv

Differential Revision: D2440796

9 years agoAdd unit test for update_missing
Kevin Hurley [Tue, 15 Sep 2015 21:06:07 +0000 (14:06 -0700)]
Add unit test for update_missing

Summary: Forgot to add a test for this when I added update_missing.  This adds a test for it

Reviewed By: @yfeldblum

Differential Revision: D2441612

9 years agoPrefer constexpr to preprocessor macros in FBString.h
Yedidya Feldblum [Tue, 15 Sep 2015 20:34:26 +0000 (13:34 -0700)]
Prefer constexpr to preprocessor macros in FBString.h

Summary: [Folly] Prefer `constexpr` to preprocessor macros in `FBString.h`.

Reviewed By: @JoelMarcey

Differential Revision: D2441709

9 years agoNo need to export global instances of folly::AsciiCase(Ins|S)ensitive
Yedidya Feldblum [Tue, 15 Sep 2015 20:21:25 +0000 (13:21 -0700)]
No need to export global instances of folly::AsciiCase(Ins|S)ensitive

Summary: [Folly] No need to export global instances of `folly::AsciiCaseInsensitive` and `folly::AsciiCaseSensitive`.

Calling code may simply construct instances at will - the compiler will optimize away the object construction since it has only the default ctor/dtor, no storage, and no vtable.

Reviewed By: @fugalh

Differential Revision: D2437419

9 years agoDrop support for GCC < 4.8 in folly/FBString.h
Yedidya Feldblum [Tue, 15 Sep 2015 05:48:41 +0000 (22:48 -0700)]
Drop support for GCC < 4.8 in folly/FBString.h

Summary: [Folly] Drop support for GCC < 4.8 in `folly/FBString.h`.

Can't stay indefinitely backcompat with older compilers.

Reviewed By: @JoelMarcey

Differential Revision: D2441747

9 years agoImplement clock_getres for Windows
Orvid King [Mon, 14 Sep 2015 22:53:54 +0000 (15:53 -0700)]
Implement clock_getres for Windows

Summary: Because the windows version of pthread very definitely does not implement the clock. The Cygwin and MinGW headers might, so let them still fall back to the pthread includes.
Closes https://github.com/facebook/folly/pull/280

Reviewed By: @yfeldblum

Differential Revision: D2419067

Pulled By: @JoelMarcey

9 years agoAdd benchmarks for SpookyHashV2 and constify SpookyHashV2::Final
Giuseppe Ottaviano [Mon, 14 Sep 2015 21:07:26 +0000 (14:07 -0700)]
Add benchmarks for SpookyHashV2 and constify SpookyHashV2::Final

Summary: `Final` is idempotent, no reason for it not to be `const`. Checked with a benchmark that the new version does not affect performance.

Reviewed By: @yfeldblum

Differential Revision: D2397377

9 years agoCodeMod: Drop FOLLY_OVERRIDE and FOLLY_FINAL
Yedidya Feldblum [Mon, 14 Sep 2015 19:57:14 +0000 (12:57 -0700)]
CodeMod: Drop FOLLY_OVERRIDE and FOLLY_FINAL

Summary: [Folly] CodeMod: Drop `FOLLY_OVERRIDE` and `FOLLY_FINAL`. These were to support GCC 4.6 and earlier, and MSVC 2008 and earlier.

Reviewed By: @philippv

Differential Revision: D2438309

9 years agoLift the test for ext/random and sfmt19937
Yedidya Feldblum [Mon, 14 Sep 2015 19:51:19 +0000 (12:51 -0700)]
Lift the test for ext/random and sfmt19937

Summary: [Folly] Lift the test for `<ext/random>` and `__gnu_cxx::sfmt19937`.

Put an actual check in `folly/configure.ac` and use the result in `folly/Random.h`.

Reviewed By: @Gownta

Differential Revision: D2438420

9 years agoMSVC 2015 constexpr in Bits.h
Orvid King [Mon, 14 Sep 2015 17:53:41 +0000 (10:53 -0700)]
MSVC 2015 constexpr in Bits.h

Summary: MSVC 2015 supports constexpr intrinsics, and, as the GCC builtins are implemented by #287, we can remove all the MSVC specific intrinsic stuff that isn't constexpr supported.
Closes https://github.com/facebook/folly/pull/288

Reviewed By: @yfeldblum

Differential Revision: D2419064

Pulled By: @JoelMarcey

9 years agoDon't use weird format specifiers
Orvid King [Mon, 14 Sep 2015 17:54:21 +0000 (10:54 -0700)]
Don't use weird format specifiers

Summary: MSVC doesn't like it when you use `%1$s`, and actually emits a warning at compile time, so pass the second arugment twice instead.
Closes https://github.com/facebook/folly/pull/302

Reviewed By: @yfeldblum

Differential Revision: D2371262

Pulled By: @JoelMarcey

9 years agoPort fbstring_core to big-endian architectures.
Christopher Cole [Mon, 14 Sep 2015 17:47:12 +0000 (10:47 -0700)]
Port fbstring_core to big-endian architectures.

Summary: There's 2 ways this could be implemented - either as a series of preprocessor blocks depending on target architecture (as I have implemented it here), or by encapsulating access to MediumLarge::capacity_ within getters/setters as in a similar manner to setSmallSize() and smallSize(). The first option makes the code a bit harder to read, but the second option changes the existing control flow a bit which could slightly alter performance.

I opted for the first so as to keep the existing amd64 flow untouched, but can easily change the pull request to the second option to keep code readability a priority.
Closes https://github.com/facebook/folly/pull/244

Reviewed By: @Gownta

Differential Revision: D2306568

Pulled By: @JoelMarcey

9 years agoHandle MSVC correctly in detail/IPAddress.h
Orvid King [Mon, 14 Sep 2015 17:47:08 +0000 (10:47 -0700)]
Handle MSVC correctly in detail/IPAddress.h

Summary: Winsock's structures don't have a member named `s6_addr16`, so use the name for Winsock's structure.
Closes https://github.com/facebook/folly/pull/279

Reviewed By: @yfeldblum

Differential Revision: D2284223

Pulled By: @JoelMarcey

9 years agoSupport unique_ptr instances with custom deleters in to_shared_ptr
Yedidya Feldblum [Mon, 14 Sep 2015 17:21:23 +0000 (10:21 -0700)]
Support unique_ptr instances with custom deleters in to_shared_ptr

Summary: [Folly] Support `unique_ptr` instances with custom deleters in `to_shared_ptr`.

We now support `unique_ptr<T, D>` rather than just `unique_ptr<T>`.

Reviewed By: @lbrandy

Differential Revision: D2438217

9 years agoThe ' in folly/configure.ac messes with Vim's highlighting
Yedidya Feldblum [Sun, 13 Sep 2015 23:44:39 +0000 (16:44 -0700)]
The ' in folly/configure.ac messes with Vim's highlighting

Summary: [Folly] The ' in `folly/configure.ac` messes with Vim's highlighting.

Reviewed By: @JoelMarcey

Differential Revision: D2438280

9 years agoUse .string() not .native() when working with boost::filesystem
Orvid King [Sun, 13 Sep 2015 23:06:37 +0000 (16:06 -0700)]
Use .string() not .native() when working with boost::filesystem

Summary: Because .native() produces a wstring under MSVC.
Closes #306

Reviewed By: @yfeldblum

Differential Revision: D2419043

Pulled By: @JoelMarcey

9 years agoDrop support for GCC < 4.8 in folly/experimental/Bits.h
Yedidya Feldblum [Sun, 13 Sep 2015 20:23:17 +0000 (13:23 -0700)]
Drop support for GCC < 4.8 in folly/experimental/Bits.h

Summary: [Folly] Drop support for GCC < 4.8 in folly/experimental/Bits.h

Can't stay indefinitely backcompat with older compilers.

Reviewed By: @meyering

Differential Revision: D2420133

9 years agoPort Folly to PPC64
Leonardo Bianconi [Sun, 13 Sep 2015 16:47:49 +0000 (09:47 -0700)]
Port Folly to PPC64

Summary: This patch adds PPC64 specific code in order to support this architecture and be able to be used by HHVM.
Closes #299

Reviewed By: @simpkins

Differential Revision: D2340723

Pulled By: @JoelMarcey

9 years agoAdd support for std::string in folly::Hash
Giuseppe Ottaviano [Sat, 12 Sep 2015 00:37:20 +0000 (17:37 -0700)]
Add support for std::string in folly::Hash

Reviewed By: @​liviu

Differential Revision: D2436705

9 years agoTyping changes in the LockFreeRingBuffer to support 64/32 bit iOS architectures.
Bryce Redd [Fri, 11 Sep 2015 22:46:05 +0000 (15:46 -0700)]
Typing changes in the LockFreeRingBuffer to support 64/32 bit iOS architectures.

Summary: The capacity variable changed types, and turn function needed an explicit cast.

Reviewed By: @BurntBrunch

Differential Revision: D2421421

9 years agoAdd ability to merge dynamic objects
Kevin Hurley [Fri, 11 Sep 2015 17:25:14 +0000 (10:25 -0700)]
Add ability to merge dynamic objects

Summary: Adding ability to merge dynamic objects with another object.  It will just overwrite duplicate keys

Reviewed By: @yfeldblum

Differential Revision: D2413628

9 years agoMarking a bunch of AsyncSSLSocket methods as "const".
Maxim Georgiev [Fri, 11 Sep 2015 16:38:28 +0000 (09:38 -0700)]
Marking a bunch of AsyncSSLSocket methods as "const".

Summary: folly::AsyncSSLSocket class has a number of "get..." methods which don't change the object's state, but are not marked as "const". As a result these methods can't be called on objects passed through const pointer or referrence. Adding "const" modificator for these methods.

Reviewed By: @yfeldblum

Differential Revision: D2430134

9 years agoRevert "use thread_local instead of ThreadLocal for some statics in Random"
Sara Golemon [Fri, 11 Sep 2015 15:18:59 +0000 (08:18 -0700)]
Revert "use thread_local instead of ThreadLocal for some statics in Random"

This reverts commit 6caa3d95ee837703f8f094ffbff5592627417711.

9 years agofolly: fix EmplaceIterCtor test
Lucian Grijincu [Fri, 11 Sep 2015 05:10:42 +0000 (22:10 -0700)]
folly: fix EmplaceIterCtor test

Reviewed By: @yfeldblum

Differential Revision: D2433146

9 years agoUse type-parameterized test cases in folly/test/SynchronizedTest.cpp
Yedidya Feldblum [Fri, 11 Sep 2015 03:22:21 +0000 (20:22 -0700)]
Use type-parameterized test cases in folly/test/SynchronizedTest.cpp

Summary: [Folly] Use type-parameterized test cases in `folly/test/SynchronizedTest.cpp`.

And some cleanups.

Reviewed By: @nbronson

Differential Revision: D2428287

9 years agoFix Build: IPv6: SocketAddressTest.SetFromStrings and AsyncSocketTest.ConnectTimeout...
Yedidya Feldblum [Fri, 11 Sep 2015 03:21:30 +0000 (20:21 -0700)]
Fix Build: IPv6: SocketAddressTest.SetFromStrings and AsyncSocketTest.ConnectTimeout failing

Summary: [Folly] Fix Build: IPv6: `SocketAddressTest.SetFromStrings` and `AsyncSocketTest.ConnectTimeout` failing

On IPv6-only machines, the `SocketAddressTest.SetFromStrings` example in `folly/test/SocketAddressTest.cpp` was failing because it tries to resolve `127.0.0.1` with `AI_ADDRCONFIG` set. Fix it by detecting which of IPv6 and IPv4 are enabled, and picking an appropriate loopback address.

On IPv6-only machines, the `AsyncSocketTest.ConnectTimeout` example in `folly/io/async/test/AsyncSocketTest2.cpp` was failing because it tries to connect `8.8.8.8`, which is IPv4. Fix it by detecting which of IPv6 and IPv4 are enabled, and picking an approprate remote address.

Reviewed By: @chipturner

Differential Revision: D2429109

9 years agouse thread_local instead of ThreadLocal for some statics in Random
Mark McDuff [Thu, 16 Jul 2015 23:27:21 +0000 (16:27 -0700)]
use thread_local instead of ThreadLocal for some statics in Random

Summary: Exit synchronization is the worst!  The worst!

Reviewed By: @​bmaurer

Differential Revision: D2253073

9 years agoMake EliasFanoReader and BitVectorReader API more consistent
Giuseppe Ottaviano [Thu, 10 Sep 2015 15:26:36 +0000 (08:26 -0700)]
Make EliasFanoReader and BitVectorReader API more consistent

Summary: This diff introduces a few breaking API changes to both EliasFanoReader and BitVectorReader in order to fix some inconsistencies:

- As initalized, `EliasFanoReader` and `BitVectorReader` held a value of `0`, thus if `0` is present in the list `skipTo(0)` would not update the position to `0`, as it happens with any other `skipTo(<first element>)`. This fixes `jumpTo` accordingly.
- `jump(i + 1)` would go at position `i`. Now `reader.jump(i)`'s postcondition is `reader.position() == i`.
- It is now illegal to retrieve `value()` from a reader in an out-of-bounds position (before-the-begin or past-the-end). Validity of a reader can be checked with `valid()`.

Reviewed By: @philippv

Differential Revision: D2420023

9 years agoAdd SharedPromise<T>::isFulfilled
Elliott Clark [Thu, 10 Sep 2015 07:31:47 +0000 (00:31 -0700)]
Add SharedPromise<T>::isFulfilled

Summary: Promise<T> has isFulFilled. This patch adds the corresponding functionality to shared promise.

Reviewed By: @jsedgwick

Differential Revision: D2417631

9 years agofolly: small_vector: emplace_back for iterator constructor (compat with std::vector)
Lucian Grijincu [Thu, 10 Sep 2015 00:22:27 +0000 (17:22 -0700)]
folly: small_vector: emplace_back for iterator constructor (compat with std::vector)

Summary: This works:
  std::vector<T*> v;
  std::vector<std::unique_ptr<T>> uv(v.begin(), v.end())

Make it work for small_vector as well.

Reviewed By: @ot, @Gownta

Differential Revision: D2426919

9 years agofolly::FunctionScheduler: Adding support for uniform interval distribution
Eugene Pekurovsky [Wed, 9 Sep 2015 19:33:59 +0000 (12:33 -0700)]
folly::FunctionScheduler: Adding support for uniform interval distribution

Summary: 1) Added uniform interval distribution functionality.
2) Added a generic API for custom interval distribution algorithms.
3) Fixed an issue with removing a canceled function.
4) Did some code cleanup along the way.

Reviewed By: @​kaanb

Differential Revision: D2339911

9 years agoHandle take(-1) better
Tom Jackson [Wed, 9 Sep 2015 19:23:51 +0000 (12:23 -0700)]
Handle take(-1) better

Summary: It's easy to accidentally pass a negative value to ##take()##, which leads to underflow on conversion to ##size_t##.

Reviewed By: @​rosephilip, @philippv

Differential Revision: D2421459

9 years agoDrop support for GCC < 4.8 in folly/Conv.h
Yedidya Feldblum [Wed, 9 Sep 2015 01:48:54 +0000 (18:48 -0700)]
Drop support for GCC < 4.8 in folly/Conv.h

Summary: [Folly] Drop support for GCC < 4.8 in folly/Conv.h.

Can't stay indefinitely backcompat with older compilers.

Reviewed By: @Gownta, @JoelMarcey

Differential Revision: D2422369

9 years agoDrop support for GCC < 4.8 in folly/test/FBStringTest.cpp
Yedidya Feldblum [Tue, 8 Sep 2015 22:24:42 +0000 (15:24 -0700)]
Drop support for GCC < 4.8 in folly/test/FBStringTest.cpp

Summary: [Folly] Drop support for GCC < 4.8 in folly/test/FBStringTest.cpp.

Can't stay indefinitely backcompat with older compilers.

Reviewed By: @tudor

Differential Revision: D2422425

9 years agoImplement assume()
Giuseppe Ottaviano [Tue, 8 Sep 2015 19:20:36 +0000 (12:20 -0700)]
Implement assume()

Summary: In some cases hinting the compiler about conditions that are guaranteed
to be true can help it generate better code. For instance, remove
unnecessary bound checks. Clang has `__builtin_assume` and on GCC we
can simulate it with `__builtin_unreachable`, so better just abstract
both into a function.

Reviewed By: @philippv

Differential Revision: D2419937

9 years agoSwitch a local from long to size_t for MSVC
Orvid King [Tue, 8 Sep 2015 01:23:38 +0000 (18:23 -0700)]
Switch a local from long to size_t for MSVC

Summary: Because MSVC would try to calculate the mmapLength as a long value, which isn't correct, and MSVC issues multiple warnings if you try to do it.
Closes #289

Reviewed By: @yfeldblum

Differential Revision: D2419061

Pulled By: @JoelMarcey

9 years agoImplement adjacent_tokens_only() for GFlagValueSemanticBase
Tom Hughes [Tue, 8 Sep 2015 01:23:00 +0000 (18:23 -0700)]
Implement adjacent_tokens_only() for GFlagValueSemanticBase

Summary: Needed for boost 1.59.0 which added this as a pure virtual function
in the value_semantic base class. Fixes #314.
Closes #315

Reviewed By: @yfeldblum

Differential Revision: D2419035

Pulled By: @JoelMarcey

9 years agoDeclare missing environ variable
Christopher Dolan [Tue, 8 Sep 2015 01:18:45 +0000 (18:18 -0700)]
Declare missing environ variable

Summary: Declaring `extern char** environ` in `experimental/TestUtil.cpp` fixes a compilation issue on OS X 10.10.
Closes #311

Reviewed By: @yfeldblum

Differential Revision: D2419037

Pulled By: @JoelMarcey

9 years agoDead Code: GCC-v4.6 Back Compat: folly/Chrono.h
Yedidya Feldblum [Mon, 7 Sep 2015 05:32:50 +0000 (22:32 -0700)]
Dead Code: GCC-v4.6 Back Compat: folly/Chrono.h

Summary: [Folly] Dead Code: GCC-v4.6 Back Compat: folly/Chrono.h

Reviewed By: @philippv

Differential Revision: D2419086

9 years agofix SharedPromise<T>::setWith
James Sedgwick [Fri, 4 Sep 2015 21:09:39 +0000 (14:09 -0700)]
fix SharedPromise<T>::setWith

Summary: was using the renamed/nonexistent makeTryFunction

Reviewed By: @elliottneilclark

Differential Revision: D2416178

9 years agoFix MemoryIdler::flushLocalMallocCaches() regression.
Jason Evans [Fri, 4 Sep 2015 16:12:18 +0000 (09:12 -0700)]
Fix MemoryIdler::flushLocalMallocCaches() regression.

Summary: Fix MemoryIdler::flushLocalMallocCaches() to use the correct type when querying
jemalloc's "opt.narenas" mallctl.

Reviewed By: @jobenber

Differential Revision: D2414309

9 years agoAllow (read-only) access to a fiber's stack details
Brian Watling [Thu, 3 Sep 2015 20:13:26 +0000 (13:13 -0700)]
Allow (read-only) access to a fiber's stack details

Summary: Allow users to access the stack pointer and stack size

Reviewed By: @alikhtarov

Differential Revision: D2407252

9 years agofolly: specialise makeFutureWith for functions returning futures
Sven Over [Wed, 2 Sep 2015 10:13:17 +0000 (03:13 -0700)]
folly: specialise makeFutureWith for functions returning futures

Summary: makeFutureWith executes a function and returns a future containing
set to the function's return value. This diff adds a specialisation
for the case when the function returns some type Future<T>. Instead
of returning Future<Future<T>>, makeFutureWith now just passes
on the future that the function returned, which may or may not have
a value set at this time. In case the function throws,
makeFutureWith returns a Future<T> containing the exception.

With this diff, the following two lines produce equivalent results:
  auto f1 = makeFutureWith(func);
  auto f2 = makeFuture().then(func);
except for the fact that f2 requires an additional temporary
Future<Unit> to be created and destroyed.

Reviewed By: @fugalh

Differential Revision: D2388335

9 years agoErasing Cpp2Worker in its evb thread
Haijun Zhu [Tue, 1 Sep 2015 23:50:12 +0000 (16:50 -0700)]
Erasing Cpp2Worker in its evb thread

Summary: Erasing Cpp2Worker happens in acceptor thread but when evb is
destroyed and drains its callback queue, the callbacks may access the
worker. This moves erasing Cpp2Worer to its own evb thread, after all
callbacks are run.

Reviewed By: @yfeldblum

Differential Revision: D2398089

9 years agoBump version to 57:0 v0.57.0
Sara Golemon [Mon, 31 Aug 2015 21:44:27 +0000 (14:44 -0700)]
Bump version to 57:0

9 years agoUniversally update to jemalloc 4.0.0.
Jason Evans [Sat, 29 Aug 2015 20:34:12 +0000 (13:34 -0700)]
Universally update to jemalloc 4.0.0.

Summary: Universally update to jemalloc 4.0.0.

Update MALLOC_CONF/malloc_conf configuration, and use the
"arena.<i>.purge" mallctl (replaces "arenas.purge").

Reviewed By: @​bmaurer

Differential Revision: D2355602

9 years agoLog TLS signature_algorithms extension values.
Kyle Nekritz [Fri, 28 Aug 2015 20:35:05 +0000 (13:35 -0700)]
Log TLS signature_algorithms extension values.

Summary: Provide a folly interface for logging the client supported TLS signature algorithms.

Reviewed By: @siyengar

Differential Revision: D2374528

9 years agoRevert "Mass Revert to cdb635a"
Sara Golemon [Mon, 31 Aug 2015 20:45:31 +0000 (13:45 -0700)]
Revert "Mass Revert to cdb635a"

This reverts commit 37f45ee56d52b31c662ea6e0893206ed3fabf8fb which should not have been pushed.

9 years agoBump version to 56:0 v0.56.0
Sara Golemon [Mon, 31 Aug 2015 20:26:50 +0000 (13:26 -0700)]
Bump version to 56:0

9 years agoMass Revert to cdb635a
Michael Lee [Fri, 28 Aug 2015 19:24:32 +0000 (12:24 -0700)]
Mass Revert to cdb635a

Reviewed By: @​sarals, @siyengar, @​dpittman, @​alephbax

9 years agoAdding singular token bucket implementation.
Michael Morton [Thu, 27 Aug 2015 22:44:43 +0000 (15:44 -0700)]
Adding singular token bucket implementation.

Summary: Contains only the central implementation of the TokenBucket, without moving other files to use the new implementation.

Reviewed By: @yfeldblum

Differential Revision: D2329892

9 years agomake doNotOptimizeAway work with clang
Mainak Mandal [Wed, 26 Aug 2015 17:42:59 +0000 (10:42 -0700)]
make doNotOptimizeAway work with clang

Summary: Projects depending on benchmark library fail to build with clang with the following error
```
./folly/Benchark.h:263:16: error: inline asm not supported yet: don't know how to handle tied indirect register inputs
```

I am not an expert here, just tried to take an approach similar to the MSVC compiler clause

Reviewed By: @andralex, @yfeldblum

Differential Revision: D2368670

9 years agoFix the example for folly::static_function_deleter.
Yedidya Feldblum [Wed, 26 Aug 2015 08:57:57 +0000 (01:57 -0700)]
Fix the example for folly::static_function_deleter.

Summary: [Folly] Fix the example for folly::static_function_deleter.

The problem is that

    int BIO_free(BIO*)

is not directly compatible.

So have two examples. One using `RSA_free` which is compatible, and one making a compatible wrapper around `BIO_free`.

Reviewed By: @Gownta

Differential Revision: D2381125

9 years agoBump version to 55:0 v0.55.0
Sara Golemon [Tue, 25 Aug 2015 04:47:24 +0000 (21:47 -0700)]
Bump version to 55:0

9 years agoAdding a unit test for HHWheelTimer exercising the default timeout functionality.
Maxim Georgiev [Tue, 25 Aug 2015 01:46:35 +0000 (18:46 -0700)]
Adding a unit test for HHWheelTimer exercising the default timeout functionality.

Summary: Recently default timeout logic was added to HHWheelTimer interface. This diff adds a unit test exercising this new logic.

Reviewed By: @djwatson

Differential Revision: D2376834

9 years agoGive folly/symbolizer an explicit main function
Mark Isaacson [Mon, 24 Aug 2015 21:40:00 +0000 (14:40 -0700)]
Give folly/symbolizer an explicit main function

Summary: Don't automatically generate the main function for this test.

Reviewed By: @Gownta

Differential Revision: D2375814

9 years agoCreate MockAsyncUDPSocket
Peter Griess [Mon, 24 Aug 2015 13:12:29 +0000 (06:12 -0700)]
Create MockAsyncUDPSocket

Summary: - Mark all methods in AsyncUDPSocket as virtual
- Create new gmock AsyncUDPSocket: MockAsyncUDPSocket

Reviewed By: @yfeldblum

Differential Revision: D2370668

9 years agoRename fixup from TDelayedDestruction to DelayedDestruction.
Yedidya Feldblum [Sun, 23 Aug 2015 05:09:33 +0000 (22:09 -0700)]
Rename fixup from TDelayedDestruction to DelayedDestruction.

Summary: [Folly] Rename fixup from TDelayedDestruction to DelayedDestruction.

Just affects comments.

Reviewed By: @haijunz

Differential Revision: D2373364

9 years agoRename fixup from TNotificationQueue to NotificationQueue.
Yedidya Feldblum [Sun, 23 Aug 2015 04:40:49 +0000 (21:40 -0700)]
Rename fixup from TNotificationQueue to NotificationQueue.

Summary: [Folly] Rename fixup from TNotificationQueue to NotificationQueue.

    hg locate 'folly/**/*.{h,cpp}' | xargs perl -pi -e 's/TNotificationQueue/NotificationQueue/g'

Reviewed By: @haijunz

Differential Revision: D2373334

9 years agostatic_function_deleter.
Yedidya Feldblum [Fri, 21 Aug 2015 22:24:06 +0000 (15:24 -0700)]
static_function_deleter.

Summary: [Folly] static_function_deleter.

So you can write this:

    using BIO_deleter = folly::static_function_deleter<BIO, &BIO_free>;
    auto buf = std::unique_ptr<BIO, BIO_deleter>(BIO_new(BIO_s_mem()));
    buf = nullptr;

In place of this:

    struct BIO_deleter {
      void operator()(BIO* bio) {
        BIO_free(bio);
      }
    };
    auto buf = std::unique_ptr<BIO, BIO_deleter>(BIO_new(BIO_s_mem()));
    buf = nullptr;

Reviewed By: @alandau

Differential Revision: D2364544