Maged Michael [Wed, 1 Nov 2017 14:47:41 +0000 (07:47 -0700)]
Add integrated reference counting
Summary:
Add support for reference counting integrated with the internal structures and operations of the hazard pointer library. The operations are wait-free.
The advantages of this approach over combining reference counting with hazard pointers externally are:
(1) A long list of linked objects that protected by one reference can all be reclaimed together instead of going through a potentially long series of alternating reclamation and calls to retire() for descendants.
(2) Support for iterative deletion as opposed to potential deep recursion of alternating calls to release reference count and object destructors.
Reviewed By: djwatson
Differential Revision:
D6142066
fbshipit-source-id:
02bdfcbd5a2c2d5486d937bb2f9cfb6f192f5e1a
Andrii Grynenko [Wed, 1 Nov 2017 06:06:22 +0000 (23:06 -0700)]
Clear frame cache when activating a fiber
Reviewed By: yfeldblum
Differential Revision:
D6207160
fbshipit-source-id:
57468c9d05cdb3ee6e1d10a3a254a5d1bfddc36f
Yedidya Feldblum [Wed, 1 Nov 2017 05:12:53 +0000 (22:12 -0700)]
A macro for creating member-invoke traits
Summary:
[Folly] A macro for creating member-invoke traits.
The macro creates a specialized traits container with member types and aliases mimicking `std::invoke_result` and the related traits types and aliases.
Reviewed By: aary
Differential Revision:
D6195087
fbshipit-source-id:
07c2bbab6cccb04dc8ff12e20923351e8f38abfd
Lee Howes [Tue, 31 Oct 2017 21:20:08 +0000 (14:20 -0700)]
Adding DeferredExecutor to support deferred execution of tasks on a future returned from an interface.
Summary: This adds a DeferredExecutor type that is boostable, which means that it follows the expectation we expect for C++20 that .then and get will trigger boost-blocking behaviour and ensure work makes progress. Unlike discussions for C++ this adds boost blocking to folly only in the specific case of deferring work to run on the caller's executor, to avoid the necessity to pass an executor into a library purely to ensure that finalisation work and future completion occor on a well-defined exewcutor.
Reviewed By: yfeldblum
Differential Revision:
D5828743
fbshipit-source-id:
9a4b69d7deaa33c3cecd6546651b99cc99f0c286
Martin Martin [Tue, 31 Oct 2017 21:09:59 +0000 (14:09 -0700)]
Remove unused field or local var.
Summary: Remove unused field or local var.
Reviewed By: terrelln
Differential Revision:
D6199120
fbshipit-source-id:
616a2b2549c37bcb57d2f8c530b26089f24c2973
Teng Qin [Tue, 31 Oct 2017 20:49:15 +0000 (13:49 -0700)]
Improve folly::RequestContext onSet and onUnset efficiency
Summary:
In previous discussions, it has been pointed out that `folly::RequestContext::setContext` consumes considerable amount of CPU cycles in production environments. After some investigation, we thought that might be caused by looping over all `RequestData` instances can calling the virtual `onSet` and `onUnset` callback of them. Both the iteration and invoking virtual methods are not cheap.
As you can see from this change, most of the derived classes of `RequestData` don't override the `onSet` and `onUnset` methods. Mostly likely they are only used for per-Request tracking. So the natural idea is to skip those instances when iterating and avoid the unnecessary virtual method invoke.
I have explored the solution to dynamically examine if the `RequestData` instance added has `onSet` and `onUnset` method overridden. That is possible with GCC's PMF extension, but not [[ http://lists.llvm.org/pipermail/llvm-bugs/2015-July/041164.html | for Clang ]] and probably many other compilers. This definitely won't be very good for `folly`'s probability, even if we gate it by compiler flags.
Therefore, this Diff adds the `hasCallback` method to `RequestData` class indicating whether the instance would have `onSet` and `onUnset` overridden. To make it clear to users that they need to correctly override it in order for their `onSet` and `onUnset` callback to work, making it abstract so that user must override it to something and would aware of that.
Also made some improvements on documentation.
Reviewed By: myreg
Differential Revision:
D6144049
fbshipit-source-id:
4c9fd72e9efaeb6763d55f63760eaf582ee4839e
Teng Qin [Tue, 31 Oct 2017 20:49:14 +0000 (13:49 -0700)]
Improve folly::RequestContext::get()
Summary:
Since `folly::RequestContext::get()` returns raw pointer, it could directly use the reference returned by `getStaticContext()`
I don't expect this to make much of a difference, just tiny improvements
Reviewed By: yfeldblum
Differential Revision:
D6153353
fbshipit-source-id:
1c41d4fc259aa5cb3e69e50ed24bed1ba9caf6c3
Alex Guzman [Tue, 31 Oct 2017 06:38:41 +0000 (23:38 -0700)]
Add utility function for loading certificates from a buffer
Summary: Adds a function that reads certificates in from a buffer and returns them as a vector of X509 pointers.
Reviewed By: yfeldblum
Differential Revision:
D6133332
fbshipit-source-id:
eaaaffcbd4d03f37d9d5b4c99a52b0d968b163ba
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:27 +0000 (22:00 -0700)]
Outline most throw expressions in Expected
Summary:
[Folly] Outline most `throw` expressions in `Expected`.
They are definitionally cold, but in-line `throw` statements can expand code size more than is desirable.
* Inline `throw` statement: https://godbolt.org/g/LPaf7V.
* Outline `throw` statement: https://godbolt.org/g/HZBXn6.
Reviewed By: Orvid
Differential Revision:
D6183613
fbshipit-source-id:
28240bb4aa40790d99da783a3c368db81fded124
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:25 +0000 (22:00 -0700)]
Remove dep on Format.h from GenerateFingerprintTables.cpp
Summary:
[Folly] Remove dep on `Format.h` from `GenerateFingerprintTables.cpp`.
`GenerateFingerprintTables.cpp` is a tool used to generate other sources which get built as part of the main library when using the autotools build, so it must be as free of other Folly dependencies as possible.
Reviewed By: ot, Orvid
Differential Revision:
D6183725
fbshipit-source-id:
f12b18553c78e085599a5505ae57f12bc0cd44b0
Yedidya Feldblum [Tue, 31 Oct 2017 05:00:24 +0000 (22:00 -0700)]
Remove folly/ContainerTraits.h
Summary:
[Folly] Remove `folly/ContainerTraits.h`.
It has some handly helpers, but it is not a real abstraction or utility library. Within Folly, only `folly/Padded.h` uses it, so just rewrite the bit that needs it.
Reviewed By: LeeHowes
Differential Revision:
D6183066
fbshipit-source-id:
24a223fe517d21ff531e0fa80172f15d4f963e51
Yedidya Feldblum [Tue, 31 Oct 2017 00:57:54 +0000 (17:57 -0700)]
Move folly/experimental/AsymmetricMemoryBarrier.h
Summary: [Folly] Move `folly/experimental/AsymmetricMemoryBarrier.h` to `folly/synchronization/AsymmetricMemoryBarrier.h`.
Reviewed By: Orvid
Differential Revision:
D6180676
fbshipit-source-id:
f4833318cd365181e202d5f379815e728fba168b
Christopher Dykes [Mon, 30 Oct 2017 23:01:31 +0000 (16:01 -0700)]
Fix the CMake build
Summary: By excluding the tools directory, excluding poly, and moving all the tests that have moved around to their new homes.
Reviewed By: yfeldblum
Differential Revision:
D6191644
fbshipit-source-id:
bdb39d01a796c1e52257200c0d411a4cb44116ce
Felix Leupold [Mon, 30 Oct 2017 22:26:55 +0000 (15:26 -0700)]
Allow to pass ObjC blocks into folly::Function
Summary:
In iOS blocks are initially allocated on the stack and only lazily copied to the heap (e.g when they are assigned to a variable). This means, if you pass a block as an rvalue to a C++ method that keeps moving it around instead of copy assigning it at some point, the block remains on the stack and will get freed once the original method is done (leading to use after free if the block is executed later).
This was mitigated by deleting the conversion from ObjC functions to folly functions. Given that all we need is to make sure that the block is allocated on the heap (that is it is an instance of NSMallocBlock rather than NSStackBlock), it seems drastic to ban the conversion. ObjC developers tend to be more familiar with ObjC blocks and will find it convenient to use this conversion.
This diff insteads implements the constructor and assignment operator by wrapping the ObjC block in a c++ lambda and capturing it by copy. ARC keeps track of the reference count and automatically releases the block when the lambda is deallocated. Moreover, copy only increase the retain count (instead of doing an actual copy) if the block was already stored on the heap (https://www.cocoawithlove.com/2009/10/how-blocks-are-implemented-and.html section NSMallocBlock never actually copies).
Reviewed By: ericniebler
Differential Revision:
D6109932
fbshipit-source-id:
48bb446d3a66f46affba774cfe1cfb8a60c661de
Yedidya Feldblum [Mon, 30 Oct 2017 21:52:50 +0000 (14:52 -0700)]
Simplify type_t
Summary:
[Folly] Simplify `type_t` by lifting the type to be aliased into the structure template parameter list.
May also fix curious build failures in some compilers.
Reviewed By: akrieger
Differential Revision:
D6188953
fbshipit-source-id:
96e1c3af9c11959c0899c092933158922efa7e60
Lee Howes [Mon, 30 Oct 2017 21:28:33 +0000 (14:28 -0700)]
Split SemiFuture and Future into separate types. Add BasicFuture shared between them.
Summary:
To avoid the risk of bugs caused by a Future being cast to a SemiFuture, and losing some of the properties in the process, this splits SemiFuture and Future into unrelated types, sharing a private superclass for code reuse.
* Add BasicFuture in futures::detail
* Make superclass privately inherited.
* Unset executor when constructing SemiFuture from Future.
Reviewed By: yfeldblum
Differential Revision:
D6177780
fbshipit-source-id:
dea3116aeec0572bb973c2a561e17785199e86f2
Arkady Shapkin [Mon, 30 Oct 2017 19:15:00 +0000 (12:15 -0700)]
Mention Windows (Vcpkg) build in README.md
Summary: Closes https://github.com/facebook/folly/pull/697
Reviewed By: yfeldblum
Differential Revision:
D6144274
Pulled By: Orvid
fbshipit-source-id:
a79a2e36e8fcf271925e97ece2a6adbb3c074216
Eric Niebler [Mon, 30 Oct 2017 16:26:13 +0000 (09:26 -0700)]
Folly.Poly: a library for creating type-erasing polymorphic wrappers
Summary:
`Poly` is a class template that makes it relatively easy to define a type-erasing polymorphic object wrapper.
== Type-erasure
`std::function` is one example of a type-erasing polymorphic object wrapper;
`folly::exception_wrapper` is another. Type-erasure is often used as an
alternative to dynamic polymorphism via inheritance-based virtual dispatch.
The distinguishing characteristic of type-erasing wrappers are:
* **Duck typing:** Types do not need to inherit from an abstract base
class in order to be assignable to a type-erasing wrapper; they merely
need to satisfy a particular interface.
* **Value semantics:** Type-erasing wrappers are objects that can be
passed around _by value_. This is in contrast to abstract base classes
which must be passed by reference or by pointer or else suffer from
_slicing_, which causes them to lose their polymorphic behaviors.
Reference semantics make it difficult to reason locally about code.
* **Automatic memory management:** When dealing with inheritance-based
dynamic polymorphism, it is often necessary to allocate and manage
objects on the heap. This leads to a proliferation of `shared_ptr`s and
`unique_ptr`s in APIs, complicating their point-of-use. APIs that take
type-erasing wrappers, on the other hand, can often store small objects
in-situ, with no dynamic allocation. The memory management, if any, is
handled for you, and leads to cleaner APIs: consumers of your API don't
need to pass `shared_ptr<AbstractBase>`; they can simply pass any object
that satisfies the interface you require. (`std::function` is a
particularly compelling example of this benefit. Far worse would be an
inheritance-based callable solution like
`shared_ptr<ICallable<void(int)>>`. )
== Example: Defining a type-erasing function wrapper with `folly::Poly`
Defining a polymorphic wrapper with `Poly` is a matter of defining two
things:
* An *interface*, consisting of public member functions, and
* A *mapping* from a concrete type to a set of member function bindings.
Below is a (heavily commented) example of a simple implementation of a
`std::function`-like polymorphic wrapper. Its interface has only a single
member function: `operator()`
lang=c++
// An interface for a callable object of a particular signature, Fun
// (most interfaces don't need to be templates, FWIW).
template <class Fun>
struct IFunction;
template <class R, class... As>
struct IFunction<R(As...)> {
// An interface is defined as a nested class template called
// Interface that takes a single template parameter, Base, from
// which it inherits.
template <class Base>
struct Interface : Base {
// The Interface has public member functions. These become the
// public interface of the resulting Poly instantiation.
// (Implementation note: Poly<IFunction<Sig>> will publicly
// inherit from this struct, which is what gives it the right
// member functions.)
R operator()(As... as) const {
// The definition of each member function in your interface will
// always consist of a single line dispatching to folly::call<N>.
// The "N" corresponds to the N-th member function in the
// list of member function bindings, Members, defined below.
// The first argument will always be *this, and the rest of the
// arguments should simply forward (if necessary) the member
// function's arguments.
return static_cast<R>(
folly::poly_call<0>(*this, std::forward<As>(as)...));
}
};
// The "Members" alias template is a comma-separated list of bound
// member functions for a given concrete type "T". The
// "FOLLY_POLY_MEMBERS" macro accepts a comma-separated list, and the
// (optional) "FOLLY_POLY_MEMBER" macro lets you disambiguate overloads
// by explicitly specifying the function signature the target member
// function should have. In this case, we require "T" to have a
// function call operator with the signature `R(As...) const`.
//
// If you are using a C++17-compatible compiler, you can do away with
// the macros and write this as:
//
// template <class T>
// using Members =
// folly::PolyMembers<folly::sig<R(As...) const>(&T::operator())>;
//
// And since `folly::sig` is only needed for disambiguation in case of
// overloads, if you are not concerned about objects with overloaded
// function call operators, it could be further simplified to:
//
// template <class T> using Members = folly::PolyMembers<&T::operator()>;
//
template <class T>
using Members = FOLLY_POLY_MEMBERS(
FOLLY_POLY_MEMBER(R(As...) const, &T::operator()));
};
// Now that we have defined the interface, we can pass it to Poly to
// create our type-erasing wrapper:
template <class Fun>
using Function = Poly<IFunction<Fun>>;
Given the above definition of `Function`, users can now initialize instances
of (say) `Function<int(int, int)>` with function objects like
`std::plus<int>` and `std::multiplies<int>`, as below:
lang=c++
Function<int(int, int)> fun = std::plus<int>{};
assert(5 == fun(2, 3));
fun = std::multiplies<int>{};
assert(6 = fun(2, 3));
== Defining an interface with C++17
With C++17, defining an interface to be used with `Poly` is fairly
straightforward. As in the `Function` example above, there is a struct with
a nested `Interface` class template and a nested `Members` alias template.
No macros are needed with C++17.
Imagine we were defining something like a Java-style iterator. If we are
using a C++17 compiler, our interface would look something like this:
lang=c++
template <class Value>
struct IJavaIterator {
template <class Base>
struct Interface : Base {
bool Done() const { return folly::poly_call<0>(*this); }
Value Current() const { return folly::poly_call<1>(*this); }
void Next() { folly::poly_call<2>(*this); }
};
// NOTE: This works in C++17 only:
template <class T>
using Members = folly::PolyMembers<&T::Done, &T::Current, &T::Next>;
};
template <class Value>
using JavaIterator = Poly<IJavaIterator>;
Given the above definition, `JavaIterator<int>` can be used to hold instances
of any type that has `Done`, `Current`, and `Next` member functions with the
correct (or compatible) signatures.
The presence of overloaded member functions complicates this picture. Often,
property members are faked in C++ with `const` and non-`const` member
function overloads, like in the interface specified below:
lang=c++
struct IIntProperty {
template <class Base>
struct Interface : Base {
int Value() const { return folly::poly_call<0>(*this); }
void Value(int i) { folly::poly_call<1>(*this, i); }
};
// NOTE: This works in C++17 only:
template <class T>
using Members = folly::PolyMembers<
folly::sig<int() const>(&T::Value),
folly::sig<void(int)>(&T::Value)>;
};
using IntProperty = Poly<IIntProperty>;
Now, any object that has `Value` members of compatible signatures can be
assigned to instances of `IntProperty` object. Note how `folly::sig` is used
to disambiguate the overloads of `&T::Value`.
== Defining an interface with C++14
In C++14, the nice syntax above doesn't work, so we have to resort to macros.
The two examples above would look like this:
lang=c++
template <class Value>
struct IJavaIterator {
template <class Base>
struct Interface : Base {
bool Done() const { return folly::poly_call<0>(*this); }
Value Current() const { return folly::poly_call<1>(*this); }
void Next() { folly::poly_call<2>(*this); }
};
// NOTE: This works in C++14 and C++17:
template <class T>
using Members = FOLLY_POLY_MEMBERS(&T::Done, &T::Current, &T::Next);
};
template <class Value>
using JavaIterator = Poly<IJavaIterator>;
and
lang=c++
struct IIntProperty {
template <class Base>
struct Interface : Base {
int Value() const { return folly::poly_call<0>(*this); }
void Value(int i) { return folly::poly_call<1>(*this, i); }
};
// NOTE: This works in C++14 and C++17:
template <class T>
using Members = FOLLY_POLY_MEMBERS(
FOLLY_POLY_MEMBER(int() const, &T::Value),
FOLLY_POLY_MEMBER(void(int), &T::Value));
};
using IntProperty = Poly<IIntProperty>;
== Extending interfaces
One typical advantage of inheritance-based solutions to runtime polymorphism
is that one polymorphic interface could extend another through inheritance.
The same can be accomplished with type-erasing polymorphic wrappers. In
the `Poly` library, you can use `folly::PolyExtends` to say that one interface
extends another.
lang=c++
struct IFoo {
template <class Base>
struct Interface : Base {
void Foo() const { return folly::poly_call<0>(*this); }
};
template <class T>
using Members = FOLLY_POLY_MEMBERS(&T::Foo);
};
// The IFooBar interface extends the IFoo interface
struct IFooBar : PolyExtends<IFoo> {
template <class Base>
struct Interface : Base {
void Bar() const { return folly::poly_call<0>(*this); }
};
template <class T>
using Members = FOLLY_POLY_MEMBERS(&T::Bar);
};
using FooBar = Poly<IFooBar>;
Given the above definition, instances of type `FooBar` have both `Foo()` and
`Bar()` member functions.
The sensible conversions exist between a wrapped derived type and a wrapped
base type. For instance, assuming `IDerived` extends `IBase` with `Extends`:
lang=c++
Poly<IDerived> derived = ...;
Poly<IBase> base = derived; // This conversion is OK.
As you would expect, there is no conversion in the other direction, and at
present there is no `Poly` equivalent to `dynamic_cast`.
== Type-erasing polymorphic reference wrappers
Sometimes you don't need to own a copy of an object; a reference will do. For
that you can use `Poly` to capture a //reference// to an object satisfying an
interface rather than the whole object itself. The syntax is intuitive.
lang=c++
int i = 42;
// Capture a mutable reference to an object of any IRegular type:
Poly<IRegular &> intRef = i;
assert(42 == folly::poly_cast<int>(intRef));
// Assert that we captured the address of "i":
assert(&i == &folly::poly_cast<int>(intRef));
A reference-like `Poly` has a different interface than a value-like `Poly`.
Rather than calling member functions with the `obj.fun()` syntax, you would
use the `obj->fun()` syntax. This is for the sake of `const`-correctness.
For example, consider the code below:
lang=c++
struct IFoo {
template <class Base>
struct Interface {
void Foo() { folly::poly_call<0>(*this); }
};
template <class T>
using Members = folly::PolyMembers<&T::Foo>;
};
struct SomeFoo {
void Foo() { std::printf("SomeFoo::Foo\n"); }
};
SomeFoo foo;
Poly<IFoo &> const anyFoo = foo;
anyFoo->Foo(); // prints "SomeFoo::Foo"
Notice in the above code that the `Foo` member function is non-`const`.
Notice also that the `anyFoo` object is `const`. However, since it has
captured a non-`const` reference to the `foo` object, it should still be
possible to dispatch to the non-`const` `Foo` member function. When
instantiated with a reference type, `Poly` has an overloaded `operator->`
member that returns a pointer to the `IFoo` interface with the correct
`const`-ness, which makes this work.
The same mechanism also prevents users from calling non-`const` member
functions on `Poly` objects that have captured `const` references, which
would violate `const`-correctness.
Sensible conversions exist between non-reference and reference `Poly`s. For
instance:
lang=c++
Poly<IRegular> value = 42;
Poly<IRegular &> mutable_ref = value;
Poly<IRegular const &> const_ref = mutable_ref;
assert(&poly_cast<int>(value) == &poly_cast<int>(mutable_ref));
assert(&poly_cast<int>(value) == &poly_cast<int>(const_ref));
== Non-member functions (C++17)
If you wanted to write the interface `ILogicallyNegatable`, which captures
all types that can be negated with unary `operator!`, you could do it
as we've shown above, by binding `&T::operator!` in the nested `Members`
alias template, but that has the problem that it won't work for types that
have defined unary `operator!` as a free function. To handle this case,
the `Poly` library lets you use a free function instead of a member function
when creating a binding.
With C++17 you may use a lambda to create a binding, as shown in the example
below:
lang=c++
struct ILogicallyNegatable {
template <class Base>
struct Interface : Base {
bool operator!() const { return folly::poly_call<0>(*this); }
};
template <class T>
using Members = folly::PolyMembers<
+[](T const& t) -> decltype(!t) { return !t; }>;
};
This requires some explanation. The unary `operator+` in front of the lambda
is necessary! It causes the lambda to decay to a C-style function pointer,
which is one of the types that `folly::PolyMembers` accepts. The `decltype` in
the lambda return type is also necessary. Through the magic of SFINAE, it
will cause `Poly<ILogicallyNegatable>` to reject any types that don't support
unary `operator!`.
If you are using a free function to create a binding, the first parameter is
implicitly the `this` parameter. It will receive the type-erased object.
== Non-member functions (C++14)
If you are using a C++14 compiler, the definition of `ILogicallyNegatable`
above will fail because lambdas are not `constexpr`. We can get the same
effect by writing the lambda as a named free function, as show below:
lang=c++
struct ILogicallyNegatable {
template <class Base>
struct Interface : Base {
bool operator!() const { return folly::poly_call<0>(*this); }
};
template <class T>
static auto negate(T const& t) -> decltype(!t) { return !t; }
template <class T>
using Members = FOLLY_POLY_MEMBERS(&negate<T>);
};
As with the example that uses the lambda in the preceding section, the first
parameter is implicitly the `this` parameter. It will receive the type-erased
object.
== Multi-dispatch
What if you want to create an `IAddable` interface for things that can be
added? Adding requires //two// objects, both of which are type-erased. This
interface requires dispatching on both objects, doing the addition only
if the types are the same. For this we make use of the `Self` template
alias to define an interface that takes more than one object of the the
erased type.
lang=c++
struct IAddable {
template <class Base>
struct Interface : Base {
friend Self<Base>
operator+(Self<Base> const& a, Self<Base> const& b) const {
return folly::poly_call<0>(a, b);
}
};
template <class T>
using Members = folly::PolyMembers<
+[](T const& a, T const& b) -> decltype(a + b) { return a + b; }>;
};
Given the above defintion of `IAddable` we would be able to do the following:
lang=c++
Poly<IAddable> a = 2, b = 3;
Poly<IAddable> c = a + b;
assert(poly_cast<int>(c) == 5);
If `a` and `b` stored objects of different types, a `BadPolyCast` exception
would be thrown.
== Move-only types
If you want to store move-only types, then your interface should extend the
`IMoveOnly` interface.
== Implementation notes
`Poly` will store "small" objects in an internal buffer, avoiding the cost of
of dynamic allocations. At present, this size is not configurable; it is
pegged at the size of two `double`s.
`Poly` objects are always nothrow movable. If you store an object in one that
has a potentially throwing move contructor, the object will be stored on the
heap, even if it could fit in the internal storage of the `Poly` object.
(So be sure to give your objects nothrow move constructors!)
`Poly` implements type-erasure in a manner very similar to how the compiler
accomplishes virtual dispatch. Every `Poly` object contains a pointer to a
table of function pointers. Member function calls involve a double-
indirection: once through the v-pointer, and other indirect function call
through the function pointer.
Reviewed By: yfeldblum
Differential Revision:
D4897112
fbshipit-source-id:
ff1c1156316bfbdd8f2205c4f57932c0067cacac
Alexander Pronchenkov [Mon, 30 Oct 2017 12:13:02 +0000 (05:13 -0700)]
Remove a few memory allocations in ThreadWheelTimekeeper.after()
Summary:
This diff reduces number of memory allocation in folly::ThreadWheelTimekeeper.after() method for a bit.
* std::shared_ptr(new T) is replaced with std::make_shared<T>()
* folly::Promise is stored by value
Reviewed By: yfeldblum
Differential Revision:
D6172017
fbshipit-source-id:
41bf123f10570c76d64eaac1800b7e65fe381110
Murali Vilayannur [Mon, 30 Oct 2017 05:07:46 +0000 (22:07 -0700)]
Override TemporaryFile's default move constructor
Summary:
A default move constructor/move assignment operator caused a bunch
of bugs in my test that was depending on the move constructor to either
not close the underlying file descriptor or to ensure that the TemporaryFile
object doesn't have a dangling reference to a file descriptor that has already
been closed. The current implementation caused both the moved' from and to
object to share the same underlying file descriptor and when the former object
is destroyed it ends up closing the file descriptor leaving the latter
with a dangling file descriptor which could be reused for some other
socket(s)/file descriptor by the kernel and ends up causing seg-faults
when the latter object is eventually destroyed due to it releasing
a file descriptor that now belongs to some other resource.
I changed the move constructor/move assignment operator
to have the former semantics to not close the underlying file descriptor of
the move'd from object (by releasing it and assigning it to the move'd to
object). I am not sure if anyone would ever want the alternative
semantics because the TemporaryFile object would not be usable
without a valid underlying file descriptor
Reviewed By: yfeldblum
Differential Revision:
D6182075
fbshipit-source-id:
bc809d704449c1c1182d76cdfa2f7d17b38a719a
Yedidya Feldblum [Sun, 29 Oct 2017 20:55:29 +0000 (13:55 -0700)]
CodeMod: Replace includes of folly/Hash.h with folly/hash/Hash.h
Summary: CodeMod: Replace includes of `folly/Hash.h` with `folly/hash/Hash.h`.
Reviewed By: luciang
Differential Revision:
D6156195
fbshipit-source-id:
0941b3c9cf1d17d7cc62595111e506c06ee51236
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:15 +0000 (03:33 -0700)]
Make SysBufferDeleter::operator() inlineable
Summary: [Folly] Make `SysBufferDeleter::operator()` inlineable, specifically, into `SysBufferUniquePtr`.
Reviewed By: luciang
Differential Revision:
D6182999
fbshipit-source-id:
e0409c0019f21ed44d7d4c531ebc11a239f25831
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:14 +0000 (03:33 -0700)]
Move folly/Launder.h
Summary: [Folly] Move `folly/Launder.h` to `folly/lang/`.
Reviewed By: luciang
Differential Revision:
D6182882
fbshipit-source-id:
97e46bd4e4d6f212d8234209ee90a41e7850ecb9
Yedidya Feldblum [Sun, 29 Oct 2017 10:33:10 +0000 (03:33 -0700)]
Move folly/Array.h
Summary: [Folly] Move `folly/Array.h` to `folly/container/`.
Reviewed By: luciang
Differential Revision:
D6182858
fbshipit-source-id:
59340b96058cc6d0c7a0289e316bbde98c15d724
Yedidya Feldblum [Sun, 29 Oct 2017 04:19:52 +0000 (21:19 -0700)]
Move folly/Assume.h
Summary: [Folly] Move `folly/Assume.h` to `folly/lang/`.
Reviewed By: luciang, ot
Differential Revision:
D6181983
fbshipit-source-id:
25564bb07daa1a6765651cd919b4778efb931446
Andrew Krieger [Sat, 28 Oct 2017 20:47:17 +0000 (13:47 -0700)]
Break out BitIterator into its own header
Summary:
BitIterator is a standalone construct that has a heavy boost
dependency, but not common usage. Breaking it out into its own header
will help a lot of transitive dependendencies, because it is included
in Hash.h which is included in a variety of other common headers.
This reduces the number of transitively included headers by 248 (!)
Reviewed By: yfeldblum, ot, luciang
Differential Revision:
D6178564
fbshipit-source-id:
1380154b012615b7b8c73bc15ab0ac62f6b990d3
Yedidya Feldblum [Sat, 28 Oct 2017 18:31:23 +0000 (11:31 -0700)]
Move folly/detail/AtomicUtils.h
Summary: [Folly] Move `folly/detail/AtomicUtils.h` to `folly/synchronization/detail/`.
Reviewed By: Orvid
Differential Revision:
D6180482
fbshipit-source-id:
5671c149a59eea824db2935ffabcf85a2f78b690
Scott Michelson [Sat, 28 Oct 2017 00:33:45 +0000 (17:33 -0700)]
LOG_EVERY_N instead of every time
Summary: when queues fill, this starts blowing up, affecting server performance and making logs useless. It's useful to know queues are full, but we don't need the log every time we try to append
Reviewed By: yfeldblum
Differential Revision:
D6175784
fbshipit-source-id:
b4e6966087c4a6f9fba51d7f9193b9f41e13b899
Yedidya Feldblum [Fri, 27 Oct 2017 04:24:48 +0000 (21:24 -0700)]
Fix FunctionScheduler::resetFunctionTimer concurrency bug
Summary:
[Folly] Fix `FunctionScheduler::resetFunctionTimer` concurrency bug.
The original code from the blamed diff code has a comment with this correct assessment of the bug:
> TODO: This moves out of RepeatFunc object while folly:Function can potentially be executed. This might be unsafe.
Namely, when the method is invoked with the id of a scheduled function which is concurrently being executed, the function object (including, say, lambda captures) will be moved while possibly being accessed. If the function object is small enough to be placed in-situ within `folly::Function` (48 bytes on x64), then that access to a moved-from object can happen. It might or might not, depending on the particular instance of the race in question. Or, worse, the access might be to a half-moved-from object!
The new test case for `resetFunctionTimer` passes after the fix, but is guaranteed to fail before the fix because we manage to control the concurrency enough to force the bad version of the race to happen. In the test, we just capture a `std::shared_ptr` (we could have capatured, e.g., a `std::unique_ptr` or a long-enough `std::string`) and check that it is not empty - if it is moved from, it will be empty, and the test expectation will fail.
Reviewed By: simpkins
Differential Revision:
D6158722
fbshipit-source-id:
33a7ae699bb3b22089fddbebb6d922737668309d
Yedidya Feldblum [Fri, 27 Oct 2017 02:35:23 +0000 (19:35 -0700)]
type_t, a generalization of void_t
Summary: [Folly] `type_t`, a generalization of `void_t`.
Reviewed By: ericniebler
Differential Revision:
D6082913
fbshipit-source-id:
f9557b5da1f6684b12d570b6c1bd52c102cb0703
Christopher Dykes [Thu, 26 Oct 2017 19:15:55 +0000 (12:15 -0700)]
Just use a volatile static in Malloc.h
Summary: As suggested late in the initial diff (
D5840883), just mark it volatile, as it works under all supported platforms.
Reviewed By: davidtgoldblatt, yfeldblum
Differential Revision:
D6155241
fbshipit-source-id:
00c07a11dc7fc2e33c2d1f9a45fd28006eeff6f9
Christopher Dykes [Thu, 26 Oct 2017 19:14:23 +0000 (12:14 -0700)]
Fix build with Windows SDK v10.0.16232
Summary:
It defines `MSG_ERRQUEUE` which breaks things.
There's a github PR to do this in a different way, but it's faster to just do it myself.
Closes https://github.com/facebook/folly/pull/689
Reviewed By: yfeldblum
Differential Revision:
D6155606
fbshipit-source-id:
f1c6b247efc452b4005ad3b6d82fabfd5a92f49f
Dan Melnic [Thu, 26 Oct 2017 15:37:29 +0000 (08:37 -0700)]
Disable zerocopy if we're notified about deferred copies, add a isZeroCopyWriteInProgress method, replace pair with a proper struct
Summary: Add zeroWriteDone callback
Reviewed By: djwatson
Differential Revision:
D6097129
fbshipit-source-id:
b82a942557680c3a7a3be8f81ee6f2886e99e165
Yedidya Feldblum [Thu, 26 Oct 2017 03:27:41 +0000 (20:27 -0700)]
Expected coroutines support
Summary:
[Folly] `Expected` coroutines support.
Copied from `Optional` coroutines support.
Reviewed By: ericniebler, Orvid
Differential Revision:
D5923792
fbshipit-source-id:
8661012c65762a0e540a4af2fd2fc237a8cb87a1
Kenny Yu [Thu, 26 Oct 2017 01:11:02 +0000 (18:11 -0700)]
folly: Fix data race in folly::Codel
Summary:
Data race reported by TSAN:
```
WARNING: ThreadSanitizer: data race (pid=608219)
Read of size 1 at 0x7b5800000c29 by thread T314:
#0 0x60b3441 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ./folly/executors/Codel.cpp:76
#1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
#2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
#3 0x619739d in __tsan_thread_start_func crtstuff.c:?
Previous write of size 1 at 0x7b5800000c29 by thread T315:
#0 0x60b33e4 in folly::Codel::overloaded(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) ??:?
#1 0x5c1222 in apache::thrift::concurrency::ThreadManager::ImplT<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::Worker<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >::run() ./thrift/lib/cpp/concurrency/ThreadManager.tcc:119
#2 0x5d803e7 in apache::thrift::concurrency::PthreadThread::threadMain(void*) ./thrift/lib/cpp/concurrency/PosixThreadFactory.cpp:200
#3 0x619739d in __tsan_thread_start_func crtstuff.c:?
Location is heap block of size 768 at 0x7b5800000c00 allocated by main thread:
#0 0x616ab83 in operator new(unsigned long) ??:?
#1 0x53cb92 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > >, std::allocator<apache::thrift::concurrency::SimpleThreadManager<folly::LifoSemImpl<std::atomic, folly::Baton<std::atomic, true, true> > > >, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*)
...
```
It looks like there are multiple threads reading and writing the `overloaded_` bool. To fix it, wrap it in a `std::atomic`.
Reviewed By: yfeldblum, meyering
Differential Revision:
D6149766
fbshipit-source-id:
605b29fa2f602d2ed2dfc22e46b739ef169f914e
Luca Niccolini [Wed, 25 Oct 2017 21:28:42 +0000 (14:28 -0700)]
rework reads/writes
Summary:
the current implementation had problems with scheduling reads and writes and
it would sometimes get stuck when transfering large chunks of data
here I am restructuring the code to look more like the one in HTTPSession
session flow control is not implemented yet really, it's coming next
Depends on:
D6048238
Reviewed By: afrind
Differential Revision:
D6048238
fbshipit-source-id:
ae601e771154a7f1a669a58a6e05c9e3720e7017
Marko Novakovic [Wed, 25 Oct 2017 17:15:40 +0000 (10:15 -0700)]
Supply an explicit default dtor impl
Reviewed By: yfeldblum
Differential Revision:
D6142252
fbshipit-source-id:
9ac98585a92299ca5915982c65c7d2cfa68bf60f
Yedidya Feldblum [Wed, 25 Oct 2017 05:05:23 +0000 (22:05 -0700)]
Flesh out Optional members swap, reset, emplace, has_value
Summary:
[Folly] Flesh out `Optional` members `swap`, `reset`, `emplace`, `has_value`.
* `swap` as a member and deriving `noexcept`-ness to mimic `std::optional::swap`.
* `reset` v.s. `clear` to mimic `std::optional::reset`.
* `emplace` returning ref and overload taking initializer list to mimic `std::optional::emplace`.
* `has_value` v.s. `hasValue` to mimic `std::optional::has_value`.
Reviewed By: WillerZ
Differential Revision:
D6132775
fbshipit-source-id:
34c58367b9dc63289e4b9721c5e79b1c41ba31e4
Yedidya Feldblum [Wed, 25 Oct 2017 00:16:58 +0000 (17:16 -0700)]
Simplify IsUniquePtrToSL in IOBuf.h
Summary: [Folly] Simplify `IsUniquePtrToSL` in `IOBuf.h`.
Reviewed By: Orvid
Differential Revision:
D6131231
fbshipit-source-id:
b054ef7ef9f313943a3ac1022ca6a23874a464df
Yedidya Feldblum [Tue, 24 Oct 2017 21:12:03 +0000 (14:12 -0700)]
Move folly/Hash.h to folly/hash/, leaving a shim
Summary: [Folly] Move `folly/Hash.h` to `folly/hash/`, leaving a shim.
Reviewed By: Orvid
Differential Revision:
D6132955
fbshipit-source-id:
dc789e9c6daa28116be6a5d83c3cfbb40e247114
Igor Sugak [Tue, 24 Oct 2017 17:55:58 +0000 (10:55 -0700)]
fix build with asan and static linking
Reviewed By: meyering
Differential Revision:
D6117783
fbshipit-source-id:
048b056e119bf89ab88c33b1233297d197e8acb9
Christopher Dykes [Tue, 24 Oct 2017 02:26:29 +0000 (19:26 -0700)]
Prefer bool literals rather than integers in boolean contexts
Summary: Via clang-tidy's modernize-use-bool-literals
Reviewed By: yfeldblum
Differential Revision:
D6130384
fbshipit-source-id:
359d5195897f04612c9b9042cf69383050a2ec7a
Christopher Dykes [Tue, 24 Oct 2017 00:34:54 +0000 (17:34 -0700)]
Consistently have the namespace closing comment
Summary: It's done in a lot of places, but not all. clang-tidy gives us llvm-namespace-comment, which can force this consistently.
Reviewed By: yfeldblum
Differential Revision:
D6108129
fbshipit-source-id:
1b44c5a26250364f9edf70f84173e9ba6389f06c
Christopher Dykes [Mon, 23 Oct 2017 23:52:19 +0000 (16:52 -0700)]
Modernize uses of std::make_shared
Summary: Via the modernize-make-shared from clang-tidy
Reviewed By: yfeldblum
Differential Revision:
D6129464
fbshipit-source-id:
04f560c6beeb2b8631b819fd4e6a2d51b37eeb4b
Lee Howes [Mon, 23 Oct 2017 20:36:48 +0000 (13:36 -0700)]
Add a full drain for folly's ManualExecutor.
Summary: ManualExecutor::run() is stable, which means that in cases where we want to fully drain the executor we have to loop over it. This adds ManualExecutor::drain() which does that internally.
Reviewed By: yfeldblum
Differential Revision:
D6126840
fbshipit-source-id:
e36cba5c373a57fe01de244977ec852636b58dbd
James Sedgwick [Mon, 23 Oct 2017 19:19:27 +0000 (12:19 -0700)]
move Iterator, Enumerate, EvictingCacheMap, Foreach, Merge, and
Summary: this is all non-hphp includes that are going in container/
Reviewed By: mzlee, yfeldblum
Differential Revision:
D6121745
fbshipit-source-id:
b024bde8835fc7f332686793d75eb8e71591c912
Qi Wang [Mon, 23 Oct 2017 19:09:40 +0000 (12:09 -0700)]
MemoryIdler: use mallctl directly for tcache.flush
Summary:
tcache.flush may fail if tcache is disabled. Avoid using mallctlCall
which throws on error.
Reviewed By: davidtgoldblatt
Differential Revision:
D6115419
fbshipit-source-id:
39411c80af08dc7c855efd43297809b749f935bf
Christopher Dykes [Mon, 23 Oct 2017 18:49:39 +0000 (11:49 -0700)]
Modernize use of std::make_unique
Summary: Via the clang-tidy check modernize-make-unique.
Reviewed By: yfeldblum
Differential Revision:
D6107790
fbshipit-source-id:
1cf186feae511cbd91f44893059737a85778b6cf
Jon Maltiel Swenson [Mon, 23 Oct 2017 17:16:53 +0000 (10:16 -0700)]
Rename unique_lock variables in `TimedMutex` in order to avoid shadowing
Summary: Rename `std::unique_lock` variables named `lock` to `ulock` in `folly::fibers::TimedMutex` member functions in order to avoid shadowing.
Reviewed By: andreazevedo
Differential Revision:
D6123449
fbshipit-source-id:
5fa331bb1541ac995d9b69360ee09923c14f6698
Andrew Krieger [Mon, 23 Oct 2017 16:36:28 +0000 (09:36 -0700)]
Gate std::invoke_result et. al. to appropriate MSVC versions.
Summary: Only available in >= 2017 15.3, which is 1911+.
Reviewed By: aary, Orvid
Differential Revision:
D6117237
fbshipit-source-id:
255804af5bfd0c743fd225b8a4fddf3cfc9cfeaf
James Sedgwick [Mon, 23 Oct 2017 14:27:04 +0000 (07:27 -0700)]
move Arena, ThreadCachedArena, and Malloc to memory/
Summary: all memory/-destined components besides hphp includes
Reviewed By: yfeldblum, mzlee
Differential Revision:
D6121822
fbshipit-source-id:
6c6214d84dcdefe4789ed5200399ae27203d6340
James Sedgwick [Sun, 22 Oct 2017 20:18:56 +0000 (13:18 -0700)]
move futures/test/ExecutorTest.cpp to executors/
Summary: also gotta split it up/rename it, that's coming later
Reviewed By: yfeldblum
Differential Revision:
D6121525
fbshipit-source-id:
9c6dbabd47323d94657508a0f75a0c6e7f988ace
James Sedgwick [Sat, 21 Oct 2017 22:28:58 +0000 (15:28 -0700)]
move InlineExecutor, ManualExecutor, and GlobalThreadPoolList to
Summary:
That's everything that's going in executors/ except for Executor.h
itself, which is included in hphp so will have to wait
Reviewed By: mzlee
Differential Revision:
D6100274
fbshipit-source-id:
6be37892b1ad7f46828acfa6b2951e51b157a86a
James Sedgwick [Sat, 21 Oct 2017 20:00:55 +0000 (13:00 -0700)]
move MemoryMapping, Shell, ThreadId, ThreadName, and VersionCheck to system/
Summary:
Everything that's going in system/ besides CpuId and Subprocess,
which are included in hphp
Reviewed By: mzlee
Differential Revision:
D6102263
fbshipit-source-id:
564ef584c341a4ac79db14a9d58fe23ce51e78b3
James Sedgwick [Sat, 21 Oct 2017 17:25:08 +0000 (10:25 -0700)]
move io/Compression and io/compression/* to compression/
Summary:
as above
(Note: this ignores all push blocking failures!)
Reviewed By: yfeldblum
Differential Revision:
D6099826
fbshipit-source-id:
20152487135aa8eaf6d2e99369801b6dde4992aa
Yedidya Feldblum [Sat, 21 Oct 2017 05:25:03 +0000 (22:25 -0700)]
Use folly/portability/GTest.h in folly/executurs/test/
Summary:
[Folly] Use `folly/portability/GTest.h` in `folly/executurs/test/`.
Applying the general rule for folly tests to some recently-introduced violations.
Reviewed By: Orvid
Differential Revision:
D6113707
fbshipit-source-id:
f9a2961ac845489b85b5a539595e4c82df03d922
James Sedgwick [Fri, 20 Oct 2017 20:30:03 +0000 (13:30 -0700)]
move executor task queues and thread factories into subdirectories
Summary:
as title, see moves
(Note: this ignores all push blocking failures!)
Reviewed By: mzlee
Differential Revision:
D6112001
fbshipit-source-id:
1eb10b44ae8ee1f90a10e05c29e48c99d824afa5
Christopher Dykes [Fri, 20 Oct 2017 17:29:30 +0000 (10:29 -0700)]
Drop redundant void parameters from function declarations
Summary:
This is C++, not C, so let's keep things modern.
This is applying the modernize-redundant-void-arg clang-tidy check to Folly.
Reviewed By: yfeldblum
Differential Revision:
D6106921
fbshipit-source-id:
06a56f036f59426df5dd475bf91a3f4a335266f5
Christopher Dykes [Fri, 20 Oct 2017 05:48:29 +0000 (22:48 -0700)]
Match commented argument names with actual names
Summary: Via the clang-tidy check misc-argument-comment.
Reviewed By: yfeldblum
Differential Revision:
D6107482
fbshipit-source-id:
a1fe6215c31fae472ad3b6e05abea974d706794e
Christopher Dykes [Thu, 19 Oct 2017 23:13:58 +0000 (16:13 -0700)]
Use nullptr rather than 0 for a null pointer
Summary: This time aided by clang-tidy's modernize-use-nullptr check.
Reviewed By: yfeldblum
Differential Revision:
D6102739
fbshipit-source-id:
aeb4bd0a8078d81cc88b766e0a034a37dd25fd1f
James Sedgwick [Thu, 19 Oct 2017 21:08:04 +0000 (14:08 -0700)]
move CallOnce to synchronization/
Summary: as above
Reviewed By: knekritz
Differential Revision:
D6088687
fbshipit-source-id:
0efbb7f5fa33b5f553c0c2019658370fc6e8613f
Alex Yarmula [Thu, 19 Oct 2017 20:35:27 +0000 (13:35 -0700)]
Handle timekeeperSingleton being nullptr in within()
Summary: When timekeeper singleton no longer exists during shutdown and folly::Singleton::try_get() can return nullptr, make sure nullptr is handled gracefully.
Reviewed By: yfeldblum
Differential Revision:
D6101311
fbshipit-source-id:
fefeddfbd048d1a7632688bb3526db15b685dd72
Yedidya Feldblum [Thu, 19 Oct 2017 19:33:58 +0000 (12:33 -0700)]
Add InlineExecutor.cpp to Makefile.am
Summary: [Folly] Add `InlineExecutor.cpp` to `Makefile.am`.
Reviewed By: Orvid
Differential Revision:
D6101859
fbshipit-source-id:
4cd0f875dedb548189722fd719f4424d485e1b02
Yedidya Feldblum [Thu, 19 Oct 2017 18:37:54 +0000 (11:37 -0700)]
Fix fibers build on older boost
Summary:
[Folly] Fix `folly/fibers/` build on older boost.
Where `jump_fcontext` takes `intptr_t` rather than an actual pointer, and we warn about implicit conversions from pointers to integral types.
Reviewed By: Orvid
Differential Revision:
D6098230
fbshipit-source-id:
61bd7bdff5de728d3febe8b35c97d024ab6f236a
Christopher Dykes [Thu, 19 Oct 2017 18:04:53 +0000 (11:04 -0700)]
Ensure curly-braces around control-flow
Summary: The style guidelines say control flow should always have curly braces, and we follow that, mostly. This just uses clang-tidy to clean up everywhere that we weren't.
Reviewed By: markisaa, luciang
Differential Revision:
D6097377
fbshipit-source-id:
bfe6766c37bd863ecf68851ef93265a200d4259d
James Sedgwick [Thu, 19 Oct 2017 14:45:51 +0000 (07:45 -0700)]
move ApplyTuple to functional/
Summary: as above
Reviewed By: yfeldblum
Differential Revision:
D6086563
fbshipit-source-id:
ab7f50ba46ebd1dbef6438f956258b2fbb13cb5c
Peter Alexander [Thu, 19 Oct 2017 09:46:56 +0000 (02:46 -0700)]
Add ProducerConsumerQueue::capacity()
Summary: Simple addition. Easy to track externally, but might as well provide it in the class if it is readily available.
Reviewed By: yfeldblum
Differential Revision:
D6093826
fbshipit-source-id:
9d8c02891b2cea9ce0d3f6ea78e1e0055b536eb8
Yedidya Feldblum [Thu, 19 Oct 2017 09:30:31 +0000 (02:30 -0700)]
Revert
D6050464: [Folly] Move folly/Hash.h to folly/hash/
Summary:
This reverts commit
64eb65aac8e3e7cd0126e65ca3998bfe167e2d73
bypass-lint
Differential Revision:
D6050464
fbshipit-source-id:
1ed63f30837dc11ae57b316f1f7cb233a210894a
Walker Mills [Thu, 19 Oct 2017 07:57:03 +0000 (00:57 -0700)]
Add window overload that takes an executor to prevent stack overflow
Summary:
AIUI, if there is no executor available, then callbacks are executed inline. `folly::window` uses a recursive helper function (`spawn`) to handle chaining callbacks. So if `window` is used on a large enough collection of `Future`s without executors (e.g., created by `makeFuture`, or have otherwise already completed), you get a stack overflow. A minimal repro looks like:
```
int main(int argc, char** argv) {
std::vector<int> v(100000);
for(int i=0; i < v.size(); i++) {
v[i] = i;
}
std::vector<folly::Future<folly::Unit>> f =
folly::window(
std::move(v),
[](int /* unused */) { return folly::makeFuture(); },
1);
folly::collectAll(f).get();
}
```
This diff resolves the issue by adding an overload of `folly::window` which takes an executor as its first parameter. The executor-less `window` overload calls through to the new function using an `InlineExecutor` as the default executor.
Reviewed By: yfeldblum
Differential Revision:
D6038733
fbshipit-source-id:
5dcab575592650efa2e106f12632ec06817a0009
Yedidya Feldblum [Thu, 19 Oct 2017 07:37:48 +0000 (00:37 -0700)]
Move folly/Hash.h to folly/hash/
Summary: [Folly] Move `folly/Hash.h` to `folly/hash/`.
Reviewed By: jsedgwick
Differential Revision:
D6050464
fbshipit-source-id:
64eb65aac8e3e7cd0126e65ca3998bfe167e2d73
Arkady Shapkin [Thu, 19 Oct 2017 02:35:00 +0000 (19:35 -0700)]
Enable bug workaround also for MSVC 2017.4
Summary:
Workaround a bug in template instantiation in MSVC 2017 U3/4 with /permissive-
https://developercommunity.visualstudio.com/content/problem/81223/incorrect-error-c5037-with-permissive.html
/cc Orvid
Closes https://github.com/facebook/folly/pull/691
Reviewed By: yfeldblum
Differential Revision:
D6085652
Pulled By: Orvid
fbshipit-source-id:
f068dc3f8e474163815fc497b05410fe76834a52
Boliu Xu [Thu, 19 Oct 2017 02:08:13 +0000 (19:08 -0700)]
Fix incorrect example usage in stop_watch
Summary: As titled.
Reviewed By: yfeldblum
Differential Revision:
D6087099
fbshipit-source-id:
1c346fa6f65b5ea58e728759905560ae5ad9cc9d
Richard Fillman [Thu, 19 Oct 2017 00:19:59 +0000 (17:19 -0700)]
Move small fields to avoid padding in AsyncSocket.h and AsyncSSLSocket.h
Summary: Compiled mcrouter with the 'Wpadded' flag to try to find any wasted space. Mcrouter itself did not have anything, but there was a lot in AsyncSocket.h and AsyncSSLSocket.h so looked into those and removed a bit of wasted space
Reviewed By: yfeldblum, brianwatling
Differential Revision:
D6033977
fbshipit-source-id:
2dc127208e09980be6a5db576b45d30ac6e044ff
James Sedgwick [Wed, 18 Oct 2017 23:32:35 +0000 (16:32 -0700)]
move futures/QueuedImmediateExecutor to executors/QueuedImmediateExecutor
Summary: as above
Reviewed By: yfeldblum
Differential Revision:
D6076779
fbshipit-source-id:
4c223ab9fce3be8544f6f807781c3d0a99b61dad
Andrew Krieger [Wed, 18 Oct 2017 22:44:01 +0000 (15:44 -0700)]
Workaround MSVC bug with referencing template constructors before definition
Summary:
MSVC has a bug where it is unable to match an out-of-line constructor (and
possibly other kind of member) to the definition if it is implicitly referenced
before the definition is processed. The guilty method is
SemiFuture<Unit> makeSemiFuture(). Moving it after the constructor definitions
resolves the issue.
Reviewed By: yfeldblum
Differential Revision:
D6042277
fbshipit-source-id:
97fe97c0edf3df3d9e3b808968b450c73959b600
James Sedgwick [Wed, 18 Oct 2017 22:01:33 +0000 (15:01 -0700)]
move futures/ScheduledExecutor to executors/ScheduledExecutor
Summary: see title
Reviewed By: yfeldblum
Differential Revision:
D6062601
fbshipit-source-id:
edd9a5e85f4ebecd1a6f1004a4d3b8b43b935c2b
Yedidya Feldblum [Wed, 18 Oct 2017 21:18:47 +0000 (14:18 -0700)]
Move folly/MallctlHelper.h to folly/memory/
Summary: [Folly] Move `folly/MallctlHelper.h` to `folly/memory/`.
Reviewed By: aary
Differential Revision:
D6087216
fbshipit-source-id:
4e0fa4aea976e2578127d3c340e0e9b559a224ca
James Sedgwick [Wed, 18 Oct 2017 18:49:02 +0000 (11:49 -0700)]
move Partial to functional/
Summary: as above
Reviewed By: ngoyal
Differential Revision:
D6087941
fbshipit-source-id:
948ff4f2faa87dd34f87d14ea01c83335f850a27
Christopher Dykes [Wed, 18 Oct 2017 16:49:45 +0000 (09:49 -0700)]
Use nullptr rather than 0 when initializing pointers
Summary:
Because we do this in a few places, and `nullptr` makes it far clearer what the intention is.
Note that with `-Wzero-as-null-pointer-constant` under GCC, doing `std::function<void()> f = {}` initializes `f` with a `0` rather than `nullptr`, triggering the warning, so I've enabled it there as well.
It is not currently possible to actually enable `-Wzero-as-null-pointer-constant`, because GCC 5 reports conversions resulting from a default value as occuring at the call-site rather than at the location where the parameter is defined, and the default allocator in libstdc++ is not clean for this particular warning -_-...
Reviewed By: yfeldblum
Differential Revision:
D6046746
fbshipit-source-id:
6135bb20a503c861838575cf973324d74d75ca69
James Sedgwick [Wed, 18 Oct 2017 16:44:09 +0000 (09:44 -0700)]
move futures/DrivableExecutor to executors/DrivableExecutor
Summary: as title
Reviewed By: yfeldblum
Differential Revision:
D6062437
fbshipit-source-id:
4f99e779e280fdb0b1f035013caff18764e86ab5
Yedidya Feldblum [Wed, 18 Oct 2017 06:36:06 +0000 (23:36 -0700)]
Lift the invoke helper in Function.h
Summary: [Folly] Lift the `invoke` helper in `Function.h`.
Reviewed By: aary
Differential Revision:
D6050569
fbshipit-source-id:
da07901b8d058b0199d23db675c0fb9082fdf67d
James Sedgwick [Wed, 18 Oct 2017 03:21:36 +0000 (20:21 -0700)]
move python/NotificationQueueExecutor to futures/
Summary: as above
Reviewed By: yfeldblum
Differential Revision:
D6076757
fbshipit-source-id:
afe144129e8a0242ba6baee96a84a9084e6e2571
Yedidya Feldblum [Wed, 18 Oct 2017 02:28:49 +0000 (19:28 -0700)]
Simplify impl of setThreadName
Summary: [Folly] Simplify impl of `setThreadName`.
Reviewed By: Orvid, ot
Differential Revision:
D6075179
fbshipit-source-id:
720f29cc688f97b936813898238b8eb26b8a6141
Dan Melnic [Tue, 17 Oct 2017 23:27:56 +0000 (16:27 -0700)]
Do not set WriteFlags::WRITE_MSG_ZEROCOPY if the buffer ! isManaged()
Summary: Do not set WriteFlags::WRITE_MSG_ZEROCOPY if the buffer isShared()
Reviewed By: yfeldblum
Differential Revision:
D6068711
fbshipit-source-id:
fff14dcd4fcb20c9dbb60794420845042518922c
Yedidya Feldblum [Tue, 17 Oct 2017 23:12:20 +0000 (16:12 -0700)]
Un-templatize Range::str
Summary: [Folly] Un-templatize `Range::str`.
Reviewed By: ot
Differential Revision:
D6075642
fbshipit-source-id:
08fc399c43750d34463de7634be4fe386b5db97c
Yedidya Feldblum [Tue, 17 Oct 2017 23:09:36 +0000 (16:09 -0700)]
Refactor ShutdownSocketSet atomic state machine
Summary:
[Folly] Refactor `ShutdownSocketSet` atomic state machine.
* Format.
* Use `while` over `goto`.
* Avoid `memory_order_acq_rel` as the single-argument memory order; some platforms fail to build that.
* Use `memory_order_relaxed` for all atomic operations because stronger memory orders are not actually required: the atomic state never serves as a barrier for synchronizing loads and stores of associated data because there is no associated data.
Reviewed By: davidtgoldblatt
Differential Revision:
D6058292
fbshipit-source-id:
d45d7fcfa472e6e393a5f980e75ad9ea3358bab3
Yedidya Feldblum [Tue, 17 Oct 2017 20:09:48 +0000 (13:09 -0700)]
An InlineExecutor singleton
Summary:
[Folly] An `InlineExecutor` singleton.
Using the Leaky Meyers Singleton pattern, so that it is always available whenever required.
Differential Revision:
D6074534
fbshipit-source-id:
bd4c9cd6a1e60c80de5d2eef1cb6a1e7f16b4e50
Andrew Krieger [Tue, 17 Oct 2017 16:17:30 +0000 (09:17 -0700)]
Don't make copies of std::string or fbstring when converting.
Summary:
This overload of estimateSpaceNeeded was taking a Src by
value, but Src is constrained by IsSomeString which only returns
true for std::string or fbstring, so this was inducing a copy
in any situation where folly::to<> is used with varargs which
contain fb/string arguments.
Reviewed By: yfeldblum
Differential Revision:
D6059517
fbshipit-source-id:
adc239f9049e161fc4b750bae0e3de5dbdcd1bfc
Yedidya Feldblum [Tue, 17 Oct 2017 03:10:35 +0000 (20:10 -0700)]
Make Range.h and FBString.h mutually independent
Summary:
[Folly] Make `Range.h` and `FBString.h` mutually independent.
This means that `Range` cannot directly know about `fbstring`, so any interactions between the two types must be indirected through templates.
Motivation: `FBString.h` is a relatively heaviweight `#include` for things that need `Range.h` but which do not use `fbstring`.
Reviewed By: ericniebler
Differential Revision:
D6062434
fbshipit-source-id:
e2f21c33f482eadffd0a8679eff4ece59bab53e9
Yedidya Feldblum [Tue, 17 Oct 2017 02:25:47 +0000 (19:25 -0700)]
Shrink integral hasher specialization implementations
Summary:
[Folly] Shrink integral `hasher` specialization implementations.
Instead of giving them bodies, just use inheritance.
Reviewed By: luciang
Differential Revision:
D6067757
fbshipit-source-id:
f71bd36132e1b66002493474825894b03754b34f
Chad Austin [Mon, 16 Oct 2017 23:22:25 +0000 (16:22 -0700)]
fix typo in invalid log level error message
Reviewed By: simpkins
Differential Revision:
D6068742
fbshipit-source-id:
a3b3b7fe4a6da8f250e5b9593f66b9f8a9345136
qiao hai-jun [Mon, 16 Oct 2017 18:18:55 +0000 (11:18 -0700)]
fix typo
Summary: Closes https://github.com/facebook/folly/pull/692
Differential Revision:
D6066294
Pulled By: yfeldblum
fbshipit-source-id:
a533527058b8b5bc71fb6f40bad31a50e8c3f585
Yedidya Feldblum [Mon, 16 Oct 2017 17:56:48 +0000 (10:56 -0700)]
4-way overloads for SemiFuture::value
Summary:
[Folly] 4-way overloads for `SemiFuture::value`.
Overload on the receiver reference category and `const`-qualification, deriving the return type reference category and `const`-qualification. Like `Optional`, `Try`, etc.
Differential Revision:
D6062006
fbshipit-source-id:
d7396cd4d4bb62e99445d5f61cb360898fa1c3f3
Ognjen Dragoljevic [Mon, 16 Oct 2017 14:24:15 +0000 (07:24 -0700)]
Fix hash overloads for integral types
Summary:
`folly/Hash.h:379:12: error: implicit instantiation of undefined template 'folly::hasher<unsigned long, void>'`
So, folly is unable to hash the very type it returns: `size_t`. Namely, folly has overloads for `uint32_t` and `uint64_t` which on my Mac map to `int` and `long long` respectively. `size_t` on the other hand maps to `long` which is neither.
Rather than overloading library types (which are just typedefs), we should overload all the built-in types: `char`, `short`, `int`, `long`, `long long`, `signed` and `unsigned` variants (with special treatment of `char`).
Reviewed By: yfeldblum, luciang
Differential Revision:
D6051600
fbshipit-source-id:
d59569dab963cbe0329aa589ff321cfb22308193
Andrew Krieger [Mon, 16 Oct 2017 04:30:56 +0000 (21:30 -0700)]
constexpr estimateSpaceNeeded for string literals.
Summary:
Partially inspired by WillerZ's investigation into faster compiling
StringPiece work, I thought this would be an easy drop in that short
circuits some compilation logic (or possibly is strictly better, since the
StringPiece conversion method isn't constexpr).
Reviewed By: yfeldblum
Differential Revision:
D6059537
fbshipit-source-id:
072f56e58aa47db10b54825cac8a05dc035b295c
Marc Celani [Sun, 15 Oct 2017 21:24:44 +0000 (14:24 -0700)]
Move retrying method to separate header in folly/futures
Summary: folly/futures depends on folly/Random.h, which in turn depends on <random>, which is a fairly large header. Most users of folly::futures do not use retrying, so separate it into a separate header.
Reviewed By: yfeldblum
Differential Revision:
D6028468
fbshipit-source-id:
d8155fe2ddff1a65c265a18f040ee6f1be3f3f0a
Yedidya Feldblum [Fri, 13 Oct 2017 23:14:37 +0000 (16:14 -0700)]
Style fixes for folly/experimental/gdb/README.md
Summary:
[Folly] Style fixes for `folly/experimental/gdb/README.md`.
Should have no effect on Github rendering, but should affect PHabricator rendering.
Reviewed By: Orvid, kennyyu
Differential Revision:
D6055017
fbshipit-source-id:
54204a2b77beeb884cdc486207809f78ac80afe2
Neel Goyal [Fri, 13 Oct 2017 19:22:08 +0000 (12:22 -0700)]
Add AsyncSocketException.cpp to build sources
Summary: Add AsyncSocketException.cpp to the list of files to build in Makefile.am
Reviewed By: knekritz
Differential Revision:
D6051989
fbshipit-source-id:
72083a609fc994770eca078bfef5a0ed04322bfc
Vitaly Berov [Fri, 13 Oct 2017 16:30:02 +0000 (09:30 -0700)]
Replace ShutdownSocketSet to singleton
Summary:
We recently found out that ShutdownSocketSet consumes 150+MB for our service, which uses duplex channels. The problem is that we create ~1000 of ThriftServers, and each of the creates its own ShutdownSocketSet.
In reality, ShutdownSocketSet is only needed to kill all socket's FD in emergency before crash dump is taken, so they don't hand around waiting for crash dump to complete. There is no need to keep a SSS per ThriftServer, singleton should work just fine.
There is a problem here, though. Currently a ThriftServer has 'immediateShutdown' method, which kills all sockets from SSS. So, if SSS becomes a singleton, and we have more than one ThriftServer, calling 'immediateShutdown' on one will kill sockets from the other one. First, it's a quite surprising behavior, and second, it complicates unit tests, which emulate thrift servers running in different processes.
As a result,
1. ShutdownSocketSet is created as a singleton, but each ThriftServer still keeps weak ptr to it (mostly for unit tests support).
2. replaceShutdownSocketSet method is added to ThriftServer.h, so unit tests could set different SSS for different ThriftServers.
3. method immediateShutdown is removed from ThriftServer, because its behavior would be 'surprising'.
There still may be unexpected consequences of this change for the tests because of Singleton, but let's see.
Reviewed By: yfeldblum
Differential Revision:
D6015576
fbshipit-source-id:
dab70dbf82d01bcc71bbe063f983e862911ceb24
Lucian Grijincu [Fri, 13 Oct 2017 08:58:11 +0000 (01:58 -0700)]
folly: AsyncSocketException: move implementation to .cpp, refactor
Reviewed By: yfeldblum
Differential Revision:
D6042832
fbshipit-source-id:
c716ee672c4acfa39cab9f10f3b3f88ca770cd20