Nicholas Ormrod [Thu, 20 Nov 2014 19:06:39 +0000 (11:06 -0800)]
Fix build break
Summary: Diff that had @override-unit-failures broke the build
Test Plan: fbconfig -r folly && fbmake dbg && fbmake runtests
Reviewed By: robbert@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1694388
Tasks:
5662947
Signature: t1:
1694388:
1416510278:
d50f4896814a34d73d55be292961f5ecd74b1bb4
Blame Revision:
D1680735
James Sedgwick [Thu, 20 Nov 2014 18:02:13 +0000 (10:02 -0800)]
merge event base tests
Summary: someone made another event base test while the big set still lived in thrift. merge it now that the old tests have been moved.
Test Plan: fbconfig folly/io/async/test && fbmake runtests --extended-tests
Reviewed By: davejwatson@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1690764
Signature: t1:
1690764:
1416413748:
f3ae64d509432844bb2e1b689a75d66825269d15
Dave Watson [Thu, 20 Nov 2014 17:32:17 +0000 (09:32 -0800)]
Bump version to 17:0
Dave Watson [Wed, 19 Nov 2014 17:41:06 +0000 (09:41 -0800)]
folly/io/async documentation
Summary: Documentation brain-dump. The sources already have pretty good doxygen-style comments, but could use a good high-level overview.
Test Plan: Reviewers get to edit
Reviewed By: jsedgwick@fb.com, afrind@fb.com, dcsommer@fb.com, alandau@fb.com
Subscribers: fugalh, subodh, kmdent, seanc, vloh, sjenkins, kparton, doug, net-systems@, njormrod, folly-diffs@, sgolemon, alikhtarov, andrii, tudorb
FB internal diff:
D1690976
Tasks:
5648428
Signature: t1:
1690976:
1416419706:
3894976c32904f040d518b9a4ce26ccb40d7ed80
Andrei Bajenov [Thu, 20 Nov 2014 01:47:07 +0000 (17:47 -0800)]
HHWheelTimer - fix scheduling timeouts within callbacks
Summary:
This one is a little tricky, but it explains what happened in t5388156. Basically what we observed in the stack traces is that the HHWheelTimer count_ is 0, but there is still a timeout scheduled on the AsyncTimeout.
How I can see this happening is the following:
- SASL times out
- We fall back to insecure mode in the callback and schedule a send of a real message to the server. Here we schedule another timeout on the same HHWheelTimer object. Since count_ is 0 at this point, we schedule a timeout in AsyncTimeout.
- Now the initial SASL timeout was late, so the HHWheelTimer is in a catchup state. This means that it's possible for the callback on the timeout that was just scheduled to fire on the same "tick".
- This callback invokes SR's callbacks which try to detach the event base from the channel. But it can't detach because we have something scheduled on the AsyncTimeout.
- We crash.
Test Plan:
- will wait for contbuild
- Something like this repros the problem: https://phabricator.fb.com/P17220063. Note I hacked the catchup part to get it to work. Now running loadgen with security = permitted causes the crash.
// hphp tests are stalled, have to do this...
Reviewed By: mshneer@fb.com
Subscribers: bmatheny, mshneer, sandeepkk, trunkagent, njormrod, folly-diffs@, alandau, mhorowitz, srenfro, hitesh, wstefancik
FB internal diff:
D1680735
Tasks:
5388156
Signature: t1:
1680735:
1416252123:
e76668860ccda9380a87996b4fa3de957e129404
Dave Watson [Wed, 19 Nov 2014 20:54:03 +0000 (12:54 -0800)]
Bump version to 16:0
Dave Watson [Mon, 10 Nov 2014 18:14:41 +0000 (10:14 -0800)]
OS merges
Summary:
All pretty trivial:
https://github.com/facebook/folly/issues/99
https://github.com/facebook/folly/issues/98
https://github.com/facebook/folly/issues/45
Test Plan: Will watch jenkins fbthrift build
Reviewed By: dcsommer@fb.com
Subscribers: doug, njormrod, folly-diffs@
FB internal diff:
D1669952
Signature: t1:
1669952:
1415643677:
906234f0a89f38645b0072d3c88762d8fa2729dc
Dave Watson [Tue, 28 Oct 2014 21:46:19 +0000 (14:46 -0700)]
server bootstrap
Summary:
ServerBootstrap a la netty.
This should be enough for some refactoring of thrift server and proxygen servers - but there are still lots of TODOs left to do
Test Plan:
Unittests included
Depends on
D1638358
Reviewed By: jsedgwick@fb.com
Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, jsedgwick, afrind, dcsommer
FB internal diff:
D1649521
Tasks:
5488516
Signature: t1:
1649521:
1416256073:
fc003fd471bdfd137160dd6d7befd933ee8addd2
Daniel Sommermann [Tue, 18 Nov 2014 22:15:57 +0000 (14:15 -0800)]
Run the HHWheelTimer and EventBase tests
Summary: Fix the TARGETS from the refactor
Test Plan: ran it locally
Reviewed By: davejwatson@fb.com
Subscribers: doug, njormrod, folly-diffs@
FB internal diff:
D1689151
Signature: t1:
1689151:
1416353673:
3d3575b10f963e9f558954b8f3eba8bf48cdaa74
Edward Chin [Tue, 18 Nov 2014 21:21:00 +0000 (13:21 -0800)]
add custom getLoad() for multifeed leaf
Summary:
getLoad() is called in servicerouter loadbalancing when a load_counter is not specified. I added a custom getLoad() for leaf and made it so that different metrics can be tested without code pushes.
Changes:
1) Overloaded PriorityThreadManager::getCodel() so you can specify a PRIORITY. aggr->leaf thrift calls are marked as HIGH priority.
2) The value of FLAGS_load_metric determines what metric the custom getLoad() will use (codel, max requests, or fallback to ThriftServer::getLoad()).
3) Smc listener for FLAGS_load_metric.
4) Removed server->setIsUnevenLoad(false) since there isn't reason this need to be set, and activeRequests_ and getPendingCount() are not tracked when isUnevenLoad_ is false.
Test Plan: Run leaf and check that smc listener works. Custom getLoad() won't be called until t5625774 is resolved.
Reviewed By: hitesh@fb.com
Subscribers: njormrod, folly-diffs@, trunkagent, varunk, fugalh, alandau, nkgupta, bmatheny, jaredg, xie, mshneer, yguo
FB internal diff:
D1685215
Signature: t1:
1685215:
1416339297:
0d1f8cb0cec27242d95f0067337d80149b0a720e
Dave Watson [Thu, 13 Nov 2014 20:51:43 +0000 (12:51 -0800)]
Clean up runBeforeLoop API
Summary: Clean up API per stepan's comments in
D1641057: runBeforeLoop callbacks should be run before the if block, so runInLoop calls will work.
Test Plan: added unittest
Reviewed By: hans@fb.com
Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1679773
Signature: t1:
1679773:
1415916488:
f20dd98bf40dd4cb19a4b11875dc0c9fa3e7870e
Blame Revision:
D1641057
Daniel Sommermann [Thu, 13 Nov 2014 18:49:31 +0000 (10:49 -0800)]
Move some tests to folly
Summary:
These lived in fbthrift previously, but they should move with the
main code too.
Test Plan: unit tests
Reviewed By: davejwatson@fb.com
Subscribers: doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1683229
Signature: t1:
1683229:
1416010730:
36fb7e4c9916ae7a9b5972cd476f82014c5f4c78
James Sedgwick [Tue, 18 Nov 2014 18:05:10 +0000 (10:05 -0800)]
top-level via()
Summary: as discussed
Test Plan: compiles :)
Reviewed By: steveo@fb.com
Subscribers: fugalh, njormrod, folly-diffs@
FB internal diff:
D1687921
Signature: t1:
1687921:
1416331366:
4e58a69daf39797e0d456ba86a463a9da2fd0369
subodh [Sun, 16 Nov 2014 01:17:53 +0000 (17:17 -0800)]
Improve initialization of openssl
Summary:
SSLContext takes it upon itself to
perform initialization of openssl along with
cleaning it up.
However in applications where there are multiple
libraries using openssl, cleanup is not really
desirable. Thus the application should own init
and cleanup of openssl, rather than SSLContext.
This is bad because openssl's init apis are not thread
safe.
This diff modifies SSLContext with 3 changes to
make this possible in the context of apps:
1. Add macro SSLCONTEXT_NO_REFCOUNT, for apps to be
able to disable refcounting
2. Move all steps of init into initializeOpenSSL.
As a result, apps can choose when to call
SSLContext::intitializeOpenSSL and if it has
been called before, openssl will not be reinitialized.
3. Move randomize as a static method to make it a part
of the initialization process.
Test Plan: Unit tests.
Reviewed By: ranjeeth@fb.com
Subscribers: ranjeeth, folly-diffs@, ssl-diffs@, njormrod, seanc, trunkagent, dihde
FB internal diff:
D1686397
Signature: t1:
1686397:
1416270565:
de805794d65af1c164ed334ff7ba93fe60b2b78a
Nicholas Ormrod [Mon, 17 Nov 2014 23:47:05 +0000 (15:47 -0800)]
Fix build break
Summary:
Change counting index to unsigned to avoid unsigned-signed
comparison errors.
@build-break
Test Plan:
fbconfig -r folly && fbmake dbg
fbmake runtests wangle
Reviewed By: robbert@fb.com, davejwatson@fb.com
Subscribers: sdwilsh, fugalh, njormrod, folly-diffs@
FB internal diff:
D1686315
Signature: t1:
1686315:
1416267974:
d8f6075550b9d611790263338dd4f49e7ef451cc
Blame Revision:
D1595200
James Sedgwick [Mon, 17 Nov 2014 23:26:00 +0000 (15:26 -0800)]
Indefinite observers and inline observers
Summary:
provides new APIs observe(ObserverPtr<T>) and observe(Observer<T>*) that
are not associated with a subscription. The latter method avoids allocations at the cost of disowning responsiblity for object lifetimes.
It could be used for batch-allocating filter chains, for instance. I hacked up some recursive templates that would make that easy to do, but
haven't included them. I'd rather see the first use case and then decide whether to let users set them up themselves or to provide an
abstraction that makes it easier.
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@, fugalh, njormrod
FB internal diff:
D1595200
Tasks:
5002278
Signature: t1:
1595200:
1416266578:
95c0532af42443fd83e2f94d30790c3c0831ce65
James Sedgwick [Mon, 17 Nov 2014 23:23:18 +0000 (15:23 -0800)]
OutputBufferingHandler
Summary:
see
D1618704 and
D1604575
Really need to make a decision about making a push for EventBaseManager everywhere. Getting/setting EBs when there's only ever one seems so silly, but in (e.g.) tests where one-off EBs are used without using the EBM, stuff like this will break.
Test Plan: definitely compiles and sorta works, will write unit tests if the dust settles
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@, trunkagent, fugalh, njormrod
FB internal diff:
D1618727
Alex Landau [Fri, 14 Nov 2014 04:50:30 +0000 (20:50 -0800)]
CompactProtocol: readVarint optimizations
Summary:
* Unroll readVarint() - no loops in the fast path
* Split Cursor::skip() into fast and slow paths
Test Plan:
```
------ old ------ ------ new ------
Benchmark time/iter iters/s time/iter iters/s win
=========================================================================================
CompactProtocol_read_Empty 13.21ns 75.69M 13.21ns 75.71M 0.03%
CompactProtocol_read_SmallInt 29.95ns 33.39M 28.96ns 34.53M 3.41%
CompactProtocol_read_BigInt 67.83ns 14.74M 36.77ns 27.19M 84.46%
CompactProtocol_read_SmallString 55.72ns 17.95M 54.04ns 18.50M 3.06%
CompactProtocol_read_BigString 342.93ns 2.92M 332.18ns 3.01M 3.08%
CompactProtocol_read_BigBinary 186.18ns 5.37M 181.45ns 5.51M 2.61%
CompactProtocol_read_LargeBinary 190.65ns 5.25M 184.85ns 5.41M 3.05%
CompactProtocol_read_Mixed 101.97ns 9.81M 97.06ns 10.30M 4.99%
CompactProtocol_read_SmallListInt 148.09ns 6.75M 139.52ns 7.17M 6.22%
CompactProtocol_read_BigListInt 71.69us 13.95K 63.34us 15.79K 13.19%
CompactProtocol_read_BigListMixed 1.26ms 790.67 1.18ms 845.66 6.95%
CompactProtocol_read_LargeListMixed 139.80ms 7.15 134.68ms 7.42 3.78%
```
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1681244
Signature: t1:
1681244:
1416265579:
f630c53f9c31788f847d0af72198bbd9e5ea43f1
James Sedgwick [Mon, 17 Nov 2014 23:04:23 +0000 (15:04 -0800)]
implicit constructor exception_wrapper(std::exception)
Summary: this should address some of the wordiness issues with exception_wrapper, i.e. always having to use make_exception_wrapper
Test Plan: wrote a little test to make sure everything compiles (or doesn't) as expected, not worth committing
Reviewed By: davejwatson@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1679460
Signature: t1:
1679460:
1416265170:
5cd72d95dd855cd4e594dbbc49d0c53d012fbc99
James Sedgwick [Mon, 17 Nov 2014 23:04:35 +0000 (15:04 -0800)]
fix potential race/memory corruption in IOThreadPoolExecutor
Summary:
In unusual but possible circumstances, the EventBase and thus pending tasks will outlive the pool, so we shouldn't keep references of any kind to the pool in the task.
The only reference we were keeping was used to access the task stats rx subject. Store the subject as a shared ptr and give a copy of the ptr to the Thread object, which is itself
owned by a shared ptr and captured by every task. I thought this had to do with the thread local leak in mentioned in the test plan of
D1682860 but this patch doesn't actually fix that :(
Thankfully, while task surfing I saw @phillip's awesome
D1682698. Patching that in fixes the leak! Woo. Either way, this is more correct.
Test Plan: unit under clang/asan
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, philipp
FB internal diff:
D1683221
Tasks:
5336655
Signature: t1:
1683221:
1416264933:
946d29b5a3eb22ed08812f2adefb7284b1899e4e
James Sedgwick [Mon, 17 Nov 2014 22:44:10 +0000 (14:44 -0800)]
priority CPU thread pool
Summary:
just extend CPUThreadPoolExecutor to use a queue that is itself composed of N mpmc queues, one per priority
the verbosity is starting to kill me, i had thought before of truncating Executor of all these pool types and now I'm definitely going to do that unless someone fights me.
Test Plan: added unit; maybe i'm not being clever enough as i couldn't think of many ways to test this reliably so there's just a basic preemption test
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, bmatheny
FB internal diff:
D1676452
Tasks:
5002392
Signature: t1:
1676452:
1416263990:
cdf5d44e4a50a6180ba547a3ed4c0c24d4ffdd8f
Dave Watson [Thu, 13 Nov 2014 20:31:47 +0000 (12:31 -0800)]
Fix format bug
Summary: Fix addition bug instead of string concat
Test Plan: ?? It compiles? The only call paths are when syscalls fail, it would be a lot of work to stub them out
Reviewed By: mshneer@fb.com
Subscribers: trunkagent, doug, njormrod, folly-diffs@
FB internal diff:
D1679713
Tasks:
5604354
Signature: t1:
1679713:
1415925209:
ed05d3d1ce3e7e8db32c4afc86c5f24748d9c65b
Dave Watson [Thu, 23 Oct 2014 17:26:08 +0000 (10:26 -0700)]
Move Acceptor to wangle
Summary:
Initial pass at moving acceptor to wangle. Involves moving most of the config stuff from proxygen/lib/services, and *all* of the ssl stuff from proxygen/lib/ssl.
Only minor changes:
* Acceptor can be overriden to use thrift socket types, so I don't have to change TTransportException everywhere just yet
* proxygen::Exception to std::runtime_exception in a few spots - looks like it is entirely bad config exceptions, so it should be okay
* Just used std::chrono directly instead of stuff in Time.h (which is just typedefs and simple helpers)
Test Plan:
used in
D1539327
fbconfig -r proxygen/httpserver; fbmake runtests
Probably other projects are broken, will iterate to fix
None of the failling tests look related
Reviewed By: dcsommer@fb.com
Subscribers: oleksandr, netego-diffs@, hphp-diffs@, ps, trunkagent, doug, fugalh, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1638358
Tasks:
5002353
Signature: t1:
1638358:
1414526683:
87a405e3c24711078707c00b62a50b0e960bf126
Philip Pronin [Fri, 14 Nov 2014 22:00:47 +0000 (14:00 -0800)]
fix potential memory leak in ThreadLocal
Summary:
See this LSan abort: https://phabricator.fb.com/P17233565.
Destructor of the object stored in `folly::ThreadLocal` itself may be using
`folly::ThreadLocal` with the same tag, with the current implementation these
objects may escape cleanup happening on thread exit.
Test Plan:
_build/opt/folly/test/thread_local_test --gtest_filter=ThreadLocalPtr.CreateOnThreadExit
Reviewed By: lucian@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1682698
Tasks:
5596043
Signature: t1:
1682698:
1416006810:
100aaa5c17cecceeea568165d552d9d7907f38d0
Alex Landau [Thu, 13 Nov 2014 01:19:15 +0000 (17:19 -0800)]
CompactProtocol: more improvements
Summary:
Add a specialized Cursor method that reads exactly 1 byte, in
addition to the existing one that reads into a buffer. Use it when
reading varints and standalone bytes in a CompactProtocol struct.
Test Plan:
```
------ old ------ ------ new ------
Benchmark time/iter iters/s time/iter iters/s win
=========================================================================================
CompactProtocol_read_Empty 18.68ns 53.54M 13.21ns 75.69M 41.37%
CompactProtocol_read_SmallInt 42.60ns 23.47M 29.95ns 33.39M 42.27%
CompactProtocol_read_BigInt 83.62ns 11.96M 68.40ns 14.62M 22.24%
CompactProtocol_read_SmallString 67.33ns 14.85M 55.62ns 17.98M 21.08%
CompactProtocol_read_BigString 353.83ns 2.83M 330.19ns 3.03M 7.07%
CompactProtocol_read_BigBinary 190.82ns 5.24M 182.90ns 5.47M 4.39%
CompactProtocol_read_LargeBinary 200.95ns 4.98M 187.00ns 5.35M 7.43%
CompactProtocol_read_Mixed 137.42ns 7.28M 102.98ns 9.71M 33.38%
CompactProtocol_read_SmallListInt 203.98ns 4.90M 146.68ns 6.82M 39.18%
CompactProtocol_read_BigListInt 120.50us 8.30K 71.56us 13.97K 68.31%
CompactProtocol_read_BigListMixed 1.62ms 617.07 1.26ms 795.60 28.93%
CompactProtocol_read_LargeListMixed 177.50ms 5.63 140.73ms 7.11 26.29%
```
Reviewed By: haijunz@fb.com
Subscribers: trunkagent, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1678077
Signature: t1:
1678077:
1415923409:
22accee6b62b6e2bf471f3758a290f71978a8c4e
Andrii Grynenko [Tue, 11 Nov 2014 19:13:30 +0000 (11:13 -0800)]
folly::Singleton-based McrouterManager
Summary: Introduces a Singleton which keeps a map persistence_id => mcrouter_t*. Makes mcrouter instance not know if it's managed by McrouterManager.
Test Plan: unit tests
Reviewed By: pavlo@fb.com
Subscribers: trunkagent, alikhtarov, njormrod, folly-diffs@
FB internal diff:
D1673274
Signature: t1:
1673274:
1415735863:
c990a6a526f9525c68cc23892d690a9b3cb94ace
Hans Fugal [Wed, 12 Nov 2014 18:01:50 +0000 (10:01 -0800)]
Have EventBase implement wangle::Executor
Summary:
It already does the work (`runInEventBaseThread`) but it will now be convenient to pass an `EventBase` where wangle wants an `Executor`.
Had to rip off the `boost::noncopyable` from `wangle::Executor` which is an interface and does not require non-copyability so that didn't really belong there in the first place I think. (Without this change, you get an obscure compiler error because of the double-inheritance from `boost::noncopyable`).
Test Plan: Things build, tests pass
Reviewed By: davejwatson@fb.com
Subscribers: jsedgwick, trunkagent, fugalh, exa, njormrod, folly-diffs@, andrii
FB internal diff:
D1671500
Signature: t1:
1671500:
1415727572:
a7dba33c669ca122aecaee3c700f9e53e54838d1
Nick Burrett [Wed, 12 Nov 2014 13:30:43 +0000 (05:30 -0800)]
folly::EventBase: wrap libevent calls to prevent race-condition
Summary:
Patch
D1585087 exposes two flaws in EventBase(). It introduces IO
worker threads to the ThriftServer which are constructed/destructed in
parallel.
Within the construction phase, a new EventBase() is instantiated for
each thread and unwound in destruction.
When using the BaseControllerTask (in Python), the following sequence
is observed:
a = event_init() [ThriftServer]
b = event_init() [IO worker 1]
c = event_init() [IO worker 2]
...
event_base_free(c)
event_base_free(b)
event_base_free(a) -> segfault
1. event_init() should only ever be called once. It internally
modifies a global variable in libevent, current_base to match the
return value. event_base_free() will set current_base back to NULL if
the passed in arg matches current_base. Therefore subsequent calls
must use event_base_new().
2. Since current_base is a global and EventBase() is called by multiple
threads, it is important to guard with a mutex. The guard itself also
exposed the bug because:
a = event_init() [current_base = a]
b = event_init() [current_base = b]
...
event_base_free(b) [b == current_base -> current_base = NULL]
So current_base ends up prematurely set to NULL.
Test Plan:
Run dba/core/daemons/dbstatus/dbstatus_tests.lpar, which no longer
segfaults
Reviewed By: jsedgwick@fb.com, davejwatson@fb.com
Subscribers: dihde, evanelias, trunkagent, njormrod, ncoffield, lachlan, folly-diffs@
FB internal diff:
D1663654
Tasks:
5545819
Signature: t1:
1663654:
1415732265:
d51c4c4cae99c1ac371460bf18d26d4f917a3c52
Blame Revision:
D1585087
Alecs King [Tue, 11 Nov 2014 21:53:09 +0000 (13:53 -0800)]
Use folly::IPAddress::hash instead of gethostid
Summary: find a non-loopback ipv4 or ipv6 address and feed it to folly::IPAddress::hash
Test Plan:
1)
fbconfig -r mcrouter
fbmake runtests
2)
compare hostids on different hosts
Reviewed By: pavlo@fb.com
Subscribers: njormrod, folly-diffs@, trunkagent, ps, bmatheny, alikhtarov
FB internal diff:
D1668944
Tasks:
5557721
Signature: t1:
1668944:
1415736928:
fb4b042a575c0b00f52780f3abf54bf7630b3a97
Dave Watson [Thu, 6 Nov 2014 17:59:27 +0000 (09:59 -0800)]
Clean up RequestContext
Summary: Using a ThreadLocal cleans up the code quite a bit. Also reuse the shared_ptr in create instead of creating a new one saves an allocation
Test Plan:
fbconfig thrift/lib/cpp/test:RequestContextTest; fbmake runtests
fbconfig common/services/cpp/test:trace_test; fbmake runtests
Reviewed By: hans@fb.com
Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@, vloh
FB internal diff:
D1663960
Signature: t1:
1663960:
1415390250:
36d9b772016d2a12d804e98edbc1725af882e507
Dave Watson [Mon, 10 Nov 2014 18:14:41 +0000 (10:14 -0800)]
OS merges
Summary:
All pretty trivial:
https://github.com/facebook/folly/issues/99
https://github.com/facebook/folly/issues/98
https://github.com/facebook/folly/issues/45
Test Plan: Will watch jenkins fbthrift build
Reviewed By: dcsommer@fb.com
Subscribers: doug, njormrod, folly-diffs@
FB internal diff:
D1669952
Signature: t1:
1669952:
1415643677:
906234f0a89f38645b0072d3c88762d8fa2729dc
Dmitry Panin [Sat, 8 Nov 2014 01:31:02 +0000 (17:31 -0800)]
Add optional parameter pruneHook to EvictingCacheMap::set(..)
Summary:
Inside `set()` we can do pruning, but it will happen
with default pruneHook.
Adding it as an optional param makes API more convenient.
(Instead, the users of API could just call `setPruneHook(pruneHook)` before `set`,
and then `setPruneHook(nullptr)` afterwards -- but it looks too ugly)
Test Plan:
```
fbconfig -r folly/ && fbmake runtests
```
passes:
```
Summary (total time 60.11s):
PASS: 1758
FAIL: 0
SKIP: 0
FATAL: 0
TIMEOUT: 0
```
Reviewed By: njormrod@fb.com
Subscribers: trunkagent, agartrell, njormrod, folly-diffs@
FB internal diff:
D1665690
Tasks:
5551091
Signature: t1:
1665690:
1415391406:
e4d2a956f9212aed70ab518159dbb19553764ce4
Pavlo Kushnir [Sat, 8 Nov 2014 02:43:32 +0000 (18:43 -0800)]
Bump version to 15:0
Nicholas Ormrod [Fri, 31 Oct 2014 17:48:19 +0000 (10:48 -0700)]
Fix folly lint errors
Summary:
Fix EOF whitespace
for file in `find folly -type f` ; do if [ -z "`tail -n1 $file`" ] ; then sed -i '$d' $file ; fi ; done
Test Plan: unit tests
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644130
Tasks:
5486739
Signature: t1:
1644130:
1414715392:
b6c783851aa030ad1148f84a98139a5dca207da0
Misha Shneerson [Fri, 7 Nov 2014 20:33:14 +0000 (12:33 -0800)]
isDetachable for HHWheelTimer
Summary: title
Test Plan: unit tests
Reviewed By: andrei.bajenov@fb.com
Subscribers: trunkagent, mcduff, hitesh, alandau, bmatheny, njormrod, mshneer, folly-diffs@, andreib, davejwatson
FB internal diff:
D1666499
Tasks:
5563183
Signature: t1:
1666499:
1415347256:
d58c8bbe952385c1c96f7f8cc6ae7f02216c56bb
Dave Watson [Tue, 14 Oct 2014 18:10:48 +0000 (11:10 -0700)]
Move SSL socket to folly
Summary: One of the last thrift -> folly moves. The only change was the exception types - there are small wrapper classes in thrift/lib/cpp/async left to convert from AsyncSocketException to TTransportException.
Test Plan: run unit tests
Reviewed By: dcsommer@fb.com
Subscribers: jdperlow, trunkagent, doug, bmatheny, ssl-diffs@, njormrod, mshneer, folly-diffs@, fugalh, jsedgwick, andrewcox, alandau
FB internal diff:
D1632425
Signature: t1:
1632425:
1414526483:
339ae107bacb073bdd8cf0942fd0f6b70990feb4
Tom Jackson [Fri, 7 Nov 2014 01:20:41 +0000 (17:20 -0800)]
Remove volatile from MemoryMappingTest
Summary: Not needed, I don't know why I put them there before.
Test Plan: Run the unit test
Reviewed By: njormrod@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1665689
Tasks:
5487902
Signature: t1:
1665689:
1415323144:
0998e7f700a3b40652615a36c3b9c9f661fbdadf
Daniel Sommermann [Thu, 6 Nov 2014 21:44:13 +0000 (13:44 -0800)]
Add some SocketAddress tests
Summary: There was a bit of gap in test coverage for bracketed ipv6
Test Plan: unit tests
Reviewed By: viswanath@fb.com
Subscribers: doug, ps, bmatheny, njormrod, folly-diffs@
FB internal diff:
D1664783
Signature: t1:
1664783:
1415316694:
b17e0dc1fcfece06c6e04e5f65e2095c66d28cc4
Dmitry Panin [Thu, 6 Nov 2014 20:47:43 +0000 (12:47 -0800)]
Fix integer overflow in folly/Benchmark.cpp
Summary:
`bm_max_secs` has type int32
So when this value is more than 2, expression
```
FLAGS_bm_max_secs *
1000000000;
```
overflows
Test Plan:
I did put a
```
LOG(INFO) << timeBudgetInNs;
```
and observed correct values (before fix they were overflowed)
Reviewed By: antonl@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1663247
Signature: t1:
1663247:
1415261814:
c9154ffde183b2a4f5403e534e47e52e8276c61b
Philip Pronin [Tue, 4 Nov 2014 18:21:34 +0000 (10:21 -0800)]
BMI1 support in EliasFanoCoding
Summary:
This diff updates `folly::CpuId` with support of extended features (EAX =
7, ECX = 0) to provide detection logic for BMI1 introduced in Haswell, and
provides support for `BLSR` instruction in `EliasFanoReader`.
Test Plan:
I used clang to compile the logic and run unittests
Reviewed By: lucian@fb.com
Subscribers: fbcode-common-diffs@, trunkagent, chaoyc, search-fbcode-diffs@, unicorn-diffs@, njormrod, folly-diffs@
FB internal diff:
D1658100
Signature: t1:
1658100:
1415126635:
d1820b8eb41c9e9786b5c8062b801cf1e2049a97
Hans Fugal [Wed, 5 Nov 2014 23:44:06 +0000 (15:44 -0800)]
(wangle) fix a race condition in Core::maybeCallback
Summary:
`calledBack_` could be seen as true by both threads in this conditional. Classic rookie mistake. :-/
Test Plan: run unit tests
Reviewed By: darshan@fb.com
Subscribers: trunkagent, hannesr, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1661199
Tasks:
5542938,
5506504
Signature: t1:
1661199:
1415215840:
fb69f56c8cf6f59beeca809724ce015b5260d9ad
Blame Revision:
D1636487
Brian Watling [Wed, 5 Nov 2014 23:17:10 +0000 (15:17 -0800)]
Remove memory leak from readRandomDevice by using a raw fd
Summary: Switch from dynamically allocating a File to using a raw fd for readRandomDevice. This prevents races at shutdown while also eliminating the memory leak
Test Plan: run unit tests
Reviewed By: meyering@fb.com
Subscribers: njormrod, folly-diffs@, tao-eng@
FB internal diff:
D1662151
Signature: t1:
1662151:
1415229242:
525b6294b27bb68b5dda70aadb8d3ba1cc61b815
Brian Watling [Wed, 5 Nov 2014 22:50:37 +0000 (14:50 -0800)]
Fix crashes in readRandomDevice()
Summary: The comment in readRandomDevice() lies - the PCHECK fails with "Bad file descriptor" in some tests. This is most likely due to a race where a background thread is calling readRandomDevice() while the main thread shuts down. Another possibility is bad destruction order, where rnadomDevice is destroyed before some other static object whose destructor calls readRandomDevice(). Either way, this fixes it without having to chace down every instance.
Test Plan: run unit tests
Reviewed By: meyering@fb.com
Subscribers: antonl, trunkagent, meyering, simpkins, njormrod, folly-diffs@, tao-eng@
FB internal diff:
D1660903
Signature: t1:
1660903:
1415215895:
7f1f96ab7416b45c4dc18e78173fb59eb27bd03b
Hans Fugal [Wed, 5 Nov 2014 19:21:49 +0000 (11:21 -0800)]
(wangle) fix a race in whenAll
Summary:
Race in `++ctx->count == ctx->total`. This ordering, while not very obvious or likely, is possible:
T1 T2
-- --
++ctx->count
++ctx->count
ctx->total
==
setValue
delete ctx
ctx->total
==
setValue
delete ctx
Test Plan:
That's the idea, anyway. I need some sleep, and it takes 20 minutes to build and test. I had a more convoluted fix (using `shared_ptr`) and it did seem to fix the error we were seeing, but I was seeing another error.
Reviewed By: darshan@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1660663
Tasks:
5506504
Signature: t1:
1660663:
1415207632:
49dc224363cec27736fc71fb211fa846be9e170f
Blame Revision:
D1636487
Lucian Grijincu [Wed, 5 Nov 2014 00:51:03 +0000 (16:51 -0800)]
folly: wangle: NamedThreadFactory: store of the thread prefix locally, don't assume they'll live forever
Summary: There's no real reason to store this as a StringPiece, is there?
Test Plan: n/a
Reviewed By: sdoroshenko@fb.com, robot9@fb.com
Subscribers: fugalh, njormrod, folly-diffs@
FB internal diff:
D1659558
Tasks:
5538418
Signature: t1:
1659558:
1415147560:
2c5b0c996893854c3ea9d0ad02b006bcc5960ffa
Nicholas Ormrod [Tue, 4 Nov 2014 00:18:41 +0000 (16:18 -0800)]
Fix licenses
Summary:
There are two types of licenses used in folly, only one of
which is accepted by the linter.
This diff changes the license notices in pre-existing folly files!
Many folly/io/async/* files have the second type of license, but
without a Copyright notice. I have added copyright notices to these
files.
I have also added a compliant notice to
folly/test/function_benchmark/benchmark_impl.h, which was the sole file
in folly/test/function_benchmark/ that didn't have a standard license.
Test Plan:
The changes to folly are comment only.
Run all of folly against the linter, see no more license errors.
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@, sjenkins
FB internal diff:
D1648489
Tasks:
5486739
Signature: t1:
1648489:
1415035522:
3d8bd9611eb7c7117b70d5e7f68de5768639a727
Nicholas Ormrod [Mon, 3 Nov 2014 20:39:11 +0000 (12:39 -0800)]
Fix global statics
Summary:
Statics in headers are bad.
Test Plan: run unit tests
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1646906
Tasks:
5486739
Signature: t1:
1646906:
1415042805:
dc4d1cec54e9320f1e609808a73622c731a4cdc9
Nicholas Ormrod [Mon, 3 Nov 2014 18:06:41 +0000 (10:06 -0800)]
rm protected inheritance
Summary:
Remove the protected inheritance from folly.
The code looks like the protected might be appropriate: there are some
derived classes which might want access to the AsyncTimeout.
Test Plan: run unit tests
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1649686
Tasks:
5486739
Signature: t1:
1649686:
1415035288:
18efd2cf9aae8caab66d8303c22cdc26c6b54ae5
James Sedgwick [Mon, 3 Nov 2014 17:37:35 +0000 (09:37 -0800)]
netty-like channels
Summary:
I wanted some foundational abstractions to start building out codecs on top of.
I also know that Blake gets very amused when I shamelessly copy Java/Netty abstractions, and I live to amuse Blake so I did it again.
So here's an implementation of something very similar to Netty's ChannelAdapters/ChannelPipelines
Only read() and write() for now, everything/anything else can easily be bolted on once the design is settled (if this is at all the direction we want to go)
I have a lot of thoughts about this design but I'm going to save my fingers and leave that to ad hoc discussions once folks take a look at this
A couple of things, though:
- It should be possible to dynamically add handlers to the chain. How I envision this working is that each original adapters keeps two lists of shared/unique ptrs to adapters of the correct type to sit next to them on either side, and dispatch to them appropriately when they're there.
- I was trying to do without separate ChannelHandlerContext objects altogether (keep the interface, but have ChannelPipeline act as the context itself) but ran into issues with virtual multiple inheritance. I might have another go at this.
- Only movable types are permitted. I hope this won't be too restrictive, because it would be a PITA to support both move-only and copy-only types.
- Why no Rx? Seems to me that any handlers that actually needs Rx (e.g. stats fanout or something) can deal with it themselves.
- If it turned out to be useful to nest these (for more flexible composition) that would also be doable. i.e. ChannelPipeline<ChannelPipeline<Handler1, Handler2>, ChannelPipeline<Handler3, Handler4>>
Test Plan: super basic test compiles and runs as expected
Reviewed By: davejwatson@fb.com
Subscribers: ajitb, folly-diffs@, ldbrandy, trunkagent, fugalh, njormrod
FB internal diff:
D1604575
Tasks:
5002299
Signature: t1:
1604575:
1415034767:
bc3b12fae726890aa6a55ed391286917ae23e56e
Daniel Sommermann [Mon, 20 Oct 2014 19:52:01 +0000 (12:52 -0700)]
Fix -Wsign-compare
Summary: Folly should be able to compile with strict flags.
Test Plan: unit tests, review
Reviewed By: meyering@fb.com
Subscribers: meyering, trunkagent, doug, fugalh, njormrod, folly-diffs@
FB internal diff:
D1627280
Signature: t1:
1627280:
1414792755:
004f5a737ece1e93bcf4331718a98afc57e4f80c
James Sedgwick [Mon, 3 Nov 2014 00:28:49 +0000 (16:28 -0800)]
Name prefix setter for NamedThreadFactory
Summary: it's useful to update the prefix after the construction.
Test Plan:
unit, that's it :/
Reviewed By: davejwatson@fb.com
Subscribers: mshneer, folly-diffs@, wch, atlas2-eng@, everstore-dev@, wormhole-diffs@, ads-dsp-eng@, bwester, trunkagent, fugalh, alandau, njormrod, bmatheny
FB internal diff:
D1585087
Dave Watson [Fri, 31 Oct 2014 19:49:57 +0000 (12:49 -0700)]
Fix bad rebase for SSLContext
Summary: Put back previous liger diffs
Test Plan: It builds
Reviewed By: seanc@fb.com
Subscribers: doug, ssl-diffs@, njormrod, folly-diffs@
FB internal diff:
D1652754
Signature: t1:
1652754:
1414785984:
df0fc7bf59dc2e89defd2c1a4ffe3b288238ba58
Nicholas Ormrod [Fri, 31 Oct 2014 21:54:30 +0000 (14:54 -0700)]
Fix line too long lint errors
Summary:
Many lines that are too long contain URLs. This diff adds
url-like lines to the list of exceptions to 80-char limits. It also
fixed the locations in folly with line-too-long errors.
Test Plan: arc lint on folly, see no line-too-long errors.
Reviewed By: robbert@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644151
Tasks:
5486739
Signature: t1:
1644151:
1414779408:
76bcec1d14daaa8ed071c715bf26b108c8fe4b87
Mark Drayton [Fri, 31 Oct 2014 20:42:40 +0000 (13:42 -0700)]
Increase size of demangled symbol buffer
Summary:
Symbols which demangle to a string longer than 1024 bytes are
quite common in our code. This diff increases the size of the output
buffer to accommodate them.
Test Plan: run it
Reviewed By: lucian@fb.com
Subscribers: trunkagent, njormrod, folly-diffs@
FB internal diff:
D1639801
Tasks:
5464222
Signature: t1:
1639801:
1414784601:
2f59d5a58e434f4cf9df5b25b917c5094c8b133f
Sachin Kadloor [Fri, 31 Oct 2014 18:13:13 +0000 (11:13 -0700)]
Undo a local change
Summary: I had committed a change by mistake. Reverting it.
Test Plan: build it.
Reviewed By: lakshmiganesh@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1652341
Signature: t1:
1652341:
1414779013:
9f363acba95fbac1988200081659edf71ac63eec
Nicholas Ormrod [Fri, 31 Oct 2014 18:07:44 +0000 (11:07 -0700)]
volatile override
Summary:
Some uses of volatile are legit. Add a hidden override.
Test Plan: run unit tests
Run flint against folly/Malloc.cpp, see no more volatile warning.
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, louisk, njormrod, folly-diffs@
FB internal diff:
D1644493
Tasks:
5486739
Signature: t1:
1644493:
1414715317:
491d0f631d8152a5b7ec66237e00c404530ab590
Dave Watson [Fri, 31 Oct 2014 17:37:55 +0000 (10:37 -0700)]
fix build
Summary: add missing ssl dep
Test Plan: build it
Reviewed By: dcsommer@fb.com
Subscribers: doug, njormrod, folly-diffs@
FB internal diff:
D1652234
Signature: t1:
1652234:
1414778507:
4504ea06c3076e0cdfa7772166cf48bc564c9076
Tomislav Novak [Fri, 24 Oct 2014 18:46:51 +0000 (11:46 -0700)]
Fix large size class category handling in folly::goodMallocSize()
Summary:
jemalloc seems to now use `4064 * 1024` as `arena_maxclass`. Therefore,
allocations that fall in the range `(4064*1024, 4072*1024]` will trigger an
assert in `folly::goodMallocSize()` (`nallocx()` will round up sizes within
that range up to the next multiple of 4 MB (chunk size)).
Test Plan: Call to `folly::goodMallocSize(
4161568)` no longer triggeres an assert.
Reviewed By: je@fb.com
Subscribers: aalexandre, trunkagent, njormrod, folly-diffs@
FB internal diff:
D1638370
Signature: t1:
1638370:
1414703429:
f3ed2cc8dcb183ff4e8367e2c1f884e122605cba
Sachin Kadloor [Fri, 31 Oct 2014 17:58:47 +0000 (10:58 -0700)]
SocketAddress logging
Summary:
Better logging in SocketAddress
Test Plan: run it
Reviewed By: qwu@fb.com
Subscribers: ps, bmatheny, xning
FB internal diff:
D1644573
Tasks:
5406091
Signature: t1:
1644573:
1414626398:
1e23ee85ec8ee0be89cb6bdc07da23978b4d988a
Nicholas Ormrod [Fri, 31 Oct 2014 17:55:44 +0000 (10:55 -0700)]
Fix/override bad symbol names
Summary:
Symbols containing two underscores, or starting with underscore
followed by a capital letter, are reserved by the implementation. Fix
the inappropriate cases, and nolint the ones which we are using
correctly
Test Plan: run unit tests
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1650435
Tasks:
5486739
Signature: t1:
1650435:
1414713329:
7844e7802ebabcac7ef21aef0becf71ba513afb0
Nicholas Ormrod [Fri, 31 Oct 2014 17:52:41 +0000 (10:52 -0700)]
Comparing the same tokens
Summary:
Avoid comparing-the-same-tokens error by parenthesizing one
side.
This diff only touches folly tests
Test Plan: run unit tests
Reviewed By: robbert@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1650386
Tasks:
5486739
Signature: t1:
1650386:
1414713837:
cbb578d23f759aa049f4246113d9da6d40504da4
Nicholas Ormrod [Fri, 31 Oct 2014 17:50:05 +0000 (10:50 -0700)]
throw() specs deprecated
Summary: Switch to noexcept
Test Plan: run unit tests
Reviewed By: robbert@fb.com, andrei.alexandrescu@fb.com
Subscribers: aalexandre, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644135
Tasks:
5486739
Signature: t1:
1644135:
1414715537:
22e1baf91ab7e3250b0b2a460a12d56783f2baed
Nicholas Ormrod [Fri, 31 Oct 2014 17:48:19 +0000 (10:48 -0700)]
Fix folly lint errors
Summary:
Fix EOF whitespace
for file in `find folly -type f` ; do if [ -z "`tail -n1 $file`" ] ; then sed -i '$d' $file ; fi ; done
Test Plan: run unit tests
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644130
Tasks:
5486739
Signature: t1:
1644130:
1414715392:
b6c783851aa030ad1148f84a98139a5dca207da0
Dave Watson [Wed, 1 Oct 2014 17:51:40 +0000 (10:51 -0700)]
SSLContext
Summary:
Move SSLContext to folly.
Test Plan: It builds.
Reviewed By: dcsommer@fb.com
Subscribers: jdperlow, atlas2-eng@, wormhole-diffs@, bwester, trunkagent, doug, ps, bmatheny, ssl-diffs@, alikhtarov, njormrod, mshneer, folly-diffs@, andrewcox, alandau, jsedgwick, fugalh
FB internal diff:
D1631924
Signature: t1:
1631924:
1414616562:
9a67dbf20f00eb8fbcb35880efcb94c0fae07dcc
Dave Watson [Thu, 30 Oct 2014 23:16:56 +0000 (16:16 -0700)]
save/restore request context in future
Summary:
Generic 'threadlocal' like object that follows async calls around.
Note: Finagle's futures do this also. It is super useful.
Test Plan: I should write a unittest?
Reviewed By: haijunz@fb.com, hans@fb.com
Subscribers: trunkagent, dawidp, doug, fugalh, njormrod, folly-diffs@
FB internal diff:
D1650843
Tasks:
4698780
Signature: t1:
1650843:
1414711295:
c7439733680ab4903eb9c05dcf8bf52100bf3f07
Hans Fugal [Thu, 30 Oct 2014 23:20:17 +0000 (16:20 -0700)]
(wangle-test) fix double ifndef in thens.rb
Summary: Looks like a bad git merge
Test Plan: builds
Reviewed By: hannesr@fb.com
Subscribers: hannesr, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1649773
Tasks:
5501131
Signature: t1:
1649773:
1414700920:
cc691283884e9654803bba3ffe78f4553194752d
Nicholas Ormrod [Thu, 30 Oct 2014 21:15:02 +0000 (14:15 -0700)]
operator::* needs implicit
Summary:
operator::* needs to be explicit, or explicitly labeled
implicit. The build breaks with explicit, so label.
Test Plan: run unit tests
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644176
Tasks:
5486739
Signature: t1:
1644176:
1414693471:
e3a08e27ba27a5ae0e4b9a5e009acf0a16728c45
Nicholas Ormrod [Thu, 30 Oct 2014 20:55:13 +0000 (13:55 -0700)]
Make dynamic noexcept
Summary:
Many dynamic operations, including its move constructor, are
noexcept but not labeled so. Labeled some important functions as
noexcept.
Test Plan: run unit tests
Reviewed By: delong.j@fb.com
Subscribers: tudorb, philipp, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644380
Tasks:
5486739
Signature: t1:
1644380:
1414618757:
fb910d8a3fe3e634da4215c432577edf7371be61
Daniel Sommermann [Fri, 17 Oct 2014 20:38:00 +0000 (13:38 -0700)]
Fix -Wsign-compare
Summary:
Fixed unsigned/signed compares.
I had to suppress the warning in one place, but don't worry, clang
supports `#pragma GCC diagnostic ...` (see http://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas)
Test Plan: unit tests
Reviewed By: afrind@fb.com
Subscribers: alandau, mshneer, trunkagent, doug, jdperlow, bmatheny, njormrod, cgheorghe, folly-diffs@
FB internal diff:
D1624249
Tasks:
5140804
Signature: t1:
1624249:
1414620209:
5399e8d90d8ca32b30794a7b2a4a7c2d7d437dda
Nicholas Ormrod [Thu, 30 Oct 2014 18:38:36 +0000 (11:38 -0700)]
Override #define of keyword
Summary:
Add a hidden option to suppress ###define [keyword]##, which is
occasionally appropriate.
Test Plan:
Build flint, run it on FBString.h, see no error.
Ran flint tests locally, see no errors.
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, louisk, njormrod, folly-diffs@
FB internal diff:
D1644182
Tasks:
5486739
Signature: t1:
1644182:
1414616294:
4b6004d358cba865455366f5a943644b3542ae51
Nicholas Ormrod [Thu, 30 Oct 2014 18:26:45 +0000 (11:26 -0700)]
Override for include-guard
Summary:
In certain circumstances, it is appropriate to omit and include
guard. Add an option to allow this to be overriden.
Test Plan: Build flint, run it on folly, see no include-guard errors.
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, louisk, njormrod, folly-diffs@
FB internal diff:
D1644170
Tasks:
5486739
Signature: t1:
1644170:
1414616595:
7ac52f474c1312a0c28e89255b1151d56c680acf
Nicholas Ormrod [Thu, 30 Oct 2014 18:16:39 +0000 (11:16 -0700)]
Allow catch-int to be nolinted
Summary:
Add a hidden nolint option to suppress this lint warning. It is
appropriate in this case.
Test Plan:
Build flint, run it on ExceptionWrapperTest.cpp, see no catch
lint failure
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, louisk, njormrod, folly-diffs@
FB internal diff:
D1644165
Tasks:
5486739
Signature: t1:
1644165:
1414616664:
a6b9dc34660df84b33ed8faaf48ec048a02bad01
Nicholas Ormrod [Thu, 30 Oct 2014 17:18:27 +0000 (10:18 -0700)]
Move constructors should be noexcept
Summary:
The compiler is able to make some nice optimizations when it
knows that move constructors are noexcept. (In particular, it helps a
lot inside of std::vector if you don't need to worry about the
possibility of exception during reallocation.) Some move constructors in
folly are obviously moveable: change them.
Test Plan: unit tests
Reviewed By: delong.j@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644296
Tasks:
5486739
Signature: t1:
1644296:
1414618605:
d9a0db5193c82650b96e9c62b019d5da218b15c5
James Sedgwick [Wed, 29 Oct 2014 23:33:14 +0000 (16:33 -0700)]
Future<T>::then([](T&&)), aka next()
Summary:
variants of then() that bypass Try and forward any exceptions up to the next future. like next() from http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3865.pdf
this could go a long way towards reducing spurious rethrows wherever people don't ever actually catch exceptions in their continuations, which is probably often enough.
anyone know if there's something in folly or standard library that does what my silly VoidWrapper struct does?
there's a lot of copypasta here but i'm not sure consolidating into helpers would actually be useful considering the amount of template crap, i don't feel strongly about it though.
Test Plan: added unit
Reviewed By: hans@fb.com
Subscribers: trunkagent, sammerat, fugalh, njormrod, folly-diffs@, bmatheny
FB internal diff:
D1641776
Signature: t1:
1641776:
1414610434:
c742563b8061a748fca9292fc2765081edcf9d52
dcsommer [Wed, 29 Oct 2014 23:08:59 +0000 (16:08 -0700)]
Bump version to 14:0
Sean Cannella [Wed, 29 Oct 2014 21:02:55 +0000 (14:02 -0700)]
Really fix the clang warning in Format-inl.h
Summary:
The previous diff added the safety check but didn't actually
remove the compiler warning. This is what I get for doing an incremental
compile.
Test Plan: fbmake clean ; fbmake dbg with clang:dev
Reviewed By: meyering@fb.com
Subscribers: trunkagent, mathieubaudet, njormrod, folly-diffs@, bmatheny, ranjeeth, subodh, kmdent, fma, benyluo
FB internal diff:
D1646519
Signature: t1:
1646519:
1414602292:
cfb908ae094caaef02e64eb2c42544fd34fa1389
Nicholas Ormrod [Wed, 29 Oct 2014 20:16:38 +0000 (13:16 -0700)]
Fix spelling mistakes
Summary: @override-unit-failures
Test Plan: n/a
Reviewed By: rhysparry@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644139
Tasks:
5486739
Signature: t1:
1644139:
1414553949:
626fdca07a583c3d5017abe19408a5e6f6d7674a
Nicholas Ormrod [Wed, 29 Oct 2014 20:10:39 +0000 (13:10 -0700)]
Fix multiple includes
Summary:
Add nolint comments to appropriate multiply-included files.
Removed raw duplicates from StlVectorTest.
Test Plan:
fbconfig -r folly && fbmake runtests
Ran arc lint, see no "included multiple times" messages
Reviewed By: robbert@fb.com
Subscribers: sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1644144
Tasks:
5486739
Signature: t1:
1644144:
1414599280:
1e403ea3675d2173b249c4ce6db0dad621a139bc
Nicholas Ormrod [Wed, 29 Oct 2014 20:03:02 +0000 (13:03 -0700)]
operator bool()
Summary:
Among all of the implicit conversions, conversion to bool is
particularly tricky. For this reason, it has a special call-out in flint
among the casts, and canno be overriden.
However, there are legitimate use cases for operator bool. In this case,
we have a iterator that logically represents a bool but in practice
represents a packed integral. This class needs to be implicitly
convertible to bool, and warrants an override.
Push blocking tests have had over a day to complete, but haven't.
Further, the folly change is comment-only.
Test Plan:
Build flint, run it on BitIteratorDetail.h, see no more lint
errors
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: sdwilsh, louisk, njormrod, folly-diffs@
FB internal diff:
D1644141
Tasks:
5486739
Signature: t1:
1644141:
1414607351:
29740da758b75187f4f7b6b5a5ad28a523e7080d
Nicholas Ormrod [Wed, 29 Oct 2014 19:55:12 +0000 (12:55 -0700)]
__attribute__s need double underscore
Summary: See
D1508946 for the rationale behind this lint message.
Test Plan:
fbconfig -r folly && fbmake dbg
Reviewed By: hans@fb.com
Subscribers: sdwilsh, fugalh, njormrod, folly-diffs@
FB internal diff:
D1644190
Tasks:
5486739
Signature: t1:
1644190:
1414532704:
87d5851f363cf37c58f7e7c3d57c8210a792d4a1
Hans Fugal [Wed, 29 Oct 2014 18:30:02 +0000 (11:30 -0700)]
move clang disable of Thens.cpp to thens.rb
Summary: So when we re-generate code we don't cause clang test failures (until #
4412111 is done)
Test Plan: git diff
Reviewed By: jsedgwick@fb.com
Subscribers: net-systems@, fugalh, mathieubaudet, exa, njormrod, folly-diffs@
FB internal diff:
D1644550
Tasks:
4412111
Signature: t1:
1644550:
1414604128:
8184e7ce1f3e417c170ef67346d553caecc1f013
Dave Watson [Tue, 28 Oct 2014 17:52:49 +0000 (10:52 -0700)]
Move AsyncSocket to folly (try 2)
Summary:
Test Plan:
Reviewed By: dcsommer@fb.com
Subscribers: trunkagent, mcduff, hitesh, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@
FB internal diff:
D1644071
Signature: t1:
1644071:
1414526899:
c41dd55e2957a7e1fcc54508e20cdb4a9c8c30d4
Sean Cannella [Wed, 29 Oct 2014 02:09:35 +0000 (19:09 -0700)]
Address clang sign warning in Format-inl.h
Summary:
This has been bothering me for a while when I've been trying to
verify builds with clang. This generates a ton of spew and the current
code can't actually handle non-default negatives so let's just throw if
we see one.
Test Plan: fbmake runtests
Reviewed By: meyering@fb.com
Subscribers: mathieubaudet, njormrod, folly-diffs@, bmatheny, benyluo, shikong, kmdent, fma, ranjeeth, subodh
FB internal diff:
D1643881
Signature: t1:
1643881:
1414523777:
085035ce8c280fbd6481b2ce0354b53b96fdc50e
Hans Fugal [Tue, 28 Oct 2014 21:49:16 +0000 (14:49 -0700)]
kill threadgate
Test Plan: This diff is actually the test plan. It was contbuild bait. I fixed things that broke (in separate diffs), and then I rebased this on top, everything is green, and we pulled the trigger.
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1632937
Tasks:
5409538
Signature: t1:
1632937:
1414531009:
4dbaa9513fbfba45a9c63e7be3de3646a6e926a2
Dave Watson [Mon, 27 Oct 2014 17:11:03 +0000 (10:11 -0700)]
Add MemoryIdler suppot to IOThreadPoolExecutor
Summary:
Idle memory in IO threads. If loop is unused for a period of time, free associated memory, and call epoll again.
Had to add a new list of callbacks that don't make the loop nonblocking (i.e. using runInLoop() instead would use the nonblocking version of epoll).
Could bake this in to EventBase directly, but that seems like the wrong abstraction, since EventBase doesn't actually control the thread - for example, that approach would also free up memory for stack-allocated EventBases where they are used synchronously by clients.
This diff doesn't change IO scheduling at all - current IO work is round robin, so this probably only helps if the whole server is idle (at least until we add smarter scheduling)
Test Plan:
Based on top of
D1585087.
fbconfig thrift/perf/cpp; fbmake dbg
_bin/thrift/perf/cpp/ThriftServer
_bin/thrift/perf/cpp/loadgen -num_threads=100 -weight_sendrecv=1 -cpp2 -async
Ran loadgen for a while, watched res memory in top. Stopped loadgen. After ~5 sec, res memory was much reduced.
Reviewed By: jsedgwick@fb.com
Subscribers: trunkagent, doug, fugalh, njormrod, folly-diffs@
FB internal diff:
D1641057
Tasks:
5002425
Viswanath Sivakumar [Tue, 28 Oct 2014 02:38:10 +0000 (19:38 -0700)]
Revert "
D1587625 [thrift->folly] AsyncSocket"
Summary:
Test Plan:
Reviewed By: cgheorghe@fb.com
Subscribers:
FB internal diff:
D1642334
Blame Revision:
D1587625
Adam Simpkins [Sun, 28 Sep 2014 01:24:18 +0000 (18:24 -0700)]
improve io::Appender functionality
Summary:
Add an operator()(StringPiece) method to Appender, so it can be used
directly with folly::Formatter objects. Also add printf() and vprintf()
methods to Appender, for places that need to use existing printf-style
formatting.
This also includes versions of push() and pushAtMost() that accept
ByteRange objects. Previously we only had push() implementations that
took separate buffer and size arguments.
Test Plan: Added new unit tests to IOBufCursorTest.cpp
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, doug, net-systems@, exa, njormrod
FB internal diff:
D1583684
Dave Watson [Mon, 27 Oct 2014 21:25:00 +0000 (14:25 -0700)]
Add missing headers
Summary: Jenkins ci build was broken for a while, so we missed these in contbuild results
Test Plan: watch ci build for fbthrift
Reviewed By: hans@fb.com
Subscribers: doug, fugalh, njormrod, folly-diffs@
FB internal diff:
D1641515
Dave Watson [Fri, 26 Sep 2014 20:49:11 +0000 (13:49 -0700)]
AsyncSocket
Summary:
Move async socket to folly.
Changes:
* Made an AsyncSocketException type instead of TTransportException: Some of the exceptions didn't fit nicely in to std::exception types (like TIMED_OUT). There are some wrappers in thrift/lib/cpp/async to convert back to TTransportException, so all existing code still compiles.
* Moved read/write callbacks out of AsyncTransport: filters are going to want to do the read/write stuff separately (see revproxy/tunnel/filters, and discussions in
D1483148).
Test Plan:
fbconfig -r thrift; fbmake runtests
contbuild should catch everything else - exception types shouldn't change for existing code
Reviewed By: dcsommer@fb.com
Subscribers: mshneer, folly-diffs@, trunkagent, doug, alandau, bmatheny, njormrod, fugalh, jsedgwick
FB internal diff:
D1587625
Hans Fugal [Mon, 27 Oct 2014 15:53:23 +0000 (08:53 -0700)]
(wangle) fix race in Core::detachOne()
Summary:
In
D1618240 I introduced a race condition in `detachOne()`, where `detached_` is incremented and then tested. If the promise and future are racing, they can both see `detached_ == 2` in the conditional, and then they'll both try to free the Core object. This fixes that.
It also fixes a related problem (which actually showed up more often with the test I wrote), where we transition into the Done state before setting the value, and then `maybeCallback()` observes the state is Done (because we're just reading an atomic, not grabbing the lock, which is intentional), tries to execute the callback, but `folly::Optional` throws an exception because the value hasn't been set yet (at least in debug it does). I should have listened to my gut and kept the state assignment after the transition action in the first place.
Test Plan: New unit test
Reviewed By: jsedgwick@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@, mnd
FB internal diff:
D1636490
Tasks:
5438209
Blame Revision:
D1618240
Hans Fugal [Mon, 27 Oct 2014 15:53:20 +0000 (08:53 -0700)]
unrevert "(wangle) express current Core functionality with a state machine""
Summary: Reverts
D1633874. Companion to
D1636490 which fixes the bug.
Test Plan:
git reverting code that was git reverted and hasn't changed in the interim
Won't be checked in without the companion bugfix diff (
D1636490)
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1636487
Tasks:
5438209
Blame Revision:
D1633874
Dave Watson [Thu, 23 Oct 2014 22:21:23 +0000 (15:21 -0700)]
Remove some leftover apache paths
Summary: EventBaseManager was moved to folly, we shouldn't have deps on apache
Test Plan: fbconfig -r folly/experimental/wangle/concurrency; fbmake runtests
Reviewed By: hans@fb.com
Subscribers: doug, fugalh, njormrod, folly-diffs@
FB internal diff:
D1636006
Hans Fugal [Thu, 23 Oct 2014 01:11:34 +0000 (18:11 -0700)]
Revert "(wangle) express current Core functionality with a state machine"
Summary:
Test Plan:
Reviewed By: harishs@fb.com
Subscribers: net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1633874
Tasks:
5438209
Hans Fugal [Wed, 22 Oct 2014 17:14:45 +0000 (10:14 -0700)]
fix Future<const T>::value()
Summary: I'm still a little mystified how one makes a Future<const T> (I tried to make one explicitly in a test and failed), but this is clearly the bug @reedriley sees in https://phabricator.fb.com/
D1620805#21
Test Plan:
fbconfig titan/cachius/test
fbmake
Reviewed By: reedriley@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, reedriley, folly-diffs@
FB internal diff:
D1630378
Blame Revision:
D1620805
Dave Watson [Thu, 25 Sep 2014 17:17:20 +0000 (10:17 -0700)]
move asyncserversocket to folly
Summary:
Changes:
* namespace to folly
* some std::chrono replacesments
* Moves shutdownSocketSet stuff to thriftserver instead of in the socket itself.
* Changes exception types. I don't see anywhere that uses the TAsync*SERVER*socket exceptions depending on the TTransport type, but I could be wrong? I don't really know what to do about the exception types
unittests still postponed overnight.
Test Plan:
fbconfig -r thrift; fbmake runtests
fbconfig -r proxygen; fbmake runtests
Reviewed By: dcsommer@fb.com
Subscribers: hphp-diffs@, ps, folly-diffs@, anca, trunkagent, jsedgwick, fugalh, doug, alandau, bmatheny, njormrod
FB internal diff:
D1579187
James Sedgwick [Wed, 22 Oct 2014 01:04:44 +0000 (18:04 -0700)]
add PortableSpinLock.h to Makefile, unbreak fbthrift build
Summary:
fix fbthrift build, broken in
D1621717
I wonder if it'd be possible to lint against this somehow. The jenkins failures are easy to miss especially on folly diffs with lots of failure noise (in this case it was still postponed)
Test Plan: wait for jenkins
Reviewed By: davejwatson@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1630833
Andrii Grynenko [Tue, 21 Oct 2014 20:55:56 +0000 (13:55 -0700)]
Schedule destroyInstances in registrationComplete
Summary: This makes it more likely that SingletonVaultDestructor won't be created after atexit calls.
Test Plan: unit test
Reviewed By: chip@fb.com
Subscribers: hphp-diffs@, ps, njormrod, folly-diffs@
FB internal diff:
D1629804
Tasks:
5353022
Hans Fugal [Tue, 21 Oct 2014 22:50:02 +0000 (15:50 -0700)]
(wangle) fix after-delete assert
Summary:
This would cause debug builds to do a bad thing (access the variable `this->detached_` within an assert, after `delete this`).
Test Plan: unit tests
Hopefully now that we have a dummy cpp file in `folly/wangle/detail` contbuild
will pick it up and all the dependencies will also run their tests.
Right now, we suspect maybe maelstrom (@wez) and adinvoicer and zookeeper
(@jying) and probably others are seeing this in unit test failures (esp. if
they use asan, which is rightly detecting read after free). Hoping contbuild
will catch them.
Reviewed By: davejwatson@fb.com
Subscribers: net-systems@, fugalh, exa, njormrod, folly-diffs@, wez, dcapel, jying, cgheorghe
FB internal diff:
D1630301
Tasks:
5424546,
5435720
Blame Revision:
D1618240
Hans Fugal [Tue, 21 Oct 2014 18:55:52 +0000 (11:55 -0700)]
waitFor race workaround
Summary: Fixing the race will take some more thought. Probably a condvar. For now, I'm turning waitFor into a spinlock instead.
Test Plan: new unit test that deadlocks before the workaround
Reviewed By: jsedgwick@fb.com
Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@
FB internal diff:
D1628015
Tasks:
5427828
Hans Fugal [Tue, 21 Oct 2014 17:24:14 +0000 (10:24 -0700)]
(wangle) Fix a couple compilation issues
Summary: If you try to include files in a weird (but not incorrect) order you get compilation errors. No longer!
Test Plan: new dummy `cpp_binary` target
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, folly-diffs@, net-systems@, fugalh, exa, njormrod
FB internal diff:
D1621083