folly.git
9 years agos/makeFuture().via/via/
Hans Fugal [Fri, 5 Dec 2014 17:33:35 +0000 (09:33 -0800)]
s/makeFuture().via/via/

Summary:
codemod to replace all `makeFuture().via` with just `via`.

Test Plan: contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, zeus-diffs@, targeting-diff-backend@, abirchall, fugalh, msk, exa, benj, njormrod, folly-diffs@

FB internal diff: D1715876

Signature: t1:1715876:1417712302:b882916d394f90caa23bd73fa68b8f786af649e3

9 years agovia and activate/deactivate chaining
Hans Fugal [Fri, 5 Dec 2014 17:33:30 +0000 (09:33 -0800)]
via and activate/deactivate chaining

Summary:
Better support and test chaining of `via` and `activate`/`deactivate`.

The real problem is that without the ref qualifier a multiple chain of lvalue references can end up with a destructed object being referenced after.

https://akrzemi1.wordpress.com/2014/06/02/ref-qualifiers/

Test Plan:
This is mostly new tests that would fail and now pass.
I think maybe the tests are a bit weak, it would be good to find a way to ensure we aren't going to see the access-after-free bugs in these tests.

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, fugalh, exa, njormrod, folly-diffs@

FB internal diff: D1714873

Tasks: 5489801

Signature: t1:1714873:1417628538:9e610c5ba5e0a22c19a11d53aa956be45d585058

9 years agoRemove Later
Hans Fugal [Fri, 5 Dec 2014 17:33:25 +0000 (09:33 -0800)]
Remove Later

Summary: dalek-exterminate

Test Plan:
Moved `Later` tests to `via` tests.
fbgs
contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: fbcode-common-diffs@, adityab, lins, trunkagent, fugalh, exa, njormrod, folly-diffs@, hannesr

FB internal diff: D1714862

Tasks: 5409538

Signature: t1:1714862:1417621949:f63f49e1093a021170d2346e8e673db042d2bc56

9 years agoHHWheelTimer::scheduleTimeoutFn
Hans Fugal [Fri, 5 Dec 2014 17:29:04 +0000 (09:29 -0800)]
HHWheelTimer::scheduleTimeoutFn

Summary:
cpp11ify

I'm gonna use this for Wangle timeouts.

Test Plan:
new unit test
_bin/folly/io/async/test/HHWheelTimerTest

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, njormrod, folly-diffs@

FB internal diff: D1720014

Tasks: 5002969

Signature: t1:1720014:1417732753:ddcad431b6c305c9ba27f6b9e3454ab158c13f4b

9 years agoRevert "Make folly::Singleton's destruction happen earlier"
Bartosz Nitka [Fri, 5 Dec 2014 13:30:32 +0000 (05:30 -0800)]
Revert "Make folly::Singleton's destruction happen earlier"

Summary: This reverts commit 6584412293cc5b9aad2004e99d550e3478df1e5d.

Test Plan: thisisrevert

Reviewed By: smarlow@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1721739

Signature: t1:1721739:1417786147:01aaaf50b679da7942b2a30ad039eb0ac1031430

9 years agoinclude gflags header in SSLSessionCacheManager
Nicholas Ormrod [Thu, 4 Dec 2014 22:08:43 +0000 (14:08 -0800)]
include gflags header in SSLSessionCacheManager

Summary:
SSLSessionCacheManager uses glfags but doesn't include it. This
breaks on some systems.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, fugalh, ssl-diffs@, njormrod, folly-diffs@

FB internal diff: D1716997

Signature: t1:1716997:1417720032:548c2ad8f54e41d42585c331324f2065733595c1

9 years agoMake folly::Singleton's destruction happen earlier
Andrii Grynenko [Thu, 4 Dec 2014 01:26:45 +0000 (17:26 -0800)]
Make folly::Singleton's destruction happen earlier

Summary: Schedule destroyInstances to be executed via atexit in registrationComplete. registrationComplete is called from main(), so this makes sure folly::Singleton's will be destoyed before all singletons constructed before main() (which should cover all third-party libraries' singletons).

Test Plan: unit test

Reviewed By: chip@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1717963

Tasks: 5666654

Signature: t1:1717963:1417659478:c50c271d1786be75499565f6ab9c0a1a6f6f347d

9 years agoCorrect getsockopt call made by AsyncSocket
Andre Pinto [Thu, 4 Dec 2014 19:52:53 +0000 (11:52 -0800)]
Correct getsockopt call made by AsyncSocket

Summary:
The getsockopt's last parameter (optlen) is a value-result parameter
and AsyncSocket::getSockOpt was passing a value as argument.

Test Plan: Unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1717463

Tasks: 4867290

Signature: t1:1717463:1417664828:6c7a74ff31725121f892ce1adba2653e70728192

9 years agoadd consumeUntilDrained API to NotificationQueue::Consumer
James Sedgwick [Thu, 4 Dec 2014 18:29:24 +0000 (10:29 -0800)]
add consumeUntilDrained API to NotificationQueue::Consumer

Summary:
... and employ it in EventBase's destructor to stop leaking memory from unexecuted NotificationQueue-variety runInEventBaseThread() callbacks

in addition to the attached task, this should also fix the root cause of the already-worked-around #5564342

Test Plan:
test no longer leaks.* on the other hand, IOThreadPoolExecutor::stop() no longer actually stops, it joins. I added comments to stop() in headers indicating that it should be treated as best-effort.

* hilariously, this actually isn't true - there's a new leak of a thread local inside the io pool because callbacks can outlive the pool now. i'll leave a full explanation for the upcoming patch.

Reviewed By: davejwatson@fb.com

Subscribers: alandau, bmatheny, mshneer, trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1682860

Tasks: 5336655

Signature: t1:1682860:1416347774:ac5f31fb72373992f425c93ac284a0cd27608db4

9 years agoReplace Later in tcc
Hans Fugal [Wed, 3 Dec 2014 17:38:47 +0000 (09:38 -0800)]
Replace Later in tcc

Summary:
s/`sendAsyncLater`/`sendAsyncVia`/ and change it to return a cold future with equivalent semantics. We have to call `via` twice in the implementation—once to gate the result and again to make sure the caller will get a future that executes in that executor's context also.

NB this is a slight semantic change - if the executor is, say, a threadpool then now it will go through the threadpool queue (and maybe execute in two different threads) whereas before the whole later chain would execute in the same thread. I don't *think* this is a problem, but something to think about.

Test Plan:
stuff we fbconfig'd builds
contbuild
unit tests
thinking really hard

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, net-systems@, ldbrandy, hannesr, fugalh, zhuohuang, exa, watashi, smarlow, akr, bnitka, jcoens, darshan, njormrod, anfarmer, folly-diffs@

FB internal diff: D1644012

Tasks: 5409538

Signature: t1:1644012:1417625401:99b1b7df6de4cfcdd945eed7104d4c82e8c0b78f

9 years agoFixing static initilize fiasco in Benchmark.cpp
Maxim Sokolov [Tue, 2 Dec 2014 00:03:36 +0000 (16:03 -0800)]
Fixing static initilize fiasco in Benchmark.cpp

Summary:
Benchmark.cpp code is dependent on initialization order, which leads
to benchmarks not running. The root cause is that on some linkers the code which
adds benchmarks to the benchmarks vector is run before the vector initialization.

The original bug report: https://github.com/facebook/folly/issues/18
See: http://www.parashift.com/c++-faq-lite/static-init-order.html

Test Plan: run benchmark_test and compare result with master output

Reviewed By: andrei.alexandrescu@fb.com, njormrod@fb.com

Subscribers: trunkagent, sdwilsh, folly-diffs@

FB internal diff: D1710588

Signature: t1:1710588:1417468175:fd2705a573cef5c4ff020b60c6aec3d83bcdbbab

9 years agoReadd unit test
Andrei Bajenov [Mon, 1 Dec 2014 21:52:38 +0000 (13:52 -0800)]
Readd unit test

Summary:
Description:
Last time a merge error broke stuff, fixing and readding

Test Plan:
Ran unit test

This reverts commit 1075ab9d6f62cad36a228bd3fc7203a75fdf5baf.

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@, wstefancik

FB internal diff: D1695920

Signature: t1:1695920:1416529609:3f726ac5df4e33f254075738ea6655a6fc01bb7f

9 years agoFix dynamic::insert
Pavlo Kushnir [Thu, 27 Nov 2014 03:28:18 +0000 (19:28 -0800)]
Fix dynamic::insert

Summary: folly::dynamic::insert is broken. When we try to insert an rvalue and the key already exists in dynamic, insert will replace the value with empty dynamic.

Test Plan: unit test attached

Reviewed By: stepan@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1704995

Signature: t1:1704995:1417053631:8e0163693df721c54b016f8c27e81e7bf60194cb

9 years agofix varint decoding in case the first buffer is smaller than encoded varint
Mainak Mandal [Wed, 26 Nov 2014 02:27:13 +0000 (18:27 -0800)]
fix varint decoding in case the first buffer is smaller than encoded varint

Summary: This is not a super clean solution, given that I have just copied the code over form Varint.h. Another way would be to add a peek(IOBuf&, size_t) method to CursorBase and use that to peek deeper into the chain.

Test Plan: unit tests

Reviewed By: tulloch@fb.com

Subscribers: trunkagent, njormrod, dcapel, benr, folly-diffs@

FB internal diff: D1689872

Signature: t1:1689872:1416964989:b7f12a2686233f161401288ffcb8c51926b01fdf

9 years agoIOBuf::moveToFbString legacy 4GiB limit
Nicholas Ormrod [Wed, 26 Nov 2014 01:42:05 +0000 (17:42 -0800)]
IOBuf::moveToFbString legacy 4GiB limit

Summary:
IOBufs now support larger than 4GiB chunks. One of the checks
was not removed. Tracing through the capacity, its use if all 64-bit
compatible.

Test Plan: moveToFbString a 4GiB+ IOBuf element.

Reviewed By: maxim@fb.com

Subscribers: net-systems@, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1705316

Tasks: 5648445

Signature: t1:1705316:1416965241:d9d064bbde53e033d32bfe872b3445ee59005f5e

9 years agozlib compression fails on large IOBufs
Nicholas Ormrod [Wed, 26 Nov 2014 00:36:20 +0000 (16:36 -0800)]
zlib compression fails on large IOBufs

Summary:
If a single IOBuf has size exceeding 2^32, then our zlib
compression algorithm fails. Specifically, zlib z_stream.avail_in is
only 32 bytes (I think it's a long?
http://www.gzip.org/zlib/zlib_faq.html#faq32), and so a too-big IOBuf
will overflow the z_stream and cause data loss.

This diff breaks up large IOBufs into smaller chunks.

Test Plan:
fbconfig -r folly && fbmake runtests

Also compressed biggrep's configerator blob, which is how this bug was
caught. It now works. See the associated task.

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1702925

Tasks: 5648445

Signature: t1:1702925:1416958232:459d498ff1db13e1a20766855e6f2f97da8cde8c

9 years agoMake SSLContext's exit() handling more graceful
Subodh Iyengar [Tue, 25 Nov 2014 21:07:16 +0000 (13:07 -0800)]
Make SSLContext's exit() handling more graceful

Summary:
Some platforms that SSLContext run on call exit() despite best practices.
This cause the static structures to be destructed and cause race conditions
and crashes.

This new's the static structures so that they won't be destructed on exit()

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: trunkagent, ssl-diffs@, seanc, njormrod, folly-diffs@

FB internal diff: D1702186

Tasks: 5572637

Signature: t1:1702186:1416941649:c5bdfd8cc604fc3618f51bdb8b910b5b2cf350ad

9 years agofix Makefile.am
James Sedgwick [Fri, 21 Nov 2014 23:23:56 +0000 (15:23 -0800)]
fix Makefile.am

Summary: also kill duplicated test file

Test Plan: run all exp/wangle tests

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1698503

Signature: t1:1698503:1416612139:7139c6aa2ca79eabdaec6c33192194df984af348

9 years agofix build
Dave Watson [Fri, 21 Nov 2014 23:06:53 +0000 (15:06 -0800)]
fix build

Summary: bad merge with D1661169 caused a unittest to fail.  Update unittest

Test Plan: fbconfig -r folly/experimental/wangle; fbmake runtests

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, njormrod, folly-diffs@

FB internal diff: D1698435

Signature: t1:1698435:1416611458:555b6b50baa1bc5850b10e19c9c94978f93f2a5a

9 years agoChannelPipeline tests and fixes
James Sedgwick [Fri, 21 Nov 2014 21:06:05 +0000 (13:06 -0800)]
ChannelPipeline tests and fixes

Summary: As above. This paid off with a couple bugfixes.

Test Plan: run em all

Reviewed By: hans@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1695106

Signature: t1:1695106:1416522038:0c3345aadf954bf346d35b99877e7f8dfcf3ceff

9 years agoAsyncSocketHandler
James Sedgwick [Fri, 21 Nov 2014 20:59:49 +0000 (12:59 -0800)]
AsyncSocketHandler

Summary:
mostly copypasta from TAsyncTransportHandler

Test Plan: compiles

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1690973

Signature: t1:1690973:1416529528:4feb187a68ad5405662b9b0efb160edd253a2977

9 years agotest for OutputBufferingHandler
James Sedgwick [Fri, 21 Nov 2014 19:55:49 +0000 (11:55 -0800)]
test for OutputBufferingHandler

Summary:
MLE: my first custom matcher

revealed some include issues, also fixed

Test Plan: guess

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1686223

Signature: t1:1686223:1416334120:41bce88d56dd9ca107f3ddc3927c6be9419370cf

9 years agomodifiable channel pipelines
James Sedgwick [Fri, 21 Nov 2014 19:30:14 +0000 (11:30 -0800)]
modifiable channel pipelines

Summary:
Basically the same interface as before, but you must specify the read and write types for the ends of pipeline. Implementation is cleaner as well; there's fewer levels of indirection

This dynamic casts shit all over the place and is less typesafe then the previous iteration, but I think with some carefully placed static_asserts, could be just as safe (in the case where you don't do any modification, anyway)

Right now you can only add to the front or back of the pipeline but the way it's set up you could add any number of mutations, including ones that are triggered by handlers. But this should (might?) be enough for Tunnel, which was the motivation.

Test Plan: basic test compiles, thrift2 diff still works with a one line change

Reviewed By: hans@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, bmatheny

FB internal diff: D1661169

Tasks: 5002299

Signature: t1:1661169:1416521727:1f126279796c0b09d1905b9f7dbc48a9e5540271

9 years agoAdd process group leader option
Alexey Spiridonov [Tue, 12 Aug 2014 19:24:40 +0000 (12:24 -0700)]
Add process group leader option

Summary: Now a subprocess can be reliably made a group leader -- good for job control.

Test Plan:
unit test, checked that the pgid test worked in bash, too (only OKAY_XXX is printed)

==> XXX.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_XXX
./YYY.sh

==> YYY.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_YYY
./ZZZ.sh

==> ZZZ.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_ZZZ

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, njormrod, folly-diffs@, simpkins

FB internal diff: D1492526

Signature: t1:1492526:1416528620:3cf98b1c1e334a7d551b2c2f3e76b1c70f07ad7c

9 years agoFix build break
Nicholas Ormrod [Thu, 20 Nov 2014 19:06:39 +0000 (11:06 -0800)]
Fix build break

Summary: Diff that had @override-unit-failures broke the build

Test Plan: fbconfig -r folly && fbmake dbg && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: sdwilsh, njormrod, folly-diffs@

FB internal diff: D1694388

Tasks: 5662947

Signature: t1:1694388:1416510278:d50f4896814a34d73d55be292961f5ecd74b1bb4

Blame Revision: D1680735

9 years agomerge event base tests
James Sedgwick [Thu, 20 Nov 2014 18:02:13 +0000 (10:02 -0800)]
merge event base tests

Summary: someone made another event base test while the big set still lived in thrift. merge it now that the old tests have been moved.

Test Plan: fbconfig folly/io/async/test && fbmake runtests --extended-tests

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1690764

Signature: t1:1690764:1416413748:f3ae64d509432844bb2e1b689a75d66825269d15

9 years agoBump version to 17:0
Dave Watson [Thu, 20 Nov 2014 17:32:17 +0000 (09:32 -0800)]
Bump version to 17:0

9 years agofolly/io/async documentation
Dave Watson [Wed, 19 Nov 2014 17:41:06 +0000 (09:41 -0800)]
folly/io/async documentation

Summary: Documentation brain-dump.  The sources already have pretty good doxygen-style comments, but could use a good high-level overview.

Test Plan: Reviewers get to edit

Reviewed By: jsedgwick@fb.com, afrind@fb.com, dcsommer@fb.com, alandau@fb.com

Subscribers: fugalh, subodh, kmdent, seanc, vloh, sjenkins, kparton, doug, net-systems@, njormrod, folly-diffs@, sgolemon, alikhtarov, andrii, tudorb

FB internal diff: D1690976

Tasks: 5648428

Signature: t1:1690976:1416419706:3894976c32904f040d518b9a4ce26ccb40d7ed80

9 years agoHHWheelTimer - fix scheduling timeouts within callbacks
Andrei Bajenov [Thu, 20 Nov 2014 01:47:07 +0000 (17:47 -0800)]
HHWheelTimer - fix scheduling timeouts within callbacks

Summary:
This one is a little tricky, but it explains what happened in t5388156. Basically what we observed in the stack traces is that the HHWheelTimer count_ is 0, but there is still a timeout scheduled on the AsyncTimeout.

How I can see this happening is the following:
- SASL times out
- We fall back to insecure mode in the callback and schedule a send of a real message to the server. Here we schedule another timeout on the same HHWheelTimer object. Since count_ is 0 at this point, we schedule a timeout in AsyncTimeout.
- Now the initial SASL timeout was late, so the HHWheelTimer is in a catchup state. This means that it's possible for the callback on the timeout that was just scheduled to fire on the same "tick".
- This callback invokes SR's callbacks which try to detach the event base from the channel. But it can't detach because we have something scheduled on the AsyncTimeout.
- We crash.

Test Plan:
- will wait for contbuild
- Something like this repros the problem: https://phabricator.fb.com/P17220063. Note I hacked the catchup part to get it to work. Now running loadgen with security = permitted causes the crash.

// hphp tests are stalled, have to do this...

Reviewed By: mshneer@fb.com

Subscribers: bmatheny, mshneer, sandeepkk, trunkagent, njormrod, folly-diffs@, alandau, mhorowitz, srenfro, hitesh, wstefancik

FB internal diff: D1680735

Tasks: 5388156

Signature: t1:1680735:1416252123:e76668860ccda9380a87996b4fa3de957e129404

9 years agoBump version to 16:0
Dave Watson [Wed, 19 Nov 2014 20:54:03 +0000 (12:54 -0800)]
Bump version to 16:0

9 years agoOS merges
Dave Watson [Mon, 10 Nov 2014 18:14:41 +0000 (10:14 -0800)]
OS merges

Summary:
All pretty trivial:

https://github.com/facebook/folly/issues/99
https://github.com/facebook/folly/issues/98
https://github.com/facebook/folly/issues/45

Test Plan: Will watch jenkins fbthrift build

Reviewed By: dcsommer@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1669952

Signature: t1:1669952:1415643677:906234f0a89f38645b0072d3c88762d8fa2729dc

9 years agoserver bootstrap
Dave Watson [Tue, 28 Oct 2014 21:46:19 +0000 (14:46 -0700)]
server bootstrap

Summary:
ServerBootstrap a la netty.

This should be enough for some refactoring of thrift server and proxygen servers - but there are still lots of TODOs left to do

Test Plan:
Unittests included

Depends on D1638358

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, jsedgwick, afrind, dcsommer

FB internal diff: D1649521

Tasks: 5488516

Signature: t1:1649521:1416256073:fc003fd471bdfd137160dd6d7befd933ee8addd2

9 years agoRun the HHWheelTimer and EventBase tests
Daniel Sommermann [Tue, 18 Nov 2014 22:15:57 +0000 (14:15 -0800)]
Run the HHWheelTimer and EventBase tests

Summary: Fix the TARGETS from the refactor

Test Plan: ran it locally

Reviewed By: davejwatson@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1689151

Signature: t1:1689151:1416353673:3d3575b10f963e9f558954b8f3eba8bf48cdaa74

9 years agoadd custom getLoad() for multifeed leaf
Edward Chin [Tue, 18 Nov 2014 21:21:00 +0000 (13:21 -0800)]
add custom getLoad() for multifeed leaf

Summary:
getLoad() is called in servicerouter loadbalancing when a load_counter is not specified.  I added a custom getLoad() for leaf and made it so that different metrics can be tested without code pushes.

Changes:
1) Overloaded PriorityThreadManager::getCodel() so you can specify a PRIORITY.  aggr->leaf thrift calls are marked as HIGH priority.
2) The value of FLAGS_load_metric determines what metric the custom getLoad() will use (codel, max requests, or fallback to ThriftServer::getLoad()).
3) Smc listener for FLAGS_load_metric.
4) Removed server->setIsUnevenLoad(false) since there isn't reason this need to be set, and activeRequests_ and getPendingCount() are not tracked when isUnevenLoad_ is false.

Test Plan: Run leaf and check that smc listener works.  Custom getLoad() won't be called until t5625774 is resolved.

Reviewed By: hitesh@fb.com

Subscribers: njormrod, folly-diffs@, trunkagent, varunk, fugalh, alandau, nkgupta, bmatheny, jaredg, xie, mshneer, yguo

FB internal diff: D1685215

Signature: t1:1685215:1416339297:0d1f8cb0cec27242d95f0067337d80149b0a720e

9 years agoClean up runBeforeLoop API
Dave Watson [Thu, 13 Nov 2014 20:51:43 +0000 (12:51 -0800)]
Clean up runBeforeLoop API

Summary: Clean up API per stepan's comments in D1641057: runBeforeLoop callbacks should be run before the if block, so runInLoop calls will work.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1679773

Signature: t1:1679773:1415916488:f20dd98bf40dd4cb19a4b11875dc0c9fa3e7870e

Blame Revision: D1641057

9 years agoMove some tests to folly
Daniel Sommermann [Thu, 13 Nov 2014 18:49:31 +0000 (10:49 -0800)]
Move some tests to folly

Summary:
These lived in fbthrift previously, but they should move with the
main code too.

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1683229

Signature: t1:1683229:1416010730:36fb7e4c9916ae7a9b5972cd476f82014c5f4c78

9 years agotop-level via()
James Sedgwick [Tue, 18 Nov 2014 18:05:10 +0000 (10:05 -0800)]
top-level via()

Summary: as discussed

Test Plan: compiles :)

Reviewed By: steveo@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1687921

Signature: t1:1687921:1416331366:4e58a69daf39797e0d456ba86a463a9da2fd0369

9 years agoImprove initialization of openssl
subodh [Sun, 16 Nov 2014 01:17:53 +0000 (17:17 -0800)]
Improve initialization of openssl

Summary:
SSLContext takes it upon itself to
perform initialization of openssl along with
cleaning it up.

However in applications where there are multiple
libraries using openssl, cleanup is not really
desirable. Thus the application should own init
and cleanup of openssl, rather than SSLContext.

This is bad because openssl's init apis are not thread
safe.

This diff modifies SSLContext with 3 changes to
make this possible in the context of apps:

1. Add macro SSLCONTEXT_NO_REFCOUNT, for apps to be
able to disable refcounting
2. Move all steps of init into initializeOpenSSL.
As a result, apps can choose when to call
SSLContext::intitializeOpenSSL and if it has
been called before, openssl will not be reinitialized.
3. Move randomize as a static method to make it a part
of the initialization process.

Test Plan: Unit tests.

Reviewed By: ranjeeth@fb.com

Subscribers: ranjeeth, folly-diffs@, ssl-diffs@, njormrod, seanc, trunkagent, dihde

FB internal diff: D1686397

Signature: t1:1686397:1416270565:de805794d65af1c164ed334ff7ba93fe60b2b78a

9 years agoFix build break
Nicholas Ormrod [Mon, 17 Nov 2014 23:47:05 +0000 (15:47 -0800)]
Fix build break

Summary:
Change counting index to unsigned to avoid unsigned-signed
comparison errors.

@build-break

Test Plan:
fbconfig -r folly && fbmake dbg
fbmake runtests wangle

Reviewed By: robbert@fb.com, davejwatson@fb.com

Subscribers: sdwilsh, fugalh, njormrod, folly-diffs@

FB internal diff: D1686315

Signature: t1:1686315:1416267974:d8f6075550b9d611790263338dd4f49e7ef451cc

Blame Revision: D1595200

9 years agoIndefinite observers and inline observers
James Sedgwick [Mon, 17 Nov 2014 23:26:00 +0000 (15:26 -0800)]
Indefinite observers and inline observers

Summary:
provides new APIs observe(ObserverPtr<T>) and observe(Observer<T>*) that
are not associated with a subscription. The latter method avoids allocations at the cost of disowning responsiblity for object lifetimes.

It could be used for batch-allocating filter chains, for instance. I hacked up some recursive templates that would make that easy to do, but
haven't included them. I'd rather see the first use case and then decide whether to let users set them up themselves or to provide an
abstraction that makes it easier.

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, fugalh, njormrod

FB internal diff: D1595200

Tasks: 5002278

Signature: t1:1595200:1416266578:95c0532af42443fd83e2f94d30790c3c0831ce65

9 years agoOutputBufferingHandler
James Sedgwick [Mon, 17 Nov 2014 23:23:18 +0000 (15:23 -0800)]
OutputBufferingHandler

Summary:
see D1618704 and D1604575
Really need to make a decision about making a push for EventBaseManager everywhere. Getting/setting EBs when there's only ever one seems so silly, but in (e.g.) tests where one-off EBs are used without using the EBM, stuff like this will break.

Test Plan: definitely compiles and sorta works, will write unit tests if the dust settles

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, trunkagent, fugalh, njormrod

FB internal diff: D1618727

9 years agoCompactProtocol: readVarint optimizations
Alex Landau [Fri, 14 Nov 2014 04:50:30 +0000 (20:50 -0800)]
CompactProtocol: readVarint optimizations

Summary:
* Unroll readVarint() - no loops in the fast path
* Split Cursor::skip() into fast and slow paths

Test Plan:
```
------ old ------   ------ new ------
Benchmark                               time/iter  iters/s  time/iter  iters/s        win
=========================================================================================
CompactProtocol_read_Empty                13.21ns   75.69M    13.21ns   75.71M      0.03%
CompactProtocol_read_SmallInt             29.95ns   33.39M    28.96ns   34.53M      3.41%
CompactProtocol_read_BigInt               67.83ns   14.74M    36.77ns   27.19M     84.46%
CompactProtocol_read_SmallString          55.72ns   17.95M    54.04ns   18.50M      3.06%
CompactProtocol_read_BigString           342.93ns    2.92M   332.18ns    3.01M      3.08%
CompactProtocol_read_BigBinary           186.18ns    5.37M   181.45ns    5.51M      2.61%
CompactProtocol_read_LargeBinary         190.65ns    5.25M   184.85ns    5.41M      3.05%
CompactProtocol_read_Mixed               101.97ns    9.81M    97.06ns   10.30M      4.99%
CompactProtocol_read_SmallListInt        148.09ns    6.75M   139.52ns    7.17M      6.22%
CompactProtocol_read_BigListInt           71.69us   13.95K    63.34us   15.79K     13.19%
CompactProtocol_read_BigListMixed          1.26ms   790.67     1.18ms   845.66      6.95%
CompactProtocol_read_LargeListMixed      139.80ms     7.15   134.68ms     7.42      3.78%
```

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1681244

Signature: t1:1681244:1416265579:f630c53f9c31788f847d0af72198bbd9e5ea43f1

9 years agoimplicit constructor exception_wrapper(std::exception)
James Sedgwick [Mon, 17 Nov 2014 23:04:23 +0000 (15:04 -0800)]
implicit constructor exception_wrapper(std::exception)

Summary: this should address some of the wordiness issues with exception_wrapper, i.e. always having to use make_exception_wrapper

Test Plan: wrote a little test to make sure everything compiles (or doesn't) as expected, not worth committing

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1679460

Signature: t1:1679460:1416265170:5cd72d95dd855cd4e594dbbc49d0c53d012fbc99

9 years agofix potential race/memory corruption in IOThreadPoolExecutor
James Sedgwick [Mon, 17 Nov 2014 23:04:35 +0000 (15:04 -0800)]
fix potential race/memory corruption in IOThreadPoolExecutor

Summary:
In unusual but possible circumstances, the EventBase and thus pending tasks will outlive the pool, so we shouldn't keep references of any kind to the pool in the task.
The only reference we were keeping was used to access the task stats rx subject. Store the subject as a shared ptr and give a copy of the ptr to the Thread object, which is itself
owned by a shared ptr and captured by every task. I thought this had to do with the thread local leak in mentioned in the test plan of D1682860 but this patch doesn't actually fix that :(
Thankfully, while task surfing I saw @phillip's awesome D1682698. Patching that in fixes the leak! Woo. Either way, this is more correct.

Test Plan: unit under clang/asan

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, philipp

FB internal diff: D1683221

Tasks: 5336655

Signature: t1:1683221:1416264933:946d29b5a3eb22ed08812f2adefb7284b1899e4e

9 years agopriority CPU thread pool
James Sedgwick [Mon, 17 Nov 2014 22:44:10 +0000 (14:44 -0800)]
priority CPU thread pool

Summary:
just extend CPUThreadPoolExecutor to use a queue that is itself composed of N mpmc queues, one per priority

the verbosity is starting to kill me, i had thought before of truncating Executor of all these pool types and now I'm definitely going to do that unless someone fights me.

Test Plan: added unit; maybe i'm not being clever enough as i couldn't think of many ways to test this reliably so there's just a basic preemption test

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, bmatheny

FB internal diff: D1676452

Tasks: 5002392

Signature: t1:1676452:1416263990:cdf5d44e4a50a6180ba547a3ed4c0c24d4ffdd8f

9 years agoFix format bug
Dave Watson [Thu, 13 Nov 2014 20:31:47 +0000 (12:31 -0800)]
Fix format bug

Summary: Fix addition bug instead of string concat

Test Plan: ?? It compiles?  The only call paths are when syscalls fail, it would be a lot of work to stub them out

Reviewed By: mshneer@fb.com

Subscribers: trunkagent, doug, njormrod, folly-diffs@

FB internal diff: D1679713

Tasks: 5604354

Signature: t1:1679713:1415925209:ed05d3d1ce3e7e8db32c4afc86c5f24748d9c65b

9 years agoMove Acceptor to wangle
Dave Watson [Thu, 23 Oct 2014 17:26:08 +0000 (10:26 -0700)]
Move Acceptor to wangle

Summary:
Initial pass at moving acceptor to wangle.  Involves moving most of the config stuff from proxygen/lib/services, and *all* of the ssl stuff from proxygen/lib/ssl.

Only minor changes:
* Acceptor can be overriden to use thrift socket types, so I don't have to change TTransportException everywhere just yet
* proxygen::Exception to std::runtime_exception in a few spots - looks like it is entirely bad config exceptions, so it should be okay
* Just used std::chrono directly instead of stuff in Time.h (which is just typedefs and simple helpers)

Test Plan:
used in D1539327

fbconfig -r proxygen/httpserver; fbmake runtests

Probably other projects are broken, will iterate to fix

None of the failling tests look related

Reviewed By: dcsommer@fb.com

Subscribers: oleksandr, netego-diffs@, hphp-diffs@, ps, trunkagent, doug, fugalh, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1638358

Tasks: 5002353

Signature: t1:1638358:1414526683:87a405e3c24711078707c00b62a50b0e960bf126

9 years agofix potential memory leak in ThreadLocal
Philip Pronin [Fri, 14 Nov 2014 22:00:47 +0000 (14:00 -0800)]
fix potential memory leak in ThreadLocal

Summary:
See this LSan abort: https://phabricator.fb.com/P17233565.

Destructor of the object stored in `folly::ThreadLocal` itself may be using
`folly::ThreadLocal` with the same tag, with the current implementation these
objects may escape cleanup happening on thread exit.

Test Plan:
_build/opt/folly/test/thread_local_test --gtest_filter=ThreadLocalPtr.CreateOnThreadExit

Reviewed By: lucian@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1682698

Tasks: 5596043

Signature: t1:1682698:1416006810:100aaa5c17cecceeea568165d552d9d7907f38d0

9 years agoCompactProtocol: more improvements
Alex Landau [Thu, 13 Nov 2014 01:19:15 +0000 (17:19 -0800)]
CompactProtocol: more improvements

Summary:
Add a specialized Cursor method that reads exactly 1 byte, in
addition to the existing one that reads into a buffer. Use it when
reading varints and standalone bytes in a CompactProtocol struct.

Test Plan:
```
------ old ------   ------ new ------
Benchmark                               time/iter  iters/s  time/iter  iters/s        win
=========================================================================================
CompactProtocol_read_Empty                18.68ns   53.54M    13.21ns   75.69M     41.37%
CompactProtocol_read_SmallInt             42.60ns   23.47M    29.95ns   33.39M     42.27%
CompactProtocol_read_BigInt               83.62ns   11.96M    68.40ns   14.62M     22.24%
CompactProtocol_read_SmallString          67.33ns   14.85M    55.62ns   17.98M     21.08%
CompactProtocol_read_BigString           353.83ns    2.83M   330.19ns    3.03M      7.07%
CompactProtocol_read_BigBinary           190.82ns    5.24M   182.90ns    5.47M      4.39%
CompactProtocol_read_LargeBinary         200.95ns    4.98M   187.00ns    5.35M      7.43%
CompactProtocol_read_Mixed               137.42ns    7.28M   102.98ns    9.71M     33.38%
CompactProtocol_read_SmallListInt        203.98ns    4.90M   146.68ns    6.82M     39.18%
CompactProtocol_read_BigListInt          120.50us    8.30K    71.56us   13.97K     68.31%
CompactProtocol_read_BigListMixed          1.62ms   617.07     1.26ms   795.60     28.93%
CompactProtocol_read_LargeListMixed      177.50ms     5.63   140.73ms     7.11     26.29%
```

Reviewed By: haijunz@fb.com

Subscribers: trunkagent, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1678077

Signature: t1:1678077:1415923409:22accee6b62b6e2bf471f3758a290f71978a8c4e

9 years agofolly::Singleton-based McrouterManager
Andrii Grynenko [Tue, 11 Nov 2014 19:13:30 +0000 (11:13 -0800)]
folly::Singleton-based McrouterManager

Summary: Introduces a Singleton which keeps a map persistence_id => mcrouter_t*. Makes mcrouter instance not know if it's managed by McrouterManager.

Test Plan: unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, alikhtarov, njormrod, folly-diffs@

FB internal diff: D1673274

Signature: t1:1673274:1415735863:c990a6a526f9525c68cc23892d690a9b3cb94ace

9 years agoHave EventBase implement wangle::Executor
Hans Fugal [Wed, 12 Nov 2014 18:01:50 +0000 (10:01 -0800)]
Have EventBase implement wangle::Executor

Summary:
It already does the work (`runInEventBaseThread`) but it will now be convenient to pass an `EventBase` where wangle wants an `Executor`.

Had to rip off the `boost::noncopyable` from `wangle::Executor` which is an interface and does not require non-copyability so that didn't really belong there in the first place I think. (Without this change, you get an obscure compiler error because of the double-inheritance from `boost::noncopyable`).

Test Plan: Things build, tests pass

Reviewed By: davejwatson@fb.com

Subscribers: jsedgwick, trunkagent, fugalh, exa, njormrod, folly-diffs@, andrii

FB internal diff: D1671500

Signature: t1:1671500:1415727572:a7dba33c669ca122aecaee3c700f9e53e54838d1

9 years agofolly::EventBase: wrap libevent calls to prevent race-condition
Nick Burrett [Wed, 12 Nov 2014 13:30:43 +0000 (05:30 -0800)]
folly::EventBase: wrap libevent calls to prevent race-condition

Summary:
Patch D1585087 exposes two flaws in EventBase().  It introduces IO
worker threads to the ThriftServer which are constructed/destructed in
parallel.

Within the construction phase, a new EventBase() is instantiated for
each thread and unwound in destruction.

When using the BaseControllerTask (in Python), the following sequence
is observed:

a = event_init() [ThriftServer]
b = event_init() [IO worker 1]
c = event_init() [IO worker 2]
...
event_base_free(c)
event_base_free(b)
event_base_free(a)  -> segfault

1. event_init() should only ever be called once.  It internally
modifies a global variable in libevent, current_base to match the
return value.  event_base_free() will set current_base back to NULL if
the passed in arg matches current_base.   Therefore subsequent calls
must use event_base_new().

2. Since current_base is a global and EventBase() is called by multiple
threads, it is important to guard with a mutex.  The guard itself also
exposed the bug because:

a = event_init()  [current_base = a]
b = event_init()  [current_base = b]
...
event_base_free(b) [b == current_base -> current_base = NULL]

So current_base ends up prematurely set to NULL.

Test Plan:
Run dba/core/daemons/dbstatus/dbstatus_tests.lpar, which no longer
segfaults

Reviewed By: jsedgwick@fb.com, davejwatson@fb.com

Subscribers: dihde, evanelias, trunkagent, njormrod, ncoffield, lachlan, folly-diffs@

FB internal diff: D1663654

Tasks: 5545819

Signature: t1:1663654:1415732265:d51c4c4cae99c1ac371460bf18d26d4f917a3c52

Blame Revision: D1585087

9 years agoUse folly::IPAddress::hash instead of gethostid
Alecs King [Tue, 11 Nov 2014 21:53:09 +0000 (13:53 -0800)]
Use folly::IPAddress::hash instead of gethostid

Summary: find a non-loopback ipv4 or ipv6 address and feed it to folly::IPAddress::hash

Test Plan:
1)
fbconfig -r mcrouter
fbmake runtests

2)
compare hostids on different hosts

Reviewed By: pavlo@fb.com

Subscribers: njormrod, folly-diffs@, trunkagent, ps, bmatheny, alikhtarov

FB internal diff: D1668944

Tasks: 5557721

Signature: t1:1668944:1415736928:fb4b042a575c0b00f52780f3abf54bf7630b3a97

9 years agoClean up RequestContext
Dave Watson [Thu, 6 Nov 2014 17:59:27 +0000 (09:59 -0800)]
Clean up RequestContext

Summary: Using a ThreadLocal cleans up the code quite a bit.  Also reuse the shared_ptr in create instead of creating a new one saves an allocation

Test Plan:
fbconfig thrift/lib/cpp/test:RequestContextTest; fbmake runtests

fbconfig common/services/cpp/test:trace_test; fbmake runtests

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@, vloh

FB internal diff: D1663960

Signature: t1:1663960:1415390250:36d9b772016d2a12d804e98edbc1725af882e507

9 years agoOS merges
Dave Watson [Mon, 10 Nov 2014 18:14:41 +0000 (10:14 -0800)]
OS merges

Summary:
All pretty trivial:

https://github.com/facebook/folly/issues/99
https://github.com/facebook/folly/issues/98
https://github.com/facebook/folly/issues/45

Test Plan: Will watch jenkins fbthrift build

Reviewed By: dcsommer@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1669952

Signature: t1:1669952:1415643677:906234f0a89f38645b0072d3c88762d8fa2729dc

9 years agoAdd optional parameter pruneHook to EvictingCacheMap::set(..)
Dmitry Panin [Sat, 8 Nov 2014 01:31:02 +0000 (17:31 -0800)]
Add optional parameter pruneHook to EvictingCacheMap::set(..)

Summary:
Inside `set()` we can do pruning, but it will happen
with default pruneHook.
Adding it as an optional param makes API more convenient.
(Instead, the users of API could just call `setPruneHook(pruneHook)` before `set`,
and then `setPruneHook(nullptr)` afterwards -- but it looks too ugly)

Test Plan:
```
fbconfig -r folly/ && fbmake runtests
```
passes:
```
Summary (total time 60.11s):
PASS: 1758
FAIL: 0
SKIP: 0
FATAL: 0
TIMEOUT: 0
```

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, agartrell, njormrod, folly-diffs@

FB internal diff: D1665690

Tasks: 5551091

Signature: t1:1665690:1415391406:e4d2a956f9212aed70ab518159dbb19553764ce4

10 years agoBump version to 15:0
Pavlo Kushnir [Sat, 8 Nov 2014 02:43:32 +0000 (18:43 -0800)]
Bump version to 15:0

10 years agoFix folly lint errors
Nicholas Ormrod [Fri, 31 Oct 2014 17:48:19 +0000 (10:48 -0700)]
Fix folly lint errors

Summary:
Fix EOF whitespace
for file in `find folly -type f` ; do if [ -z "`tail -n1 $file`" ] ; then sed -i '$d' $file ; fi ; done

Test Plan: unit tests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644130

Tasks: 5486739

Signature: t1:1644130:1414715392:b6c783851aa030ad1148f84a98139a5dca207da0

10 years agoisDetachable for HHWheelTimer
Misha Shneerson [Fri, 7 Nov 2014 20:33:14 +0000 (12:33 -0800)]
isDetachable for HHWheelTimer

Summary: title

Test Plan: unit tests

Reviewed By: andrei.bajenov@fb.com

Subscribers: trunkagent, mcduff, hitesh, alandau, bmatheny, njormrod, mshneer, folly-diffs@, andreib, davejwatson

FB internal diff: D1666499

Tasks: 5563183

Signature: t1:1666499:1415347256:d58c8bbe952385c1c96f7f8cc6ae7f02216c56bb

10 years agoMove SSL socket to folly
Dave Watson [Tue, 14 Oct 2014 18:10:48 +0000 (11:10 -0700)]
Move SSL socket to folly

Summary: One of the last thrift -> folly moves.  The only change was the exception types - there are small wrapper classes in thrift/lib/cpp/async left to convert from AsyncSocketException to TTransportException.

Test Plan: run unit tests

Reviewed By: dcsommer@fb.com

Subscribers: jdperlow, trunkagent, doug, bmatheny, ssl-diffs@, njormrod, mshneer, folly-diffs@, fugalh, jsedgwick, andrewcox, alandau

FB internal diff: D1632425

Signature: t1:1632425:1414526483:339ae107bacb073bdd8cf0942fd0f6b70990feb4

10 years agoRemove volatile from MemoryMappingTest
Tom Jackson [Fri, 7 Nov 2014 01:20:41 +0000 (17:20 -0800)]
Remove volatile from MemoryMappingTest

Summary: Not needed, I don't know why I put them there before.

Test Plan: Run the unit test

Reviewed By: njormrod@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1665689

Tasks: 5487902

Signature: t1:1665689:1415323144:0998e7f700a3b40652615a36c3b9c9f661fbdadf

10 years agoAdd some SocketAddress tests
Daniel Sommermann [Thu, 6 Nov 2014 21:44:13 +0000 (13:44 -0800)]
Add some SocketAddress tests

Summary: There was a bit of gap in test coverage for bracketed ipv6

Test Plan: unit tests

Reviewed By: viswanath@fb.com

Subscribers: doug, ps, bmatheny, njormrod, folly-diffs@

FB internal diff: D1664783

Signature: t1:1664783:1415316694:b17e0dc1fcfece06c6e04e5f65e2095c66d28cc4

10 years agoFix integer overflow in folly/Benchmark.cpp
Dmitry Panin [Thu, 6 Nov 2014 20:47:43 +0000 (12:47 -0800)]
Fix integer overflow in folly/Benchmark.cpp

Summary:
`bm_max_secs` has type int32
So when this value is more than 2, expression
```
FLAGS_bm_max_secs * 1000000000;
```
overflows

Test Plan:
I did put a
```
LOG(INFO) << timeBudgetInNs;
```
and observed correct values (before fix they were overflowed)

Reviewed By: antonl@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1663247

Signature: t1:1663247:1415261814:c9154ffde183b2a4f5403e534e47e52e8276c61b

10 years agoBMI1 support in EliasFanoCoding
Philip Pronin [Tue, 4 Nov 2014 18:21:34 +0000 (10:21 -0800)]
BMI1 support in EliasFanoCoding

Summary:
This diff updates `folly::CpuId` with support of extended features (EAX =
7, ECX = 0) to provide detection logic for BMI1 introduced in Haswell, and
provides support for `BLSR` instruction in `EliasFanoReader`.

Test Plan:
I used clang to compile the logic and run unittests

Reviewed By: lucian@fb.com

Subscribers: fbcode-common-diffs@, trunkagent, chaoyc, search-fbcode-diffs@, unicorn-diffs@, njormrod, folly-diffs@

FB internal diff: D1658100

Signature: t1:1658100:1415126635:d1820b8eb41c9e9786b5c8062b801cf1e2049a97

10 years ago(wangle) fix a race condition in Core::maybeCallback
Hans Fugal [Wed, 5 Nov 2014 23:44:06 +0000 (15:44 -0800)]
(wangle) fix a race condition in Core::maybeCallback

Summary:
`calledBack_` could be seen as true by both threads in this conditional. Classic rookie mistake. :-/

Test Plan: run unit tests

Reviewed By: darshan@fb.com

Subscribers: trunkagent, hannesr, net-systems@, fugalh, exa, njormrod, folly-diffs@

FB internal diff: D1661199

Tasks: 55429385506504

Signature: t1:1661199:1415215840:fb69f56c8cf6f59beeca809724ce015b5260d9ad

Blame Revision: D1636487

10 years agoRemove memory leak from readRandomDevice by using a raw fd
Brian Watling [Wed, 5 Nov 2014 23:17:10 +0000 (15:17 -0800)]
Remove memory leak from readRandomDevice by using a raw fd

Summary: Switch from dynamically allocating a File to using a raw fd for readRandomDevice. This prevents races at shutdown while also eliminating the memory leak

Test Plan: run unit tests

Reviewed By: meyering@fb.com

Subscribers: njormrod, folly-diffs@, tao-eng@

FB internal diff: D1662151

Signature: t1:1662151:1415229242:525b6294b27bb68b5dda70aadb8d3ba1cc61b815

10 years agoFix crashes in readRandomDevice()
Brian Watling [Wed, 5 Nov 2014 22:50:37 +0000 (14:50 -0800)]
Fix crashes in readRandomDevice()

Summary: The comment in readRandomDevice() lies - the PCHECK fails with "Bad file descriptor" in some tests. This is most likely due to a race where a background thread is calling readRandomDevice() while the main thread shuts down. Another possibility is bad destruction order, where rnadomDevice is destroyed before some other static object whose destructor calls readRandomDevice(). Either way, this fixes it without having to chace down every instance.

Test Plan: run unit tests

Reviewed By: meyering@fb.com

Subscribers: antonl, trunkagent, meyering, simpkins, njormrod, folly-diffs@, tao-eng@

FB internal diff: D1660903

Signature: t1:1660903:1415215895:7f1f96ab7416b45c4dc18e78173fb59eb27bd03b

10 years ago(wangle) fix a race in whenAll
Hans Fugal [Wed, 5 Nov 2014 19:21:49 +0000 (11:21 -0800)]
(wangle) fix a race in whenAll

Summary:
Race in `++ctx->count == ctx->total`. This ordering, while not very obvious or likely, is possible:

T1            T2
--            --
++ctx->count
++ctx->count
ctx->total
==
setValue
delete ctx
ctx->total
==
setValue
delete ctx

Test Plan:
That's the idea, anyway. I need some sleep, and it takes 20 minutes to build and test. I had a more convoluted fix (using `shared_ptr`) and it did seem to fix the error we were seeing, but I was seeing another error.

Reviewed By: darshan@fb.com

Subscribers: trunkagent, net-systems@, fugalh, exa, njormrod, folly-diffs@

FB internal diff: D1660663

Tasks: 5506504

Signature: t1:1660663:1415207632:49dc224363cec27736fc71fb211fa846be9e170f

Blame Revision: D1636487

10 years agofolly: wangle: NamedThreadFactory: store of the thread prefix locally, don't assume...
Lucian Grijincu [Wed, 5 Nov 2014 00:51:03 +0000 (16:51 -0800)]
folly: wangle: NamedThreadFactory: store of the thread prefix locally, don't assume they'll live forever

Summary: There's no real reason to store this as a StringPiece, is there?

Test Plan: n/a

Reviewed By: sdoroshenko@fb.com, robot9@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1659558

Tasks: 5538418

Signature: t1:1659558:1415147560:2c5b0c996893854c3ea9d0ad02b006bcc5960ffa

10 years agoFix licenses
Nicholas Ormrod [Tue, 4 Nov 2014 00:18:41 +0000 (16:18 -0800)]
Fix licenses

Summary:
There are two types of licenses used in folly, only one of
which is accepted by the linter.

This diff changes the license notices in pre-existing folly files!
Many folly/io/async/* files have the second type of license, but
without a Copyright notice. I have added copyright notices to these
files.

I have also added a compliant notice to
folly/test/function_benchmark/benchmark_impl.h, which was the sole file
in folly/test/function_benchmark/ that didn't have a standard license.

Test Plan:
The changes to folly are comment only.

Run all of folly against the linter, see no more license errors.

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@, sjenkins

FB internal diff: D1648489

Tasks: 5486739

Signature: t1:1648489:1415035522:3d8bd9611eb7c7117b70d5e7f68de5768639a727

10 years agoFix global statics
Nicholas Ormrod [Mon, 3 Nov 2014 20:39:11 +0000 (12:39 -0800)]
Fix global statics

Summary:
Statics in headers are bad.

Test Plan: run unit tests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1646906

Tasks: 5486739

Signature: t1:1646906:1415042805:dc4d1cec54e9320f1e609808a73622c731a4cdc9

10 years agorm protected inheritance
Nicholas Ormrod [Mon, 3 Nov 2014 18:06:41 +0000 (10:06 -0800)]
rm protected inheritance

Summary:
Remove the protected inheritance from folly.

The code looks like the protected might be appropriate: there are some
derived classes which might want access to the AsyncTimeout.

Test Plan: run unit tests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1649686

Tasks: 5486739

Signature: t1:1649686:1415035288:18efd2cf9aae8caab66d8303c22cdc26c6b54ae5

10 years agonetty-like channels
James Sedgwick [Mon, 3 Nov 2014 17:37:35 +0000 (09:37 -0800)]
netty-like channels

Summary:
I wanted some foundational abstractions to start building out codecs on top of.
I also know that Blake gets very amused when I shamelessly copy Java/Netty abstractions, and I live to amuse Blake so I did it again.
So here's an implementation of something very similar to Netty's ChannelAdapters/ChannelPipelines
Only read() and write() for now, everything/anything else can easily be bolted on once the design is settled (if this is at all the direction we want to go)
I have a lot of thoughts about this design but I'm going to save my fingers and leave that to ad hoc discussions once folks take a look at this
A couple of things, though:
- It should be possible to dynamically add handlers to the chain. How I envision this working is that each original adapters keeps two lists of shared/unique ptrs to adapters of the correct type to sit next to them on either side, and dispatch to them appropriately when they're there.
- I was trying to do without separate ChannelHandlerContext objects altogether (keep the interface, but have ChannelPipeline act as the context itself) but ran into issues with virtual multiple inheritance. I might have another go at this.
- Only movable types are permitted. I hope this won't be too restrictive, because it would be a PITA to support both move-only and copy-only types.
- Why no Rx? Seems to me that any handlers that actually needs Rx (e.g. stats fanout or something) can deal with it themselves.
- If it turned out to be useful to nest these (for more flexible composition) that would also be doable. i.e. ChannelPipeline<ChannelPipeline<Handler1, Handler2>, ChannelPipeline<Handler3, Handler4>>

Test Plan: super basic test compiles and runs as expected

Reviewed By: davejwatson@fb.com

Subscribers: ajitb, folly-diffs@, ldbrandy, trunkagent, fugalh, njormrod

FB internal diff: D1604575

Tasks: 5002299

Signature: t1:1604575:1415034767:bc3b12fae726890aa6a55ed391286917ae23e56e

10 years agoFix -Wsign-compare
Daniel Sommermann [Mon, 20 Oct 2014 19:52:01 +0000 (12:52 -0700)]
Fix -Wsign-compare

Summary: Folly should be able to compile with strict flags.

Test Plan: unit tests, review

Reviewed By: meyering@fb.com

Subscribers: meyering, trunkagent, doug, fugalh, njormrod, folly-diffs@

FB internal diff: D1627280

Signature: t1:1627280:1414792755:004f5a737ece1e93bcf4331718a98afc57e4f80c

10 years agoName prefix setter for NamedThreadFactory
James Sedgwick [Mon, 3 Nov 2014 00:28:49 +0000 (16:28 -0800)]
Name prefix setter for NamedThreadFactory

Summary: it's useful to update the prefix after the construction.

Test Plan:
unit, that's it :/

Reviewed By: davejwatson@fb.com

Subscribers: mshneer, folly-diffs@, wch, atlas2-eng@, everstore-dev@, wormhole-diffs@, ads-dsp-eng@, bwester, trunkagent, fugalh, alandau, njormrod, bmatheny

FB internal diff: D1585087

10 years agoFix bad rebase for SSLContext
Dave Watson [Fri, 31 Oct 2014 19:49:57 +0000 (12:49 -0700)]
Fix bad rebase for SSLContext

Summary: Put back previous liger diffs

Test Plan: It builds

Reviewed By: seanc@fb.com

Subscribers: doug, ssl-diffs@, njormrod, folly-diffs@

FB internal diff: D1652754

Signature: t1:1652754:1414785984:df0fc7bf59dc2e89defd2c1a4ffe3b288238ba58

10 years agoFix line too long lint errors
Nicholas Ormrod [Fri, 31 Oct 2014 21:54:30 +0000 (14:54 -0700)]
Fix line too long lint errors

Summary:
Many lines that are too long contain URLs. This diff adds
url-like lines to the list of exceptions to 80-char limits. It also
fixed the locations in folly with line-too-long errors.

Test Plan: arc lint on folly, see no line-too-long errors.

Reviewed By: robbert@fb.com

Subscribers: sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644151

Tasks: 5486739

Signature: t1:1644151:1414779408:76bcec1d14daaa8ed071c715bf26b108c8fe4b87

10 years agoIncrease size of demangled symbol buffer
Mark Drayton [Fri, 31 Oct 2014 20:42:40 +0000 (13:42 -0700)]
Increase size of demangled symbol buffer

Summary:
Symbols which demangle to a string longer than 1024 bytes are
quite common in our code. This diff increases the size of the output
buffer to accommodate them.

Test Plan: run it

Reviewed By: lucian@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1639801

Tasks: 5464222

Signature: t1:1639801:1414784601:2f59d5a58e434f4cf9df5b25b917c5094c8b133f

10 years agoUndo a local change
Sachin Kadloor [Fri, 31 Oct 2014 18:13:13 +0000 (11:13 -0700)]
Undo a local change

Summary: I had committed a change by mistake. Reverting it.

Test Plan: build it.

Reviewed By: lakshmiganesh@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1652341

Signature: t1:1652341:1414779013:9f363acba95fbac1988200081659edf71ac63eec

10 years agovolatile override
Nicholas Ormrod [Fri, 31 Oct 2014 18:07:44 +0000 (11:07 -0700)]
volatile override

Summary:
Some uses of volatile are legit. Add a hidden override.

Test Plan: run unit tests

Run flint against folly/Malloc.cpp, see no more volatile warning.

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: sdwilsh, louisk, njormrod, folly-diffs@

FB internal diff: D1644493

Tasks: 5486739

Signature: t1:1644493:1414715317:491d0f631d8152a5b7ec66237e00c404530ab590

10 years agofix build
Dave Watson [Fri, 31 Oct 2014 17:37:55 +0000 (10:37 -0700)]
fix build

Summary: add missing ssl dep

Test Plan: build it

Reviewed By: dcsommer@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1652234

Signature: t1:1652234:1414778507:4504ea06c3076e0cdfa7772166cf48bc564c9076

10 years agoFix large size class category handling in folly::goodMallocSize()
Tomislav Novak [Fri, 24 Oct 2014 18:46:51 +0000 (11:46 -0700)]
Fix large size class category handling in folly::goodMallocSize()

Summary:
jemalloc seems to now use `4064 * 1024` as `arena_maxclass`. Therefore,
allocations that fall in the range `(4064*1024, 4072*1024]` will trigger an
assert in `folly::goodMallocSize()` (`nallocx()` will round up sizes within
that range up to the next multiple of 4 MB (chunk size)).

Test Plan: Call to `folly::goodMallocSize(4161568)` no longer triggeres an assert.

Reviewed By: je@fb.com

Subscribers: aalexandre, trunkagent, njormrod, folly-diffs@

FB internal diff: D1638370

Signature: t1:1638370:1414703429:f3ed2cc8dcb183ff4e8367e2c1f884e122605cba

10 years agoSocketAddress logging
Sachin Kadloor [Fri, 31 Oct 2014 17:58:47 +0000 (10:58 -0700)]
SocketAddress logging

Summary:
Better logging in SocketAddress

Test Plan: run it

Reviewed By: qwu@fb.com

Subscribers: ps, bmatheny, xning

FB internal diff: D1644573

Tasks: 5406091

Signature: t1:1644573:1414626398:1e23ee85ec8ee0be89cb6bdc07da23978b4d988a

10 years agoFix/override bad symbol names
Nicholas Ormrod [Fri, 31 Oct 2014 17:55:44 +0000 (10:55 -0700)]
Fix/override bad symbol names

Summary:
Symbols containing two underscores, or starting with underscore
followed by a capital letter, are reserved by the implementation. Fix
the inappropriate cases, and nolint the ones which we are using
correctly

Test Plan: run unit tests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1650435

Tasks: 5486739

Signature: t1:1650435:1414713329:7844e7802ebabcac7ef21aef0becf71ba513afb0

10 years agoComparing the same tokens
Nicholas Ormrod [Fri, 31 Oct 2014 17:52:41 +0000 (10:52 -0700)]
Comparing the same tokens

Summary:
Avoid comparing-the-same-tokens error by parenthesizing one
side.

This diff only touches folly tests

Test Plan: run unit tests

Reviewed By: robbert@fb.com

Subscribers: sdwilsh, njormrod, folly-diffs@

FB internal diff: D1650386

Tasks: 5486739

Signature: t1:1650386:1414713837:cbb578d23f759aa049f4246113d9da6d40504da4

10 years agothrow() specs deprecated
Nicholas Ormrod [Fri, 31 Oct 2014 17:50:05 +0000 (10:50 -0700)]
throw() specs deprecated

Summary: Switch to noexcept

Test Plan: run unit tests

Reviewed By: robbert@fb.com, andrei.alexandrescu@fb.com

Subscribers: aalexandre, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644135

Tasks: 5486739

Signature: t1:1644135:1414715537:22e1baf91ab7e3250b0b2a460a12d56783f2baed

10 years agoFix folly lint errors
Nicholas Ormrod [Fri, 31 Oct 2014 17:48:19 +0000 (10:48 -0700)]
Fix folly lint errors

Summary:
Fix EOF whitespace
for file in `find folly -type f` ; do if [ -z "`tail -n1 $file`" ] ; then sed -i '$d' $file ; fi ; done

Test Plan: run unit tests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644130

Tasks: 5486739

Signature: t1:1644130:1414715392:b6c783851aa030ad1148f84a98139a5dca207da0

10 years agoSSLContext
Dave Watson [Wed, 1 Oct 2014 17:51:40 +0000 (10:51 -0700)]
SSLContext

Summary:
Move SSLContext to folly.

Test Plan: It builds.

Reviewed By: dcsommer@fb.com

Subscribers: jdperlow, atlas2-eng@, wormhole-diffs@, bwester, trunkagent, doug, ps, bmatheny, ssl-diffs@, alikhtarov, njormrod, mshneer, folly-diffs@, andrewcox, alandau, jsedgwick, fugalh

FB internal diff: D1631924

Signature: t1:1631924:1414616562:9a67dbf20f00eb8fbcb35880efcb94c0fae07dcc

10 years agosave/restore request context in future
Dave Watson [Thu, 30 Oct 2014 23:16:56 +0000 (16:16 -0700)]
save/restore request context in future

Summary:
Generic 'threadlocal' like object that follows async calls around.

Note: Finagle's futures do this also.  It is super useful.

Test Plan: I should write a unittest?

Reviewed By: haijunz@fb.com, hans@fb.com

Subscribers: trunkagent, dawidp, doug, fugalh, njormrod, folly-diffs@

FB internal diff: D1650843

Tasks: 4698780

Signature: t1:1650843:1414711295:c7439733680ab4903eb9c05dcf8bf52100bf3f07

10 years ago(wangle-test) fix double ifndef in thens.rb
Hans Fugal [Thu, 30 Oct 2014 23:20:17 +0000 (16:20 -0700)]
(wangle-test) fix double ifndef in thens.rb

Summary: Looks like a bad git merge

Test Plan: builds

Reviewed By: hannesr@fb.com

Subscribers: hannesr, net-systems@, fugalh, exa, njormrod, folly-diffs@

FB internal diff: D1649773

Tasks: 5501131

Signature: t1:1649773:1414700920:cc691283884e9654803bba3ffe78f4553194752d

10 years agooperator::* needs implicit
Nicholas Ormrod [Thu, 30 Oct 2014 21:15:02 +0000 (14:15 -0700)]
operator::* needs implicit

Summary:
operator::* needs to be explicit, or explicitly labeled
implicit. The build breaks with explicit, so label.

Test Plan: run unit tests

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644176

Tasks: 5486739

Signature: t1:1644176:1414693471:e3a08e27ba27a5ae0e4b9a5e009acf0a16728c45

10 years agoMake dynamic noexcept
Nicholas Ormrod [Thu, 30 Oct 2014 20:55:13 +0000 (13:55 -0700)]
Make dynamic noexcept

Summary:
Many dynamic operations, including its move constructor, are
noexcept but not labeled so. Labeled some important functions as
noexcept.

Test Plan: run unit tests

Reviewed By: delong.j@fb.com

Subscribers: tudorb, philipp, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644380

Tasks: 5486739

Signature: t1:1644380:1414618757:fb910d8a3fe3e634da4215c432577edf7371be61

10 years agoFix -Wsign-compare
Daniel Sommermann [Fri, 17 Oct 2014 20:38:00 +0000 (13:38 -0700)]
Fix -Wsign-compare

Summary:
Fixed unsigned/signed compares.

I had to suppress the warning in one place, but don't worry, clang
supports `#pragma GCC diagnostic ...` (see http://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas)

Test Plan: unit tests

Reviewed By: afrind@fb.com

Subscribers: alandau, mshneer, trunkagent, doug, jdperlow, bmatheny, njormrod, cgheorghe, folly-diffs@

FB internal diff: D1624249

Tasks: 5140804

Signature: t1:1624249:1414620209:5399e8d90d8ca32b30794a7b2a4a7c2d7d437dda

10 years agoOverride #define of keyword
Nicholas Ormrod [Thu, 30 Oct 2014 18:38:36 +0000 (11:38 -0700)]
Override #define of keyword

Summary:
Add a hidden option to suppress ###define [keyword]##, which is
occasionally appropriate.

Test Plan:
Build flint, run it on FBString.h, see no error.
Ran flint tests locally, see no errors.

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: sdwilsh, louisk, njormrod, folly-diffs@

FB internal diff: D1644182

Tasks: 5486739

Signature: t1:1644182:1414616294:4b6004d358cba865455366f5a943644b3542ae51

10 years agoOverride for include-guard
Nicholas Ormrod [Thu, 30 Oct 2014 18:26:45 +0000 (11:26 -0700)]
Override for include-guard

Summary:
In certain circumstances, it is appropriate to omit and include
guard. Add an option to allow this to be overriden.

Test Plan: Build flint, run it on folly, see no include-guard errors.

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: sdwilsh, louisk, njormrod, folly-diffs@

FB internal diff: D1644170

Tasks: 5486739

Signature: t1:1644170:1414616595:7ac52f474c1312a0c28e89255b1151d56c680acf

10 years agoAllow catch-int to be nolinted
Nicholas Ormrod [Thu, 30 Oct 2014 18:16:39 +0000 (11:16 -0700)]
Allow catch-int to be nolinted

Summary:
Add a hidden nolint option to suppress this lint warning. It is
appropriate in this case.

Test Plan:
Build flint, run it on ExceptionWrapperTest.cpp, see no catch
lint failure

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: sdwilsh, louisk, njormrod, folly-diffs@

FB internal diff: D1644165

Tasks: 5486739

Signature: t1:1644165:1414616664:a6b9dc34660df84b33ed8faaf48ec048a02bad01

10 years agoMove constructors should be noexcept
Nicholas Ormrod [Thu, 30 Oct 2014 17:18:27 +0000 (10:18 -0700)]
Move constructors should be noexcept

Summary:
The compiler is able to make some nice optimizations when it
knows that move constructors are noexcept. (In particular, it helps a
lot inside of std::vector if you don't need to worry about the
possibility of exception during reallocation.) Some move constructors in
folly are obviously moveable: change them.

Test Plan: unit tests

Reviewed By: delong.j@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1644296

Tasks: 5486739

Signature: t1:1644296:1414618605:d9a0db5193c82650b96e9c62b019d5da218b15c5

10 years agoFuture<T>::then([](T&&)), aka next()
James Sedgwick [Wed, 29 Oct 2014 23:33:14 +0000 (16:33 -0700)]
Future<T>::then([](T&&)), aka next()

Summary:
variants of then() that bypass Try and forward any exceptions up to the next future. like next() from http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3865.pdf

this could go a long way towards reducing spurious rethrows wherever people don't ever actually catch exceptions in their continuations, which is probably often enough.

anyone know if there's something in folly or standard library that does what my silly VoidWrapper struct does?

there's a lot of copypasta here but i'm not sure consolidating into helpers would actually be useful considering the amount of template crap, i don't feel strongly about it though.

Test Plan: added unit

Reviewed By: hans@fb.com

Subscribers: trunkagent, sammerat, fugalh, njormrod, folly-diffs@, bmatheny

FB internal diff: D1641776

Signature: t1:1641776:1414610434:c742563b8061a748fca9292fc2765081edcf9d52

10 years agoBump version to 14:0
dcsommer [Wed, 29 Oct 2014 23:08:59 +0000 (16:08 -0700)]
Bump version to 14:0

10 years agoReally fix the clang warning in Format-inl.h
Sean Cannella [Wed, 29 Oct 2014 21:02:55 +0000 (14:02 -0700)]
Really fix the clang warning in Format-inl.h

Summary:
The previous diff added the safety check but didn't actually
remove the compiler warning. This is what I get for doing an incremental
compile.

Test Plan: fbmake clean ; fbmake dbg with clang:dev

Reviewed By: meyering@fb.com

Subscribers: trunkagent, mathieubaudet, njormrod, folly-diffs@, bmatheny, ranjeeth, subodh, kmdent, fma, benyluo

FB internal diff: D1646519

Signature: t1:1646519:1414602292:cfb908ae094caaef02e64eb2c42544fd34fa1389