Tom Jackson [Wed, 4 Feb 2015 20:51:11 +0000 (12:51 -0800)]
Fixing find_first_of O(n) case
Summary:
The `memchr()`-based `find_first_of()` behaves extremely badly when it's used in a loop and the input string doesn't contain all the needles. This was discovered when a reasonable line-breaking routine tried to use it to break lines by a mix of '\r' and '\n'. The entire remainder of the file was scanned each time a line was read.
Before:
```
CountDelimsBase 1.26s 794.86m
CountDelimsNoSSE 100.03% 1.26s 795.12m
CountDelimsStd 40501.63% 3.11ms 321.93
CountDelimsMemchr 98.31% 1.28s 781.41m
CountDelimsByteSet 23162.41% 5.43ms 184.11
```
After:
```
CountDelimsBase 3.20ms 312.08 <-- Base impl no longer considers memchr
CountDelimsNoSSE 102.37% 3.13ms 319.47
CountDelimsStd 103.19% 3.11ms 322.05
CountDelimsMemchr 0.25% 1.27s 788.39m
CountDelimsByteSet 59.68% 5.37ms 186.27
```
Test Plan: Benchmarks
Reviewed By: njormrod@fb.com
Subscribers: folly-diffs@, yfeldblum
FB internal diff:
D1823536
Signature: t1:
1823536:
1423081687:
bb2ec8cdea477ee9b9c8d8ad2bfdecdc52657515
Sara Golemon [Wed, 11 Feb 2015 01:54:30 +0000 (17:54 -0800)]
Revert "fix configure script to check pthread_atfork(3) in the correct way"
This reverts commit
462e13b135cfa772403fddc16105bf23ea78105a.
Marcin [Mon, 9 Feb 2015 20:02:49 +0000 (20:02 +0000)]
Merge pull request #125 from 0mok/fix-configure-pthread-atfork
fix configure script to check pthread_atfork(3) in the correct way
Brad Kim [Mon, 9 Feb 2015 05:04:00 +0000 (14:04 +0900)]
Merge commit '
64f2f2734ad853784bdd260bcf31e065c47c0741' into fix-configure-pthread-atfork
Andrew Cox [Wed, 4 Feb 2015 21:04:02 +0000 (13:04 -0800)]
Bump version to 24:0
Andrii Grynenko [Wed, 4 Feb 2015 20:01:39 +0000 (12:01 -0800)]
Fix folly::Singleton error message
Test Plan: fbmake runtests
Reviewed By: chip@fb.com, pavlo@fb.com
Subscribers: folly-diffs@, yfeldblum
FB internal diff:
D1825369
Signature: t1:
1825369:
1423080185:
8507643a3f8860bae6a2240e0ae28fbaf2885b12
Dave Watson [Wed, 4 Feb 2015 18:52:49 +0000 (10:52 -0800)]
Future constructor
Summary:
As a replacement for makeFuture(). The main advantage is for non-void futures, implicit conversion works pretty well.
See unittest for examples
Test Plan: fbconfig -r folly/futures; fbmake runtests
Reviewed By: hannesr@fb.com
Subscribers: yfeldblum, trunkagent, doug, folly-diffs@, jsedgwick
FB internal diff:
D1806575
Signature: t1:
1806575:
1422465608:
6099f791591b70ce1bcda439b49307b8f3187d89
Tom Jackson [Wed, 4 Feb 2015 18:52:34 +0000 (10:52 -0800)]
Adding demonstrative test of O(n^2) string split case
Summary:
Even though the benchmark only keeps the first 10 lines, it gets slower the bigger the "file" is.
```
folly/gen/test/StringBenchmark.cpp relative time/iter iters/s
----------------------------------------------------------------------------
Lines_Gen(1e3) 5.53us 180.94K
Lines_Gen(2e3) 66.43% 8.32us 120.21K
Lines_Gen(3e3) 48.26% 11.45us 87.33K
```
Test Plan: Benchmarks
Reviewed By: lesha@fb.com
Subscribers: folly-diffs@, yfeldblum
FB internal diff:
D1823545
Tasks:
6155600
Signature: t1:
1823545:
1423023712:
86fdb3dadbec44195e4b3596cf793cea80ae3a76
Philip Pronin [Tue, 3 Feb 2015 07:53:46 +0000 (23:53 -0800)]
fix AsyncServerSocket::bind issue
Summary:
When closing sockets on retry, we should use `shutdownSocketSet_`
if it is set (so socket will properly be removed from the set).
Also I changed all `::close()` and `::shutdown()` calls to `*NoInt()`
version.
Test Plan:
fbconfig unicorn/test:basic_compression_test && fbmake opt -j32
while _bin/unicorn/test/test:basic_compression_test; do echo DONE; done
Reviewed By: mcduff@fb.com
Subscribers: trunkagent, folly-diffs@, yzhan, yfeldblum
FB internal diff:
D1821104
Tasks:
4752579,
6123510
Signature: t1:
1821104:
1423001050:
5bc09ffdd6666c2884ea82dbd70831a56513cfc9
Blame Revision:
D1795120
Tianjiao Yin [Tue, 3 Feb 2015 22:03:18 +0000 (14:03 -0800)]
print better error message
Summary:
before:
E0203 12:32:34.548096
3354863 [admonitor-local] Singleton.cpp:72] Singleton of type N8facebook12configerator22ConfigeratorStaticDataE has a living reference at destroyInstances time; beware! Raw pointer is 0x7fd6ccc81000. It is very likely that some other singleton is holding a shared_ptr to it. Make dependencies between these singletons are properly defined.
after:
E0203 13:48:09.013022
3913115 Singleton.cpp:72] Singleton of type facebook::configerator::ConfigeratorStaticData has a living reference at destroyInstances time; beware! Raw pointer is 0x7f6f7dc4c000. It is very likely that some other singleton is holding a shared_ptr to it. Make dependencies between these singletons are properly defined.
Test Plan: run it
Reviewed By: chip@fb.com
Subscribers: folly-diffs@, yfeldblum
FB internal diff:
D1822466
Signature: t1:
1822466:
1423000686:
345f40fa706701476256a7157468521bc69166a0
Dave Watson [Mon, 2 Feb 2015 22:10:44 +0000 (14:10 -0800)]
kill asyncsslserversocket
Summary: This was only used in a unittest, every real use case uses AsyncServerSocket, and decides on its own to do SSL or not.
Test Plan: fbconfig -r thrift/lib/cpp/test; fbmake runtests
Reviewed By: alandau@fb.com
Subscribers: fugalh, trunkagent, doug, alandau, bmatheny, ssl-diffs@, mshneer, jsedgwick, folly-diffs@
FB internal diff:
D1806807
Signature: t1:
1806807:
1422396209:
02333c736e1ef10963e3fe0b4ed6ecf6122475bb
Dave Watson [Tue, 27 Jan 2015 21:05:18 +0000 (13:05 -0800)]
Remove request context enable flag
Summary:
Originally meant as a kill switch. It's turned on in everything except adfinder, so we can probably remove this flag after running a couple adfinder canaries
I have no idea who should review this in ads team, guessed and added some people
Test Plan: https://our.intern.facebook.com/intern/tupperware/canary/view/?experiment_id=32937
Reviewed By: haijunz@fb.com
Subscribers: trunkagent, doug, ruibalp, nli, alandau, bmatheny, wormhole-diffs@, mshneer, folly-diffs@
FB internal diff:
D1809255
Tasks:
5645329
Signature: t1:
1809255:
1422490068:
5b4551e6c126d86cac8f8e7b06da6474e6b8a25a
Andrii Grynenko [Wed, 28 Jan 2015 04:09:27 +0000 (20:09 -0800)]
Wait for some time if Singleton isn't destroyed immediately
Summary: This diff introduces 5 seconds wait time to let other threads release Singleton which may be temporarily locked. This helps prevent most of "Singleton object alive after destruction" warnings in cases where weak_ptr API is used correctly. Abusive use of folly::Singletons, where dependencies between singletons are not properly defined will still cause a warning.
Test Plan: unit test
Reviewed By: chip@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1808371
Signature: t1:
1808371:
1422487261:
573eb40b6a260e428d96be476659335250c7ea76
Haijun Zhu [Mon, 2 Feb 2015 19:41:38 +0000 (11:41 -0800)]
Set CLOEXEC for connected fd
Summary:
AsyncSocket sets CLOEXEC if it connects an address, but won't
if it uses a connected fd. This sets CLOEXEC for such fd.
Test Plan: fbconfig folly/io/async/test && fbmake runtests
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, folly-diffs@, zacm
FB internal diff:
D1809300
Signature: t1:
1809300:
1422900997:
40fcc84506582ac67076fd975a36d094522c35c2
Yedidya Feldblum [Mon, 2 Feb 2015 19:37:23 +0000 (11:37 -0800)]
EventBase::runInEventLoopThreadAndWait.
Summary:
[Folly] EventBase::runInEventLoopThreadAndWait.
Useful for when some code needs to be run in the event loop thread, but another thread needs to trigger the code and then wait for it to be done.
Test Plan:
Unit tests:
* `folly/io/async/test/EventBaseTest.cpp`
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, folly-diffs@, brettp, dougw
FB internal diff:
D1810764
Signature: t1:
1810764:
1422900654:
7ff0aa7feb2792266f620b344cf8a1110a09f7ef
Brad Kim [Wed, 4 Feb 2015 15:27:55 +0000 (00:27 +0900)]
fix configure script to check pthread_atfork(3) in the correct way
woo [Mon, 2 Feb 2015 21:15:32 +0000 (13:15 -0800)]
Bump version to 23:0
Marcin Pawlowski [Wed, 28 Jan 2015 00:01:46 +0000 (16:01 -0800)]
fixing readme and configure script links to double-conv
Summary:
Fixing following the issues raised by nsuke on github
following his diff (https://github.com/facebook/folly/pull/112)
Test Plan: visual inspection, tested build on arch linux
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1807378
Tasks:
5909189
Signature: t1:
1807378:
1422403093:
ca024f396a27ecd9b304c57643f0107764a38ca1
Nathan Bronson [Mon, 2 Feb 2015 16:46:18 +0000 (08:46 -0800)]
add some missing methods to DeterministicAtomic
Summary:
fetch_and and fetch_or are actually required in some instances,
since the operation they perform isn't reversible. This also adds some
other missing functions, ^= and fetch_{add,sub,xor}.
Test Plan: use from other code's unit tests
Reviewed By: mssarang@fb.com
Subscribers: yfeldblum, folly-diffs@
FB internal diff:
D1798924
Signature: t1:
1798924:
1422892374:
a9859039075ddcac54f75259b9bc6a29bb963a09
Chip Turner [Sat, 31 Jan 2015 03:51:20 +0000 (19:51 -0800)]
Revert "[folly::gen] Making 'just()' reference arguments like 'from()'"
Summary: This reverts commit
90da92b05762f8032560d0d6a66237ab2772d7f2.
Test Plan: compiles
Reviewed By: tjackson@fb.com
Subscribers: lins, anca, folly-diffs@, yfeldblum
FB internal diff:
D1817376
Tasks:
6118752
Hans Fugal [Fri, 30 Jan 2015 23:23:24 +0000 (15:23 -0800)]
Rework the Future::Core state machine
Summary:
There was a race reading `callback_` in `maybeCallback` and setting `callback_` in `setCallback`. This diff reworks the state machine to make this unpossible. To avoid the explosion of states due to the cross-product of has-interrupt-handler/has-been-interrupted/etc. I introduce a separate lock for setting interrupt handler and interruption, since this is primarily orthogonal. Other attributes (active, for example) are still atomic variables, and while somewhat tied into the state machine logically (e.g. transitioning from Armed to Done only happens when active) they are mostly independent, keeping the state machine simple (and probably faster).
I think it may even be possible to do some things cheaper. In some states, we may not need to protect the writing of `callback_` and `result_`. But we'd need to enforce some ordering so I'm not going to try to tackle that. But that could be some speedup if we can do it cheaply.
Test Plan:
Builds and all existing tests pass.
Reviewed By: rockyliu4@fb.com
Subscribers: yfeldblum, stepan, trunkagent, exa, folly-diffs@, jsedgwick
FB internal diff:
D1807854
Tasks:
6087856
Signature: t1:
1807854:
1422656713:
25b62706cd7952b2dde06dab08074f8030db456b
Nathan Bronson [Thu, 29 Jan 2015 20:02:45 +0000 (12:02 -0800)]
cautionary comment in futexWake
Summary:
Add a description of the fault-after-unlock problem that can
occur when a synchronization object is used to guard its own
destruction.
Test Plan: comment change only
Reviewed By: alikhtarov@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1808446
Signature: t1:
1808446:
1422481173:
83c0369e2277cd5b8cf6377fcd92b132019611a7
Tom Jackson [Thu, 29 Jan 2015 18:56:30 +0000 (10:56 -0800)]
Making 'just()' reference arguments like 'from()'
Summary: That is, for all inputs except r-values.
Test Plan:
Run tests
@override-test-failures
Reviewed By: ajaymenon@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1807789
Signature: t1:
1807789:
1422410689:
442d50ab17f8e3b08e34f33318a4dc7f23b4c6cc
Shijin Kong [Thu, 29 Jan 2015 18:54:53 +0000 (10:54 -0800)]
enable ssl false start with Next Protocol Negotiation (NPN) extension
Summary:
This speeds up TLS handshake and might be a factor of liger perf regression.
The enabling is guarded by an #ifdef. The condition itself is defined in an
openssl patch.
Test Plan: folly unit tests pass. Tried on devices as well and from tcpdump data was sent before the new ticket was received, essentially speeded up the handshake process.
Reviewed By: subodh@fb.com
Subscribers: trunkagent, kmdent, seanc, benyluo, ssl-diffs@, ranjeeth, folly-diffs@
FB internal diff:
D1806856
Tasks:
5284979
Signature: t1:
1806856:
1422494521:
0a048ea9001da13b5d698b5a764d1e66dcbedc99
Tom Jackson [Thu, 29 Jan 2015 18:22:59 +0000 (10:22 -0800)]
indirect(), for making pointers out of refs
Test Plan: Unit tests
Reviewed By: ajaymenon@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1808023
Signature: t1:
1808023:
1422410709:
8db2d73d5b4c0c2eab563643e5fa1557ebfd4730
Sean Cannella [Wed, 28 Jan 2015 23:28:58 +0000 (15:28 -0800)]
Cleanup duplicate include
Summary:
Included climits and limits.h due to non-conflicting changes,
clean this up.
Test Plan: compiled
Reviewed By: kelarini@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1810133
Signature: t1:
1810133:
1422483209:
0826d2198769efed86607de6c5b28ce0bc4ac04b
James Sedgwick [Wed, 28 Jan 2015 21:55:45 +0000 (13:55 -0800)]
fix deprecated Promise::setException()
Summary: should have been using fulfilTry anyways. and fulfilTry needs to take the Try by value so fix that too
Test Plan: unit
Reviewed By: hans@fb.com
Subscribers: fugalh, folly-diffs@, jsedgwick
FB internal diff:
D1808923
Tasks:
6098987
Signature: t1:
1808923:
1422478981:
d4b9727394339c996ebccb7841b94e0c7b2bffb4
Kyle Dent [Wed, 28 Jan 2015 21:47:43 +0000 (13:47 -0800)]
Fixing break in multifeed. CHAR_BIT was previously unfound
Summary: Now CHAR_BIT will be found
Test Plan: ran multifeed and folly
Reviewed By: seanc@fb.com
Subscribers: seanc, folly-diffs@
FB internal diff:
D1810027
Signature: t1:
1810027:
1422480946:
4568c5190e318ff097d33b3d0434593b72f8205e
Khalid El-Arini [Wed, 28 Jan 2015 21:47:44 +0000 (13:47 -0800)]
fix missing header in Range.h
Summary: ^
Test Plan: fbmake
Reviewed By: woo@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1810061
Signature: t1:
1810061:
1422481560:
089c9186834f2bf87ac6d10b71d53784299b7992
Blame Revision:
D1806632
Viswanath Sivakumar [Wed, 28 Jan 2015 20:08:36 +0000 (12:08 -0800)]
Convert TransportInfo SSL fields to shared_ptrs
Summary:
We do a lot of copying of TransportInfo in proxygen, and in most cases
the SSL structs don't change after connection establishment. We could
cut down on memory usage by sharing these huge strings. This is
especially true with SPDY where all streams belonging to a session could
share these fields.
Facebook:
Test Plan: Unit tests, will canary
Reviewed By: afrind@fb.com
Subscribers: fugalh, bmatheny, ssl-diffs@, folly-diffs@, jsedgwick, woo
FB internal diff:
D1807557
Tasks:
5343753
Signature: t1:
1807557:
1422472932:
53038345fca620632097586fb9e410bca8fe748d
Kyle Dent [Wed, 28 Jan 2015 19:33:49 +0000 (11:33 -0800)]
Fixing a -Wshorten-64-32 issues in folly for liger.
Summary: Normally I would use the folly::to<> function, but it would cause a circular dependency, so I just added a static cast
Test Plan: Ran on iOS and ran the folly tests
Reviewed By: seanc@fb.com
Subscribers: lucian, mpawlowski, marcelo, tudorb, aalexandre, seanc, folly-diffs@
FB internal diff:
D1806632
Signature: t1:
1806632:
1422416646:
b8104f18f90eb7457f2f358428f2bd800f8f1db5
James Sedgwick [Wed, 28 Jan 2015 19:01:44 +0000 (11:01 -0800)]
kill a couple unnecessary rethrows
Summary: as above, there were less of these than I expected, nice
Test Plan: unit
Reviewed By: hans@fb.com
Subscribers: trunkagent, folly-diffs@, jsedgwick
FB internal diff:
D1789332
Tasks:
5949939
Signature: t1:
1789332:
1421878033:
d7c2979a77b51a5257b8bcd910ad9296ca1aa7e0
Sean Cannella [Wed, 28 Jan 2015 16:47:51 +0000 (08:47 -0800)]
Fix Conv.h compilation on Android
Summary:
std::to_string doesn't exist on Android so don't use it.
Facebook: Did a sync to fbandroid and confirmed liger compiles with this fix.
Test Plan: existing tests
Reviewed By: ranjeeth@fb.com
Subscribers: trunkagent, folly-diffs@, shikong, kmdent, fma, pgriess
FB internal diff:
D1808037
Signature: t1:
1808037:
1422410556:
d78e0633a1554254b1a1f25bef49a4550a1817c6
James Sedgwick [Wed, 28 Jan 2015 16:05:30 +0000 (08:05 -0800)]
Another stab at waitWithSemaphore -> Future<T>::wait()
Summary:
See
D1785572. Check out the new Future test and the commented portion of wait(Duration) for the fix
The test only fails a few times out of a hundred before the fix, but hasn't failed yet after
Test Plan: futures unit, wait for contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, rushix, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick
FB internal diff:
D1803526
Tasks:
5940008,
6059995
Signature: t1:
1803526:
1422309486:
3613c59a708ecac312d241723828763feb2a57aa
Andrei Alexandrescu [Wed, 28 Jan 2015 00:21:22 +0000 (16:21 -0800)]
Add writeFile function to folly
Summary: Reciprocal of readFile
Test Plan: fbmake runtests
Reviewed By: tudorb@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1807551
Signature: t1:
1807551:
1422410565:
f5bda294deb788da9f3881c19bb20cfa1f588c09
Marcin Pawlowski [Wed, 28 Jan 2015 00:01:46 +0000 (16:01 -0800)]
fixing readme and configure script links to double-conv
Summary:
Fixing following the issues raised by nsuke on github
following his diff (https://github.com/facebook/folly/pull/112)
Test Plan: visual inspection, tested build on arch linux
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1807378
Tasks:
5909189
Signature: t1:
1807378:
1422403093:
ca024f396a27ecd9b304c57643f0107764a38ca1
Claudiu Gheorghe [Mon, 26 Jan 2015 22:59:46 +0000 (14:59 -0800)]
make getFD() a const method on AsyncUDPServerSocket
Summary: no need to be mutable, since it only calls AsyncUDPSocket::getFD() which is a const method
Test Plan: just compiled -- simple change
Reviewed By: afrind@fb.com
Subscribers: trunkagent, doug, folly-diffs@
FB internal diff:
D1802765
Signature: t1:
1802765:
1422307972:
eab6ab4b16407df6e5034cef78c79f45cd9fa46f
Marcin Pawlowski [Mon, 26 Jan 2015 21:57:21 +0000 (13:57 -0800)]
fix folly build under python 3
Summary:
Make python scripts compatible woth both python 3 and python 2.
Added ignore rule for test/glog directory.
Facebook:
While verifying some issues reported on github i noticed that
folly does not build when python 3 is the default (at least on arch
linux). As it turns out it is fairly easy to make our python files
compatible with both python 2 and python 3.
I also added test/glog*/* to gitignore so that it is easier to work
on folly outside of facebook.
Test Plan:
unit tests in fb envirnoment and unit test on arch linux
(python 3.4.2, gcc 4.9.2)
Reviewed By: njormrod@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1801368
Signature: t1:
1801368:
1422294777:
4e7b6b5634e8dccd849194e171c9a342bd1cb8b8
Lucian Grijincu [Mon, 26 Jan 2015 21:43:36 +0000 (13:43 -0800)]
folly: to: make exceptions more informative
Summary: print the value which failed conversion to aid debugging.
Test Plan: run code which throws in to<> and contbuild
Reviewed By: tudorb@fb.com, andrei.alexandrescu@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1786294
Signature: t1:
1786294:
1421795912:
00cc10923990e5aac0a15eedec09deb8e8d470d1
Hannes Roth [Mon, 26 Jan 2015 20:53:42 +0000 (12:53 -0800)]
(Wangle) Then Magic
Summary: All is good now.
Test Plan: Run all the tests.
Reviewed By: hans@fb.com
Subscribers: jsedgwick, trunkagent, fugalh, folly-diffs@
FB internal diff:
D1758272
Signature: t1:
1758272:
1421889405:
a015d30783715e106a1e6667e971f7cf47c8392d
Hannes Roth [Mon, 26 Jan 2015 17:08:49 +0000 (09:08 -0800)]
(Wangle) Don't add an extra value() call for makeFuture(Try)
Summary: Not sure why we'd need that? This avoids a move, I think.
Test Plan:
Run all the tests. Also unbreaks this code, which used to work at some
point.
https://phabricator.fb.com/diffusion/FBCODE/browse/master/taoThriftService/TaoServiceHandler.cpp;
a27bc2ef36cd671d65ae0fd2cc1fb1f823e68ae4$246
Reviewed By: hans@fb.com
Subscribers: trunkagent, folly-diffs@, jsedgwick
FB internal diff:
D1800619
Signature: t1:
1800619:
1422053903:
f9e65a0be3d820a9a9989b3cf5dfaf2a61e2e900
Mark McDuff [Sat, 24 Jan 2015 09:23:20 +0000 (01:23 -0800)]
fix memory leak in AsyncServerSocket
Test Plan: g-unittest
Reviewed By: philipp@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1801665
Signature: t1:
1801665:
1422093104:
eecf86ea8c1ab39dcb3d01442ed66ae45ef3bede
Mark McDuff [Thu, 22 Jan 2015 00:18:25 +0000 (16:18 -0800)]
make AsyncServerSocket bind to same port on ipv4 and ipv6 with port=0
Summary: I'm in unfamiliar territory, so shout if I'm doing something dumb. Perhaps it's a bad assumption that if the ipv4 port is free that the ipv6 port is also free?
Test Plan: g-unittest
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, ps, bmatheny, folly-diffs@
FB internal diff:
D1795120
Signature: t1:
1795120:
1422034693:
bd315023ab6cd9e9bda12161d05dd781dc401546
Woo Xie [Fri, 23 Jan 2015 18:21:38 +0000 (10:21 -0800)]
specify connection's idle tiemout
Summary: Now all connections are always scheduled with connectionManager's default timeout. This diff enables us to specify the different timeouts for each managed connections.
Test Plan: prospect testing
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick
FB internal diff:
D1797193
Tasks:
5343753
Signature: t1:
1797193:
1422034092:
ed67b96efe8af8f8d1355d3f86fb1289daafb178
Dave Watson [Fri, 23 Jan 2015 17:50:35 +0000 (09:50 -0800)]
Finagle interfaces
Summary: Future service interfaces similar to finagle. Service creators for client, filters
Test Plan: Unittests included - also sets up a simple pipeline to test a full stack client/server.
Reviewed By: hans@fb.com
Subscribers: jsedgwick, trunkagent, njormrod, folly-diffs@, doug, fugalh
FB internal diff:
D1573086
Tasks:
5002456
Signature: t1:
1573086:
1421970698:
328453c4a980bb6950fc9aeed6a2b6d9819c20db
Dave Watson [Thu, 22 Jan 2015 20:32:26 +0000 (12:32 -0800)]
Revert "[futures] waitWithSemaphore -> Future<T>::wait()"
Summary:
Subject:
This reverts commit
d8eeb9a12f932b74d268c74ce7ce2610dc14b152.
Test Plan: Watch contbuild?
Reviewed By: njormrod@fb.com
Subscribers: trunkagent, doug, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick
FB internal diff:
D1797163
Tasks:
5940008
Signature: t1:
1797163:
1421959040:
c9d528367fed2de2f7dafdac9416aa01bdb47df9
Andrii Grynenko [Thu, 22 Jan 2015 19:07:08 +0000 (11:07 -0800)]
Fix ASAN failures in OBCClient
Summary:
This also adds a dependency from all folly::Singletons on RequestContext. Better fix would probaly be to switch RequestContext to be managed by folly::Singleton, however that would require us to switch all singletons which use it to be folly::Singletons too.
Facebook:
Making sure we initialize SR singletons when OBCClient singleton is created, so that SR is destoyed after OBCClient.
Test Plan:
fbconfig -r --sanitize=address --platform-all=gcc-4.8.1-glibc-2.17 dragon/
fbmake runtests
Reviewed By: meyering@fb.com, davejwatson@fb.com
Subscribers: trunkagent, folly-diffs@, meyering
FB internal diff:
D1796847
Tasks:
5986816
Signature: t1:
1796847:
1421954531:
e9ec177d5289a33c12fe0947426d89963c8e262c
Adam Simpkins [Fri, 16 Jan 2015 02:35:56 +0000 (18:35 -0800)]
Add additional instantiations in Instantiations.cpp
Summary:
Histogram::getPercentileBucketIdx() and getPercentileEstimate() are
implemented using template methods of HistogramBuckets.
This updates Instantiations.cpp to also instantiate the versions of
these template methods that are required by Histogram<int64_t>. Without
this, anyone using these methods was required to also include
Histogram-defs.h, or otherwise they may get link errors. This makes it
so they no longer need to include Histogram-defs.h
The C++ syntax required is pretty horrendous. We could potentially
avoid this by refactoring the code so that these methods are no longer
templates (and require that the Bucket class provide methods for this
purpose). At the moment adding instantiations seems simpler, and
doesn't change any public APIs.
Test Plan:
Successfully linked code using
Histogram<int64_t>::getPercentileEstimate() without including
Histogram-defs.h
Reviewed By: ldbrandy@fb.com, meyering@fb.com
Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@
FB internal diff:
D1786346
Signature: t1:
1786346:
1421429734:
da17b7ea326c64a1e158fec3b87dc49c3db3d848
Adam Simpkins [Thu, 15 Jan 2015 00:00:31 +0000 (16:00 -0800)]
add IOBuf::appendToIov()
Summary:
This splits out some code from the existing getIov() method, to support
appending to an existing iovec array, rather than always returning a new
one.
I have a use case where I have multiple IOBuf chains to write at once,
preceded by some header data. appendToIov() makes it easier to populate
a single iovec array with this data, and then perform a single writev()
call.
Test Plan: Updated the unit tests to exercise appendToIov()
Reviewed By: jasmeetbagga@fb.com
Subscribers: trunkagent, doug, exa, net-systems@, folly-diffs@
FB internal diff:
D1783090
Tasks:
Signature: t1:
1783090:
1421450476:
d5c89d6e1adbd2f2239057238e020b302df09f23
Blame Revision:
James Sedgwick [Wed, 21 Jan 2015 21:39:03 +0000 (13:39 -0800)]
getVia() and waitVia()
Summary:
Introduce ProgressableExecutor, which is an Executor that can be driven somehow. Examples include EventBase and ManualExecutor
Then introduce Future<T>::getVia(ProgressableExecutor*) and Future<T>::waitVia(ProgressableExecutor*) that drive the given executor until the Future is complete, with the usual semantics of get and wait respectively
This is a really common pattern in tests and you can see in the various changes to other projects lends sopme nice redness and cleanliness
Some notes:
1. I don't like ProgressableExecutor::makeProgress() that much. Too verbose. Maybe DrivableExecutor::drive()? Something else? Thoughts?
2. I still need to integrate this with some stuff in Zeus (SessionFuture) and Zookeeper (ZookeeperFuture) but I'm going to do that in a separate diff because it's going to be a little more intrusive
3. These APIs take a raw ptr so that they are consistent with via()
4. See inline note on ManualExecutor
5. See inline note in added unit tests
Test Plan: add unit for new API, wait for contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, dresende, pzq, tdimson, fbcode-common-diffs@, targeting-diff-backend@, alandau, apollo-diffs@, bmatheny, everstore-dev@, zhuohuang, laser-diffs@, mshneer, folly-diffs@, hannesr, jsedgwick
FB internal diff:
D1789122
Tasks:
5940008
Signature: t1:
1789122:
1421868315:
6ea2fc2702be1dc283c24a46d345fb5da3935b32
James Sedgwick [Wed, 21 Jan 2015 19:41:50 +0000 (11:41 -0800)]
use wait()/get() instead of while(!f.isReady()) { ... }
Summary: as above. nice redness. couple of comments inline though
Test Plan: contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, zeus-diffs@, cold-storage-diffs@, targeting-diff-backend@, apollo-diffs@, micha, folly-diffs@, jsedgwick
FB internal diff:
D1787776
Tasks:
5940008
Signature: t1:
1787776:
1421776843:
13772348538a0af27ceb5a363b818ece0cdfffc6
James Sedgwick [Wed, 21 Jan 2015 19:35:42 +0000 (11:35 -0800)]
waitWithSemaphore -> Future<T>::wait()
Summary:
see task. also adjust all callsites to get() or wait() as appropriate. this is a lot better, especially in various tests
Facebook:
cc @rushix, @wez as this was quite nice for docstore and novak, and you might want to be aware of new conventions
Test Plan: futures unit, wait for contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick, wez, rushix
FB internal diff:
D1785572
Tasks:
5940008
Signature: t1:
1785572:
1421866794:
a879de4d0bc14e96c434f623ed2a74361e25f28c
Dave Watson [Wed, 21 Jan 2015 16:12:33 +0000 (08:12 -0800)]
codecs
Summary:
Start of codec framework. Copied the frame based codecs almost exactly from netty, but made to fit our pipeline management.
BytesToMessageCodec is slightly different: Netty preprocesses all the available data to a List<Message>, while this codec a) Only does one message at a time to avoid queueing issues, and b) doesn't template the message type, and just passes an IOBuf
I'm fighting the type system to get the pipelines to play nice to each other: I'd rather template the message type, but it prevents stacking outgoing handlers on top of it, since you ahve to specify both the input/output type for each handler, even if you only care about the output type. Suggestions to fix? Netty gets around this by lots of dynamic casting to Object type, but we can't do that in C++ since we don't have a base object type
Test Plan: Includes lots of tests
Reviewed By: jsedgwick@fb.com
Subscribers: jsedgwick, doug, fugalh, folly-diffs@
FB internal diff:
D1758189
Tasks:
5002361,
5002316
Signature: t1:
1758189:
1421170225:
6bc8cc6a0bb461a965665bc88f7009033b215da9
Bob Haarman [Tue, 20 Jan 2015 20:36:40 +0000 (12:36 -0800)]
folly: added C interface to SingletonVault
Test Plan:
fbconfig folly/experimental/test:singletonvault_c_test && fbmake runtests
buck test folly/experimental/test:singletonvault_c_test
Reviewed By: andrii@fb.com
Subscribers: trunkagent, sdwilsh, folly-diffs@
FB internal diff:
D1780054
Signature: t1:
1780054:
1421187023:
b18c1d5575b2864b3d3df001a0024b91eb321c46
Haijun Zhu [Tue, 20 Jan 2015 18:54:59 +0000 (10:54 -0800)]
Try to fix IdleTime unit test
Summary:
IdleTime unit test has been failing randomly (#
5996886). The
event base loop calculates an exponentially moving average of loop busy
time in each loop. In the test, the busy time is initialized to 5900 and
each loop runs more than 8000 us. In normal case decaying of the
previous loop busy time will take 6 loops to move this average to >6000.
But if the test is running on a heavily loaded machine the busy+idle time
a loop is longer than it should be, causing the decaying happen faster.
This diff skips this test if this happens.
Test Plan:
It won't affect anything if running on my devserver. It only
helps on a heavily loaded contbuild host.
Reviewed By: alandau@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1788450
Tasks:
5996886
Signature: t1:
1788450:
1421460774:
1ec575c50f881e10b5a0208717fe68164f0d0f57
Alexey Spiridonov [Sat, 17 Jan 2015 00:04:22 +0000 (16:04 -0800)]
Fix comment typo
Test Plan: checked this is the right function name
Reviewed By: ngbronson@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1788386
Signature: t1:
1788386:
1421453582:
82d0c4685c1bd2d888f5974fd2a998ebbc27a9b5
Nathan Bronson [Fri, 16 Jan 2015 23:44:39 +0000 (15:44 -0800)]
fix and improve MPMCQueue comment
Summary:
The big comment block at the top of MPMCQueue had a non-sense
sentence resulting from a merge conflict, and also didn't include
details about ordering or recommended patterns for handling consumer
shutdown.
Test Plan: comment change
Reviewed By: lesha@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1788230
Signature: t1:
1788230:
1421451643:
7c768c2b127499e3f2b06ee241fba851b4ed2a29
Satadru Pan [Fri, 16 Jan 2015 20:05:10 +0000 (12:05 -0800)]
apply all sockopts to listening sockets
Summary:
We want all socket options to be applied to the listen sockets too.
The old code here that only applied SO_KEEPALIVE is a bit superstitious.
Not all options will be inherited by accept()ed sockets, but that's
fine. We set them again post-accept().
Applying all these options to the listening sockets will fix at least
one long standing bug, which is that QoS is not set correctly for the
first few packets in a flow. This also will let us do the right thing
when we gain the ability to set TCP_MAXSEG in
D1741753.
Test Plan: fbmake runtests & canary in prod
Reviewed By: afrind@fb.com, davejwatson@fb.com
Subscribers: jsedgwick, satadru, trunkagent, fugalh, exa, folly-diffs@, agartrell
FB internal diff:
D1745182
Tasks:
2911597
Signature: t1:
1745182:
1418860157:
045680a91b153482bcd4a014894fb28059955d06
Jim Meyering [Sat, 10 Jan 2015 06:28:00 +0000 (22:28 -0800)]
folly: Bits: restore type of bitOffset_ to shorter "unsigned int"
Summary:
My initial change to avoid the -Wsign-compare error neglected
the fact that the difference of two unsigned types is also
unsigned. That broke some unicorn tests, as seen in
D1776343,
which worked around the problem by changing the type of
bitOffset_ like this: s/unsigned int/size_t.
The real problem lay in the distance_to function, which computed
the signed difference of two bitOffset_ values.
To work properly with the narrower unsigned type,
we could either first widen each operand to ssize_t
before computing their parenthesized difference, or
(as Philip suggested and I prefer), just drop the
parentheses and let the signedness of the preceding
operands ensure that each successive intermediate
result is also signed.
Test Plan:
Choose one of the failing tests, and ensure it now passes:
fbconfig -r unicorn/diskindex4/test:doc_map_test
fbmake runtests
Also run all unicorn tests, for good measure (still waiting):
fbconfig -r unicorn && fbmake runtests
Reviewed By: philipp@fb.com, andrei.alexandrescu@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1776624
Tasks:
5941250
Signature: t1:
1776624:
1420872874:
b1ea8a3f23f21269aa65cbe1647e86a8f622a710
Blame Revision:
D1770613
James Sedgwick [Fri, 16 Jan 2015 14:37:21 +0000 (06:37 -0800)]
remove outer Try from whenAll/whenN/whenAny callbacks
Summary:
it's redundant and we shouldn't encourage it, so i'm changing all callsites
i changed docs as well
Test Plan: wait for contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, targeting-diff-backend@, adityab, everstore-dev@, zhuohuang, darshan, micha, folly-diffs@, wch, lins, tingy, jsedgwick
FB internal diff:
D1784667
Tasks:
5936469
Signature: t1:
1784667:
1421364620:
83169739320e5342d28a744e3689794f16108fea
James Sedgwick [Fri, 16 Jan 2015 14:33:13 +0000 (06:33 -0800)]
kill remaining Futures-related Wangle references
Summary: grep around for wangle, adjust where appropriate
Test Plan: contbuild
Reviewed By: hans@fb.com
Subscribers: trunkagent, fbcode-common-diffs@, ruibalp, zeus-diffs@, vikas, targeting-diff-backend@, alandau, mwa, jgehring, zhguo, jying, darshan, fuegen, mshneer, folly-diffs@, lins, maxwellsayles, jsedgwick
FB internal diff:
D1777581
Tasks:
5960242
Signature: t1:
1777581:
1421361395:
2ee8bfeca76dafe7376a217c66e0a4eaf3ef416a
Jim Meyering [Thu, 15 Jan 2015 03:54:27 +0000 (19:54 -0800)]
MPMCQueue.h: fix a bug introduced while accommodating -Wsign-compare
Summary:
* folly/MPMCQueue.h (waitForTurn): My changes to accommodate
-Wsign-compare introduced a bug that could cause lots of wasted CPU.
(See @scottf's comments on the blamed patch.)
Subtraction involving an unsigned type will always yield an unsigned
result, and there was at least this one instance where the result
would sometimes have to be a small negative number. Instead, we'd
get approximately UINT32_MAX. The fix is to explicitly cast the
difference to a signed type.
Test Plan:
Run this and note there is no error:
(this ensures it remains -Wsign-compare-clean with gcc-4.9)
fbconfig --platform-all=gcc-4.9-glibc-2.20 tao/server && fbmake dbgo
Before this change, running the mpmc_queue_test test would get
timeout errors ~8 in 100 runs. Confirm that with this patch, now
there is no type of failure. I.e., run these commands and ensure
the grep prints nothing:
fbconfig folly/test:mpmc_queue_test
for i in $(seq 100); do fbmake runtests_dbgo >& makerr-$i; done
grep -E '(TIMEOUT|FATAL|FAIL).* [1-9]' makerr-*
Reviewed By: hans@fb.com
Subscribers: trunkagent, scottf, folly-diffs@
FB internal diff:
D1783672
Tasks:
5941250
Signature: t1:
1783672:
1421350260:
d7d306e8cf3ed3ee636d06228e1f24a2c2796b54
Blame Revision:
D1766722
Dave Watson [Thu, 15 Jan 2015 21:25:02 +0000 (13:25 -0800)]
'fix' threadlocal fork test
Summary:
configure seems to think ubuntu 14.04 doesn't have pthread_atfork, which seems wrong?! But this test seems to fail if we don't have it on some platform anyway. Not sure the right fix, just disable it for now?
Test Plan: watch https://ci-builds.fb.com/job/folly/
Reviewed By: hans@fb.com
Subscribers: doug, folly-diffs@
FB internal diff:
D1783046
Signature: t1:
1783046:
1421348575:
d279877abcc597a04c285092a5e5e53406922d5a
Blame Revision:
D1768669
Ranjeeth Dasineni [Thu, 15 Jan 2015 01:06:40 +0000 (17:06 -0800)]
fix the manual config and flipped condition in SpinLock.h
Summary:
i) FOLLY_HAVE_INT128_T definition is duplicated and is getting set to 1 on android. deleted that.
ii) we dont have pthread_spinlock_t and pthread_atfork on android, the former is absent apple as well
iii) a condition got flipped in
D1768669. __ANDROID__ implies !FOLLY_HAVE_PTHREAD_SPINLOCK_T. fixed that
Test Plan:
- fbconfig -r folly && fbmake
- ran folly build with buck as well
- synced to fb4a and it builds
Reviewed By: seanc@fb.com
Subscribers: doug, seanc, folly-diffs@
FB internal diff:
D1783579
Signature: t1:
1783579:
1421350138:
9301303f77784e7bdb851e3f643751078457b025
James Sedgwick [Wed, 14 Jan 2015 15:08:50 +0000 (07:08 -0800)]
global executors with weak_ptr semantics
Summary:
unfortunately, can't use atomics now that a weak_ptr is stored instead of a raw ptr, hence the additional singleton locks
might want to make an overload of via() that takes shared_ptr to integrate more easily? or change via() to store a shared_ptr (and make it non-owning if a raw ptr is passed)
Test Plan: unit
Reviewed By: davejwatson@fb.com
Subscribers: jsedgwick, trunkagent, fugalh, folly-diffs@
FB internal diff:
D1764359
Tasks:
5002442
Signature: t1:
1764359:
1420845340:
349ea88091d7ca4ee386b54aec599647341fadd4
Louis Kruger [Tue, 13 Jan 2015 20:03:43 +0000 (12:03 -0800)]
Add folly::sorted_vector_map::at()
Summary: http://www.cplusplus.com/reference/map/map/at/
Test Plan: unittest
Reviewed By: ldbrandy@fb.com
Subscribers: trunkagent, jkong, folly-diffs@
FB internal diff:
D1773996
Signature: t1:
1773996:
1421176834:
33c8b39f03bf085ca2dd45df113cba948d0f2dd0
Viswanath Sivakumar [Tue, 13 Jan 2015 19:02:44 +0000 (11:02 -0800)]
Bump version to 22:0
Hans Fugal [Tue, 6 Jan 2015 00:21:50 +0000 (16:21 -0800)]
(folly) osx bootstrap script
Summary:
This is a bootstrap script for building folly on osx. I attempted to make it Just Workâ„¢ for anyone who uses Homebrew (http://brew.sh/) but even if it doesn't work out of the box it should be instructive for the motivated developer on OSX.
Facebook:
In #
5134151 Wangle used to not build, because something was using a futex. But that seems to have been rectified (or the build problems are hiding in headers that aren't used in the libfolly build).
I'll also update https://our.intern.facebook.com/intern/wiki/Folly/BuildingOnMac to point people to this script instead.
Test Plan:
Tested on my laptop which is running Yosemite (10.10.1)
Build, then inspect `otool -L .libs/*dylib`
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@, exa
FB internal diff:
D1765248
Tasks:
5134151
Signature: t1:
1765248:
1420500905:
bc5da97a0446f502cb2badc85c9004ff0a62027a
Viswanath Sivakumar [Tue, 13 Jan 2015 18:57:03 +0000 (10:57 -0800)]
Add libgtest to Makefile to unbreak OSS build
Summary:
SpookyHashTest now needs libgtestmain to be linked
(https://phabricator.fb.com/
D1775555).
https://ci-builds.fb.com/job/folly/1570/console
Test Plan:
Don't have Ubuntu setup, but this should fix it. Will kick off jenkins
build.
Reviewed By: andrewjcg@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1779604
Signature: t1:
1779604:
1421174608:
de865529d728e943288b52c0d63ebb712eda094f
Blame Revision:
D1775555
James Sedgwick [Mon, 12 Jan 2015 14:48:32 +0000 (06:48 -0800)]
move Futures from folly::wangle to folly
Summary: Namespace change. Next up is to extract the Executors into folly/executors/
Test Plan: wait for contbuild
Reviewed By: davejwatson@fb.com
Subscribers: jsedgwick, trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, lars, davejwatson, ruibalp, hero-diffs@, zeus-diffs@, andrewcox, vikas, mcduff, cold-storage-diffs@, unicorn-diffs@, ldbrandy, targeting-diff-backend@, netego-diffs@, abirchall, fugalh, adamsyta, atlas2-eng@, chenzhimin, mpawlowski, alandau, bmatheny, adityab, everstore-dev@, zhuohuang, mwa, jgehring, prometheus-diffs@, smarlow, akr, bnitka, jcoens, benj, laser-diffs@, zhguo, jying, darshan, micha, apodsiadlo, alikhtarov, fuegen, dzhulgakov, jeremyfein, mshneer, folly-diffs@, wch, lins, tingy, hannesr, maxwellsayles
FB internal diff:
D1772779
Tasks:
5960242
Signature: t1:
1772779:
1420751149:
bc0b9220be25f1d46d9cef3fdeaa9c3681217aff
Andrew Gallagher [Sat, 10 Jan 2015 03:25:31 +0000 (19:25 -0800)]
folly: make remaining TARGETS files auto converitble to BUCK files
Summary: Also add the `@buckify` tag to keep these automatically synced.
Test Plan: built and tested with buck and fbconfig/fbmake
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, fugalh, folly-diffs@, jsedgwick
FB internal diff:
D1775555
Signature: t1:
1775555:
1420840948:
e9a2e86379be71b7d953fa53a727e51d2541b011
Lucian Grijincu [Sat, 10 Jan 2015 01:18:36 +0000 (17:18 -0800)]
folly: Bits: make bitsPerBlock 64 bit back
Summary:
D1770613 made bitsPerBlock 32 bit. Sadpanda.
Test Plan: _bin/unicorn/diskindex4/test/doc_map_test --gtest_filter=DocMap/DocMapTest.Deleted/23
Reviewed By: meyering@fb.com
Subscribers: net-systems@, folly-diffs@
FB internal diff:
D1776343
Tasks:
5968993
Signature: t1:
1776343:
1420852923:
f35166dd9023a701ff16467710131f821810215a
Blame Revision:
D1770613
Jim Meyering [Wed, 7 Jan 2015 16:16:07 +0000 (08:16 -0800)]
folly/dynamic-inl.h: avoid -Wsign-compare error
Summary:
* folly/dynamic-inl.h (format): Add an explicit
int-to-size_t cast (that is ok because we've just ensured
it is non-negative) to avoid this error from gcc-4.9:
folly/dynamic-inl.h:953:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: delong.j@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1773242
Tasks:
5941250
Signature: t1:
1773242:
1420822301:
b1e3c065cd25728c77957ce9e1f52e884ba8693b
Jim Meyering [Wed, 7 Jan 2015 21:45:19 +0000 (13:45 -0800)]
folly: ProducerConsumerQueue: avoid many -Wsign-compare errors
Summary:
There were many templates instantiated with signed types.
Changing them to unsigned avoids the compilation failures
noted below. However, we'll have to watch arc-diff-spawned
tests for new failures, just in case.
This is by far the largest undecomposable patch of those
I've written for this task so far. It is possible that
some subset of these changes accomplishes the same goal,
but once I managed to get a successful compilation, I have
only re-reviewed it once and did not try to shrink it.
Test Plan:
Without these changes, we would get the following 32 errors from gcc-4.9:
https://phabricator.fb.com/P19692642
With them, (and all of the other patches attached to the parent task,
all of fbcode compiles and passes its tests (modulo preexisting
failures and aborts).
Reviewed By: delong.j@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1773461
Tasks:
5941250
Signature: t1:
1773461:
1420821964:
61635f3a2efc95e8d4457c36e7a5ee3a0a50df23
James Sedgwick [Fri, 9 Jan 2015 16:27:02 +0000 (08:27 -0800)]
move wangle/futures to futures
Summary:
First step of moving futures out of wangle into folly
Namespace change will be another diff
This was just a straight codemod of "wangle/futures" to "futures"
Test Plan: wait for contbuild
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, lars, ruibalp, hero-diffs@, zeus-diffs@, vikas, mcduff, hitesh, cold-storage-diffs@, unicorn-diffs@, ldbrandy, targeting-diff-backend@, netego-diffs@, fugalh, adamsyta, atlas2-eng@, alandau, apollo-diffs@, bmatheny, adityab, everstore-dev@, zhuohuang, sweeney, mwa, jgehring, prometheus-diffs@, smarlow, akr, bnitka, jcoens, zhguo, jying, darshan, apodsiadlo, alikhtarov, fuegen, dzhulgakov, alihussains, jeremyfein, mshneer, folly-diffs@, wch, lins, tingy, maxwellsayles, hannesr
FB internal diff:
D1772041
Tasks:
5960242
Signature: t1:
1772041:
1420739169:
36126b1264c5d1747d241921d02b13e306c73fe1
Jim Meyering [Wed, 7 Jan 2015 03:25:48 +0000 (19:25 -0800)]
folly/Bits.h (BitIterator): avoid -Wsign-compare error
Summary:
* folly/Bits.h (BitIterator) [bitOffset_]: Change type of this
member from ssize_t to "unsigned int".
(bitsPerBlock): Change return type from size_t to "unsigned int".
These changes avoid some -Wsign-compare-highlighted issues, e.g.,
folly/Bits.h:456:18: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
I've also run "fbmake runtests", and it failed precisely the way it does
without this patch.
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1770613
Tasks:
5941250
Signature: t1:
1770613:
1420757397:
7ee7e5de500c8991af2597a6245d12d8f286b4b7
Jim Meyering [Tue, 6 Jan 2015 17:33:03 +0000 (09:33 -0800)]
folly: fix many -Wsign-compare warning/errors reported by gcc-4.9
Summary:
With the upgrade to gcc-4.9 fbcode's default is to enable
-Wsign-compare by default. That exposes nontrivial technical debt.
This is part of that clean-up. Some of these changes fix the problem
where there's an "int" for-loop index, yet an unsigned limit.
The fix is to use an unsigned type when the limit is also unsigned.
I usually choose size_t because of the general recommendation (when
writing portable code) to avoid size-tied types like uint32_t and
uint64_t unless you have a very good reason to require them.
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 tao/server && fbmake dbgo
Also, run this, compiling and running tests with gcc-4.8.1.
Verify that there are no more failures than without this patch:
fbmake opt -k && fbmake runtests_opt
Here's the tail of that process, (same with and without the patch):
Summary (total time 47.45s):
PASS: 1949
FAIL: 1
SKIP: 0
FATAL: 9
TIMEOUT: 0
Reviewed By: hans@fb.com
Subscribers: trunkagent, fugalh, njormrod, folly-diffs@
FB internal diff:
D1766722
Tasks:
5941250
Signature: t1:
1766722:
1420762240:
a8545648ddb4fd0b2adf0d3147a84409c0f706a8
Hans Fugal [Thu, 8 Jan 2015 22:55:34 +0000 (14:55 -0800)]
(folly) Account for different vsnprintf behavior on OSX in test
Summary: Apparently there is some dispute about the correct thing for `snprintf(buf, len, "bogus%", "foo")` to return. OSX says 5, Linux says -1. The code in `folly/String.cpp` depends on -1 to throw the exception this test was expecting.
Test Plan: make check on OSX
Reviewed By: chip@fb.com
Subscribers: exa, folly-diffs@
FB internal diff:
D1770970
Tasks:
5936226
Signature: t1:
1770970:
1420680750:
bafae2aa8327a52013e68af64a163323244ca03b
Hans Fugal [Thu, 8 Jan 2015 22:49:01 +0000 (14:49 -0800)]
(folly) make check on OSX Yosemite (10.10)
Summary:
This switches from gcc-4.9 provided by Homebrew to the default g++ (clang) from Yosemite.
$ g++ --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.51) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix
Absolute path for double-conversion, so make in `test/` works.
gtest-1.7.0 (Using 1.6.0 causes compilation errors in Yosemite)
Various portability fixes.
Enhancements/fixes to `configure.ac` and associated source code for better portability.
Test Plan:
It builds and runs (yay)
Two tests fail, not sure whether that's specific to osx. Will investigate and make any fixes in a follow-up diff
Reviewed By: pgriess@fb.com
Subscribers: trunkagent, folly-diffs@, exa
FB internal diff:
D1768669
Tasks:
5936226
Signature: t1:
1768669:
1420728580:
dd6919c21e0edf30788e523b16e3a5b923e2d7f0
Hans Fugal [Thu, 8 Jan 2015 22:46:26 +0000 (14:46 -0800)]
avoid std::_Hash_impl
Summary:
Default Yosemite clang has no `std::_Hash_impl`, which is an internal implementation detail anyway.
@davejwatson you might have a different suggestion about how to implement this without that function, or how to test if it exists and do this only if it doesn't. This is probably not the most efficient approach, since it copies the string.
Test Plan:
builds
inspection
fbgs doesn't turn up many results (only recent wangle ssl code really) so I don't think this will be a big perf regression if we just go with it. But if I'm wrong, we can gate it on `#if __APPLE__` or something.
Reviewed By: davejwatson@fb.com
Subscribers: folly-diffs@, fugalh, exa, davejwatson
FB internal diff:
D1767052
Signature: t1:
1767052:
1420738784:
e219ebff7aec8682b24c15a03b47077e1559c1ab
Andrei Alexandrescu [Thu, 8 Jan 2015 18:59:51 +0000 (10:59 -0800)]
Mini-refactoring prompted by https://phabricator.fb.com/
D1770460
Summary: I got obliged into this one :o).
Test Plan: runtests
Reviewed By: meyering@fb.com
Subscribers: njormrod, folly-diffs@
FB internal diff:
D1772328
Signature: t1:
1772328:
1420744114:
2b81e46d77800d247293cb667bbef86e9d6d4755
Jim Meyering [Wed, 7 Jan 2015 19:11:07 +0000 (11:11 -0800)]
folly/stats/Histogram-defs.h: avoid -Wsign-compare error
Summary:
* folly/stats/Histogram-defs.h (HistogramBuckets): Take the unusual
approach of converting "numBuckets" to signed, because we cannot
cast "max-min" (a known positive value) to an unsigned type --
we don't have an unsigned variant of ValueType.
This avoids the following error from gcc-4.9:
folly/stats/Histogram-defs.h:41:31: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: ldbrandy@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1770129
Tasks:
5941250
Signature: t1:
1770129:
1420745311:
c447e59f1730abafdf0b962e70f0886ae1d48db1
Jim Meyering [Tue, 6 Jan 2015 17:24:30 +0000 (09:24 -0800)]
folly/.../FileUtilDetail.h: avoid -Wsign-compare warnings
Summary:
* folly/detail/FileUtilDetail.h (wrapvFull): Change type of result
variable from ssize_t to size_t: this required changing r == -1 to
r == (size_t)-1.
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Also run fbmake runtests, and confirm that the results are
the same with and without this change.
Reviewed By: lucian@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1772171
Tasks:
5941250
Signature: t1:
1772171:
1420743142:
1f9e02006e2d77a0c9451bae7965fb28617d95d6
Jim Meyering [Tue, 6 Jan 2015 23:46:38 +0000 (15:46 -0800)]
folly/wangle/futures/Future-inl.h: avoid -Wsign-compare errors
Summary:
* folly/wangle/futures/Future-inl.h (whenAll): Do not compare
signed (std::distance) with unsigned(size_t): add an assertion
to eliminate the possibility of negative, then assign to a new
size_t "n".
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: jsedgwick@fb.com
Subscribers: trunkagent, fugalh, folly-diffs@
FB internal diff:
D1770668
Tasks:
5941250
Signature: t1:
1770668:
1420736965:
fc722657a2ec523052fd96c33d65d5ca24aae61c
Jim Meyering [Wed, 7 Jan 2015 16:02:21 +0000 (08:02 -0800)]
folly/FBString.h: avoid -Wsign-compare error
Summary:
* folly/FBString.h (operator>>): Since istream.width() is signed,
handle the case of a negative width. This avoids risk that "n"
will be less than 0 when compared against unsigned "extracted".
However, to suppress the following warning, we must also explicitly
convert "n" to size_t before comparing:
folly/FBString.h:2326:22: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: trunkagent, net-systems@, njormrod, folly-diffs@
FB internal diff:
D1770460
Tasks:
5941250
Signature: t1:
1770460:
1420679684:
b262eb31e6511b6311bebf0fbc29099471648b36
James Sedgwick [Thu, 8 Jan 2015 16:42:04 +0000 (08:42 -0800)]
stop returning move(...) when unnecessary
Summary: as discussed in another diff, this might help with NRVO
Test Plan: unit
Reviewed By: hans@fb.com
Subscribers: trunkagent, fugalh, folly-diffs@
FB internal diff:
D1764921
Signature: t1:
1764921:
1420677311:
c6b63d7bf04c0e17283117b3edef798c288e5fbb
James Sedgwick [Thu, 8 Jan 2015 16:12:39 +0000 (08:12 -0800)]
fix shadowed variables
Summary: as above, fixes builds
Test Plan: compiled thrift + mcrouter
Reviewed By: seanc@fb.com
Subscribers: seanc, fugalh, folly-diffs@
FB internal diff:
D1771910
Signature: t1:
1771910:
1420733490:
7f33e57a5853395cab786e5d35d4c15c75a15c5a
James Sedgwick [Thu, 8 Jan 2015 15:51:15 +0000 (07:51 -0800)]
doxygen comments for Try
Summary:
One header at a time, starting with the small fry
I'm allowed to drink while I do this right? Good, because I don't think I ever had a choice.
Test Plan: generated doxygen and looked decent http://home.fburl.com/~jsedgwick/html/classfolly_1_1wangle_1_1Try.html
Reviewed By: davejwatson@fb.com
Subscribers: trunkagent, fugalh, njormrod, folly-diffs@
FB internal diff:
D1677730
Tasks:
5480262
Signature: t1:
1677730:
1415899240:
bdd03569a82be1c8def8c13786eae39b4a2a2f94
James Sedgwick [Thu, 8 Jan 2015 15:48:23 +0000 (07:48 -0800)]
exception_ptr -> exception_wrapper migration
Summary:
integrate exception_wrapper everywhere, and deprecate public methods that take ptrs directly to discourage their use
note that this will break for throwing non-exceptions, which is probably fine
this change opens the door to interesting optimizations for those interested, e.g. Future::then<Exn1, Exn2>(/* func throwing Exn1 and Exn2 */) that autowraps the given types into the resultant future
new benchmark:
```
throwAndCatch 23.69us 42.21K
throwAndCatchWrapped 119.53% 19.82us 50.45K
throwWrappedAndCatchWrapped 350.16% 6.77us 147.80K
```
Test Plan: existing unit tests, suspected potential perf wins confirmed by benchmark, will wait for windtunnel to see other wins/regressions
Reviewed By: hans@fb.com
Subscribers: search-fbcode-diffs@, apodsiadlo, alikhtarov, andrii, trunkagent, fugalh, njormrod, folly-diffs@, bmatheny
FB internal diff:
D1644912
Signature: t1:
1644912:
1420731849:
3dc658dc03bfd6e75d61158808c7dad96092ecfb
Jim Meyering [Wed, 7 Jan 2015 16:05:04 +0000 (08:05 -0800)]
folly/String-inl.h (humanify): avoid -Wsign-compare error
Summary:
* folly/String-inl.h (humanify): Count with a signed type.
Otherwise, gcc-4.9 fails with e.g.,
folly/String-inl.h:596:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
folly/String-inl.h:601:32: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: philipp@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1770451
Tasks:
5941250
Signature: t1:
1770451:
1420704354:
c841ca453a5586fbf474535ca26cef17f95427dd
Jim Meyering [Tue, 6 Jan 2015 20:40:30 +0000 (12:40 -0800)]
folly/AtomicHashArray: use an unsigned type for each of two counters
Summary:
* folly/AtomicHashArray.h (numEntries_, numPendingEntries_):
Use an unsigned type for each of these. They count things, can
never go below 0, and are compared to unsigned values.
Otherwise, gcc-4.9 would emit this:
folly/AtomicHashArray-inl.h:153:38: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: philipp@fb.com
Subscribers: trunkagent, folly-diffs@
FB internal diff:
D1770695
Tasks:
5941250
Signature: t1:
1770695:
1420683354:
bfa4775bc2f8aab74e34772308a5c8b1779243b8
Jim Meyering [Tue, 6 Jan 2015 20:45:11 +0000 (12:45 -0800)]
folly/AtomicHashMap-inl.h: avoid -Wsign-compare warning/error (easy)
Summary:
* folly/AtomicHashMap-inl.h (insertInternal): Do not declare
nextMapIdx to be signed ("int"), since we compare it against
variables of unsigned type. Otherwise, gcc-4.9 would give this
diagnostic:
folly/AtomicHashMap-inl.h:99:18: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
complaining about this comparison:
if (nextMapIdx >= kNumSubMaps_ ||
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: philipp@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1770683
Tasks:
5941250
Signature: t1:
1770683:
1420683505:
4452c7cb4f14f098be089601be906884036348ed
Jim Meyering [Tue, 6 Jan 2015 18:00:14 +0000 (10:00 -0800)]
Histogram and Timeseries vs gcc-4.9 and -Wsign-compare
Summary:
Address multiple -Werror=sign-compare issues exposed when building
tao with gcc-4.9.
In most of these changes I've changed the type of the index in
a for-loop from int to size_t. The important thing is to use
an unsigned type when the limit is also unsigned. I choose size_t
because the general recommendation (when writing portable code)
to avoid size-tied types like uint32_t and uint64_t unless you
have a very good reason to require them.
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 tao/server && fbmake dbgo
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1766651
Tasks:
5941250
Signature: t1:
1766651:
1420594537:
56ef53ca233e1649469db9570942c1d5dd47cf6d
Jim Meyering [Wed, 7 Jan 2015 04:54:29 +0000 (20:54 -0800)]
folly/test/ForeachTest.cpp: avoid -Wsign-compare error
Summary:
* folly/test/ForeachTest.cpp: Change a "1" to "1U", so it matches
the signedness of the size_t upper bound.
Otherwise, gcc-4.9 fails with e.g.,
folly/Foreach.h:194:16: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: folly-diffs@
FB internal diff:
D1770603
Tasks:
5941250
Signature: t1:
1770603:
1420679246:
56ef62ac7fa4413a4ad6310c3381a12bdc59e64c
Jim Meyering [Wed, 7 Jan 2015 17:30:43 +0000 (09:30 -0800)]
folly/stats/BucketedTimeSeries-defs.h: avoid -Wsign-compare error
Summary:
* folly/stats/BucketedTimeSeries-defs.h (BucketedTimeSeries):
Assume that maxDuration is never negative, and cast the signed
duration_.count() to size_t, to avoid this error from gcc-4.9:
folly/stats/BucketedTimeSeries-defs.h:38:18: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: andrei.alexandrescu@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1770171
Tasks:
5941250
Signature: t1:
1770171:
1420669115:
61e6b8d3248f88b25736322c7066af015209fba5
Jim Meyering [Wed, 7 Jan 2015 16:09:26 +0000 (08:09 -0800)]
folly/Format-inl.h: use the correct type for fwrite return value (easy)
Summary:
* folly/Format-inl.h (folly): fwrite returns size_t, not ssize_t.
Fixing this avoids the following diagnostic from gcc-4.9:
folly/Format-inl.h:299:11: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: philipp@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1770155
Tasks:
5941250
Signature: t1:
1770155:
1420668733:
62593231d35f5874521dba575f23b68891b2a4f9
Jim Meyering [Wed, 7 Jan 2015 16:13:57 +0000 (08:13 -0800)]
folly/Format-inl.h: avoid -Wsign-compare error
Summary:
* folly/Format-inl.h (IndexableTraitsSeq::at): Add an explicit
int-to-size_t cast (ok here, since we've just confirmed
that the value is not negative) to avoid this error from gcc-4.9:
folly/Format-inl.h:947:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: tjackson@fb.com
Subscribers: trunkagent, net-systems@, folly-diffs@
FB internal diff:
D1770193
Tasks:
5941250
Signature: t1:
1770193:
1420670569:
83ac19c2ca8cd408d7c86d7dce49e2d4b418941a
Jim Meyering [Wed, 7 Jan 2015 05:15:27 +0000 (21:15 -0800)]
folly/test/SpinLockTest.cpp: avoid -Wsign-compare error (trivial)
Summary:
* folly/test/SpinLockTest.cpp (trylockTestThread):
Change parameter type from int to size_t, to fix these:
folly/test/SpinLockTest.cpp:67:25: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
folly/test/SpinLockTest.cpp:82:60: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: philipp@fb.com
Subscribers: net-systems@, folly-diffs@
FB internal diff:
D1770564
Tasks:
5941250
Signature: t1:
1770564:
1420674373:
fa3020398e2df66590eb71f798419b6f555d07c4
Dave Watson [Wed, 7 Jan 2015 22:55:17 +0000 (14:55 -0800)]
Fix CPUObserver test
Summary: CPUObserver observe events happen in different threads than the IOThreadPool ones.
Test Plan: fbconfig folly/wangle/concurrent/test && fbmake runtests --stress-runs 48
Reviewed By: jsedgwick@fb.com
Subscribers: doug, fugalh, folly-diffs@
FB internal diff:
D1769345
Tasks:
5948465
Signature: t1:
1769345:
1420654316:
196121ca30c905497a0acade05b9d877c98f6eeb
Jim Meyering [Wed, 7 Jan 2015 21:40:41 +0000 (13:40 -0800)]
folly/wangle/bootstrap/ServerBootstrap.h: avoid -Wsign-compare error (trivial)
Summary:
* folly/wangle/bootstrap/ServerBootstrap.h:
Change type of a for-loop index from int to size_t.
Test Plan:
Run this and note there are fewer errors than before:
fbconfig --platform-all=gcc-4.9-glibc-2.20 -r folly && fbmake dbgo
Reviewed By: jsedgwick@fb.com
Subscribers: fugalh, folly-diffs@
FB internal diff:
D1770091
Tasks:
5941250
Signature: t1:
1770091:
1420667597:
fbe9a5ba56a0e34e0480c82c8fb61128406ded95