folly.git
9 years agorefactor the interpretation of SSL errors into a function
Woo Xie [Thu, 11 Jun 2015 20:25:03 +0000 (13:25 -0700)]
refactor the interpretation of SSL errors into a function

Summary: This fucntion will be re-used for different SSL write functions in future.

Reviewed By: @djwatson

Differential Revision: D2117529

9 years agouse LifoSem instead of sem_t for ManualExecutor
Hans Fugal [Thu, 11 Jun 2015 18:52:33 +0000 (11:52 -0700)]
use LifoSem instead of sem_t for ManualExecutor

Summary: OSX apparently knows better than all of us plebes, and got rid of anonymous semaphores. Rather than try to work around that directly, I just switched to folly::LifoSem, which seems like a not-terrible idea anyway.

Reviewed By: @jsedgwick

Differential Revision: D2143566

9 years agofolly::via(Executor*, Func)
Hans Fugal [Thu, 11 Jun 2015 18:51:06 +0000 (11:51 -0700)]
folly::via(Executor*, Func)

Summary: ((not yet) more performant) sugar for `via(&x).then(func)`

Reviewed By: @​hannesr

Differential Revision: D2131246

9 years agoMake IOBuf copyable
Tudor Bosman [Thu, 11 Jun 2015 18:18:41 +0000 (11:18 -0700)]
Make IOBuf copyable

Summary: Give IOBuf a copy constructor, which clones the whole chain. (IOBufs have
shared pointer semantics). You could argue for cloning one single IOBuf,
but cloning the whole chain has the nice side effect of making Thrift
structures with IOBufs in them copyable, so there's that.

Reviewed By: @djwatson

Differential Revision: D2127209

9 years agoRequestEventBaseCache -> IOObjectCache
James Sedgwick [Thu, 11 Jun 2015 17:42:18 +0000 (10:42 -0700)]
RequestEventBaseCache -> IOObjectCache

Summary: Next step towards killing RequestEventBase hackery.
Use EBs from global IO executor instead

Reviewed By: @fugalh

Differential Revision: D2123142

9 years agowangle::getEventBase()
James Sedgwick [Thu, 11 Jun 2015 17:41:43 +0000 (10:41 -0700)]
wangle::getEventBase()

Summary: this is going to be a really common pattern, so I'm providing a shortcut

Reviewed By: @fugalh

Differential Revision: D2146008

9 years agooptimize makeFuture and Future<T>::Future()
James Sedgwick [Wed, 10 Jun 2015 23:29:08 +0000 (16:29 -0700)]
optimize makeFuture and Future<T>::Future()

Summary: No reason to go through the whole Promise rigamarole. Add an appropriate Core ctor and use that to make a completed Future with just the core alloc

Note the big win in the `constantFuture` benchmark.

```
Before:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                             120.50ns    8.30M
promiseAndFuture                                  91.99%   130.98ns    7.63M
withThen                                          28.17%   427.77ns    2.34M
----------------------------------------------------------------------------
oneThen                                                    430.48ns    2.32M
twoThens                                          58.03%   741.86ns    1.35M
fourThens                                         31.85%     1.35us  739.97K
hundredThens                                       1.61%    26.80us   37.32K
----------------------------------------------------------------------------
no_contention                                                4.58ms   218.48
contention                                        83.70%     5.47ms   182.86
----------------------------------------------------------------------------
throwAndCatch                                                8.09us  123.55K
throwAndCatchWrapped                              94.43%     8.57us  116.67K
throwWrappedAndCatch                             154.69%     5.23us  191.12K
throwWrappedAndCatchWrapped                      614.06%     1.32us  758.70K
----------------------------------------------------------------------------
throwAndCatchContended                                     967.54ms     1.03
throwAndCatchWrappedContended                    103.48%   935.04ms     1.07
throwWrappedAndCatchContended                    148.24%   652.70ms     1.53
throwWrappedAndCatchWrappedContended            14313.28%     6.76ms   147.94
============================================================================

After:
============================================================================
folly/futures/test/Benchmark.cpp                relative  time/iter  iters/s
============================================================================
constantFuture                                              69.11ns   14.47M
promiseAndFuture                                  55.12%   125.37ns    7.98M
withThen                                          16.49%   419.18ns    2.39M
----------------------------------------------------------------------------
oneThen                                                    370.39ns    2.70M
twoThens                                          55.11%   672.05ns    1.49M
fourThens                                         29.00%     1.28us  782.89K
hundredThens                                       1.23%    30.22us   33.09K
----------------------------------------------------------------------------
no_contention                                                4.56ms   219.46
contention                                        82.82%     5.50ms   181.77
----------------------------------------------------------------------------
throwAndCatch                                                8.30us  120.42K
throwAndCatchWrapped                              96.40%     8.61us  116.08K
throwWrappedAndCatch                             162.66%     5.11us  195.89K
throwWrappedAndCatchWrapped                      680.39%     1.22us  819.36K
----------------------------------------------------------------------------
throwAndCatchContended                                     979.17ms     1.02
throwAndCatchWrappedContended                    103.09%   949.84ms     1.05
throwWrappedAndCatchContended                    153.55%   637.71ms     1.57
throwWrappedAndCatchWrappedContended            10468.47%     9.35ms   106.91
============================================================================
```

Reviewed By: @fugalh, @​hannesr

Differential Revision: D2144664

9 years agoFix libc++ use
Praveen Kumar [Wed, 10 Jun 2015 22:21:07 +0000 (15:21 -0700)]
Fix libc++ use

Summary: Use of std::max from libc++ resulted in error.
std::max found type of its arguments diffrent
whereas it expected them to be same.
This diff fix the error.
Closes #223

Reviewed By: @yfeldblum

Differential Revision: D2144639

Pulled By: @sgolemon

9 years agoFix capitalization error in README.md
Yadwinder Grewal [Wed, 10 Jun 2015 21:19:33 +0000 (14:19 -0700)]
Fix capitalization error in README.md

Summary: Closes #211

Reviewed By: @Gownta, @lbrandy

Differential Revision: D2144702

Pulled By: @sgolemon

9 years agoFixing typos in README.md
Austin McKinley [Wed, 10 Jun 2015 21:25:41 +0000 (14:25 -0700)]
Fixing typos in README.md

Summary: Closes #202

Reviewed By: @paulbiss

Differential Revision: D2144850

Pulled By: @sgolemon

9 years agoUpdate FBVector.md
Ed Baunton [Wed, 10 Jun 2015 21:24:55 +0000 (14:24 -0700)]
Update FBVector.md

Summary: Minor typo when referencing `std::allocator`
Closes #84

Reviewed By: @yfeldblum

Differential Revision: D2144830

Pulled By: @sgolemon

9 years agoUpdate github.com/floitsch/double-conversion to new location
Stewart Henderson [Wed, 10 Jun 2015 20:56:44 +0000 (13:56 -0700)]
Update github.com/floitsch/double-conversion to new location

Summary: Closes #212

Reviewed By: @JoelMarcey, @paulbiss

Differential Revision: D2144749

Pulled By: @sgolemon

9 years agofix Baton.h typo
Praveen Kumar [Wed, 10 Jun 2015 21:13:43 +0000 (14:13 -0700)]
fix Baton.h typo

Summary: checked the sense
Closes #224

Reviewed By: @paulbiss

Differential Revision: D2144508

Pulled By: @sgolemon

9 years ago(Wangle) Re-add race test
Hannes Roth [Wed, 10 Jun 2015 16:06:50 +0000 (09:06 -0700)]
(Wangle) Re-add race test

Summary: I figured this test would be useless since it tests a check added before
everything was converted to FSM. But since it appears to fail on Mac OS
X, it might be useful to have after all.

Reviewed By: @fugalh

Differential Revision: D2143500

9 years agoBump version to 45:0 v0.45.0
woo [Tue, 9 Jun 2015 23:12:18 +0000 (16:12 -0700)]
Bump version to 45:0

9 years agofix typo in SharedPromise comment
James Sedgwick [Tue, 9 Jun 2015 20:27:00 +0000 (13:27 -0700)]
fix typo in SharedPromise comment

Summary: ^

Test Plan: OK

Reviewed By: yfeldblum@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2140212

Signature: t1:2140212:1433877717:86b7bf372091b455cdb25a377451e1ed4e21f50d

9 years agoBump version to 44:0
Sara Golemon [Tue, 9 Jun 2015 20:11:31 +0000 (13:11 -0700)]
Bump version to 44:0

9 years agoAsyncSSLSocket StartTLS
Dave Watson [Tue, 9 Jun 2015 18:34:10 +0000 (11:34 -0700)]
AsyncSSLSocket StartTLS

Summary:
Adds a StartTLS mode to AsyncSSLSocket.  Previously I could only find anyone doing something like this by using AsyncSocket, calling detachFd, then creating a new AsyncSSLSocket, and calling sslConn/sslAccept.

That had a couple downsides: 1) All pointers to the previous AsyncSocket become invalid and similarly 2) have to be super careful reads/writes happen on the correct socket, are flushed before changing socket types, etc.

This makes it super easy to just use the same AsyncSSLSocket for everything:
a) Create AsyncSSLSocket in StartTLS mode
b) send/recv anything
c) Call sslAccept/sslConn.  Existing writes are still flushed in the correct order, any additional writes are buffered until handshake completes
d) Start receiving encrypted data.

I made it a new mode (vs. the default), since it seems bad to unintentionally send unencrypted data.

Use case is easy secure thrift upgrade (similar to how current kerberos does it)

Test Plan: New unittest

Reviewed By: afrind@fb.com

Subscribers: doug, ssl-diffs@, folly-diffs@, yfeldblum, chalfant, haijunz, andrewcox, alandau, alikhtarov, jsedgwick, simpkins

FB internal diff: D2120114

Signature: t1:2120114:1433798448:caeddc8feb6cc10fb34200ba97ea323bcaf09f7a

9 years agofix osx bootstrap for new double-conversion directory structure
Hans Fugal [Tue, 9 Jun 2015 19:33:21 +0000 (12:33 -0700)]
fix osx bootstrap for new double-conversion directory structure

Summary:
Because https://github.com/google/double-conversion/commit/64702b40a2499702f9157ef015f0da6e006114bf

If this breaks an existing checkout, it is because you have an old version of double-conversion now. Go ahead and `rm -rf` it, or start with a clean folly clone.

Test Plan: Clean repo, `./bootstr<tab>`.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2136836

Signature: t1:2136836:1433874889:6f6c6a7bd337e052c088aabf89955e9b21b1d8e1

9 years agoAdd a guard page to a limited number of stacks
Anton Likhtarov [Thu, 4 Jun 2015 23:38:15 +0000 (16:38 -0700)]
Add a guard page to a limited number of stacks

Summary:
GuardedPageAllocator now maintains a process-wide cache
of preallocated stacks with guard pages.  We can't create too many of these, since
each stack has the overhead of two memory mappings.  Once we run out of preallocated stacks
we simply fallback on the default allocator.

Test Plan:
unit tests

perflab
TAO unit tests

Reviewed By: pavlo@fb.com

Subscribers: zhuohuang, trunkagent, sameen, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129510

Tasks: 7319041

Signature: t1:2129510:1433542031:324942af3c9813344c1b331ee2c9b66c4bfa3f03

9 years ago(Wangle) variadic collect
Hannes Roth [Mon, 8 Jun 2015 20:07:01 +0000 (13:07 -0700)]
(Wangle) variadic collect

Summary:
For D2099047 (matthieu) and also for symmetry. Can re-use most of the
code, also refactored it a bit (using an empty base case).

Test Plan:
Run all the tests.

Will add some more before committing.

Reviewed By: jsedgwick@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant, matthieu

FB internal diff: D2131515

Signature: t1:2131515:1433776852:544166fbfdfabf6760fd78f87821290e17e6e4a3

9 years agoremove Singleton from io/async deps
James Sedgwick [Mon, 8 Jun 2015 20:02:55 +0000 (13:02 -0700)]
remove Singleton from io/async deps

Summary: unused, caused circular dep issues

Test Plan: compiles

Reviewed By: mzlee@fb.com

Subscribers: subodh, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2135780

Signature: t1:2135780:1433786607:015cfa9bb14472f55f80467f2cfeb7426be9f7ea

9 years agofix the compling error caused by tcpinfo of low version
Woo Xie [Mon, 8 Jun 2015 19:24:46 +0000 (12:24 -0700)]
fix the compling error caused by tcpinfo of low version

Summary: my D2097198 break the project relying on old glibc

Test Plan: compling

Reviewed By: wez@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2135297

Tasks: 7283522

Signature: t1:2135297:1433791259:8b82b8d9b16da32be54c0dff1214fa20c94840e2

9 years agoAdd internal mode for HHWheelTimer
Alan Frindell [Mon, 8 Jun 2015 16:37:42 +0000 (09:37 -0700)]
Add internal mode for HHWheelTimer

Summary:
This allows an EventBase to exit loop() even if there are pending timeouts in the timer (make sure to cancelAll before deleting!)

Test Plan: Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2111942

Signature: t1:2111942:1432934212:7ac1973bdbbfdbda714699ab373618399f29ef76

9 years agoFix haystack threading crashes
Dave Watson [Mon, 8 Jun 2015 16:32:05 +0000 (09:32 -0700)]
Fix haystack threading crashes

Summary:
proxygen/facebook/httpserver/filter/StatsFilter expects the threadlocal stats_ variable to be initialized before any requests, and destroyed on the same thread.

There were two issues:
1) addObserver was being called after bind(), so it was possible to get some requests before the stats_ variable was initialized.  Moved up the observer, should fix it (although I don't have a repro for this)

2) on server stop(), sometimes the observer onServerStop() callback was called, but the eventBase had already stopped.  The IOThreadPoolExecutor eventually deleted the eventBase, causing consumeUntilDrained to be called, but in the main thread, and not in the eventBase's thread.  I'm not as sure about this fix, hopefully @jsedgwick can chime in: I explicitly deleted the eventBase on the eventBase thread before join.  Here's the stack for this issue:

P19866015

Reviewed By: shiva@fb.com

Subscribers: doug, fugalh, bmatheny, folly-diffs@, yfeldblum, chalfant, shiva, jsedgwick

FB internal diff: D2132208

Signature: t1:2132208:1433537787:4379ba50a908ca6f19924c2339876afaf663364c

9 years agoFileServer
James Sedgwick [Mon, 8 Jun 2015 15:46:46 +0000 (08:46 -0700)]
FileServer

Summary: Based on new FileRegion interface

Test Plan: ran it, streamed lots of files. I have found an obscure way to break it but I don't think it's a FileRegion issue, rather a pipeline/bootstrap issue. Still digging

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2088094

Signature: t1:2088094:1432758904:0f1e5765dbedea5ddbe047db50e7b632cfb0c053

9 years agoAsyncSocket::writeRequest() and its first user wangle::FileRegion
James Sedgwick [Mon, 8 Jun 2015 15:41:33 +0000 (08:41 -0700)]
AsyncSocket::writeRequest() and its first user wangle::FileRegion

Summary: similar to D2050808, but move the functionality into AsyncSocket itself so that you have a consistent interface and contiguous writes for a single file

Test Plan: added unit, will hook this up to a file server example next

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, net-systems@, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2084452

Signature: t1:2084452:1433181933:175158618966706db00bf6620cc86ae145d04ecf

9 years agoConv.h: fix a type error caught by clang
Steve O'Brien [Mon, 8 Jun 2015 02:06:26 +0000 (19:06 -0700)]
Conv.h: fix a type error caught by clang

Summary: Need to be more explicit about types; it was trying to add types deduced to be `unsigned long` + `long` which didn't agree and could be an overflow.  Be explicit anyway about the types involved.  Appease Clang, and also make the code more obvious to the reader.

Test Plan: Tried with gcc 4.9, clang 3.5, clang 3.6.

Reviewed By: yfeldblum@fb.com

Subscribers: mathieubaudet, maoy, folly-diffs@, yzhan, yfeldblum, chalfant

FB internal diff: D2134814

Tasks: 7337462

Signature: t1:2134814:1433726862:5dd80b198187c610f793e26160919922863a22a2

Blame Revision: D1934777

9 years agodecimal conversion: digits10 using bit-counting instructions on x86-64
Steve O'Brien [Sun, 7 Jun 2015 22:00:02 +0000 (15:00 -0700)]
decimal conversion: digits10 using bit-counting instructions on x86-64

Summary: To estimate length of a number's base-10 length in digits, use insn `bsrq` (Bit Scan Reverse) to count significant bits.  From that, approximate base-10 length.  Tries to avoid branchiness, expensive math, and loops.

Test Plan:
1) Tested correctness by comparing results with `snprintf` and ensuring same string lengths.  Tested at each boundary condition (2^k)-1, 2^k, (2^k+1); and similar for base 10.
2) Benchmarked with gcc 4.9 and clang 3.5.

Before/after values are millions operations / sec
GCC 4.9                             Clang 3.5
1       111.09  111.7   1.005       1       115.36  393.81  3.414
2       115.36  111.7   0.968       2       115.36  393.89  3.414
3       114.91  111.34  0.969       3       111.09  393.56  3.543
4       114.91  111.34  0.969       4       111.09  393.86  3.545
5       115.36  111.36  0.965       5       111.09  392.18  3.530
6        99.99  104.32  1.043       6       103.43  393.74  3.807
7        83.31   84.71  1.017       7        81.06  268.39  3.311
8        76.9    78.23  1.017       8        76.91  268.26  3.488
9        62.48   68.26  1.093       9        65.56  190     2.898
10        59.98   63.65  1.061      10        61.17  190.54  3.115
11        50.6    55.87  1.104      11        54.54  148.03  2.714
12        47.19   51.7   1.096      12        50.84  148.57  2.922
13        40.53   46.99  1.159      13        43.33  115.91  2.675
14        40.48   43.42  1.073      14        41.5   115.97  2.794
15        34.92   40.21  1.151      15        37.27   94.89  2.546
16        33.49   37.51  1.120      16        35.77   94.88  2.653
17        29.89   35.02  1.172      17        31.7    80.67  2.545
18        29.11   32.98  1.133      18        30.76   80.63  2.621
19        26.58   31.05  1.168      19        28.22   70.15  2.486
20        25.64   29.38  1.146      20        27.96   70.16  2.509

Reviewed By: ldbrandy@fb.com

Subscribers: dancol, trunkagent, marcelo, chalfant, maoy, folly-diffs@, yzhan, yfeldblum

FB internal diff: D1934777

Signature: t1:1934777:1433523486:3acbe7ed9c9560c44194f854754529041eaa289d

9 years ago(Wangle) Clean up tests
Hannes Roth [Fri, 5 Jun 2015 15:50:18 +0000 (08:50 -0700)]
(Wangle) Clean up tests

Summary:
This always bothered me. Now:

1) Separate tests for each feature
2) `XTest.cpp` contains `TEST(X, lowerCaseCamelCase)`

Also cleaned up some header files and using statements.

Shamelessly trying to increase the number of lines that blame to me.

Test Plan:
Run all the tests.

`git grep 'TEST' -- 'folly/futures/test/*Test.cpp' | cut -d ',' -f 1 | cut -d '/' -f 4 | sort -u`

`diff <(git ls-files -- 'folly/futures/test/*Test.cpp' | cut -d '/' -f 4 | sort) <(git grep 'Test.cpp' -- folly/futures/TARGETS | cut -d '/' -f 4 | cut -d "'" -f 1 | sort)`

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2127359

Tasks: 6019442

Signature: t1:2127359:1433459532:54a91ae83d7fb2d0b3f3769f673fefea20f35435

9 years agoBump version to 43:0 v0.43.0
Noam Lerner [Fri, 5 Jun 2015 02:34:15 +0000 (19:34 -0700)]
Bump version to 43:0

9 years agodisable stack madvise optimization on non-Linux platforms
Nathan Bronson [Thu, 4 Jun 2015 23:15:43 +0000 (16:15 -0700)]
disable stack madvise optimization on non-Linux platforms

Summary:
This diff strengthens the preconditions for attempting to use
optimize by madvising away idle stacks, because it seemed to be causing
problems on OS X (https://github.com/facebook/proxygen/issues/3).

Test Plan: unit tests

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2129100

Signature: t1:2129100:1433458268:0b6b3696dde6c2bd13b89bc7ec58f0b1898be458

9 years agoAdd missing include
Josh Watzman [Thu, 4 Jun 2015 19:22:12 +0000 (12:22 -0700)]
Add missing include

Summary:
`std::runtime_error` is defined in this header. Most build
systems seem to end up including it transitively anyways, but one version
of Clang on my Mac does not.

Test Plan:
`fbconfig -r folly && fbmake dbg` still works. Builds on my
Mac with MacPorts clang 3.6.

Reviewed By: ldbrandy@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2127268

Signature: t1:2127268:1433437985:546d0de71d8973d10f8ae3fdb6c61efae59444b9

9 years agomerge dex and readme
Hans Fugal [Thu, 4 Jun 2015 15:26:12 +0000 (08:26 -0700)]
merge dex and readme

Summary:
This is still manual as in the previous DEX.md revision, but I incorporated README.md into dex and redid the manual export.

Test Plan: reading_intensifies

Reviewed By: yfeldblum@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2124555

Tasks: 6047407

Signature: t1:2124555:1433370669:91f8bb55c1e6de4dbeccbb85556f54ae49464c7a

9 years agoReduce footprint of EliasFanoReader
Giuseppe Ottaviano [Thu, 4 Jun 2015 04:31:16 +0000 (21:31 -0700)]
Reduce footprint of EliasFanoReader

Summary:
`EliasFanoReader` has a copy of `EliasFanoCompressedList` as member, but it only needs few of its members. With this diff, it only copies the members it needs.
Also, `progress_` is a duplicate of `upper_.position()`, so it was removed.
Microbenchmarks do not indicate any significant change in performance.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2125956

Signature: t1:2125956:1433381848:2a333ce7a741bec5d059e9e771309463d6018ea2

9 years agoAdd cloning constructor to folly::IOBuf
Tudor Bosman [Thu, 4 Jun 2015 00:20:40 +0000 (17:20 -0700)]
Add cloning constructor to folly::IOBuf

Summary:
``` lang=cpp
folly::IOBuf buf(folly::IOBuf::CLONE, other);
```

is more convenient (and expression-friendly) than

``` lang=cpp
folly::IOBuf buf;
other.cloneInto(buf);
```

Test Plan: used it

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2124605

Signature: t1:2124605:1433372767:c167a36ffdc6d20c740aa57895532e9445766fce

9 years agoremove useless self-assignment in folly::iobuf
Louis Brandy [Wed, 3 Jun 2015 21:24:26 +0000 (14:24 -0700)]
remove useless self-assignment in folly::iobuf

Summary:
I'm pretty sure this doesn't do anything.

Based on a cursory review, I also don't think this is a bug (ie, it should have been assigned to something else).

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: tudorb@fb.com

Subscribers: net-systems@, enis, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2124497

Tasks: 7306976

Signature: t1:2124497:1433368083:1097dc22734dacad59b30d4b721be111d021bb43

9 years agoBump version to 42:0 v0.42.0
Noam Lerner [Wed, 3 Jun 2015 16:56:48 +0000 (09:56 -0700)]
Bump version to 42:0

9 years ago(Futures) manually-generated and manually-sanitized DEX.md
Hans Fugal [Wed, 3 Jun 2015 15:50:10 +0000 (08:50 -0700)]
(Futures) manually-generated and manually-sanitized DEX.md

Summary:

Test Plan: I copied this to a test github project and see that it renders well enough.

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2120950

Tasks: 7295194

Signature: t1:2120950:1433289846:8fbf5123e71d5594742305009fadb516206e2ba6

9 years agoMore flexible constructors for Elias-Fano lists
Giuseppe Ottaviano [Wed, 3 Jun 2015 03:57:33 +0000 (20:57 -0700)]
More flexible constructors for Elias-Fano lists

Summary:
Implement constructors for EliasFanoCompressedList to read a list from a contiguous byte range given either size and upper bound, or size, lower bits width, and upper bits size.

Refactor the rest accordingly.

Test Plan: unit tests

Reviewed By: philipp@fb.com

Subscribers: trunkagent, chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2105658

Tasks: 5474196

Signature: t1:2105658:1433270469:9948b159504e08c1b00eeb4cbe327752364ec300

9 years agofix mem leak
Alecs King [Wed, 3 Jun 2015 00:04:25 +0000 (17:04 -0700)]
fix mem leak

Summary:
- use folly::ThreadLocal to work around GCC bug 57914 (with the benefit of accessAllThreads)
- clean up corresponding thread-local and global cache entries before eventbase gets destructed since there was a possible memory leak for short-term living eventbase.

Test Plan:
tests

Reviewed By: andrii@fb.com

Subscribers: smarlow, rushix, ilyam, trunkagent, folly-diffs@, yfeldblum, chalfant, jinfu

FB internal diff: D2116216

Tasks: 72910287279391

Signature: t1:2116216:1433212893:e57a7df90b15b89ccd9471469e669c6e7dc477bf

Blame Revision: D1941662

9 years ago(Wangle) Don't use std::bind in Core
Hannes Roth [Tue, 2 Jun 2015 17:09:46 +0000 (10:09 -0700)]
(Wangle) Don't use std::bind in Core

Summary:
Seems like an unnecessary indirection that might make it harder to for
the compiler to inline everything (if that's even possible with
`std::bind`)?

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2115032

Signature: t1:2115032:1433182233:3938d0498cad50df32c8b6ee4536de14ae5fd006

9 years agoMinor fix
Delyan Kratunov [Tue, 2 Jun 2015 04:35:34 +0000 (21:35 -0700)]
Minor fix

Summary:
`LockFreeRingBuffer` was using `std::unique_ptr` but not importing `memory`.
Also, Android NDK's libstdc++ doesn't provide `std::llround`, use straight
`llround`, we don't need the constexpr-ness anyway.

Test Plan: `fbmake runtests`

Reviewed By: jmkaldor@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116923

Signature: t1:2116923:1433210832:9ec0ed7f558c271a51d528b8f88fbce9b466ba3a

9 years agoAsyncServerSocket: more tries for binding to a port
Mark McDuff [Mon, 1 Jun 2015 21:57:10 +0000 (14:57 -0700)]
AsyncServerSocket: more tries for binding to a port

Summary: User reports of intermittent failures at 5.  Setting up a server is expensive.  Binding to a socket is pretty cheap.  Why not 25 tries?  Why not 100?

Test Plan: hope

Reviewed By: alandau@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2116187

Signature: t1:2116187:1433199889:09d6916eddf8521f120894ce361c3e1a96905c90

9 years agofolly: adding folly::ltrimWhitespace / folly::rtrimWhitespace
Haim Grosman [Tue, 2 Jun 2015 00:14:49 +0000 (17:14 -0700)]
folly: adding folly::ltrimWhitespace / folly::rtrimWhitespace

Summary:
folly: adding folly::trimWhitespace

extending the idea of folly::skipWhitespace.
the reason for adding it, we want to be able to do:

auto s = skipWhitespace(trimWhitespace(" aaaa "))

very similar to python's strip()

Test Plan:
adding unit tests:

# make ; _bin/folly/test/string_test --gtest_filter="String.whitespace"
# fbmake --ccache off --distcc off dbg -j 16
fbmake dbg -j 16
Fbmake run id: G62i4cDP42U
Note: Google Test filter = String.whitespace
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from String
[ RUN      ] String.whitespace
[       OK ] String.whitespace (0 ms)
[----------] 1 test from String (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (0 ms total)
[  PASSED  ] 1 test.

Reviewed By: ldbrandy@fb.com

Subscribers: ldbrandy, folly-diffs@, yfeldblum, chalfant, azhavnerchik

FB internal diff: D2109364

Signature: t1:2109364:1433192444:862e237bba1928fcb94be1f95c57a68d457939e9

9 years agoenrich network-quality HTTPHeader field with retransmission rate
Woo Xie [Mon, 1 Jun 2015 22:30:34 +0000 (15:30 -0700)]
enrich network-quality HTTPHeader field with retransmission rate

Summary:
estimating tcp retransmission rate of the socket

Test Plan: unit tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, njormrod, bmatheny, trunkagent, chalfant, yfeldblum, jsedgwick

FB internal diff: D2097198

Tasks: 4888253

Signature: t1:2097198:1433196365:16db26dfd721514481497eddfc7820a453618d33

9 years agofolly: exception_tracer: add throw() to __cxa_begin_catch
Lucian Grijincu [Mon, 1 Jun 2015 19:22:57 +0000 (12:22 -0700)]
folly: exception_tracer: add throw() to __cxa_begin_catch

Summary:
http://libcxxabi.llvm.org/spec.html

void* __cxa_begin_catch(void* exceptionObject) throw();

```
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:145:7: error: '__cxa_begin_catch' is missing exception specification 'throw()' [-Werror]
void* __cxa_begin_catch(void *excObj) {
^
throw()
folly/experimental/exception_tracer/ExceptionTracerLib.cpp:35:7: note: previous declaration is here
void* __cxa_begin_catch(void* excObj);
^
1 error generated.
11.44s error: folly/experimental/exception_tracer/ExceptionTracerLib.cpp
fbmake: *** [_build/opt/folly/experimental/exception_tracer/b187621/ExceptionTracerLib.o] Error 1
```

Test Plan: fbmake

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2115305

Tasks: 5908365

Signature: t1:2115305:1433186210:3652d81ab10effe1be6da86705322ce090fb6cd4

Blame Revision: https://phabricator.fb.com/D2114085

9 years agofolly: -Wsign-compare
Lucian Grijincu [Mon, 1 Jun 2015 05:58:40 +0000 (22:58 -0700)]
folly: -Wsign-compare

Test Plan: runtests

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, tudort, chalfant

FB internal diff: D2114063

Signature: t1:2114063:1433111051:6686508d78de39ea4ea5dc283189c472e23f8f42

9 years agoComparing against tlsExthostname_
Abhik Chatterjee [Fri, 29 May 2015 20:54:24 +0000 (13:54 -0700)]
Comparing against tlsExthostname_

Summary:
Instead of comparing tlsExthostname_ with NULL, we are comparing it
against tlsExthostname_.

Test Plan: Automated tests

Reviewed By: afrind@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, moa

FB internal diff: D2106257

Tasks: 4751985

Signature: t1:2106257:1432932687:78c6244392d7842e844a3c1e654ffc4dc0e760a2

9 years agoupdate wangle/rx/README.md
Hans Fugal [Fri, 29 May 2015 20:07:26 +0000 (13:07 -0700)]
update wangle/rx/README.md

Summary:
https://github.com/facebook/wangle/pull/1

Test Plan: reads better

Reviewed By: jsedgwick@fb.com

Subscribers: fugalh, exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2111498

Signature: t1:2111498:1432927058:1f14b38faa324d0bdef9421ae5592244a6b78928

9 years agoFix asan test crashes
Sarang Masti [Thu, 28 May 2015 23:12:21 +0000 (16:12 -0700)]
Fix asan test crashes

Summary: We need to prevent the socket from being deleted while in the loopCallback.

Test Plan: --  fbconfig '--sanitize' 'address' 'folly/io/async/test' && fbmake runtests

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2109031

Tasks: 7250307

Signature: t1:2109031:1432918801:77f0352254925d3f33f1e004c8bba89cde63e2cf

Blame Revision: D2102601

9 years agoChain executor in timeout functions
Dave Watson [Fri, 29 May 2015 17:08:22 +0000 (10:08 -0700)]
Chain executor in timeout functions

Summary: within, onTimeout, etc use the executor of the future if it is set.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2104770

Tasks: 6958727

Signature: t1:2104770:1432773599:a71c00827071bef46e5700de04c7125142e4eb17

9 years agoMake TurnSequencer non-x86-safe
Delyan Kratunov [Fri, 29 May 2015 00:31:09 +0000 (17:31 -0700)]
Make TurnSequencer non-x86-safe

Summary:
`TurnSequencer` uses inline assembly, which obviously won't work on ARM.
Guard said assembly to x86{,_64} targets.

Test Plan:
Added an always-false `static_assert` to the `ifdef` block, confirmed it gets
triggered (i.e., we're not removing the `pause` on x86_64).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069320

Signature: t1:2069320:1431663264:3b3fb4282cf2b755f9b26c76b608b91ff665e226

9 years agoMake detail/Futex.cpp bionic-safe
Delyan Kratunov [Fri, 29 May 2015 00:30:59 +0000 (17:30 -0700)]
Make detail/Futex.cpp bionic-safe

Summary:
The headers that the Android NDK exports are all sorts of screwed up.
In particular, they expose a subset of the `futex.h`, so, even though the
kernel supports newer options, we can't reference them from the NDK headers.

This diff ensures Futex.cpp redefines all the things it uses, if the headers
are lacking.

It also transitions away from `SYS_` to `__NR_` which is apparently the newer
convention.

Lastly, the duration usage is made explicitly `time_t`-safe for use on 32-bit
platforms where `time_t` is `long int` but `std::chrono:seconds` is stored as
`long long int`.

Test Plan:
Existing tests, sample app compiled and ran cleanly on Android as well
(tests not verified on Android due to folly largely being unported to Android yet).

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2069306

Signature: t1:2069306:1431721711:36c77b1afe8dd9259c1050f11a87511dcf7dd25f

9 years agoLockFreeRingBuffer
Delyan Kratunov [Fri, 29 May 2015 00:30:19 +0000 (17:30 -0700)]
LockFreeRingBuffer

Summary:
This introduces a lock-free ring buffer with the following expected semantics:

1. Writers can't block on readers
2. Writers are mostly lock-free
3. Readers can detect if they're being too slow
4. Be usable on Android (TBD but should work as-is with the armeabi-v7a ABI; armeabi (ARMv5) support is explicitly a non-goal)

Non-goals:
1. Match MPMCQueue in level of optimization. There's no need for that yet.

Test Plan: iloveunittests

Reviewed By: ngbronson@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2037718

Signature: t1:2037718:1432850250:c57963510d8cda58edc006f4c3260f5ac34d4996

9 years agoExtract TurnSequencer to detail/TurnSequencer
Delyan Kratunov [Fri, 29 May 2015 00:29:46 +0000 (17:29 -0700)]
Extract TurnSequencer to detail/TurnSequencer

Summary:
A completely mechanical transformation that moves TurnSequencer into its own
header, to be used by other collections.

Test Plan: Existing tests.

Reviewed By: jmkaldor@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2065108

Signature: t1:2065108:1431474613:a3a9d063ebd2bedb31abb37be5fd33f0fb3eca6a

9 years agoFix toString() for authority-less URIs
Tudor Bosman [Thu, 28 May 2015 20:59:56 +0000 (13:59 -0700)]
Fix toString() for authority-less URIs

Summary: Uri("foo:bar").str() would incorrectly return "foo://bar"

Test Plan: test added

Reviewed By: savasp@fb.com, markisaa@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2107530

Tasks: 7248055

Signature: t1:2107530:1432837143:c100f148c07b5b141cc036b1b39e6c8317e9bbd6

9 years agoChain interrupt handlers
Dave Watson [Thu, 28 May 2015 20:57:27 +0000 (13:57 -0700)]
Chain interrupt handlers

Summary: Chain interrupt handlers in then().  User can always choose to set a new one later.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2091569

Signature: t1:2091569:1432760579:ea7afa9b4e62e4b04b4d5f17950b9b7356a57f09

9 years agoFix AsyncSocket::handleRead
Sarang Masti [Wed, 27 May 2015 03:19:36 +0000 (20:19 -0700)]
Fix AsyncSocket::handleRead

Summary:
If openssl has buffered data read from the socket, we might not get a
read event on the socket. So, we must schedule a readCallback to ensure
before exiting from AsyncSocket::handleRead if we have exhausted the
maxReadsPerEvent_ limit.

Test Plan: -- modifying existing test to test this corner case.

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2102601

Tasks: 7168699

Signature: t1:2102601:1432837605:82e72a2a1875c08c9c1e8e831796c9c90df927fa

9 years agofolly: MemoryMapping: madvise: round the end to lower bound
Lucian Grijincu [Thu, 28 May 2015 17:42:05 +0000 (10:42 -0700)]
folly: MemoryMapping: madvise: round the end to lower bound

Test Plan: n/a

Reviewed By: ott@fb.com

Subscribers: ott, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2100535

Tasks: 4421175

Signature: t1:2100535:1432674713:6f5f40a8462851b2b8972c68d34ae23aaf1e9340

9 years agodrop V0 of EliasFanoEncoder
Philip Pronin [Thu, 28 May 2015 01:06:40 +0000 (18:06 -0700)]
drop V0 of EliasFanoEncoder

Summary: Cleanup. Drop support for V0 in favor of V1.

Test Plan: unit tests

Reviewed By: lucian@fb.com

Subscribers: fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, unicorn-diffs@, folly-diffs@, yfeldblum, tudort, chalfant

FB internal diff: D2105967

Signature: t1:2105967:1432781247:e420d8b4b8c69d28dfc229e8a2af6df8a580f979

9 years agoBump version to 41:0 v0.41.0
Pavlo Kushnir [Thu, 28 May 2015 00:54:37 +0000 (17:54 -0700)]
Bump version to 41:0

9 years agoFix cancel in ThreadWheelTimeKeeper
Dave Watson [Wed, 27 May 2015 18:20:40 +0000 (11:20 -0700)]
Fix cancel in ThreadWheelTimeKeeper

Summary: This is actually a bug, future.cancel() doesn't work with the current THreadWheelTimekeeper, because cancel() only works from the eventBase thread.

Test Plan: added unittest.  Crashes before, passes now

Reviewed By: hans@fb.com

Subscribers: doug, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2091531

Signature: t1:2091531:1432224024:4aa5dd71de15b1344034a414d47c97ffaba68949

9 years agofolly::Unit docs and use lift in is_void_or_unit
Hans Fugal [Wed, 27 May 2015 16:30:48 +0000 (09:30 -0700)]
folly::Unit docs and use lift in is_void_or_unit

Summary:
I'm not sure about the name Lift now. We are lifting, yes, but are we lifting into Unit, or into "can't be void because void is unit"? But LiftIntoNonVoid is a bit verbose. I'm totally open to other names or arrangements. We could also rename `is_void_or_unit`, but to what?

I reimplemented `is_void_or_unit` in terms of `Unit::Lift` because it's kinda cool but also to provide a little motivational example to the reader for why Lift exists in the first place.

Test Plan: Still builds and passes tests. Nothing significant depends on this yet.

Reviewed By: hannesr@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2102147

Tasks: 6847876

Signature: t1:2102147:1432742966:a03973a45882d3e9f6fa7158ef393b148cbe16fc

9 years agoInstrument EventHandlers' execution
Andre Pinto [Wed, 27 May 2015 02:06:05 +0000 (19:06 -0700)]
Instrument EventHandlers' execution

Summary:
Add the option to instrument every EventHandler that runs in a
EventBase.

Test Plan:
Unit tests

Reviewed By: davejwatson@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2044546

Tasks: 6958000

Signature: t1:2044546:1432221358:6500d6c1a6631d7a06a18e874d051f2ff3108e10

9 years agoPromise::isFulfilled()
Hans Fugal [Wed, 27 May 2015 01:10:15 +0000 (18:10 -0700)]
Promise::isFulfilled()

Summary: See task

Test Plan: runtests

Reviewed By: jsedgwick@fb.com

Subscribers: exa, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2101549

Tasks: 7225286

Signature: t1:2101549:1432688294:5fb9d7370c38c1392429a09ba48b131cac16647d

9 years agocopy wangle back into folly
James Sedgwick [Tue, 26 May 2015 22:14:41 +0000 (15:14 -0700)]
copy wangle back into folly

Summary: copy everything but example/

Test Plan: fbconfig -r folly/wangle && fbmake runtests

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100811

Tasks: 5802833

Signature: t1:2100811:1432678173:6c336fe53aa223993f6f82de4ac91b3c19beacf1

9 years agomove folly/wangle to wangle
James Sedgwick [Tue, 26 May 2015 22:14:40 +0000 (15:14 -0700)]
move folly/wangle to wangle

Summary: this preserves history and will be followed by a copy diff

Test Plan: next diff will compile

Reviewed By: hans@fb.com

Subscribers: fugalh, ps, bmatheny, folly-diffs@, jsedgwick, yfeldblum, markdrayton, chalfant

FB internal diff: D2100784

Tasks: 5802833

Signature: t1:2100784:1432677219:838a336d84918145d19c6cfa4290dd58374821fd

9 years agoremoving non-existing file from the build
Marcelo Juchem [Tue, 26 May 2015 21:54:52 +0000 (14:54 -0700)]
removing non-existing file from the build

Summary: broken by D2096146

Test Plan: arc unit

Reviewed By: andrewcox@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2100773

Signature: t1:2100773:1432676073:a72e72fa5fb915b027fdfd31dd814d02e6606858

9 years agocleaning up RequestContext
Marcelo Juchem [Tue, 26 May 2015 20:46:32 +0000 (13:46 -0700)]
cleaning up RequestContext

Test Plan: built folly

Reviewed By: mshneer@fb.com

Subscribers: davejwatson, mshneer, trunkagent, alandau, bmatheny, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2096146

Tasks: 6337419

Signature: t1:2096146:1432658670:4ff00e4274412519dc3b02d2a849b7ff289ac979

9 years agoBump version to 40:0 v0.40.0
woo [Tue, 26 May 2015 18:32:31 +0000 (11:32 -0700)]
Bump version to 40:0

9 years agoremove spurious warning
James Sedgwick [Tue, 26 May 2015 17:58:31 +0000 (10:58 -0700)]
remove spurious warning

Summary: see task

Test Plan: OK

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2099416

Tasks: 7166291

Signature: t1:2099416:1432658571:4ae632c218c4764f23e4b189167f6fdaff13dc2d

9 years agoSome optimizations
Anton Likhtarov [Mon, 18 May 2015 23:13:09 +0000 (16:13 -0700)]
Some optimizations

Summary:
1. Eliminate some string -> StringPiece -> strings conversions
2. Mcrouter: eliminated unnecessary inlining by moving slow path logic into its own method.

Using a test setup with shadow sampling enabled and shadowing some requests,
(typical prod setup), this brings down the cost from ~1.4% cpu in standalone mcrouter to ~0.2%:

```
before:

+ 0.70%          3898  mcrouter_orig  mcrouter_orig  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.13%           864  mcrouter_orig  mcrouter_orig  [.] EventGroup<ScubaRow>::processExtraSamplers
+ 0.58%          3347  mcrouter_orig  mcrouter_orig  [.] DynamicScubaSampler::getSampler

~ 1.41% total

after:

+ 0.18%          1223  mcrouter_fix  mcrouter_fix  [.] FbAdditionalProxyRequestLogger::logReply
+ 0.04%           205  mcrouter_fix  mcrouter_fix  [.] EventGroup<ScubaRow>::processSampler

~ 0.22% total
```

Fiber local optimization might have more of an effect.

Test Plan:
unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, alikhtarov, folly-diffs@, yfeldblum, darshan, chalfant

FB internal diff: D2089133

Tasks: 5414865

Signature: t1:2089133:1432338487:4158dc6b720c04f43820193e73b98d4197afcffa

9 years agomove FiberIOExecutor to wangle/concurrent
James Sedgwick [Tue, 26 May 2015 03:32:00 +0000 (20:32 -0700)]
move FiberIOExecutor to wangle/concurrent

Summary: Need to cut this dep on wangle out of folly as we're moving wangle to its own repo

Test Plan: chimera unit

Reviewed By: rushix@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2092122

Signature: t1:2092122:1432239179:9261c1b3a3a346b886b15899d25a4d0530d4e890

9 years agoRevert "Make strlcpy available in folly" (D2062632)
Chip Turner [Sat, 23 May 2015 00:48:35 +0000 (17:48 -0700)]
Revert "Make strlcpy available in folly" (D2062632)

Summary:
Reverting diff to fix the build.

Test Plan: contbuild rule that failed to build is okay now

Reviewed By: ldbrandy@fb.com

9 years agoMake strlcpy available in folly
Chip Turner [Mon, 11 May 2015 22:39:21 +0000 (15:39 -0700)]
Make strlcpy available in folly

Summary:
strncpy is bad.  strlcpy is somewhat less bad.  We already had
this function, but let's move it somewhere more reasonable.

Test Plan: runtests

Reviewed By: ldbrandy@fb.com

Subscribers: trunkagent, lins, anca, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2062632

Signature: t1:2062632:1431969926:cc7f7283073d0242fe8f361efac2557aa0b0a481

9 years agoremove unused variable
Marc Horowitz [Fri, 22 May 2015 17:51:28 +0000 (10:51 -0700)]
remove unused variable

Test Plan: search, and fbmake

Reviewed By: delyank@fb.com

Subscribers: trunkagent, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093523

Signature: t1:2093523:1432249208:84a27e2328636c2aafe43412f61a693b5645b895

9 years agofolly/experimental/fibers/EventBaseLoopController.h: include <atomic>
Jim Meyering [Thu, 21 May 2015 23:22:13 +0000 (16:22 -0700)]
folly/experimental/fibers/EventBaseLoopController.h: include <atomic>

Summary: Without <atomic>, this fails to compile with clang:dev and newer glibc.

Test Plan:
fbconfig --clang --with-project-version=clang:dev --with-project-version=llvm_compiler_rt:dev --platform-all=gcc-4.9-glibc-2.20 --sanitize=address mcrouter/lib/network:client-test-sync && fbmake dbg

Reviewed By: pavlo@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2093709

Signature: t1:2093709:1432250378:2b78c142598bf0a7b0869089c752f5ccdae9da22

9 years agoRefactor tests to verify that a single folly target can be used successfully
Marc Horowitz [Tue, 12 May 2015 02:23:45 +0000 (19:23 -0700)]
Refactor tests to verify that a single folly target can be used successfully

Summary:
This includes a change to Range to move operator<< into the
header, to achieve the goal.  Specifically, see format_test,
dynamic_test, json_test, demangle_test

Test Plan: fbconfig -r buck && fbmake runtests

Reviewed By: njormrod@fb.com

Subscribers: darshan, tjackson, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2063698

Signature: t1:2063698:1431467309:069da6d74bb5c384e7a21e6be19a4b20466bdd92

9 years agofolly refactorings to better support cross-platform
Marc Horowitz [Fri, 17 Apr 2015 01:03:39 +0000 (18:03 -0700)]
folly refactorings to better support cross-platform

Summary:
I'm looking into pulling parts of folly (right now, dynamic,
json, and their dependencies) into fbandroid for use as part of xplat.
This diff includes a few kinds of changes: portability fixes for arm;
reduce the size of the code generated by moving non-templated
functions and methods into cpp files; refactoring header usages which
require extra compiler flags on android to cpp files; and slicing up
the libraries a bit differently to reduce dependencies.  This should
all be backward-compatible, and do no harm to fbcode.

Test Plan: runtests, sandcastle

Reviewed By: njormrod@fb.com

Subscribers: darshan, davejwatson, tudorb, dancol, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2057797

Tasks: 7005344

Signature: t1:2057797:1432145435:fa10f129fc669e682da5b4b207fc96986ca035fc

9 years agofixup service filter interface
Dave Watson [Thu, 21 May 2015 17:40:26 +0000 (10:40 -0700)]
fixup service filter interface

Summary:
Based on a more thourough reading of finagle's interface:
* adds close/isAvailable, which seem very close to thrift's interfaces
* ComposedServices are hardcoded to underlying services, to simplify the code (means extra allocs?)
* Made everything a shared_ptr
* Addd ServiceFactoryFilters

Test Plan: Updated the existing unittests and added some new ones

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2037206

Signature: t1:2037206:1432147489:3464d4c12a9e434d4973febcabbf7b2b3a883257

9 years agotupleRange, tuplePrepend
Tudor Bosman [Thu, 21 May 2015 14:54:11 +0000 (07:54 -0700)]
tupleRange, tuplePrepend

Summary:
tupleRange<start, n>(tuple): return a tuple consisting of a range of elements
from the given tuple

tuplePrepend(x, tuple): return a tuple consisting of prepending x to the given
tuple.

For Lispies:

std::get<0>(tuple) is car.
tupleRange<1>(tuple) is cdr.
tuplePrepend(x, tuple) is cons.

Test Plan: test added

Reviewed By: lesha@fb.com

Subscribers: trunkagent, lesha, ilyam, folly-diffs@, yfeldblum, chalfant, jhj, alerer, kma, pamelavagata, tulloch

FB internal diff: D2087568

Signature: t1:2087568:1432164681:18795d0e8bb01f38ffc6949ac233f514ab098355

9 years ago(Wangle) Fix typo
Hannes Roth [Wed, 20 May 2015 19:10:00 +0000 (12:10 -0700)]
(Wangle) Fix typo

Summary:
This was supposed to be the `Result` type, since it's called on the
Future returned by the lambda.

Test Plan: Added tests for void and different types in vector/lambda.

Reviewed By: mhl@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2087819

Tasks: 7126300

Signature: t1:2087819:1432142435:72914fa64eff03454774b87a24c426379defab3b

Blame Revision: rFBCODEf229322bc273190a85b5e995dcd8209b1fbf0825

9 years agoBump version to 39:0 v0.39.0
Viswanath Sivakumar [Wed, 20 May 2015 17:57:38 +0000 (10:57 -0700)]
Bump version to 39:0

9 years agomake AsyncSocket::WriteRequest an interface
James Sedgwick [Wed, 20 May 2015 15:34:26 +0000 (08:34 -0700)]
make AsyncSocket::WriteRequest an interface

Summary: This will allow a subsequent diff to implement file transfers as another type of write request

Test Plan: unit

Reviewed By: davejwatson@fb.com

Subscribers: net-systems@, folly-diffs@, yfeldblum, chalfant, fugalh, bmatheny

FB internal diff: D2080257

Signature: t1:2080257:1432044566:bcc0724d349879f46e3e58ee672aff7bf37fa5f6

9 years agoMake AtomicHashMap shadow declaration clean
Brian Watling [Wed, 20 May 2015 01:38:38 +0000 (18:38 -0700)]
Make AtomicHashMap shadow declaration clean

Summary: Some of AtomicHashMap's locals shadow member functions - rename the locals to fix the warnings

Test Plan: unit tests

Reviewed By: chaoc@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant, tao-eng@

FB internal diff: D2086270

Signature: t1:2086270:1432083900:fae1be39e55e4c30b47fdc7a069bb13d75292b0a

9 years agoMake Subprocess movable
Alexey Spiridonov [Tue, 19 May 2015 23:36:35 +0000 (16:36 -0700)]
Make Subprocess movable

Summary:
Subprocess doesn't have any non-movable members, and its implementation does not take addresses of the object, so I think it's safe. Move makes a bunch of code cleaner (you no longer have to wrap it in `std::unique_ptr` with associated clumsiness).

https://phabricator.fb.com/diffusion/FBCODE/browse/master/folly/Subprocess.h

Test Plan:
- unit test
- Searched for `this` in `Subprocess.{h,cpp}`.
- Inspected member variables: `pid_`, `returnCode_`, `pipes_`
- contbuild

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@, yfeldblum, chalfant

FB internal diff: D2079167

Signature: t1:2079167:1432048688:26f96e29310298f47a9a9a7abef22dc863f68942

9 years agoKoans to the Future
Hans Fugal [Tue, 19 May 2015 19:57:36 +0000 (12:57 -0700)]
Koans to the Future

Summary:
The first file of Koans. Some simple constructor and `makeFuture` stuff, but this diff is mostly about the framework (ie the `TARGETS` and `main.cpp` and `Koan.h`, and the layout of the Koan files).

Known Issues: I am not nearly enlightened enough to write these in a particularly zen style with lots of inside zen-jokes, so I'm not even trying.

Test Plan:
Work through the koans,
then `fbmake runtests`,
reach enlightenment

Reviewed By: davejwatson@fb.com

Subscribers: cgthayer, exa, folly-diffs@, jsedgwick, yfeldblum, bmatheny, chalfant

FB internal diff: D2082141

Tasks: 6973057

Signature: t1:2082141:1432057657:273708f566154cc54f726b85f05457388357ef4e

9 years agoAdd getTotalCount() to Histogram
Hasnain Lakhani [Tue, 19 May 2015 18:26:24 +0000 (11:26 -0700)]
Add getTotalCount() to Histogram

Summary:
Add a getTotalCount() method to the Histogram class so that callers
can read out the number of values that were put into the histogram

Test Plan: Added a new test.

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, evanmao, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2078239

Tasks: 7097793

Signature: t1:2078239:1431739799:5de3a02df67ff535e06b4a1547690440cf594824

9 years agoAdd 'runnable' callback to ExecutionObserver
Brian Watling [Tue, 19 May 2015 15:30:03 +0000 (08:30 -0700)]
Add 'runnable' callback to ExecutionObserver

Summary: Add a callback when a fiber becomes runnable

Test Plan: unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov, folly-diffs@, yfeldblum, chalfant

FB internal diff: D2081306

Signature: t1:2081306:1432011152:0ee93cb2682eb2a289b99c403e91465e72dd4ee8

9 years agoframing handler pipeline stage
Dave Watson [Tue, 19 May 2015 13:43:14 +0000 (06:43 -0700)]
framing handler pipeline stage

Summary: last pipeline in the original diff (HeaderServer/Client channel can also become pipeline stages in the future...)

Test Plan:
fbconfig -r thrift/lib/cpp2; fbmake runtests

canary results will follow in a comment

Reviewed By: alandau@fb.com

Subscribers: fugalh, folly-diffs@, chalfant, trunkagent, doug, alandau, bmatheny, jsedgwick, yfeldblum

FB internal diff: D2033559

Signature: t1:2033559:1430417432:c6cf4ccbf9ef26d89e7d7c5955d103348205b365

9 years agoRFC: FutureDAG
James Sedgwick [Mon, 18 May 2015 15:42:50 +0000 (08:42 -0700)]
RFC: FutureDAG

Summary:
See task. Set up a DAG of Future-returning tasks (optionally with executors) and eventually kick them off.
One big question is ownership. Currently the user would be responsible for ensuring that the FutureDAG outlives its own completion. This requirement could go away with shared_from_this magic maybe

Test Plan: unit. I didn't bother to test via() functionality because it's too much work for now - the functionality is trivial. Same for "true-async" dags...

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2073481

Signature: t1:2073481:1431961131:82a8898502d5308f6ab3cc8cc5b84b016d3998fe

9 years agoMake folly::format no longer crash on invalid format strings
Tudor Bosman [Sat, 16 May 2015 02:29:05 +0000 (19:29 -0700)]
Make folly::format no longer crash on invalid format strings

Summary:
Yes, ideally we'd detect this at compile time, but if we can't, causing a SEV1
is not the best way to do it.

format() now behaves like formatChecked(); the old API is maintained for
backwards compatibility, but deprecated.

Test Plan: format_test

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, dpittman, jdelong, sdmich, net-systems@, dmitri, folly-diffs@, yfeldblum, andrii, mssarang, chalfant

FB internal diff: D2075829

Tasks: 7095768

Signature: t1:2075829:1431713985:b3fdec4820104b4ddc4be0b6af999db174a692d9

9 years agoFix race issue in EventBase::runInEventBaseThreadAndWait (t6779663)
Alan Frindell [Fri, 15 May 2015 17:42:48 +0000 (10:42 -0700)]
Fix race issue in EventBase::runInEventBaseThreadAndWait (t6779663)

Summary: Huge thanks to @afrind for debugging this issue with me and found the root cause. As per the comment from @afrind for the diff D1823407, there is a tricky race issue. The main thread could have left and reset the condition_variable from its stack but the EventBase thread tries to access it afterwards due to race and could be blocked indefinitely. This caused the server-side IO threads not able to pick up the incoming connections for the proxygen case. The fix is to use a simpler struct barrier and get a hold of the shared_ptr instead of the same object in a safer way.

Test Plan:
The original issue reproes very easily in HDFS XDC encryption case. Servers easily enter into bad state and we got high volume of timeouts from the client. With the fix, this does not happen anymore after the fix being deployed at 11:15PM. Here is the Scuba log before and after the fix:
https://fburl.com/109969805
And here is the correspond Scuba diagram for successful calls in the same test:
https://fburl.com/109971729
The throughput improved a lot after the fix.

Reviewed By: davejwatson@fb.com

Subscribers: doug, folly-diffs@, yfeldblum, chalfant, afrind, brettp, dougw, fma

FB internal diff: D2071646

Signature: t1:2071646:1431709609:10fb033536f9e4fb428dea8ba68f6a9a051616c0

9 years agofix clang build
James Sedgwick [Fri, 15 May 2015 16:55:52 +0000 (09:55 -0700)]
fix clang build

Summary: PipelineBase needs something virtual so we can dynamic_cast from it down to the real pipeline, as we do in Proxy

Test Plan: built proxy with clang

Reviewed By: hans@fb.com

Subscribers: fugalh, mathieubaudet, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2075855

Tasks: 70774197077417

Signature: t1:2075855:1431708780:03ce8d8f40cdb56d24baf75f4dc29004e5ea4c7a

9 years ago(Wangle) unorderedReduce
Hannes Roth [Thu, 14 May 2015 22:36:27 +0000 (15:36 -0700)]
(Wangle) unorderedReduce

Summary:
Use this if you don't need the order of the input, e.g. summing up
values. This constructs a separate Future chain to do the reducing,
because we don't want to add locking while reducing. The only lock
necessary is when adding a new Future to the chain, which should be
really quick.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015326

Tasks: 6025252

Signature: t1:2015326:1431557191:9ea2edccb0162dedf067b5b3300de2fe72a1a4c9

9 years ago(Wangle) chain -> thenMulti + thenMultiWithExecutor
Hannes Roth [Thu, 14 May 2015 18:55:27 +0000 (11:55 -0700)]
(Wangle) chain -> thenMulti + thenMultiWithExecutor

Summary:
If we make `chain` a member function we can avoid the type issues and infer everything. I also added thenMulti for symmetry. Sadly the compiler doesn't like having a thenMulti with an optional `Executor*` as the first argument, it fails after some deductions. Hence `thenMultiWithExecutor`.

itssobeautiful

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2021000

Signature: t1:2021000:1431557618:169447dc9d747b23a8a1ba830e78c43713d09a96

9 years ago(Wangle) window
Hannes Roth [Thu, 14 May 2015 18:50:06 +0000 (11:50 -0700)]
(Wangle) window

Summary: `window` creates up to `n` Futures at a time and only starts new ones when previous ones complete. A sliding window.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: bmatheny, henryf, scottstraw, juliafu, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2015310

Signature: t1:2015310:1431557556:1017006cc9c9c2562ebe2c3dabfc4dbf316ff408

9 years agovia with priority
James Sedgwick [Thu, 14 May 2015 00:45:10 +0000 (17:45 -0700)]
via with priority

Summary:
I wish I could just have an add(Func, priority) but the damned overloaded virtual warnings become a nightmare, so it's addWithPriority.
I also switched priority to a uint8_t in the hopes of reducing Core size. Turns out std::atomic<uint8_t> is 8 bytes anyways :( I left it that way because come on you really shouldn't be using > 256 priorities.
Biggest problem is the data race with the two atomics executor_ and priority_. Should we just use a microspinlock to co-protect them? Could probably save some size from the atomics that way.

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: hannesr, fugalh, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2039619

Tasks: 6928162

Signature: t1:2039619:1431551266:3b31ed2329301aaa9c32f0f41b6e61f3482d570e

9 years agotransportActive and transportInactive
James Sedgwick [Thu, 14 May 2015 00:43:44 +0000 (17:43 -0700)]
transportActive and transportInactive

Summary:
These are equivalents to Netty's channelActive and channelInactive, but we've been calling channels transports so I'm staying consistent.
I skipped integrating this into TAsyncTransportHandler because thrift still does manual CB attachment/detachment and it's unclear how that fits into this model
If my suspicions are correct, it *should* be possible to make attachReadCallback and detachReadCallback private in AsyncSocketHandler, right? And perhaps get rid of the event base modifier methods? What's our use case for those?

Test Plan: unit, employ in telnet server

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, alandau, bmatheny, folly-diffs@, jsedgwick, yfeldblum, chalfant

FB internal diff: D2044520

Signature: t1:2044520:1431551998:af1de358b5dbefcca148814015d8e9f63f458d5d