zlib compression fails on large IOBufs
Summary:
If a single IOBuf has size exceeding 2^32, then our zlib
compression algorithm fails. Specifically, zlib z_stream.avail_in is
only 32 bytes (I think it's a long?
http://www.gzip.org/zlib/zlib_faq.html#faq32), and so a too-big IOBuf
will overflow the z_stream and cause data loss.
This diff breaks up large IOBufs into smaller chunks.
Test Plan:
fbconfig -r folly && fbmake runtests
Also compressed biggrep's configerator blob, which is how this bug was
caught. It now works. See the associated task.
Reviewed By: robbert@fb.com
Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@
FB internal diff:
D1702925
Tasks:
5648445
Signature: t1:
1702925:
1416958232:
459d498ff1db13e1a20766855e6f2f97da8cde8c