Simon Pilgrim [Sat, 3 Oct 2015 22:06:06 +0000 (22:06 +0000)]
[DAGCombiner] Generalize FADD constant combines to work with vectors
Updated the FADD combines to work with vectors as well as scalars.
Differential Revision: http://reviews.llvm.org/D13416
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249251
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjay Patel [Sat, 3 Oct 2015 20:52:55 +0000 (20:52 +0000)]
add test cases that demonstrate bad behavior
These are based on PR25016 and likely caused by a bug in
MachineCombiner's definition of improvesCriticalPathLen().
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249249
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjay Patel [Sat, 3 Oct 2015 20:45:01 +0000 (20:45 +0000)]
include equal sign in debug equations; NFC
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249248
91177308-0d34-0410-b5e6-
96231b3b80d8
Davide Italiano [Sat, 3 Oct 2015 20:20:28 +0000 (20:20 +0000)]
[llvm-size] Attempt to fix a test failure on Windows.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249247
91177308-0d34-0410-b5e6-
96231b3b80d8
Davide Italiano [Sat, 3 Oct 2015 19:44:06 +0000 (19:44 +0000)]
[llvm-size] Fix time to check if time of use bug.
This was the last tool relying on this pattern.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249244
91177308-0d34-0410-b5e6-
96231b3b80d8
Simon Pilgrim [Sat, 3 Oct 2015 18:55:43 +0000 (18:55 +0000)]
[X86] Lower SEXTLOAD using SIGN_EXTEND_VECTOR_INREG. NCI.
The custom lowering in LowerExtendedLoad is doing the equivalent shuffle, so make use of existing lowering code to reduce duplication.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249243
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Sat, 3 Oct 2015 18:38:50 +0000 (18:38 +0000)]
Remove unnecessary includes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249242
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Sat, 3 Oct 2015 18:28:40 +0000 (18:28 +0000)]
Move registerSection out of line and reduce #includes. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249241
91177308-0d34-0410-b5e6-
96231b3b80d8
Simon Pilgrim [Sat, 3 Oct 2015 18:17:43 +0000 (18:17 +0000)]
[X86][SSE] Add FADD combine tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249240
91177308-0d34-0410-b5e6-
96231b3b80d8
Simon Pilgrim [Sat, 3 Oct 2015 16:26:52 +0000 (16:26 +0000)]
[DAGCombiner] Merge SIGN_EXTEND_INREG vector constant folding methods. NCI.
visitSIGN_EXTEND_INREG calls SelectionDAG::getNode to constant fold scalar constants but handles vector constants itself, despite getNode being capable of dealing with them.
This required a minor change to the getNode implementation to actually deal with cases where the scalars of a BUILD_VECTOR were wider integers than the vector type - which was the only extra ability of the visitSIGN_EXTEND_INREG implementation.
No codegen intended and all existing tests remain the same.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249236
91177308-0d34-0410-b5e6-
96231b3b80d8
Kostya Serebryany [Sat, 3 Oct 2015 07:02:05 +0000 (07:02 +0000)]
[libFuzzer] trying to fix at-exit hang
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249231
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Sat, 3 Oct 2015 02:10:28 +0000 (02:10 +0000)]
[WebAssembly] Implement the remaining conversion operations.
This is a temporary assembly syntax that will likely evolve along with
broader upcoming syntax changes.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249225
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Sat, 3 Oct 2015 00:57:12 +0000 (00:57 +0000)]
Use early return. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249224
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Sat, 3 Oct 2015 00:34:19 +0000 (00:34 +0000)]
Try to appease MSVC, NFCI.
This time by lifting the lambda's in `createNodeFromSelectLikePHI` to
the file scope. Looks like there are differences in capture rules
between clang and MSVC?
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249222
91177308-0d34-0410-b5e6-
96231b3b80d8
Tom Stellard [Sat, 3 Oct 2015 00:29:50 +0000 (00:29 +0000)]
AMDGPU/SI: Remove unused tablegen multiclass
Reviewers: arsenm
Subscribers: arsenm, llvm-commits
Differential Revision: http://reviews.llvm.org/D13395
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249221
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Sat, 3 Oct 2015 00:18:14 +0000 (00:18 +0000)]
Disallow assigning symbol a null section.
They are constructed without one and they can't go back, so this was
effectively dead code.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249220
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Sat, 3 Oct 2015 00:03:15 +0000 (00:03 +0000)]
Try to appease the MSVC bots, NFCI.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249219
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Sat, 3 Oct 2015 00:01:53 +0000 (00:01 +0000)]
[WebAssembly] Rename setlocal to set_local to match the spec.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249218
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Fri, 2 Oct 2015 23:54:03 +0000 (23:54 +0000)]
[WebAssembly] Update this test for the new loop scheme.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249217
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Fri, 2 Oct 2015 23:43:32 +0000 (23:43 +0000)]
Try to appease the MSVC bots, NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249216
91177308-0d34-0410-b5e6-
96231b3b80d8
Kostya Serebryany [Fri, 2 Oct 2015 23:34:06 +0000 (23:34 +0000)]
[libFuzzer] make LLVMFuzzerTestOneInput (the fuzzer target function) return int instead of void. The actual return value is not *yet* used (and expected to be 0). This change is API breaking, so the fuzzers will need to be updated.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249214
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Fri, 2 Oct 2015 23:24:52 +0000 (23:24 +0000)]
Fix comment ASCII art to unbreak the gcc 4.9.1 build
The trailing backslashes in some ASCII art added in r248527 cause a
"error: multi-line comment [-Werror=comment]" when building with gcc
4.9.1 -Wall. Swallow (ASCII-)artistic integrity and use pipes instead.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249212
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Fri, 2 Oct 2015 23:09:44 +0000 (23:09 +0000)]
[SCEV] Recognize simple br-phi patterns
Summary:
Teach SCEV to match patterns like
```
br %cond, label %left, label %right
left:
br label %merge
right:
br label %merge
merge:
V = phi [ %x, %left ], [ %y, %right ]
```
as "select %cond, %x, %y". Before this SCEV would match PHI nodes
exclusively to add recurrences.
This addresses PR25005.
Reviewers: joker.eph, joker-eph, atrick
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D13378
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249211
91177308-0d34-0410-b5e6-
96231b3b80d8
Chris Bieneman [Fri, 2 Oct 2015 22:28:48 +0000 (22:28 +0000)]
[CMake] We need to explicitly add llvm-config before clang so that LLVM_BUILD_EXTERNAL_COMPILER_RT can depend on llvm-config.
This patch is a required stepping stone to fix PR14109.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249202
91177308-0d34-0410-b5e6-
96231b3b80d8
Piotr Padlewski [Fri, 2 Oct 2015 22:12:22 +0000 (22:12 +0000)]
inariant.group handling in GVN
The most important part required to make clang
devirtualization works ( ͡°͜ʖ ͡°).
The code is able to find non local dependencies, but unfortunatelly
because the caller can only handle local dependencies, I had to add
some restrictions to look for dependencies only in the same BB.
http://reviews.llvm.org/D12992
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249196
91177308-0d34-0410-b5e6-
96231b3b80d8
Kostya Serebryany [Fri, 2 Oct 2015 22:00:32 +0000 (22:00 +0000)]
[libFuzzer] remove experimental flag and functionality
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249194
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Fri, 2 Oct 2015 21:11:36 +0000 (21:11 +0000)]
[WebAssembly] Fix CFG stackification of nested loops.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249187
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Fri, 2 Oct 2015 20:54:23 +0000 (20:54 +0000)]
[WebAssembly] Support calls marked as "tail", fastcc, and coldcc.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249184
91177308-0d34-0410-b5e6-
96231b3b80d8
Richard Trieu [Fri, 2 Oct 2015 20:52:14 +0000 (20:52 +0000)]
Call the correct overload.
Call the correct overload so a string literal does not get converted to a bool.
Also fix the test case to match the names given.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249183
91177308-0d34-0410-b5e6-
96231b3b80d8
Kostya Serebryany [Fri, 2 Oct 2015 20:47:55 +0000 (20:47 +0000)]
[libFuzzer] add a flag -max_total_time
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249181
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Fri, 2 Oct 2015 20:10:26 +0000 (20:10 +0000)]
[WebAssembly] Add a resize_memory intrinsic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249178
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Fri, 2 Oct 2015 19:39:59 +0000 (19:39 +0000)]
[SCEV] Refactor out a createNodeForSelect
Summary:
We will shortly re-use this for select-like br-phi pairs.
Reviewers: atrick, joker-eph, joker.eph
Subscribers: sanjoy, llvm-commits
Differential Revision: http://reviews.llvm.org/D13377
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249177
91177308-0d34-0410-b5e6-
96231b3b80d8
Michael Zolotukhin [Fri, 2 Oct 2015 19:21:52 +0000 (19:21 +0000)]
[Tests] Add one more case to LoopUnroll/pr18861.ll for better coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249174
91177308-0d34-0410-b5e6-
96231b3b80d8
Michael Zolotukhin [Fri, 2 Oct 2015 19:21:49 +0000 (19:21 +0000)]
[Tests] Give meaningful names to blocks in LoopUnroll/pr18861.ll, add a description of what's going on.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249173
91177308-0d34-0410-b5e6-
96231b3b80d8
Michael Zolotukhin [Fri, 2 Oct 2015 19:21:43 +0000 (19:21 +0000)]
[Tests] Slightly reduce test LoopUnroll/pr18861.ll.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249172
91177308-0d34-0410-b5e6-
96231b3b80d8
Dan Gohman [Fri, 2 Oct 2015 19:21:15 +0000 (19:21 +0000)]
[WebAssembly] Add a memory_size intrinsic.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249171
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Fri, 2 Oct 2015 18:58:37 +0000 (18:58 +0000)]
AMDGPU/SI: Add verifier check for exec reads
Make sure we aren't accidentally not setting
these in the instruction definitions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249170
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Fri, 2 Oct 2015 18:58:33 +0000 (18:58 +0000)]
Add way to test for generic TargetOpcodes
The alternative would be to add a bit to the target's
InstrFlags but that seems like a waste of a bit.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249169
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjoy Das [Fri, 2 Oct 2015 18:50:30 +0000 (18:50 +0000)]
[SCEV] Try to prove predicates by splitting them
Summary:
This change teaches SCEV that to prove `A u< B` it is sufficient to
prove each of these facts individually:
- B >= 0
- A s< B
- A >= 0
In practice, SCEV sometimes finds it easier to prove these facts
individually than to prove `A u< B` as one atomic step.
Reviewers: reames, atrick, nlewycky, hfinkel
Subscribers: sanjoy, llvm-commits
Differential Revision: http://reviews.llvm.org/D13042
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249168
91177308-0d34-0410-b5e6-
96231b3b80d8
Roman Divacky [Fri, 2 Oct 2015 18:25:25 +0000 (18:25 +0000)]
Actually switch the arch when we see .arch. PR21695
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249165
91177308-0d34-0410-b5e6-
96231b3b80d8
Tim Northover [Fri, 2 Oct 2015 18:07:18 +0000 (18:07 +0000)]
ARM: diagnose invalid local fixups on Thumb1
We previously stopped producing Thumb2 relaxations when they weren't supported,
but only diagnosed the case where an actual relocation was produced. We should
also tell people if local symbols aren't going to work rather than silently
overflowing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249164
91177308-0d34-0410-b5e6-
96231b3b80d8
Tim Northover [Fri, 2 Oct 2015 18:07:13 +0000 (18:07 +0000)]
ARM: correctly align constant pool value on Thumb1 targets.
Since we're using tLDRpci to access it, the constant pool's address must be 0
(mod 4).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249163
91177308-0d34-0410-b5e6-
96231b3b80d8
Hal Finkel [Fri, 2 Oct 2015 17:50:28 +0000 (17:50 +0000)]
[lit] Raise the default soft process limit when possible
It is common to have a default soft process limit, at least on some families of
Linux distributions, of 1024. This is normally more than enough, but if you
have many cores, and you're running tests that create many threads, this can
become a problem. My POWER7 development machine has 48 cores, and when running
the lld regression tests, which often want to create up to 48 threads, I run
into problems. lit, by default, will want to run 48 tests in parallel, and
48*48 < 1024, and so many tests fail like this:
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
or lit fails like this when launching a test:
OSError: [Errno 11] Resource temporarily unavailable
lit can easily detect this situation and attempt to repair it before launching
tests (by raising the soft process limit to something that will allow ncpus^2
threads to be created), and should do so to prevent spurious test failures.
This is the follow-up to this thread:
http://lists.llvm.org/pipermail/llvm-dev/2015-October/090942.html
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249161
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Fri, 2 Oct 2015 16:42:59 +0000 (16:42 +0000)]
[ARM] Typo. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249153
91177308-0d34-0410-b5e6-
96231b3b80d8
Andrea Di Biagio [Fri, 2 Oct 2015 16:08:05 +0000 (16:08 +0000)]
Reapply r249121 : "[FastISel][x86] Teach how to select SSE2/AVX bitcasts between 128/256-bit vector types."
This patch teaches FastIsel the following two things:
1) On SSE2, no instructions are needed for bitcasts between 128-bit vector types;
2) On AVX, no instructions are needed for bitcasts between 256-bit vector types.
Example:
%1 = bitcast <4 x i31> %V to <2 x i64>
Before (-fast-isel -fast-isel-abort=1):
FastIsel miss: %1 = bitcast <4 x i31> %V to <2 x i64>
Now we don't fall back to SelectionDAG and we correctly fold that computation
propagating the register associated to %V.
Originally reviewed here: http://reviews.llvm.org/D13347
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249147
91177308-0d34-0410-b5e6-
96231b3b80d8
Andrea Di Biagio [Fri, 2 Oct 2015 13:06:19 +0000 (13:06 +0000)]
Revert: [FastISel][x86] Teach how to select SSE2/AVX bitcasts between 128/256-bit vector types.
r249121 caused a Clang test failure (avx2-buitins.c).
Revert r249121 while I keep investigating on the reason why that test failed.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249124
91177308-0d34-0410-b5e6-
96231b3b80d8
Zoran Jovanovic [Fri, 2 Oct 2015 13:06:02 +0000 (13:06 +0000)]
[mips][microMIPS] Fix an issue with selecting sqrt instruction in LLVM backend
Differential Revision: http://reviews.llvm.org/D13235
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249123
91177308-0d34-0410-b5e6-
96231b3b80d8
Andrea Di Biagio [Fri, 2 Oct 2015 12:45:37 +0000 (12:45 +0000)]
[FastISel][x86] Teach how to select SSE2/AVX bitcasts between 128/256-bit vector types.
This patch teaches FastIsel the following two things:
1) On SSE2, no instructions are needed for bitcasts between 128-bit vector types;
2) On AVX, no instructions are needed for bitcasts between 256-bit vector types.
Example:
%1 = bitcast <4 x i31> %V to <2 x i64>
Before (-fast-isel -fast-isel-abort=1):
FastIsel miss: %1 = bitcast <4 x i31> %V to <2 x i64>
Now we don't fall back to SelectionDAG and we correctly fold that computation
propagating the register associated to %V.
Differential Revision: http://reviews.llvm.org/D13347
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249121
91177308-0d34-0410-b5e6-
96231b3b80d8
Richard Smith [Fri, 2 Oct 2015 00:46:33 +0000 (00:46 +0000)]
DenseMap: we're trying to call the reserved global placement allocation
function here; use "::new" to avoid accidentally picking up a class-specific
operator new.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249112
91177308-0d34-0410-b5e6-
96231b3b80d8
Adrian Prantl [Fri, 2 Oct 2015 00:27:08 +0000 (00:27 +0000)]
dsymutil: Also ignore the ByteSize when building the DeclContext cache for
clang modules.
Forward decls of ObjC interfaces don't have a bytesize.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249110
91177308-0d34-0410-b5e6-
96231b3b80d8
Ivan Krasin [Thu, 1 Oct 2015 23:23:06 +0000 (23:23 +0000)]
[LibFuzzer] test_single_input option to run a single test case.
-test_single_input flag specifies a file name with test data.
Review URL: http://reviews.llvm.org/D13359
Patch by Mike Aizatsky!
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249096
91177308-0d34-0410-b5e6-
96231b3b80d8
Bruno Cardoso Lopes [Thu, 1 Oct 2015 22:43:53 +0000 (22:43 +0000)]
[SimplifyLibCalls] Fix instruction misplacement in string/memory libcall optimization
When trying to optimize fortified library functions use the right
location to insert new instructions in order to preserve correct
def-use order.
This fixes an issue where a misplaced instruction definition would
happen to be *after* one of its use after a RAUW, forming invalid IR.
This behavior was introduced by r227250.
Differential Revision: http://reviews.llvm.org/D13301
rdar://problem/
22802369
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249092
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Thu, 1 Oct 2015 22:40:35 +0000 (22:40 +0000)]
AMDGPU: Fix unused variable warning in release build
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249091
91177308-0d34-0410-b5e6-
96231b3b80d8
Colin LeMahieu [Thu, 1 Oct 2015 22:14:05 +0000 (22:14 +0000)]
[Hexagon] XFAILing test while diagnosing backend error.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249088
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Thu, 1 Oct 2015 22:10:03 +0000 (22:10 +0000)]
AMDGPU: Move SIFixSGPRLiveRanges to be a regalloc pass
Replace LiveInterval usage with LiveVariables. LiveIntervals
computes far more information than is needed for this pass
which just needs to find if an SGPR is live out of the
defining block.
LiveIntervals are not usually available that early, requiring
computing them twice which is very expensive. The extra run of
LiveIntervals/LiveVariables/SlotIndexes was costing in total
about 5% of compile time.
Continuing to use LiveIntervals is problematic. It seems
there is an option (early-live-intervals) to run the analysis
about where it should go to avoid recomputing LiveVariables,
but it seems to be completely broken with subreg liveness
enabled. There are also problems from trying to recompute
LiveIntervals since this seems to undo LiveVariables
and clearing kill flags, causing TwoAddressInstructions
to make bad decisions.
Insert the pass right after live variables and preserve it.
The tricky case to worry about might be phis since
LiveVariables doesn't count a register as live out if
in the successor block it is only used in a phi,
but I don't think this is a concern right now
because SIFixSGPRCopies replaces SGPR phis.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249087
91177308-0d34-0410-b5e6-
96231b3b80d8
Joerg Sonnenberger [Thu, 1 Oct 2015 22:08:20 +0000 (22:08 +0000)]
Fix relocation used for GOT references in non-PIC mode. Fix relocations
for "set" pseudo op in PIC mode.
Differential Revision: http://reviews.llvm.org/D13173
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249086
91177308-0d34-0410-b5e6-
96231b3b80d8
Davide Italiano [Thu, 1 Oct 2015 21:57:09 +0000 (21:57 +0000)]
[PATCH] D13360: [llvm-objdump] Teach -d about AArch64 mapping symbols
AArch64 uses $d* and $x* to interleave between text and data.
llvm-objdump didn't know about this so it ended up printing garbage.
This patch is a first step towards a solution of the problem.
Differential Revision: http://reviews.llvm.org/D13360
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249083
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Thu, 1 Oct 2015 21:51:59 +0000 (21:51 +0000)]
AMDGPU: Merge if and switch
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249082
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Thu, 1 Oct 2015 21:51:57 +0000 (21:51 +0000)]
AMDGPU: Remove dead code
There's no point in checking VReg_1 because all uses
of it should already have been removed by SILowerI1Copies.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249081
91177308-0d34-0410-b5e6-
96231b3b80d8
Matt Arsenault [Thu, 1 Oct 2015 21:43:15 +0000 (21:43 +0000)]
AMDGPU: Make SIInsertWaits about a factor of 4 faster
This was the slowest target custom pass and was spending 80%
of the time in getMinimalPhysRegClass which was called
for every register operand.
Try to use the statically known register class when possible from
the instruction's MCOperandInfo. There are a few pseudo instructions
which are not well behaved with unknown register classes which still
require the expensive physical register class search.
There are a few other possibilities for making this even faster,
such as not inspecting implicit operands. For now those are checked
because it is technically possible to have a scalar load into
exec or vcc which can be implicitly used.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249079
91177308-0d34-0410-b5e6-
96231b3b80d8
Reid Kleckner [Thu, 1 Oct 2015 21:38:24 +0000 (21:38 +0000)]
[WinEH] Emit __C_specific_handler tables for the new IR
We emit denormalized tables, where every range of invokes in the same
state gets a complete list of EH action entries. This is significantly
simpler than trying to infer the correct nested scoping structure from
the MI. Fortunately, for SEH, the nesting structure is really just a
size optimization.
With this, some basic __try / __except examples work.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249078
91177308-0d34-0410-b5e6-
96231b3b80d8
Colin LeMahieu [Thu, 1 Oct 2015 21:19:03 +0000 (21:19 +0000)]
[Hexagon] XFAILing test while diagnosing backend error.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249075
91177308-0d34-0410-b5e6-
96231b3b80d8
Tom Stellard [Thu, 1 Oct 2015 21:16:05 +0000 (21:16 +0000)]
AMDGPU/SI: Remove assert from AMDGPUOpenCLImageTypeLowering pass
Summary:
Instead of asserting when the kernel metadata is different than we expect,
we should just skip lowering that function. This fixes assertion
failures with OpenCL argument metadata from older LLVM releases.
Reviewers: arsenm
Subscribers: arsenm, llvm-commits
Differential Revision: http://reviews.llvm.org/D13356
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249073
91177308-0d34-0410-b5e6-
96231b3b80d8
David Majnemer [Thu, 1 Oct 2015 21:04:13 +0000 (21:04 +0000)]
[WinEH] Stop BranchFolding from merging across funclets
BranchFolding would merge two funclets together, this is not OK.
Disable this and strengthen the assertion in FuncletLayout.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249069
91177308-0d34-0410-b5e6-
96231b3b80d8
Jonathan Roelofs [Thu, 1 Oct 2015 20:53:59 +0000 (20:53 +0000)]
Kill another reference to in-source builds
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249067
91177308-0d34-0410-b5e6-
96231b3b80d8
David Majnemer [Thu, 1 Oct 2015 18:44:59 +0000 (18:44 +0000)]
[WinEH] Make FuncletLayout more robust against catchret
Catchret transfers control from a catch funclet to an earlier funclet.
However, it is not completely clear which funclet the catchret target is
part of. Make this clear by stapling the catchret target's funclet
membership onto the CATCHRET SDAG node.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249052
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Thu, 1 Oct 2015 18:17:12 +0000 (18:17 +0000)]
[AArch64] Deprecate a command-line option used for testing.
Support for pairing unscaled loads and stores has been enabled since the
original ARM64 port. This feature is no longer experimental, AFAICT.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249049
91177308-0d34-0410-b5e6-
96231b3b80d8
Jonas Paulsson [Thu, 1 Oct 2015 18:12:28 +0000 (18:12 +0000)]
[SystemZ] Add some generic (floating point support) load instructions.
Add generic instructions for load complement, load negative and load positive
for fp32 and fp64, and let isel prefer them. They do not clobber CC, and so
give scheduler more freedom. SystemZElimCompare pass will convert them when it
can to the CC-setting variants.
Regression tests updated to expect the new opcodes in places where the old ones
where used. New test case SystemZ/fp-cmp-05.ll checks that
SystemZCompareElim.cpp can handle the new opcodes.
README.txt updated (bullet removed).
Note that fp128 is not yet handled, because it is relatively rare, and is a
bit trickier, because of the fact that l.dfr would operate on the sign bit of
one of the subregisters of a fp128, but we would not want to copy the other
sub-reg in case src and dst regs are not the same.
Reviewed by Ulrich Weigand.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249046
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Thu, 1 Oct 2015 17:57:31 +0000 (17:57 +0000)]
Fix printing of 64 bit values and make test more strict.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249043
91177308-0d34-0410-b5e6-
96231b3b80d8
Tom Stellard [Thu, 1 Oct 2015 17:51:34 +0000 (17:51 +0000)]
AMDGPU: Add MEM_RAT STORE_TYPED.
v2: Add test (Matt).
Fix capitalization of isEOP (Matt).
Move pattern to class parameter (Matt).
Make the instruction available to Cayman (Matt).
Change name from MEM_RAT WRITE_TYPED to MEM_RAT STORE_TYPED.
Patch by: Zoltan Gilian
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249042
91177308-0d34-0410-b5e6-
96231b3b80d8
Tom Stellard [Thu, 1 Oct 2015 17:51:29 +0000 (17:51 +0000)]
AMDGPU: Factor out EOP query.
v2: Fix brace placement and capitalization (Matt).
Patch by: Zoltan Gilian
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249041
91177308-0d34-0410-b5e6-
96231b3b80d8
NAKAMURA Takumi [Thu, 1 Oct 2015 17:01:03 +0000 (17:01 +0000)]
Reformat.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249033
91177308-0d34-0410-b5e6-
96231b3b80d8
NAKAMURA Takumi [Thu, 1 Oct 2015 17:00:56 +0000 (17:00 +0000)]
Revert r248959, "[WinEH] Emit int3 after noreturn calls on Win64"
It broke; LLVM :: CodeGen__Generic__2009-11-16-BadKillsCrash.ll
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249032
91177308-0d34-0410-b5e6-
96231b3b80d8
Rafael Espindola [Thu, 1 Oct 2015 15:22:42 +0000 (15:22 +0000)]
Use more strict types. NFC.
On 32 bit ELF these are 32 bit values.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249022
91177308-0d34-0410-b5e6-
96231b3b80d8
Arnaud A. de Grandmaison [Thu, 1 Oct 2015 14:54:31 +0000 (14:54 +0000)]
[InstCombine] Remove trivially empty lifetime start/end ranges.
Summary:
Some passes may open up opportunities for optimizations, leaving empty
lifetime start/end ranges. For example, with the following code:
void foo(char *, char *);
void bar(int Size, bool flag) {
for (int i = 0; i < Size; ++i) {
char text[1];
char buff[1];
if (flag)
foo(text, buff); // BBFoo
}
}
the loop unswitch pass will create 2 versions of the loop, one with
flag==true, and the other one with flag==false, but always leaving
the BBFoo basic block, with lifetime ranges covering the scope of the for
loop. Simplify CFG will then remove BBFoo in the case where flag==false,
but will leave the lifetime markers.
This patch teaches InstCombine to remove trivially empty lifetime marker
ranges, that is ranges ending right after they were started (ignoring
debug info or other lifetime markers in the range).
This fixes PR24598: excessive compile time after r234581.
Reviewers: reames, chandlerc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D13305
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249018
91177308-0d34-0410-b5e6-
96231b3b80d8
Ulrich Weigand [Thu, 1 Oct 2015 14:43:48 +0000 (14:43 +0000)]
[SystemZ] Add assembly instructions for obtaining clock values as well as CPU features
Provide assembler support for STCK, STCKF, STCKE, and STFLE.
Author: joncmu
Differential Revision: http://reviews.llvm.org/D13299
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249015
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Thu, 1 Oct 2015 13:43:05 +0000 (13:43 +0000)]
[AArch64] Hoist commonly failing check. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249011
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Thu, 1 Oct 2015 13:33:31 +0000 (13:33 +0000)]
[AArch64] Rename variable to improve readability. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249008
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Thu, 1 Oct 2015 13:09:44 +0000 (13:09 +0000)]
[AArch64] Update comment to reflect reality.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249007
91177308-0d34-0410-b5e6-
96231b3b80d8
Zoran Jovanovic [Thu, 1 Oct 2015 12:49:27 +0000 (12:49 +0000)]
[mips][microMIPS] Implement CACHEE, WRPGPR and WSBH instructions
Differential Revision: http://reviews.llvm.org/D10337
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249004
91177308-0d34-0410-b5e6-
96231b3b80d8
Scott Douglass [Thu, 1 Oct 2015 11:56:19 +0000 (11:56 +0000)]
[ARM] More care with Thumb1 writeback in ARMLoadStoreOptimizer
Differential Revision: http://reviews.llvm.org/D13240
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@249002
91177308-0d34-0410-b5e6-
96231b3b80d8
Jingyue Wu [Thu, 1 Oct 2015 03:51:44 +0000 (03:51 +0000)]
[NaryReassociate] SeenExprs records WeakVH
Summary:
The instructions SeenExprs records may be deleted during rewriting.
FindClosestMatchingDominator should ignore these deleted instructions.
Fixes PR24301.
Reviewers: grosser
Subscribers: grosser, llvm-commits
Differential Revision: http://reviews.llvm.org/D13315
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248983
91177308-0d34-0410-b5e6-
96231b3b80d8
Keno Fischer [Thu, 1 Oct 2015 02:45:07 +0000 (02:45 +0000)]
Fix performance problem in long-running SectionMemoryManagers
Summary:
Without this patch, the memory manager would call `mprotect` on every memory
region it ever allocated whenever it wanted to finalize memory (i.e. not just
the ones it just allocated). This caused terrible performance problems for
long running memory managers. In one particular compile heavy julia benchmark,
we were spending 50% of time in `mprotect` if running under MCJIT.
Fix this by splitting allocated memory blocks into those on which memory
permissions have been set and those on which they haven't and only running
`mprotect` on the latter.
Reviewers: lhames
Subscribers: reames, llvm-commits
Differential Revision: http://reviews.llvm.org/D13156
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248981
91177308-0d34-0410-b5e6-
96231b3b80d8
Tom Stellard [Thu, 1 Oct 2015 02:02:46 +0000 (02:02 +0000)]
AMDGPU/SI: Re-order PreloadedValue enum and number entries based on init order
Reviewers: arsenm
Subscribers: arsenm, llvm-commits
Differential Revision: http://reviews.llvm.org/D12451
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248978
91177308-0d34-0410-b5e6-
96231b3b80d8
Davide Italiano [Thu, 1 Oct 2015 01:02:37 +0000 (01:02 +0000)]
[llvm-objdump] Fix time of check to time of use bug.
There's already a test that covers this situation, so we should be
fine.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248976
91177308-0d34-0410-b5e6-
96231b3b80d8
David Blaikie [Thu, 1 Oct 2015 00:44:21 +0000 (00:44 +0000)]
Revert "Enable -Wdeprecated in the cmake build now that LLVM (& Clang, Polly, and LLD) are -Wdeprecated clean"
This reverts commit r248963.
Seems there's some standard libraries (and libcxxabi implementations)
that aren't -Wdeprecated clean... hrm.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248972
91177308-0d34-0410-b5e6-
96231b3b80d8
Dehao Chen [Thu, 1 Oct 2015 00:26:56 +0000 (00:26 +0000)]
Update sample profile propagation algorithm.
http://reviews.llvm.org/D13218
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248968
91177308-0d34-0410-b5e6-
96231b3b80d8
Ahmed Bougacha [Thu, 1 Oct 2015 00:11:07 +0000 (00:11 +0000)]
[X86] Don't custom-lower vNi32 uint_to_fp when unsafe-fp-math.
The custom code produces incorrect results if later reassociated.
Since r221657, on x86, vNi32 uitofp is lowered using an optimized
sequence:
movdqa LCPI0_0(%rip), %xmm1 ## xmm1 = [65535, ...]
pand %xmm0, %xmm1
por LCPI0_1(%rip), %xmm1 ## [0x4b000000, ...]
psrld $16, %xmm0
por LCPI0_2(%rip), %xmm0 ## [0x53000000, ...]
addps LCPI0_3(%rip), %xmm0 ## [float -5.
497642e+11, ...]
addps %xmm1, %xmm0
Since r240361, the machine combiner opportunistically reassociates
2-instruction sequences (with -ffast-math). In the new code sequence,
the ADDPS' are eligible. In isolation, for simple examples (without
reassociable users), this makes no performance difference (the goal
being to enable reassociation of longer chains).
In the trivial example (just one uitofp), the reassociation doesn't
happen, because (I think) it would require the emission of a separate
movaps for a constantpool load (instead of folding it into addps).
However, when we have multiple uitofp sequences, and the constantpool
loads are CSE'd earlier, the machine combiner can do the reassociation.
When the ADDPS' are reassociated, the resulting sequence isn't correct
anymore, as we'd be adding large (2**39) constants with comparatively
smaller values (~2**23). Given that two of the three inputs are powers
of 2 larger than 2**16, and that ulp(2**39) == 2**(39-24) == 2**15,
the reassociated chain will produce 0 for any input in [0, 2**14[.
In my testing, it also produces wrong results for 99.5% of [0, 2**32[.
Avoid this by disabling the new lowering when -ffast-math. It does
mean that we'll get slower code than without it, but at least we
won't get egregiously incorrect code.
One might argue that, considering -ffast-math is all but meaningless,
uitofp producing wrong results isn't a compiler bug. But it really is.
Fixes PR24512.
...though this is really more of a workaround.
Ideally, we'd have some sort of Machine FMF, but that's a problem
that's not worth tackling until we do more with machine IR.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248965
91177308-0d34-0410-b5e6-
96231b3b80d8
David Blaikie [Wed, 30 Sep 2015 23:36:12 +0000 (23:36 +0000)]
Enable -Wdeprecated in the cmake build now that LLVM (& Clang, Polly, and LLD) are -Wdeprecated clean
This particularly helps enforce the C++ Rule of 5 (for new move ops this
is already an error, but for a type only using C++98 features (copy
ctor/assign, dtor) it is only deprecated, not invalid)
Applying the flag for any GCC compatible compiler - GCC doesn't warn on
the Rule of 5 cases that C++11 deprecates, but it doesn't have other
false positives so far as I could see (compiling with GCC 4.8 didn't
produce any -Wdeprecated warnings I could spot).
Reviewers: aaron.ballman
Differential Revision: http://reviews.llvm.org/D13314
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248963
91177308-0d34-0410-b5e6-
96231b3b80d8
Reid Kleckner [Wed, 30 Sep 2015 23:09:23 +0000 (23:09 +0000)]
[WinEH] Emit int3 after noreturn calls on Win64
The Win64 unwinder disassembles forwards from each PC to try to
determine if this PC is in an epilogue. If so, it skips calling the EH
personality function for that frame. Typically, this means you cannot
catch an exception in the same frame that you threw it, because 'throw'
calls a noreturn runtime function.
Previously we avoided this problem with the TrapUnreachable
TargetOption, but that's a much bigger hammer than we need. All we need
is a 1 byte non-epilogue instruction right after the call. Instead,
what we got was an unconditional branch to a shared block containing the
ud2, potentially 7 bytes instead of 1. So, this reverts r206684, which
added TrapUnreachable, and replaces it with something better.
The new code pattern matches for invoke/call followed by unreachable and
inserts an int3 into the DAG. To be 100% watertight, we would need to
insert SEH_Epilogue instructions into all basic blocks ending in a call
with no terminators or successors, but in practice this is unlikely to
come up.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248959
91177308-0d34-0410-b5e6-
96231b3b80d8
Hal Finkel [Wed, 30 Sep 2015 22:34:35 +0000 (22:34 +0000)]
[PowerPC] undef Relocation names in PowerPC*.def
glibc's PowerPC /usr/include/asm/sigcontext.h, has this:
#ifdef __powerpc64__
#include <asm/elf.h>
#endif
and that contains defines of all of the relocation symbols, like this:
#define R_PPC_NONE 0
and if that file is included prior to including
include/llvm/Support/ELFRelocs/PowerPC*.def, which we cannot in general
prevent, the result will fail.
As it turns out, this happens when compiling
lld/unittests/DriverTests/GnuLdDriverTest.cpp under PPC64/Linux, because:
lld/include/lld/ReaderWriter/ELFLinkingContext.h includes
lld/unittests/DriverTests/DriverTest.h which includes
utils/unittest/googletest/include/gtest/gtest.h which includes
utils/unittest/googletest/include/gtest/internal/gtest-internal.h which includes
/usr/include/sys/wait.h which includes
/usr/include/signal.h which includes
/usr/include/bits/sigcontext.h which includes
/usr/include/asm/sigcontext.h which includes
/usr/include/asm/elf.h
the test could be fixed to include ReaderWriter/ELFLinkingContext.h before
including unittests/DriverTests/DriverTest.h, but dealing with this in the
*.def files is a more-general solution that localizes the fix to the headers
instead of requiring changes to an unbounded number of other source files (both
in-tree and external).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248957
91177308-0d34-0410-b5e6-
96231b3b80d8
Sanjay Patel [Wed, 30 Sep 2015 22:25:55 +0000 (22:25 +0000)]
[x86] enable machine combiner reassociations for 256-bit vector logical integer insts
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248955
91177308-0d34-0410-b5e6-
96231b3b80d8
Kostya Serebryany [Wed, 30 Sep 2015 22:22:37 +0000 (22:22 +0000)]
[libFuzzer] Marking exported symbols as visible. Patch by Mike Aizatsky
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248954
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Wed, 30 Sep 2015 21:10:02 +0000 (21:10 +0000)]
[AArch64] Remove an unnecessary run line and other cleanup. NFC.
Unscaled load/store combining has been enabled since the initial ARM64 port. No
need for a redundance run. Also, add CHECK-LABEL directives.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248945
91177308-0d34-0410-b5e6-
96231b3b80d8
Michael Zolotukhin [Wed, 30 Sep 2015 21:05:43 +0000 (21:05 +0000)]
[SLP] Don't vectorize loads of non-packed types (like i1, i2).
Summary:
Given an array of i2 elements, 4 consecutive scalar loads will be lowered to
i8-sized loads and thus will access 4 consecutive bytes in memory. If we
vectorize these loads into a single <4 x i2> load, it'll access only 1 byte in
memory. Hence, we should prohibit vectorization in such cases.
PS: Initial patch was proposed by Arnold.
Reviewers: aschwaighofer, nadav, hfinkel
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D13277
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248943
91177308-0d34-0410-b5e6-
96231b3b80d8
David Blaikie [Wed, 30 Sep 2015 20:37:48 +0000 (20:37 +0000)]
Fix -Wsign-compare warning
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248942
91177308-0d34-0410-b5e6-
96231b3b80d8
Evgeniy Stepanov [Wed, 30 Sep 2015 20:23:24 +0000 (20:23 +0000)]
Move dw_op_minus test to DebugInfo/X86.
The test requires X86 target support, and checks the actual debug
info contents, including register numbers which would be different on
other platforms.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248938
91177308-0d34-0410-b5e6-
96231b3b80d8
Evgeniy Stepanov [Wed, 30 Sep 2015 19:55:43 +0000 (19:55 +0000)]
Fix debug info with SafeStack.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248933
91177308-0d34-0410-b5e6-
96231b3b80d8
Chad Rosier [Wed, 30 Sep 2015 19:44:40 +0000 (19:44 +0000)]
[AArch64] Remove an unnecessary restriction on pre-index instructions.
Previously, the index was constrained to the size of the memory operation for
no apparent reason. This change removes that constraint so that we can form
pre-index instructions with any valid offset.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248931
91177308-0d34-0410-b5e6-
96231b3b80d8
Fiona Glaser [Wed, 30 Sep 2015 17:49:49 +0000 (17:49 +0000)]
DeadCodeElimination: rewrite to be faster
Same strategy as simplifyInstructionsInBlock. ~1/3 less time
on my test suite. This pass doesn't have many in-tree users,
but getting rid of an O(N^2) worst case and making it cleaner
should at least make it a viable alternative to ADCE, since
it's now consistently somewhat faster.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@248927
91177308-0d34-0410-b5e6-
96231b3b80d8