rcu: Make expedited GP CPU stoppage asynchronous
authorPeter Zijlstra <peterz@infradead.org>
Thu, 25 Jun 2015 18:27:10 +0000 (11:27 -0700)
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Fri, 17 Jul 2015 21:58:50 +0000 (14:58 -0700)
commit3a6d7c64d78a78d279851524d39999637a549363
treec45c280d3805f73b028352a4b4d9b70663a513fd
parent385b73c06f6a733547d0a7714d0c4cb4c8788b88
rcu: Make expedited GP CPU stoppage asynchronous

Sequentially stopping the CPUs slows down expedited grace periods by
at least a factor of two, based on rcutorture's grace-period-per-second
rate.  This is a conservative measure because rcutorture uses unusually
long RCU read-side critical sections and because rcutorture periodically
quiesces the system in order to test RCU's ability to ramp down to and
up from the idle state.  This commit therefore replaces the stop_one_cpu()
with stop_one_cpu_nowait(), using an atomic-counter scheme to determine
when all CPUs have passed through the stopped state.

Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
kernel/rcu/tree.c
kernel/rcu/tree.h
kernel/rcu/tree_trace.c