xprtrdma: Re-write rpcrdma_flush_cqs()
authorChuck Lever <chuck.lever@oracle.com>
Sun, 9 Nov 2014 01:14:45 +0000 (20:14 -0500)
committerAnna Schumaker <Anna.Schumaker@Netapp.com>
Tue, 25 Nov 2014 18:39:20 +0000 (13:39 -0500)
commit5c166bef4fa138da45d756dc77a8cb29fced1714
tree82c3e8832fdfa1ab5f9193d14146466f49ccc4e5
parentf1a03b76fecdb23983e2ad7e817e98d093ece9f7
xprtrdma: Re-write rpcrdma_flush_cqs()

Currently rpcrdma_flush_cqs() attempts to avoid code duplication,
and simply invokes rpcrdma_recvcq_upcall and rpcrdma_sendcq_upcall.

1. rpcrdma_flush_cqs() can run concurrently with provider upcalls.
   Both flush_cqs() and the upcalls were invoking ib_poll_cq() in
   different threads using the same wc buffers (ep->rep_recv_wcs
   and ep->rep_send_wcs), added by commit 1c00dd077654 ("xprtrmda:
   Reduce calls to ib_poll_cq() in completion handlers").

   During transport disconnect processing, this sometimes resulted
   in the same reply getting added to the rpcrdma_tasklets_g list
   more than once, which corrupted the list.

2. The upcall functions drain only a limited number of CQEs,
   thanks to the poll budget added by commit 8301a2c047cc
   ("xprtrdma: Limit work done by completion handler").

Fixes: a7bc211ac926 ("xprtrdma: On disconnect, don't ignore ... ")
BugLink: https://bugzilla.linux-nfs.org/show_bug.cgi?id=276
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
net/sunrpc/xprtrdma/verbs.c