pktgen: avoid expensive set_current_state() call in loop

Avoid calling set_current_state() inside the busy-loop in
pktgen_thread_worker().  In case of pkt_dev->delay, then it is still
used/enabled in pktgen_xmit() via the spin() call.

The set_current_state(TASK_INTERRUPTIBLE) uses a xchg, which implicit
is LOCK prefixed.  I've measured the asm LOCK operation to take approx
8ns on this E5-2630 CPU.  Performance increase corrolate with this
measurement.

Performance data with CLONE_SKB==100000, rx-usecs=30:
 (single CPU performance, ixgbe 10Gbit/s, E5-2630)
 * Prev:  5454050 pps --> 183.35ns (1/5454050*10^9)
 * Now:   5684009 pps --> 175.93ns (1/5684009*10^9)
 * Diff:  +229959 pps -->  -7.42ns

Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index fc17a9d309ac028fc61ac7db6e35a05c77513a24..b61f553689b6b3ef0ce370fc05d6230ac267d953 100644 (file)
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -3407,10 +3407,10 @@ static int pktgen_thread_worker(void *arg)
 
        pr_debug("starting pktgen/%d:  pid=%d\n", cpu, task_pid_nr(current));
 
-       set_current_state(TASK_INTERRUPTIBLE);
-
        set_freezable();
 
+       __set_current_state(TASK_RUNNING);
+
        while (!kthread_should_stop()) {
                pkt_dev = next_to_run(t);
 
@@ -3424,8 +3424,6 @@ static int pktgen_thread_worker(void *arg)
                        continue;
                }
 
-               __set_current_state(TASK_RUNNING);
-
                if (likely(pkt_dev)) {
                        pktgen_xmit(pkt_dev);
 
@@ -3456,9 +3454,8 @@ static int pktgen_thread_worker(void *arg)
                }
 
                try_to_freeze();
-
-               set_current_state(TASK_INTERRUPTIBLE);
        }
+       set_current_state(TASK_INTERRUPTIBLE);
 
        pr_debug("%s stopping all device\n", t->tsk->comm);
        pktgen_stop(t);