On Fri, May 2, 2008 at 3:40 PM, Sepherosa Ziehau
<sephe@crater.dragonflybsd.org> wrote:
quoted text
> sephe       2008/05/02 00:40:32 PDT
>
>  DragonFly src repository
>
>   Modified files:
>     sys/conf             options
>     sys/config           LINT
>     sys/dev/netif/em     Makefile if_em.c
>     sys/net              if_ethersubr.c if_var.h netisr.c netisr.h
>     sys/sys              mbuf.h
>   Log:
>   Introduce ETHER_INPUT_CHAIN option:
>   1) During RXEOF, we aggregate packets, which have same target CPU, instead of
>      calling lwkt_sendmsg() for each input packet.
>   2) At the end of RXEOF, low level ipiq sending is used to dispatch mbuf chain
>      to the target CPU.
>   3) On the target CPU, the ipi function puts mbuf to their belonging msgport.
>      Note, though lwkt_sendmsg() is used in ipi function, no further ipi activity
>      will happen, since we are on target CPU.
>
>   em(4) is made to aware of this option.
>   This option is off by default and has no effect on vlan(4) operation.

Thanks to ktr, I was managed to track down the performance drop when
packets need dispatching.  By enabling this option, I got ~150Kpps
performance boost in non-fastforwarding case under most of situation
(both MP safe and non-MP safe):
after commit: http://leaf.dragonflybsd.org/~sephe/fwd/data20080502.txt
before commit: http://leaf.dragonflybsd.org/~sephe/fwd/data20080501.txt

Under MP safe case, when if_start needs to be called on different
CPUs, there are still some unidentified causes which poison the
overall performance.  Serializer contention is suspected, but in
anycase, ktr will be our friend :)

Best Regards,
sephe

-- 
Live Free or Die