the
[Felix Marti] Ouch, and I believed linux to be a leading edge OS,
scaling from small embedded systems to hundreds of CPUs and hence
I assumed that the same 'scalability' applies to the network subsystem.
[Felix Marti] Aren't you confusing memory and bus BW here? - RDMA
enables DMA from/to application buffers removing the user-to-kernel/
kernel-to-user memory copy with is a significant overhead at the
rates we're talking about: memory copy at 20Gbps (10Gbps in and 10Gbps
out) requires 60Gbps of BW on most common platforms. So, receiving and
transmitting at 10Gbps with LRO and TSO requires 80Gbps of system
memory BW (which is beyond what most systems can do) whereas RDMA can
do with 20Gbps!
In addition, BUS improvements are really not significant (nor are buses
the bottleneck anymore with wide availability of PCI-E >= x8); TSO
avoids
the DMA of a bunch of network headers... a typical example of stateless
offload - improving performance by a few percent while offload
technologies
provide system improvements of hundreds of percent.
I know that you don't agree that TSO has drawbacks, as outlined by
Roland,
but its history showing something else: the addition of TSO took a fair
amount of time and network performance was erratic for multiple kernel
revisions and the TSO code is sprinkled across the network stack. It is
an
example of an intrusive 'improvement' whereas Steve (who started this
thread) is asking for a relatively small change (decoupling the 4-tuple
allocation from the socket). As Steve has outlined, your refusal of the
change requires RDMA users to work around the issue which pushes the
issue to the end-users and thus slowing down the acceptance of the
technology leading to a chicken-and-egg problem: you only care if there
are lots of users but you make it hard to use the technology in the
first
place, clever ;)
http://openib.org/mailman/listinfo/openib-
-
