The sendfile() facility allows a regular file to be sent out to a stream socket. The system call was first implemented in FreeBSD 3.0. This provides many performance benefits for various server appliances. Andre Opperman has implemented an improved sendfile() facility for FreeBSD that has so far shown 45% less CPU usage without TCP segmentation offload and 83% less CPU usage with TCP segmentation offload. This is a great improvement over the previous implementation.
From: Andre Oppermann [email blocked] To: freebsd-current, freebsd-net The recent addition of TSO (TCP Segmentation Offload) has highlighted some shortcommings in our sendfile(2) kernel implementation. The current code simply loops over the file, turns each 4K page into an mbuf and sends it off. This has the effect that TSO can only generate 2 packets per send instead of up to 44 at its maximum of 64K. I have rewritten kern_sendfile() to work in two loops, the inner which turns as many pages into mbufs as it can up to the free send socket buffer space. The outer loop then drops the whole mbuf chain into the send socket buffer, calls tcp_output() on it and then waits until 50% of the socket buffer are free again to repeat the cycle. This way tcp_output() gets the full amount of data to work with and can issue up to 64K sends for TSO to chop up in the network adapter without using any CPU cycles. Thus it gets very efficient especially with the readahead the VM and I/O system do. Looking at the benchmarks we see some very nice improvements (95% confidence): 45% less cpu (or 1.81 times better) with new sendfile vs. old sendfile (non-TSO) 83% less cpu (or 5.7 times better) with new sendfile vs. old sendfile (TSO) The sender is an AMD Opteron 852 (2.6GHz) with em(4) PCI-X-133 interface and the receiver is a DELL Poweredge SC1425 P-IV Xeon 3.2GHz with em(4) LOM connected back to back at 1000Base-TX full duplex. The patch is available here: http://people.freebsd.org/~andre/sendfile-20060920.diff Any testing and heavy (code) beating and reviews welcome.