I briefly checked solaris networking. Basically, they use STREAMS to implement the networking stack. Basically, a stream head, living in kernel space, is used to communicate with user space processes, providing system call interface. A STREAMS driver is used to communicate with the actual physical device. In between, multiple stream modules can be added and connected to process and deliver the data. So you can think of a STREAMS driver to be the network device driver that reads raw packets. Each protocol, like IP and TCP, is implemented as a STREAMS module. All these head, driver and modules are bi-directional.

As of Solaris 10, they coalesced all intermediate modules (IP, TCP...), and only used one huge module in between --- this is very much like linux... The reason is that "This (previous mult-module approach) can, and often does, lead to a packet being processed on more than one CPU and by more than one thread leading to excessive context switching and poor CPU data locality."

My question is, if you allocate a buffer to hold a packet (like skb in linux) and pass around the pointer to that buffer across modules, how come a packet be processed by more than one CPU? The only situation I can think of is that a packet is multiplexed to be processed in multiple modules. But in this case, the buffer should be cloned anyway...

Thanks.
-S