login
Search
Hi, As discussed on misc@, I've come across a bug in OpenBSD's handling of gratuitous Ethernet traffic, and it was suggested that I move the discussion over here. What I've found is that with certain configurations, an OpenBSD box receiving Ethernet traffic with a destination MAC address that is not its own will respond to that traffic with spoofed TCP RST packet. That is, if I try and initiate a TCP connection from machine A to machine B across a hub, with an OpenBSD machine connected to the same hub, the OpenBSD machine will spoof a TCP reset packet from B to A and kill the connection attempt. See [1] for sample tcpdump output. The following conditions are required for this to happen: - A trunk configured with one or more promiscuous slave interfaces - One or more of these slave interfaces should be receiving gratuitous unicast Ethernet frames, as for a non-switched network - The trunk must not be in promiscuous mode (meaning running tcpdump to analyze the problem makes it go away! :) - pf enabled, and the line 'set block-policy return' in pf.conf I've tracked the issue down to ether_input() (sys/net/if_ethersubr.c:530). The check at line 687 looks to see if the interface is in promisc mode before it does the destination check, but packets from a trunk interface get their received interface changed from the physical interface to the trunk around line 559. Thus, if the child interface is promisc and the trunk interface is not, packets not destined for the local machine will be passed into pf's filtering routines, with the above results. The attached patch [2] fixes the issue for me. Not knowing the OpenBSD network stack, I've tried to fix it in the least intrusive way possible, which may not be the best solution. Feedback appreciated. Cheers, Patrick -- http://www.labyrinthdata.net.au - WA Backup, Web and VPS Hosting [1] Two examples of the problem here. 10.10.50.1 sends a SYN to 10.10.50.2 (00:16:cb:d1:5b:fc), but OpenBSD (10.10.0.2/00:04:23:c9:bd:d0) ...
trunk(4) places trunkports into promiscuous mode anyway, so would it make more sense just to skip the IFF_PROMISC check here? in which case, there would be no need to it's not mentioned explicitly in style(9) but wrapping to 80 columns is preferred.
On Wed, Jun 16, 2010 at 7:46 PM, Stuart Henderson <stu@spacehopper.org> That makes sense. I wasn't sure if that promiscuous check was there Updated patch at http://patrick.ld.net.au/20100616-fix-gratuitous-reset-nopromisc.patch. I've also created bug 6404/kernel. Cheers, Patrick -- http://www.labyrinthdata.net.au - WA Backup, Web and VPS Hosting
oh, of course, this check isn't only used for trunk... I think we need to wait until Claudio's around for his input and do some testing in various scenarios.
I think something like the attached version may be the best solution.
I don't like to do the bcmp() for every unicast packet since network cards
come with nice mac filters that make this superfluous.
From code inspection bridge(4) should handle this case correctly and
trunk(4) will behave better because of this. Then there is carp(4) and
vlan(4) which reenter ether_input(). Both should do the right thing
(vlan checks the flags and carp does a lookup based on the dest mac addr).
In the end such a change needs a lot of testing.
I would like to have a better interface to program the mac address filters
since some cards are now able to filter on multiple unicast addrs at the
same time. This would allow us to not have to enable promisc mode on
carpdevs.
Another issue are unicast IP packets with multicast mac addrs. (e.g. for
carp active active clusters). We need to somehow filter those as well or
bad things happen.
Index: if_ethersubr.c
===================================================================
RCS file: /cvs/src/sys/net/if_ethersubr.c,v
retrieving revision 1.144
diff -u -p -r1.144 if_ethersubr.c
--- if_ethersubr.c 3 Jun 2010 16:15:00 -0000 1.144
+++ if_ethersubr.c 22 Jun 2010 16:07:00 -0000
@@ -677,7 +677,7 @@ ether_input(ifp0, eh, m)
* is for us. Drop otherwise.
*/
if ((m->m_flags & (M_BCAST|M_MCAST)) == 0 &&
- (ifp->if_flags & IFF_PROMISC)) {
+ (ifp->if_flags & IFF_PROMISC || ifp0->if_flags & IFF_PROMISC)) {
if (bcmp(ac->ac_enaddr, (caddr_t)eh->ether_dhost,
ETHER_ADDR_LEN)) {
m_freem(m);
On Wed, Jun 23, 2010 at 12:57 AM, Claudio Jeker Is there anything specifically you want me to test? I was planning to do some performance testing anyway; the box is not in production and won't be for a few more weeks. Cheers, Patrick -- http://www.labyrinthdata.net.au - WA Backup, Web and VPS Hosting
We need to make sure it does not break something. This diff can affect both bridge(4) and trunk(4) so both setups need to be tested with various setting of promisc interfaces.
I've done some performance testing for various combinations of trunk
and promiscuous mode, below. I can't see any difference in performance
with or without the patch.
Raw data is up at [1] for those interested. I plan to start setting
OpenBSD as our replacement router over the next few days, and it'll
get some real-world testing once I do that. I haven't noticed any odd
behaviour in the time I've been running these performance tests.
### em1
Throughput (Mb/s)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 880.50 883.75 884.40 883.50
Transmit 792.25 796.75 847.67 846.50
CPU (system %)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 56.38 61.62 57.18 58.18
Transmit 88.10 89.60 87.12 89.10
Transmit (400) 93.85 90.10 90.85 90.60
CPU (interrupts %)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 33.25 29.95 32.20 32.45
Transmit 7.40 7.40 7.15 7.40
Transmit (400) 4.25 4.25 4.50 4.75
### trunk1
Throughput (Mb/s)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 883.00 882.50 885.75 880.50
Transmit 781.50 780.50 798.50 799.25
CPU (system %)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 56.90 56.25 58.65 57.53
Transmit 91.35 89.85 90.60 89.10
CPU (interrupts %)
PROMISC PROMISC -PROMISC -PROMISC
patch no patch patch no patch
Receive 32.53 33.40 29.95 32.85
Transmit 7.40 6.65 6.90 7.65
Cheers,
Patrick
--
http://www.labyrinthdata.net.au - WA Backup, Web and VPS ...