On Tue, Jun 15, 2010 at 2:16 PM, LeviaComm Networks NOC
<NOC@leviacomm.net> wrote:
quoted text
>
> OpenBSD may be running the network in promiscuous mode, which would be why
> it is responding to MACs that it shouldn't.  If you aren't running a clean
> installation, I would recommend turning off everything except routed,
> including sshd and just use the console for now.  I would also recommend
> removing all your VLAN interfaces and reconnect only 2 then test with that,
> slowly adding VLANs back as they work.

Thanks for the suggestion there, I've managed to narrow it down. I've
connected the OpenBSD box to a 100Mb hub, which is uplinked to the
core switch I mentioned before, and connected two laptops to the hub
so I can establish TCP connections between them. The OpenBSD box now
reproduces the bug perfectly every time - every TCP connection between
the test machines gets as far as the first SYN before receiving a
spoofed RST from OpenBSD [1].

To cut a long story short, I eliminated the issue down to depend on
the following conditions:
- A trunk configured with one or more promiscuous slave ports
- Trunk must not be in promiscuous mode
- The line 'set block-policy return' in pf.conf

Had a poke around in sys/net, and the problem appears to be in
ether_input (if_ethersubr.c:530). The check at line 687 looks to see
if the interface is in promisc mode before it does the destination
check, but packets from a trunk interface get their received interface
changed from the physical interface to the trunk around line 559.
Thus, if the child interface is promisc and the trunk interface is
not, packets not destined for the local machine will be passed into
pf's filtering routines.

The attached patch [2] fixes the issue for me. Not knowing the OpenBSD
network stack, I've tried to fix it in the least intrusive way
possible, which may not be the best solution. Let me know if it'll
break anything, or if I should be submitting the patch elsewhere.

Cheers,

Patrick
--
http://www.labyrinthdata.net.au - WA Backup, Web and VPS Hosting


[1] Two examples of the problem here. 10.10.50.1 sends a SYN to
10.10.50.2, but OpenBSD (10.10.0.2/00:04:23:c9:bd:d0) spoofs a RST
response from 10.10.50.2 and closes the connection.

11:41:39.202711 00:25:00:a0:2a:6e > 00:16:cb:d1:5b:fc, ethertype IPv4
(0x0800), length 78: 10.10.50.1.51743 > 10.10.50.2.5555: Flags [S], seq
4101611714, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val
844767319 ecr 0,sackOK,eol], length 0
11:41:39.203219 00:04:23:c9:bd:d0 > 00:25:00:a0:2a:6e, ethertype IPv4
(0x0800), length 60: 10.10.50.2.5555 > 10.10.50.1.51743: Flags [R.], seq 0,
ack 4101611715, win 0, length 0
11:41:39.411521 00:25:00:a0:2a:6e > 00:16:cb:d1:5b:fc, ethertype IPv4
(0x0800), length 78: 10.10.50.1.51744 > 10.10.50.2.5555: Flags [S], seq
2093396000, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val
844767321 ecr 0,sackOK,eol], length 0
11:41:39.411987 00:04:23:c9:bd:d0 > 00:25:00:a0:2a:6e, ethertype IPv4
(0x0800), length 60: 10.10.50.2.5555 > 10.10.50.1.51744: Flags [R.], seq 0,
ack 2093396001, win 0, length 0

[2] Also up at http://patrick.ld.net.au/20100616-fix-gratuitous-reset.patch,
for less tab clobberyness.

Index: if_ethersubr.c
===================================================================
RCS file: /cvs/src/sys/net/if_ethersubr.c,v
retrieving revision 1.139
diff if_ethersubr.c
540a541
quoted text
>       struct ifnet *ifp_orig = ifp0;
568c569,570
<               /* Has been set to the trunk interface */
---
quoted text
>               /* Has been set to the trunk interface. ifp_orig preserves the
physical
quoted text
>                * input interface. */
685a688,690
quoted text
>        *
>        * Check ifp and ifp_orig, to catch promiscuous members of a
>        * non-promiscuous trunk.
688c693,695
<           (ifp->if_flags & IFF_PROMISC)) {
---
quoted text
>           ((ifp->if_flags & IFF_PROMISC) ||
>                (ifp_orig->if_flags & IFF_PROMISC))) {