I have the following network (simplified of course):
  +---+ 10.0.3.67/24    +---+ 10.0.0.180/24    +---+

  | A +-----------------+ B +------------------+ C +

  +---+   10.0.3.180/24 +---+     10.0.0.34/24 +---+
A and C are Linux hosts, while B is running OpenBSD 4.2.  B has

net.inet.ip.forwarding=1, and A and C are normally able to

communicate.  B is running pf with a state table of about 200K entries

with a limit of 800K, but none of the rules affect A<->C.
However, occasionally I see bursts of errors where when A tries to

establish a TCP connection to C, that B returns an "ICMP host

unreachable" error.  I see long strings of these even while both A and

B are able to successfully ping C.  E.g., here's a short snippet from

running tcpdump on A just a few minutes ago:
11:18:25.039132 IP 10.0.3.67 > 10.0.0.34: ICMP echo request, id 65358,

seq 10, length 64

11:18:25.039484 IP 10.0.0.34 > 10.0.3.67: ICMP echo reply, id 65358,

seq 10, length 64

11:18:25.110883 IP 10.0.3.180 > 10.0.3.67: ICMP host 10.0.0.34

unreachable, length 36

11:18:26.038676 IP 10.0.3.67 > 10.0.0.34: ICMP echo request, id 65358,

seq 11, length 64

11:18:26.038981 IP 10.0.0.34 > 10.0.3.67: ICMP echo reply, id 65358,

seq 11, length 64
Why would B send these host unreachable errors when the host is

clearly reachable?
Thanks.

It appears you're pushing a lot of traffic through the box--and on a

hunch is net.inet.ip.ifq.drops not zero?  If so, raising

net.inet.ip.ifq.maxlen might help with this--amongst other things.

From previous correspondence on this on this list, I understand 256 *

the number of interfaces is recommended, but, ideally, you just want

it high enough to where you infrequently see drops.
Hope this helps resolve your issue, but, even if it doesn't, we're

correcting an outstanding issue you've had.
Cheers.

Yeah, I haven't seen drops go up any more in the past two hours, and

haven't seen any EHOSTUNREACH errors in my logs either.  Thanks. :-)
In retrospect, it makes sense that if the interface output queue is

filling up that the forwarding code would occasionally have errors

forwarding a packet and would result in a ICMP host unreachable

response.
Of course, I've also just now noticed some other weird errors that

I'll probably need to dig into... :-/

but net.inet.ip.ifq.* is the ip input queue, not the interface output

queues :)
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

Oh... :(