Hi,

After successfully putting into testing the new firewall setup with some 
of our services, we are seeing some low congestion issues It's not 
major, but since I'm only throwing it half our expected traffic for the 
time being, I would have liked it to be at 0.

Our setup is a 4.3 i386 (Xeon 3GHz) box with 6 Intel gigabit interfaces 
(em), all of them having at least one carp interface, and 2 of them 
having trunked VLANs. NAT is only applied to outgoing traffic, which is 
very minimal. Just about all of the traffic that I'm sending to it right 
now consists of very small XML requests over HTTP, so low throughput but 
very high session count. All the interfaces have the speed and duplex 
hardcoded at the switch and system level.

Here's a link that includes some possible debugging information from 
pfctl -si, some sysctl parameters, top load and dmesg:
http://www.sjohnson.info/other/diaginfo.txt

And here's the set of PF rules that are active:
http://www.sjohnson.info/other/pf.conf

Just about all the traffic that is coming in at the moment is hitting 
that first "pass in quick" rule.

Is the congestion issue that I'm getting considered "normal" under that 
type of traffic and with the present hardware? Are there any other 
settings that I should look into tweaking?

Also, is it expected that a total of 135K sessions in our link load 
balancers give us around 550K sessions with PF? I now know it's supposed 
to be at least double because of the directional state entry, but I just 
find the number alerting, especially since it was close to a 1:1 when we 
compared them to our netfilter states (agreeing that state processing is 
completely different between the two). This is with aggressive setting, 
as I was getting passed 750K sessions with conservative setting.

Thanks again for help,
Steve Johnson