This is somewhat an extension of the thread "No SSH on External Interfaces
After pf.conf Rewrite for Load Balancing Outgoing Traffic" that Devin helped
me out with, but I started a new thread with a new title so other searching
might find it correctly.

 

I have one last issue since I moved to OpenBSD 4.7 and started doing
outbound load balancing with two internet connections.  This is with
connection to my inbound proxy from the internet to an FTP server behind my
OpenBSD firewall which fails to make a data connections.

 

Devin helped me to remember when dealing with more than one gateway, it is
necessary to use reply-to in my pass in rules to use the same gateway the
request came in on. (at least for certain services)  I used this same method
on my inbound redirects to the ftp-proxy, and my control connection is made,
but the data connection fails.  (active or passive)

 

Running ftp-proxy in debug mode 7 logging to standard output I see
essentially the same output I get in my FTP client:
using fixed server 10.52.91.10

listening on 127.0.0.1 port 21

#1 accepted connection from 75.21.149.231

#1 FTP session 1/100 started: client 75.21.149.231 to server 10.52.91.10 via
proxy 10.52.91.1

#1 server: 220 mydomain.net FTP - You're In!\r\n

#1 client: USER ftp@mydomain.net\r\n

#1 server: 331 Password required for ftp@ mydomain.net\r\n

#1 client: PASS blah_my_pass_blah\r\n

#1 server: 230 Logged on\r\n

#1 client: SYST\r\n

#1 server: 215 UNIX emulated by FileZilla\r\n

#1 client: FEAT\r\n

#1 server: 211-Features:\r\n

#1 server:  MDTM\r\n

#1 server:  REST STREAM\r\n

#1 server:  SIZE\r\n

#1 server:  MLST type*;size*;modify*;\r\n

#1 server:  MLSD\r\n

#1 server:  AUTH SSL\r\n

#1 server:  AUTH TLS\r\n

#1 server:  UTF8\r\n

#1 server:  CLNT\r\n

#1 server:  MFMT\r\n

#1 server: 211 End\r\n

#1 client: PWD\r\n

#1 server: 257 "/" is current directory.\r\n

#1 client: TYPE I\r\n

#1 server: 200 Type set to I\r\n

#1 client: ...
[ message continues ]

Does anyone have inbound ftp-proxy with multiple internet connections and
outbound load balancing working that can help me out?  I've been banging my
head against the wall for 3 days now trying every iteration of pf rules and
idea that I can come up with and I just can't get the data connection to
work.  I've tried having ftp-proxy listen on localhost with a redirect as
well as having it listen directly on the external interfaces with the same
issue.  I always get a control connection, but the data connection fails.
ftp-proxy was working fine in this configuration (same ftp-proxy settings,
same back-end FTP server settings, old pf.conf) prior to going to two
internet connections.  I need to get this working as it is for a customer.
Please help alleviate my ignorance.

----

From: dontek [mailto:dontek@gmail.com]  Sent: Thursday, June 10, 2010 3:04
PM

I have one last issue since I moved to OpenBSD 4.7 and started doing
outbound load balancing with two internet connections.  This is with
connection to my inbound proxy from the internet to an FTP server behind my
OpenBSD firewall which fails to make a data connections.

Running ftp-proxy in debug mode 7 logging to standard output I see
essentially the same output I get in my FTP client:
using fixed server 10.52.91.10
listening on 127.0.0.1 port 21
#1 accepted connection from 75.21.149.231
#1 FTP session 1/100 started: client 75.21.149.231 to server 10.52.91.10 via
proxy 10.52.91.1
#1 server: 220 mydomain.net FTP - You're In!\r\n
#1 client: USER ftp@mydomain.net\r\n
#1 server: 331 Password required for ftp@ mydomain.net\r\n
#1 client: PASS blah_my_pass_blah\r\n
#1 server: 230 Logged on\r\n
#1 client: SYST\r\n
#1 server: 215 UNIX emulated by FileZilla\r\n
#1 client: FEAT\r\n
#1 server: 211-Features:\r\n
#1 server:  MDTM\r\n
#1 server:  REST STREAM\r\n
#1 server:  SIZE\r\n
#1 server:  MLST type*;size*;modify*;\r\n
#1 server:  MLSD\r\n
#1 server:  AUTH SSL\r\n
#1 server:  AUTH TLS\r\n
#1 server:  UTF8\r\n
#1 server:  ...
[ message continues ]

Forgive me; my dumba$$ somehow forgot to add the multipath routes to my
hostname.if files for the two external interfaces.  I'm amazed everything
else was working..

Anyway, progress has been made but I'm still not 100%.

[ ftp-proxy listening on localhost with rdr-to in PF]
-----------------------------------------------------
Now I can successfully access the FTP server, but seemingly only on the
first interface for which ftp-proxy picks a route.  For instance, if I
reboot the machine, whichever external interface I connect to the FTP server
on first works, and continues to work (at least in the short term), but the
other still fails the data connection.

[ ftp-proxy(s) listening on each interface with straight pass in PF]
--------------------------------------------------------------------
I also tried running two instances of ftp-proxy, one for each interface, and
this works either on one instance or both sometimes after a reboot.  (I just
reboot every time because I can and it ensures everything is "clean")

Obviously, it seems I have a routing problem with ftp-proxy.  Anyone have
any suggestions?  I have an idea that running two instances on each
interface and using pf tagging to specify routes will probably work, but it
seems there should be a better way, or that multipath should be keeping the
source..?

-----Original Message-----
From: dontek [mailto:dontek@gmail.com] Sent: Saturday, June 12, 2010 11:52
AM

Does anyone have inbound ftp-proxy with multiple internet connections and
outbound load balancing working that can help me out?  I've been banging my
head against the wall for 3 days now trying every iteration of pf rules and
idea that I can come up with and I just can't get the data connection to
work.  I've tried having ftp-proxy listen on localhost with a redirect as
well as having it listen directly on the external interfaces with the same
issue.  I always get a control connection, but the data connection fails.
ftp-proxy was working fine in this ...
[ message continues ]