Re: pkg_add problems after 4.7 upgrade

Previous thread: Re: May 26 Current on Lenovo SL410 Laptop by Mark Kettenis on Saturday, May 29, 2010 - 8:56 am. (1 message)

Next thread: Possible bug ? Re: 4.7 Release - bgpd not passing IBGP routes by a b on Saturday, May 29, 2010 - 12:34 pm. (2 messages)
From: Casey Allen Shobe
Date: Saturday, May 29, 2010 - 10:45 am

I upgraded six machines from 4.6 to 4.7 recently, using the CD to do so.  I
then updated PKG_PATH and tried to upgrade packages.  It fails consistently
on all six machines, indicating that it can't find anything in $PKG_PATH.
So I tried setting PKG_PATH to the main openbsd mirror, with no luck.

It also fails when I specify an explicit package path, which is odd because
I'm able to access this exact URL from the same machines using ftp or links:

# pkg_add -uvvv
ftp://ftp.openbsd.org/pub/OpenBSD/4.7/packages/i386/ratpoison-1.4.4p0.tgz

Problem finding
ftp://ftp.openbsd.org/pub/OpenBSD/4.7/packages/i386/ratpoison-1.4.4p0.tgz
No packages available in the PKG_PATH

How can I resolve this or figure out more about what's going wrong?  The
-vvv options, per the man page, don't seem to do anything useful in this
case.

Cheers,
-- 
Casey Allen Shobe
casey@shobe.info

From: Jacob Meuser
Date: Saturday, May 29, 2010 - 1:54 pm

a piece of info is missing here: what do you have PKG_PATH set to,
exactly?

-- 
jakemsr@sdf.lonestar.org
SDF Public Access UNIX System - http://sdf.lonestar.org

From: Casey Allen Shobe
Date: Saturday, May 29, 2010 - 5:57 pm

I've tried a few mirrors with the same results - this was the last one  
I tried:

export PKG_PATH=ftp://obsd.cec.mtu.edu/pub/OpenBSD/4.7/packages/i386/

...or as I was trying last as indicated below:
export PKG_PATH=ftp://openbsd.org/pub/OpenBSD/4.7/packages/i386/


From: Jacob Meuser
Date: Saturday, May 29, 2010 - 6:19 pm

^^^^^^
of course that one won't work ... maybe that's a typo but that's
kinda the point.

since you're using ftp, 'ftp $PKG_PATH' should put you in the right

-- 
jakemsr@sdf.lonestar.org
SDF Public Access UNIX System - http://sdf.lonestar.org

From: Casey Allen Shobe
Date: Saturday, May 29, 2010 - 7:38 pm

Well, I tried `links $PKG_PATH	` before - that worked.  But with `ftp  
$PKG_PATH`, it appears to log in correctly, but ls never returns  
anything.  This seems to happen with ANY ftp server actually (just  
tried a bunch), and is odd because I did use the commandline ftp  
utility frequently on 4.6 without any issue...

So the problem lies with the 'ftp' utility?

Cheers,
-- 
Casey Allen Shobe
casey@shobe.info

From: Denny White
Date: Saturday, May 29, 2010 - 9:15 pm

Not really sure what happened with
ftp://obsd.cec.mtu.edu/pub/OpenBSD/4.7/packages/i386/
since it worked fine for me including 'ls'.

As for the 2nd one you tried which didn't work:
ftp://openbsd.org/pub/OpenBSD/4.7/packages/i386/

Try this:
ftp://ftp.openbsd.org/pub/OpenBSD/4.7/packages/i386/


-- 

===============================================================
Denny White - dennyboy@cableone.net
GnuPG key  : 0x1644E79A  |  http://wwwkeys.de.pgp.net
Fingerprint: D0A9 AD44 1F10 E09E 0E67  EC25 CB44 F2E5 1644 E79A
===============================================================
() ASCII ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
===============================================================

From: Casey Allen Shobe
Date: Saturday, May 29, 2010 - 9:50 pm

Well, I've found that using an HTTP mirror works.  However, this is  
concerning because all of these machines were able to use FTP properly  
before, some Debian machines on the same network can use FTP mirrors  
correctly (plus `ftp` to the same OpenBSD mirrors works from Debian),  
and the links browser on the OpenBSD machines seems to do FTP  
correctly too.

One way or the other, the 'ftp' utility seems to have gotten broken  
since 4.6.  I'd hazard to guess that it might not be supporting  
passive FTP anymore, since active FTP does not work from the network  
and this is reminiscent of what would happen if I tried using active  
FTP from behind a firewall in the past...

I've worked around the immediate problem with HTTP for now...but is  
there any more useful information that I can provide from my end?

Cheers,
-- 
Casey Allen Shobe
casey@shobe.info

From: Denny White
Date: Saturday, May 29, 2010 - 10:55 pm

If you're comfortable with it, you could try disabling pf just long
enough to see if your ftp works without it. If so you could supply
your pf.conf since there have been syntax changes and possibly you
have something outdated in it. If you've got good backups you could
try doing a fresh install on just one box and see what happens. If
the problem goes away you'll know something got screwed up during
the upgrades.


-- 

===============================================================
Denny White - dennyboy@cableone.net
GnuPG key  : 0x1644E79A  |  http://wwwkeys.de.pgp.net
Fingerprint: D0A9 AD44 1F10 E09E 0E67  EC25 CB44 F2E5 1644 E79A
===============================================================
() ASCII ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
===============================================================

From: Casey Allen Shobe
Date: Sunday, May 30, 2010 - 1:26 am

No pf here - the NAT device is actually an old WatchGuard Firefox  
outside of my control and choice, but it hasn't been touched in years...

I also upgraded my home machine (on a public IP) from 4.6 to 4.7 and  
did not run into the same problems.  I can also try a fresh install of  
4.7 on the work network - it won't take long.  I'll also do a fresh  
install of 4.6, to verify that it is something that changed in the  
release, rather than a network- or machine-specific issue.

I've been poking around a little bit since my last E-mail, and have  
found that using `ftp -A` on OpenBSD 4.7 works, whereas usual `ftp`  
does not.  This seems backwards to me as it seems like if one is going  
to work, it ought to be passive...

I also noticed that the working `ftp` on Debian (from netkit) is a  
fair bit different from the one on OpenBSD - it's man page implies  
that it uses active FTP by default, and needs a -p argument for  
passive.  But whether I just do `ftp ftp.openbsd.org` or `ftp -p ftp.openbsd.org 
` - from Debian, they both work.  I can also manually toggle passive  
on and off from within netkit-ftp, and either way, I can get a  
directory listing without problem.

I installed tnftp on Debian, and it failed to work in either active or  
passive mode, so I quickly gave up on that.

I also installed wget on one of the OpenBSD machines, and it is able  
to happily download from FTP sites whether --passive-ftp or --no- 
passive-ftp is given, and it's nicely verbose about when PASV or PORT  
are actually used so it's not just a matter of hoping the man page is  
current.

HOWEVER, I did notice one difference watching tcpdump...the clients  
that work correctly use PASV, the ones that don't use EPSV.  Indeed,  
tnftp on Debian had been clearly telling me that it was trying  
"Extended passive" and EPRT, but I didn't know that was anything  
different from PASV/PORT.

So apparently the E___ commands exist for IPv6 compatibility.  The  
tnftp that Debian packages uses ...
From: Stuart Henderson
Date: Sunday, May 30, 2010 - 5:34 am

[ reply-to set, please honour it (or change it to tech@), don't
crosspost between misc and tech. ]


Not sure why you're seeing problems now, but this hasn't changed
in years. ftp in OpenBSD 4.6 used EPSV by default too. And it does
still fallback from EPSV to PASV (you can see this with e.g. 

Yes, that's exactly it. For pkg_add I suggest switching to an http
mirror (or you could set FETCH_CMD="ftp -E" for pkg_add, but switching
to http is more straightforward).

Does anyone know the rationale for setting epsv4 = 1 by default?

On the one hand it's useful to know that the network connection is
broken, but that's about the only advantage I can see, on the other
hand if you're actually trying to fetch files (especially from a
*server* behind a broken nat device that's not under your control)
it's a real pain.

We mirror distfiles for some ports because of this...

From: Casey Allen Shobe
Date: Sunday, May 30, 2010 - 9:34 am

Alright, glad I tested this out, because I was wrong about this being a
problem that showed up in 4.7 - it is the same in 4.6.  I had PKG_PATH set
in a .profile on every machine and was using HTTP before - didn't pay
attention when I replaced it with a new FTP URL I guess, because I thought I
was using FTP before...backups proved me wrong though.  Sorry about
mistaking the old path, but I guess it's good because had I not thought it
to be a regression, I probably wouldn't have looked into it.

The rest of the diagnosis should be sound though, I think.



Well, to play devil's advocate a bit, these aren't really servers except in
the X11 sense - they're administrative workstations.  But one thing I'd like
to point out is that the NAT is not actually broken - it just doesn't take
RFC2428 into consideration, which is pretty reasonable since that RFC came
out in late 1998 and the NAT device we have I think came out the same year,
or perhaps the next one.  That's also two years after they originally
predicted IPv6 becoming widely-used, heh...

I'd happily replace it with an OpenBSD machine, but the powers that be won't
have that (I work for an international non-profit with control issues that
compliments their lack of actual security and best practices nicely).  Our
site is fine as we operate mostly-independently, but dealing with anyone up
the chain is a horrendous experience best avoided.  They also spent the
money for a lifetime support option, so we *could*, for free, upgrade the
software to be much newer and surely supporting EPSV/EPRT, but we don't have
access to do that.  Yet, that money wasted years ago is excuse for us to
never replace the thing unless it sustains hardware failure.  I'd be
readying my bat...but our nonexistent recovery plan keeps me leery... ;)


I'll be setting up a local mirror once time/bandwidth allow for it, which
should be within the week. :)

-- 
Casey Allen Shobe
casey@shobe.info

From: Richard Toohey
Date: Sunday, May 30, 2010 - 12:12 am

concerning because all of these machines were able to use FTP properly before,
some Debian machines on the same network can use FTP mirrors correctly (plus
`ftp` to the same OpenBSD mirrors works from Debian), and the links browser on
4.6.  I'd hazard to guess that it might not be supporting passive FTP anymore,
since active FTP does not work from the network and this is reminiscent of
what would happen if I tried using active FTP from behind a firewall in the
(Re-sending, don't think I cc'd list first time)

Just from my experiences ...

Two 4.6 boxes (one behind ISA, one behind a 4.3 OpenBSD box - yes urgent
upgrade planned soon!)

Both upgraded from 4.7 CD, both had packages upgraded with:

export PKG_PATH=ftp://ftp3.usa.openbsd.org/pub/OpenBSD/4.7/packages/i386
pkg_add -ui -D update -D updatedepends

(I tried the Australian mirrors but no 4.7 packages found on there when I
tried - or I did something wrong.)

Then fetched the first 5 patches by using FTP client to pull the files from
the links on the errata page.


Previous thread: Re: May 26 Current on Lenovo SL410 Laptop by Mark Kettenis on Saturday, May 29, 2010 - 8:56 am. (1 message)

Next thread: Possible bug ? Re: 4.7 Release - bgpd not passing IBGP routes by a b on Saturday, May 29, 2010 - 12:34 pm. (2 messages)