Ryan McBride recently announced that he has imported into -current support for CARP, OpenBSD's new Common Address Redundancy Protocol. He explains:

"This protocol allows multiple hosts on the same local network to share a set of IP addresses among them. Some of the functionality it provides is similar to VRRP, although CARP differs in some significant aspects: CARP has been designed to provide greater security and be protocol independent (so we can support both IPv4 and IPv6). Finally, CARP allows for some level of load balancing in addition to its high-availability functionality."

CARP is still under development, and won't be found in the upcoming November release of OpenBSD 3.4 [story]. Instead, it will be a feature to look forward to in OpenBSD 3.5. Read on for Ryan's full announcement email detailing the new protocol, as well as a few other short threads related to the recent import.

From: Ryan McBride [email blocked]
To:  misc
Subject: CARP
Date: Fri, 17 Oct 2003 21:53:10 +0000

As those of you who follow source-changes know, I have just imported
support for CARP, OpenBSD's Common Address Redundancy Protocol.

This protocol allows multiple hosts on the same local network to share a
set of IP addresses among them. Some of the functionality it provides is
similar to VRRP, although CARP differs in some significant aspects: CARP
has been designed to provide greater security and be protocol
independent (so we can support both IPv4 and IPv6). Finally, CARP allows
for some level of load balancing in addition to it's high-availability
functionality.

The basic configuration is as simple as running a command like the
following on all hosts which you want to have providing fail-over for
192.168.1.10:

# ifconfig carp0 vhid 1 192.168.1.10

One of the hosts will become "Master" of this IP address, while the
other hosts will go into "Backup" mode. If the master fails to advertise
after a certain period (3 times the advertisement interval of the Backup
host), the Backup will assume it is down and take over as Master,
responding to ARP requests for this IP address and accepting traffic
sent to it.

CARP specifies that the system which can advertise the most frequently
is most fit to be Master, one can encourage one system to be more likely
to become master by making it advertises more frequently relative to the
others. For example, the following command slows down the advertisement
frequency on this host;

# ifconfig carp0 advskew 100

Other hosts will advertise more frequently, and will be more likely to
become master.


See the man pages for more details: carp(4), ifconfig(8), sysctl(3).

A number of further developments are planned, including replay
detection, IPv6 support, and other fine-tuning.

-Ryan


P.S. If anyone has concerns about the Cisco's patent #5,473,599 and how
their claim that it applies to VRRP has forced us to design our own
incompatible protocol, don't talk to us. Instead, call Cisco's lawyer at
408-525-9706, or email him: [email blocked].



From: "STeve Andre'" [email blocked]
Subject: Re: CARP
Date: Fri, 17 Oct 2003 18:13:29 -0400

On Friday 17 October 2003 05:53 pm, Ryan McBride wrote:
> As those of you who follow source-changes know, I have just imported
> support for CARP, OpenBSD's Common Address Redundancy Protocol.
[snip]

This looks really promising, and a wonderful open implementation of
something thats been needed for a while.  Bravo to those who have
worked on this.  Finally, an alternative to VRRP...

I do not understand much of the IETF workings,  but I wonder if it
makes sense to enter this into the standards track?  Or is it worth
it?

Regardless, this is wonderful.

--STeve Andre'



From: Theo de Raadt [email blocked]
Subject: Re: CARP 
Date: Fri, 17 Oct 2003 16:30:49 -0600

> I do not understand much of the IETF workings,  but I wonder if it
> makes sense to enter this into the standards track?  Or is it worth
> it?

Why bother.  This issue came up because IETF didn't stand up to Cisco
years ago.

The entire issue of RAND "patents allowed in standards" balony came up
first over VRRP, now other groups have allowed it to happen too, or like
W3C finally cleaned up their act...

IETF is the problem.

From: Marc Balmer [email blocked]
Subject: CARP:  Know when you are the master
Date: Sat, 18 Oct 2003 22:22:21 +0200

Regarding the new CARP protocol:  Is there some way to be notified when 
a machine becomes the master, e.g. to startup services?

- mb



From: Ryan McBride [email blocked]
Subject: Re: CARP:  Know when you are the master
Date: Sat, 18 Oct 2003 21:03:31 +0000

On Sat, Oct 18, 2003 at 10:22:21PM +0200, Marc Balmer wrote:
> Regarding the new CARP protocol:  Is there some way to be notified
> when a machine becomes the master, e.g. to startup services?

Not in the direct fashion that I think you mean. It would be trivial to
write a script which calls ifconfig to check the status of the interface
(or a small program which uses the ioctl)

It should be pointed out that you should not have to start or stop
services in many (most?) configurations: services can bind to the
common address even if the host is not master. The service will simply
not recieve any traffic until the point at which the host becomes master.

But you raise a good point; I believe I'll add the appropriate logging
messages to the code so that state changes can be logged via syslog.



From: Marc Balmer [email blocked]
Subject: Re: CARP:  Know when you are the master
Date: Sat, 18 Oct 2003 23:19:33 +0200

On Sat, 18 Oct 2003 21:03:31 +0000
Ryan McBride [email blocked] wrote:

> On Sat, Oct 18, 2003 at 10:22:21PM +0200, Marc Balmer wrote:
> > Regarding the new CARP protocol:  Is there some way to be notified
> > when a machine becomes the master, e.g. to startup services?
> 
> It should be pointed out that you should not have to start or stop
> services in many (most?) configurations: services can bind to the
> common address even if the host is not master. The service will simply
> not recieve any traffic until the point at which the host becomes master.

I think there is a problem with this.  You have two machines with the same 
IP address in the network, if I understand the concept.  This is maybe not 
a problem when the machines are connected from remote.  But if a slave 
accesses the shared IP address, it will connect to itself and not to the 
master.  So the slaves access a different machine (namely themselves) than
machines not in the vhid group.

We have some installations with clustered mail gateways where a single fibre 
channel RAID array can be accessed from two servers.  Only one server, the
master, mounts the mail queue directory partition and handles e-mail.  When 
fail over occurs, the slave server mounts the partition and begins to process 
e-mail.  What would be the corresponding setting in a CARP situation?
 
> But you raise a good point; I believe I'll add the appropriate logging
> messages to the code so that state changes can be logged via syslog.

I'd prefer an ioctl with which I could register "myself" with carp to get 
notified of a state change.  A callback, so to say, or maybe a signal.

- mb

From: Marc Balmer [email blocked]
Subject: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 22:39:08 +0200

With the new CARP protocol, what exactly happens when a slave connects to 
the virtual IP address?  Will it connect to the current master or will it 
connect to it's local carp interface?

- mb



From: Ryan McBride [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 21:16:10 +0000

On Sat, Oct 18, 2003 at 10:39:08PM +0200, Marc Balmer wrote:
> With the new CARP protocol, what exactly happens when a slave connects
> to the virtual IP address?  Will it connect to the current master or
> will it connect to it's local carp interface?

The common address is not currently visible to the machine which hosts
it, so the backup system will contact the master, not itself. The
exception is if a physical interface is also configured with the
address, but you have to be careful if this is your configuration to
ensure that the host which has the physical address is master whenever
that interface is alive.

(Some changes may be made to this behaviour, but that's how it is right
now)



From: Marc Balmer [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 23:22:21 +0200

On Sat, 18 Oct 2003 21:16:10 +0000
Ryan McBride [email blocked] wrote:


> The common address is not currently visible to the machine which hosts
> it, so the backup system will contact the master, not itself. The

Ok, that solves a lot of questions in my other posting.  So the correct 
setup is to give each physical interface a unique IP adresse plus adding 
carp interfaces for the common address which is different from the physical 
addresses?

- mb



From: Ryan McBride [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 21:37:56 +0000

On Sat, Oct 18, 2003 at 11:22:21PM +0200, Marc Balmer wrote:
> Ryan McBride [email blocked] wrote:
> > The common address is not currently visible to the machine which hosts
> > it, so the backup system will contact the master, not itself. The
> 
> Ok, that solves a lot of questions in my other posting.  So the
> correct setup is to give each physical interface a unique IP adresse
> plus adding carp interfaces for the common address which is different
> from the physical addresses?

It's not "correct" per se, but certainly this will be the simplest
configuration to use in most situations.

There's no reason why you can't configure things so that the host which
has the address on it's physical interface is also the master of the
common address, by ensuring that it advertises more frequently.  This
would be desireable in situations where you have a shortage of IP
addresses (you save one IP this way), or where you want the address to
work even if CARP is turned off. 



From: Marc Balmer [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 23:43:42 +0200

On Sat, 18 Oct 2003 21:37:56 +0000
Ryan McBride [email blocked] wrote:

> There's no reason why you can't configure things so that the host which
> has the address on it's physical interface is also the master of the
> common address, by ensuring that it advertises more frequently.  This
> would be desireable in situations where you have a shortage of IP
> addresses (you save one IP this way), or where you want the address to
> work even if CARP is turned off. 

Ok, I have a host with phyiscal IF xl0 which has assigned IP 192.168.17.1 
to it.  If I do 'ifconfig carp0 vhid1 192.168.17.17 up' it apparently 
configs carp0, but I cannot ping this address although I am the master.  
Shouldn't I be able to access the virtual address now?

# ifconfig xl0
xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        address: 00:04:76:98:f2:77
        media: Ethernet autoselect (100baseTX)
        status: active
        inet 192.168.17.1 netmask 0xffffff00 broadcast 192.168.17.255
        inet6 fe80::204:76ff:fe98:f277%xl0 prefixlen 64 scopeid 0x2
# ifconfig carp0
carp0: flags=41<UP,RUNNING> mtu 1500
        carp: MASTER vhid 1 advbase 1 advskew 0
        inet 192.168.17.17 netmask 0xffffff00
# ping 192.168.17.17
PING 192.168.17.17 (192.168.17.17): 56 data bytes
--- 192.168.17.17 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
#



From: Ryan McBride [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 21:50:20 +0000

On Sat, Oct 18, 2003 at 11:43:42PM +0200, Marc Balmer wrote:
> Ok, I have a host with phyiscal IF xl0 which has assigned IP
> 192.168.17.1 to it.  If I do 'ifconfig carp0 vhid1 192.168.17.17 up'
> it apparently configs carp0, but I cannot ping this address although I
> am the master.  Shouldn't I be able to access the virtual address now?

Not the way things work right now. As I mentioned in a previous email,
there will likely be some changes made in this area.



From: Marc Balmer [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 23:55:10 +0200

On Sat, 18 Oct 2003 21:50:20 +0000
Ryan McBride [email blocked] wrote:

> On Sat, Oct 18, 2003 at 11:43:42PM +0200, Marc Balmer wrote:
> > Ok, I have a host with phyiscal IF xl0 which has assigned IP
> > 192.168.17.1 to it.  If I do 'ifconfig carp0 vhid1 192.168.17.17 up'
> > it apparently configs carp0, but I cannot ping this address although I
> > am the master.  Shouldn't I be able to access the virtual address now?
> 
> Not the way things work right now. As I mentioned in a previous email,
> there will likely be some changes made in this area.

I see.  Well, it works from remote, I can access the virtual IP address 
from my LAN, but not from the master itself which seems strange to me (or I 
missed something... :).  How is the master supposed to bind to an address it 
can't even ping?

Anyway, very interesting stuff, indeed.  I just setup a small network for 
further testing carp.

- mb



From: Ryan McBride [email blocked]
Subject: Re: CARP: Slave connects to virtual address, what happens?
Date: Sat, 18 Oct 2003 22:00:46 +0000

> I see.  Well, it works from remote, I can access the virtual IP
> address from my LAN, but not from the master itself which seems
> strange to me (or I missed something... :).

If it makes you feel better, you can consider it a bug and work around
it.  Remember that this is development code; if you want to see the
behaviour changed more quickly, send me a diff and I'll have a look.

> How is the master supposed to bind to an address it can't even ping?

It works.

From: Marc Balmer [email blocked]
Subject: carp, is this acronym a good idea?
Date: Sat, 18 Oct 2003 23:37:16 +0200

Was the name CARP deliberately choosen?  There already exists CARP since 2001, 
see RFC 3040 (Cache Array Routing Protocol)

- mb



From: Nick Holland <nick@holland-consulting.net>
Subject: Re: carp, is this acronym a good idea?
Date: Sat, 18 Oct 2003 19:44:31 -0400

Marc Balmer wrote:
> 
> Was the name CARP deliberately choosen?  There already exists CARP since 
2001, see RFC 3040 (Cache Array Routing Protocol)
> 
> - mb

Oh, that makes it a perfect name!

This is the computer industry...we like to overload our acronyms.

Anyone else wish to confess their puzzlement about why Automatic
Teller Machines (or was it Adobe Type Manager?) was such a hot topic
in the networking magazines a decade or so ago?  Do you REALLY think
they chose "Asynchronous Transfer Mode" as the name because it was
just SO descriptive, that it would be PERFECTLY clear what they were
talking about?

Part of it is, there is just a limit to how many TLAs and ETLAs[1]
exist.

I suggested a different name (same four letters, different order, took
a cheap shot at Cisco in the process), but Ryan didn't seem to care
for it for some reason. 8)


Alternate response:
Hey, OpenBSD has been using the "fish" theme since before 2000,
they're infringing on OUR name space! 8)


Yet another alternate response:
Hey, I know a few Marcs, too.  You change your name to something
short, memerable and unique, we'll talk about changing CARP? 8)


Nick.
(who's favorite restaurant has two Saras (the third one went away to
school) and two Katies as waitresses and hangs out on an IRC channel
with three Joes, two Dave, two Jeffs, two Mikes and some confusion
over Chris's.  I might be somewhat desensitized to name
overloading...  And I still gotta check my notes to remember "Todd" vs
"todd" 8-)
-- 
http://www.holland-consulting.net

• Archive of above thread
• KernelTrap interview with Theo de Raadt