login
Search
There was an interesting exchange recently on the FreeBSD hackers mailing list. A user with a FreeBSD firewall was suffering a DDoS (Distributed Denial of Service) attack and was asking for suggestions on how to lessen the effect. The resulting thread offers some good advice.
An older but also interesting story about an unrelated DDoS type attack is found in this article by Steve Gibson, making for an addicting read. Another example being the recent DDoS attacks against the DNS root servers which with ping flooding successfully crippled 7 of the 13 root servers for nearly an hour.
Each kernel and networking stack certainly has its own strengths and weaknesses against different types of attacks. What sorts of war stories do KernelTrap readers have regarding DDoS attacks, and what sorts of advice can be offered?
From: Josh Brooks
To: freebsd-hackers
Subject: DDoS attacks, packets captured ... not sure what to do.
Date: Sun, 5 Jan 2003 14:57:22 -0800 (PST)
First off, the target looks like this:
Port State Service
21/tcp open ftp
22/tcp open ssh
25/tcp open smtp
53/tcp open domain
80/tcp open http
110/tcp open pop-3
3333/tcp open dec-notes
10000/tcp open snet-sensor-mgmt
31337/tcp open Elite
(and yes, port 31337 is legit - this is a linux server and that is not a
trojan running)
The target is behind my FreeBSD firewall - and it is the firewall which
hangs and causes the DoS.
And my firewall is a 4.4-RELEASE, ~500mhz celeron with 256 megs ram. when
I run `top` I see between 17 and 50% idle CPU. I have 900+ ipfw rules in
place, and about 80% of those are just "count" rules. When the attacks
occur, my firewall simply freezes - no response, the network disappears,
and no traffic of any kind flows through it. Here is the obligatory
netstat -m on the firewall:
# netstat -m
405/4768/32768 mbufs in use (current/peak/max):
405 mbufs allocated to data
382/4524/8192 mbuf clusters in use (current/peak/max)
10240 Kbytes allocated to network (41% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
OK, so during the attack, an upstream router captured the packets. I see
two interesting things:
1. a ton of TCP SYN, [1658] -> [106] 3COM-TSMUX to ports that do
not exist on the target. See how this one goes to destination 106, but
the target is not running anything on 106 ? Once in a while the SYN
packets go to an existing port, but most of them go to nonexistent
(seemingly random) ports on the target.
2. a noticable amount of christmas tree packets aimed at the target:
TCP FIN SYN RST PSH ACK, [1400] -> [98] TAC-news
again, to ports not actually open on the target. I guess a xmas tree
packet technically has a URG flag as well, which these do not - but even
still I suspect these are bad news to be seeing.
Also some of them are not quite as xmas as other:
TCP SYN RST PSH ACK, [1230] -> [118] SQL-service
again, directed at a service that does not exist.
3. These seem less frequent, but I am seeing:
UDP, [21397] -> [2284] ^M
Source port: [21397] ^M
Destination port: [2284] ^M
UDP length: 908^M
Checksum: 0x0000 (data fragment - not able to check)^M
So .. a UDP fragment sent to a port not open on the target. This also
seems like bad news.
-----
So that's that - I see this for all three IPs that were being targeted.
So now there are two things I need to know (and ask cordially for your
help with)
1. what can I put into place on a 4.4-RELEASE ipfw firewall to combat
these items ?
2. What are 1 2 and 3 called ? For instance, is #1 a "syn flood" ?
3. will the solutions given to me actually help ? I mean, the packets
will still hit my firewall, and given the cpu utilization and config I
showed you earlier, will the fixes nullify the effect of these attacks, or
am I so underpowered that getting hit with these in any way, no matter
what precautions I have in place will disable me ?
thanks a LOT.
From: soralx
Subject: Re: DDoS attacks, packets captured ... not sure what to do.
Date: Sun, 5 Jan 2003 23:33:39 -0700
> 1. a ton of TCP SYN, [1658] -> [106] 3COM-TSMUX to ports that do
> not exist on the target.
this is not a 'SYN flood'; 'SYN flood'=TCP SYN+FIN
seems like someone is continuosly using TCP SYN "half-open"
scan to get your open ports, or just sends random SYN pakets
> 2. a noticable amount of christmas tree packets aimed at the target:
> TCP FIN SYN RST PSH ACK, [1400] -> [98] TAC-news
> again, to ports not actually open on the target.
> Also some of them are not quite as xmas as other:
> TCP SYN RST PSH ACK, [1230] -> [118] SQL-service
> again, directed at a service that does not exist.
try using 'ipfw' option 'tcpflags' to ignore such packets,
or dummynet
> 3. These seem less frequent, but I am seeing:
> UDP, [21397] -> [2284] ^M
> Source port: [21397] ^M
> Destination port: [2284] ^M
> UDP length: 908^M
> Checksum: 0x0000 (data fragment - not able to check)^M
> So .. a UDP fragment sent to a port not open on the target. This also
> seems like bad news.
UDP scan?
try "options ICMP_BANDLIM", if not already enabled
> 3. will the solutions given to me actually help ? I mean, the packets
> will still hit my firewall, and given the cpu utilization and config I
> showed you earlier, will the fixes nullify the effect of these attacks, or
Limiting ICMP pps may help. If you configure 'ipfw' to ignore such
packets (and also other trash packets that are useless), target will not send
RST for closed ports, which may also help. I don't know for certain - you need
to experiment.
05.01.2003; 23:25:10
[SorAlx] http://cydem.zp.ua/
From: Terry Lambert
Subject: Re: DDoS attacks, packets captured ... not sure what to do.
Date: Sun, 05 Jan 2003 22:51:28 -0800
soralx wrote:
> > 1. a ton of TCP SYN, [1658] -> [106] 3COM-TSMUX to ports that do
> > not exist on the target.
>
> this is not a 'SYN flood'; 'SYN flood'=TCP SYN+FIN
> seems like someone is continuosly using TCP SYN "half-open"
> scan to get your open ports, or just sends random SYN pakets
This works really well as a denial of service for FreeBSD, since
it fills up the SYN cache, forcing the use of SYN cookies.
A secondary attack, after the SYN cache has been filled once to
force the SYN cookie flag on, is to send ACK's, forcing the FreeBSD
box to do relatively expensive cryptographic calculations on the
ACK, to see if it's an ACK with a valid SYN cookie.
An exhaustive search of the sequence number space after that (to
discover a valid SYN cookie) can lead to people being able to open
connections without the SYN/SYN-ACK.
The benefit to overflowing to the point of getting a valid SYN
coookie for this exploit is that you can, in that packet, include
data payload.
By doing that, you can basically send a data payload to a FreeBSD
daemon, using a spoofed source IP address, and have it do something,
which is not something you could do, without having SYN cookies.
Aside from the other good advice in the previous posting, I suggest
that you disable SYN cookies.
Another useful thing to do is limit the number of connections per
second from a given source IP address, and to limit the total number
of connection "in progress" from a given IP address.
-- Terry
From: soralx
Subject: Re: DDoS attacks, packets captured ... not sure what to do.
Date: Mon, 6 Jan 2003 00:21:39 -0700
> Another useful thing to do is limit the number of connections per
> second from a given source IP address, and to limit the total number
> of connection "in progress" from a given IP address.
I doubt that all the packets are sent from one real IP. But, I tnink,
it may be possible to determine the IP of an attacker, because it's
not just a DoS attack. He may use other methods later. I am almost
sure he tried to scan ports earlier, probably with `nmap -v -O` to
determine the OS, and now he knows what packets to send.
BTW, what were the UDP packets for? Scanning?
06.01.2003; 00:14:26
[SorAlx] http://cydem.zp.ua/
From: Terry Lambert
Subject: Re: DDoS attacks, packets captured ... not sure what to do.
Date: Mon, 06 Jan 2003 11:16:35 -0800
soralx wrote:
> I doubt that all the packets are sent from one real IP. But, I tnink,
> it may be possible to determine the IP of an attacker, because it's
> not just a DoS attack. He may use other methods later. I am almost
> sure he tried to scan ports earlier, probably with `nmap -v -O` to
> determine the OS, and now he knows what packets to send.
Knowing his IP address is useless, if it's a denial of service,
unless you have a peering agreement with his NSP/ISP, and/or are
within driving distance, and own a shotgun.
> BTW, what were the UDP packets for? Scanning?
I don't know. You didn't characterize them well enough for anyone
to make a guess. If they were all frags, with one frag missing,
then they were an intentional denial of service on your UDP packet
reassembly buffer, which is relatively sucky in FreeBSD. Otherwise,
they might have been a Linux NFS over UDP client (same thing, really),
or some other attack (e.g. attempted DNS poisoning, etc.).
-- Terry
zilch, nada, zero. ;(
zilch, nada, zero. ;(
at least there is serious research...
Google's HTML version or original PDF version
--
I used to have a sig until the great Kahuna of FOOness
told me to dump it and use /dev/urandom instead.
Its hard to win against DDOS.
I used to run a Quake server with a friend. A lamer got banned and was pissed about it. He decided if he couldn't play on our server, no one would.
The poor server was then subjected to 100mbit/s of UDP packets coming from random source addresses and destined for the quake server's port. I say 100mbit because thats all that we were able to observe due to the 100mbit ethernet connection that led to this box. There was probably a significant number of packets dropped upstream.
It was enough bandwidth to cause others on our ethernet switch to be unreachable. Our upstream provider had multiple big pipes, so they weren't saturated, but it caused problems.
How does one win against someone who has enough drones to generate that much bandwidth? We were lucky our provider didn't just drop our connection outright.
Even if the provider cooperated with upstream providers and traced it back router-by-router, it wouldn't have done much good. They would have found hundreds of cable modem and dsl users who were totally clueless about their computer being used in a DoS attack.
I hated losing to this script kiddie, but the upstream provider requested that the quake server go away. I don't blame them.
-molo
Steve Gibson's article
While Steve's articles and writings are usually technically correct and factual, they are also very sensationalist and tabloid in nature.
You will notice he uses MASSIVE numbers of
, and liberal amounts of HTML tags.
Take the severity of what he says with a large grain of salt.
I would like suggest that he has a hidden adjenda to push .. but he doesn't sell security software, so I'm not sure why he feels it is necessary to sensationalise his articles. Maybe his personality is just a bit like "chicken-little", who thought the sky was falling.
(Steve, for a large ISP like Verio, 2 x 100 Mbps of upstream capacity is not massive at all, and you will be sharing that capacity with a large number of other customers ...)