Hello,
I just used dnsstuff to test one of my domain names and it showed me

(the first time only) that my server is an openrelay, which is obviously

not true. This is due to the default behaviour of spamd of accepting

everything, even when a spamd.alloweddomains file is present. I think

this could choke some automated tests as nearly none of them goes to the

point of actually sending data.
here is a well known spamd session:

"

telnet elrond.llorien.org 25

Trying 88.198.156.90...

Connected to elrond.llorien.org.

Escape character is '^]'.

220 elrond.llorien.org ESMTP ; Tue May 22 09:09:33 2007

ehlo test

250 Hello, spam sender. Pleased to be wasting your time.

mail from:<>

250 You are about to try to deliver spam. Your time will be spent, for

nothing.

rcpt to:<postmaster@invaliddomain.com>

250 This is hurting you more than it is hurting me.

"
I know that I can configure spamd to send a 550 error to the client, but

only after DATA, which will clearly almost never happen in automated

tests. So I think it could probably be a good idea to add an option

which makes the 550 reply at RCPT TO for domains not being in

spamd.alloweddomains. This would still allow to make spamtraps but only

those sent at alloweddomains would waste the most time to the sender.
What are your feelings bout this?

Any automated test I've ever set up for open relay, (and I run

them) as well as any sane ones I ever see test for open relay by

actually relaying a message not looking at the smtp dialoge.
	You're making much ado over nothing and spreading FUD -

the tester you are using is just making stupid assumptions.
	-Bob

It should also be noted that at least some versions of Mdaemon interpret

a 4xx error code at DATA as a permanent error. I know, the problem is on

their side too.

This was certainly not my intention to spread FUD and I am sorry if I

did. Maybe I am a little bit too paranoid. I just wanted people to share

their experiences with this.
However, there is clearly a problem with MS exchange and current spamd

behavior.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

I would say that a more accurate description of spamd's behavior with

respect to relay checkers would be 'appears to accept but does not

forward'.  What you are seeing is most likely that the relay checker

performs a limited parse of the SMTP dialogue but does not check if

its test message is actually forwarded.  This is AFAIK the intended

behavior, and it might even fool gullible spammers.
--

Peter N. M. Hansteen, member of the first RFC 1149 implementation team

http://www.blug.linux.no/rfc1149/ http://www.datadok.no/ http://www.nuug.no/

"First, we kill all the spammers" The Usenet Bard, "Twice-forwarded tales"

delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

Indeed, but it could cause you to get blacklisted by some automated

checkers, which is clearly something you don't want. I know this kind of

checker is not accurate, but some local checkers will do it that way and

you will end up with the problems.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

no, that is bullshit.

those "checkers" do not exist any more (or they went irrelevant).

it has been proven many many many moons ago that this kind of test is

useless, and this is accepted knowledge ever since. fortunately.
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

After reading your original message, I looked around the first 20-odd

relay checkers and lists of open relays google could find for me

(search string: "mail relay test").  Some these sites in turn link to

extensive lists of publicly available lists of open relays, but I

never found any indication that any of our servers (all spamd

protected) were on any of them.  
I take this as an indication that at least the more commonly used ones

do not behave as you suspect.  If other, less common ones or or pay to

use lists are more trigger happy and as a consequence offer less

accurate data than the free ones, that is of course unfortunate.
--

Peter N. M. Hansteen, member of the first RFC 1149 implementation team

http://www.blug.linux.no/rfc1149/ http://www.datadok.no/ http://www.nuug.no/

"First, we kill all the spammers" The Usenet Bard, "Twice-forwarded tales"

delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.

I speak mostly of SMTP-time checkers. Imagine you are sending a mail to

someone and while you are doing the SMTP transaction, the remote host

also connects to your server to see if it may be an openrelay. Given

current spamd behaviour and the time the remote host has to check your

server, it will judge it as an openrelay as it won't be able to pass

through the data phase.
As a secondary effect, sender callouts made from a remote server will

also be accepted (at least the first time) even if the recipient doesn't

exist on your server. But that's probably not really that important.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

They are broken then... Workaround: use different mailer instances on

different IP addresses for incoming and outgoing mail (this is often a
that's exactly why it changed from rejecting at rcpt to: stage.

http://www.openbsd.org/cgi-bin/cvsweb.cgi/src/libexec/spamd/spamd.c#rev1.85

This workaround only works if the checker connects to your MX, not to

the host sending the mail. I know they are somewhat broken but there is

no point in contacting the sender domain server if you want to check for

an openrelay as the from header is more than likely a fake.
Also, MS exchange servers don't like 4xx errors at DATA time and may

forbid the mail from being delivered until the exchange instance is
Yes, but that means callouts that should not succeed will (at least the

first time).
I know no scheme is perfect, so the point is it could be handy to have a

flag to determine when the mail should be greylisted and let people choose.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

You wouldn't need spamd on the address of a send-only instance..

(if mail's only submitted on 587/465 or from known address ranges, it
Yeuch... I didn't know about that. Found it here (needs user-agent:

googlebot) - http://www.windowsitpro.com/Article/ArticleID/95332/95332.html
   When Exchange 2003 sends a message to a server using greylisting,

   it gets back a 4xx "try again later" code. Instead of waiting a

   reasonable interval, Exchange tries again after only a few

   seconds. This attempt generally fails too, and Exchange doesn't

   try again.
   ... The message isn't delivered, and it doesn't appear in any

   queues.  Exchange won't try to redeliver it again until you

   restart the SMTP service. The message just disappears, except
Unless you teach spamd the valid usernames, the alternative is to have

*no* callout succeeding unless the sender is already grey/whitelisted.
Either way, that doesn't help the MSexchange problem, and callout is

broken by design anyway (DoS problem), it's not worth burning extra cpu
How about: --i-dont-want-to-receive-mail-from-people-using-exchange-2003

and --i-dont-want-to-receive-mail-from-people-using-callout-verification
I think a better solution would be for *more* people to use greylisting

implementations which do this, so that more MSexchange users will either

bother Microsoft to fix their bug, or script 'net stop smtpsvc;net start

smtpsvc' to run a few times a day so they can send mail to others too.
You can always revert r1.85 manually and recompile if you need...

I have only seen this when the 4xx error is sent at DATA time, not when
Most of the time with people running exchange, they don't care and don't

have a clue about what happens and argue that _your_ server is broken

because they don't have problems elsewhere.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

they're mutually exclusive:
4xx at RCPT, break callout verification.

4xx at DATA, break msexchange 2003 direct-to-mx delivery.

Well, 4xx at RCPT doesn't really break callout, it just delays the mail

a little bit further. Unless the callout is broken and answers the

sending server with a 5xx when it receives a 4xx as response from the

callout. But to be sure not to delay or break callouts, MAIL FROM:<>

should be redirected to the real server directly. However, this is quite

tricky to do as the communication with spamd has already started and you

could not just pipe the input to the real server.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

lol! i encounter this phenomenon on a regular basis: clueless people

misapplying blame for problems they are themselves the cause of.
when implementing some new STL code on a printing press, anything that

went wrong immediately thereafter was (incorrectly) attributed to my

code changes. this is a testament to the cluelessness of the people who

operate the machine. these situations remind me of a recent thread about

US crypto export laws ;).
i do end up having to manually whitelist a number of sender IPs and i

believe i now know why the emails didn't get through the greyfilter,

thanks for the info y'all. had a microsloth software distributor talk to

me for a while about the "value added" by having an all microsloth shop.

more like "cluelessness added" infrastructure: everybody should sell

their state-owned infrastructure to nepotistic private companies, it's

Unfortunately, this little MS-behaviour is very likely to be the "last

straw" that gets our greylisting turned off here.

Despite my logs that prove that greylisting has removed over 95% of

incoming spam before spamassassin has to deal with it, the fact that

some legitimate mail is lost or overly delayed has been deemed

unacceptable to the corporate masters.  The people inconvenienced by

this pay more in taxes than I make in a year so they need to be kept

happy.  And the mail that is often missed is quite often something

time-sensitive.  It really is a shame.  Greylisting has made such a huge

difference in the spam-volume here.  We receive about 10 complaints per

week about either mail that never came in or mail that came in too late

to act on.  These missing emails have sometimes cost us tens of

thousands of dollars in lost profits.  So that makes the tens of

thousands of blocked emails per day seem a lot less significant.  I have

whitelisted source IPs where possible but there is always some new

complaint right around the corner.  They appreciate the reduction in

spam that gets through but they are the first to complain if mail is

delayed or if they don't get something.  In the financial trading

sector, you would be shocked at the number of small, one-man analyst

companies operate from home and send out mail to subscribers from

dynamic IP addresses.  Couple that with lots of non-standard mailers and

it's a wonder any of their mail makes it past a decent SMTP

sanity-checker...
/J

Well, I think greylisting is still useful. It is just that if you want

to avoid losing mail or having it too much delayed, you should adjust

the settings for greylisting from 1h/4h to 9min/36h. Many mailers have

their queue runners at 15mins. Putting 36hours allows you to get mails

from servers with common pools or weird retry delays. These values were

just deduced from trial and error. Also greylisting should happen at

RCPT TO, and probably not at DATA as there are some widely used MTAs

that are buggy and choke when a 4xx error is sent in the DATA phase.

I've been running this at DATA for months, and not seen any

issues with it. 
	anyone here got hard evidence of such bugs - please show

me. Or is this just uninformed speculation?
	-Bob

err, wait, are you giving a 4xx in reply to DATA?

that is invalid.
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

eh, I wanted to send that in private mail.. too late ;(
rfc 2821 specifically forbids this behaviour.
<quote>

The DATA command can fail at only two points in the protocol exchange:
   -  If there was no MAIL, or no RCPT, command, or all such commands

      were rejected, the server MAY return a "command out of sequence"

      (503) or "no valid recipients" (554) reply in response to the DATA

      command.  If one of those replies (or any other 5yz reply) is

      received, the client MUST NOT send the message data; more

      generally, message data MUST NOT be sent unless a 354 reply is

      received.
   -  If the verb is initially accepted and the 354 reply issued, the

      DATA command should fail only if the mail transaction was

      incomplete (for example, no recipients), or if resources were

      unavailable (including, of course, the server unexpectedly

      becoming unavailable), or if the server determines that the

      message should be rejected for policy or other reasons.

</quote>
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

and that paragraph says right there, the server can decide it doesn't

have the resources to deal with it. no problem.  The RFC does not

forbid it
and 821 (which ain't dead yet) allows explicitly in the state transactions.

2821 is simply vague. 
-Bob

yes, but not in response to the DATA command (what I was talking about) 
well, 2821 is somewhat vague, 821 is kinda the definition of vague :)
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

no, you're wrong. right from rfc 2821:

----8<----

DATA

      I: 354 -> data -> S: 250

                        E: 552, 554, 451, 452

      E: 451, 554, 503

----8<----

explicitly - if I decide something is wrong after recieveing a data command

I may return a 451.

the table totally contradicts the text... kind of funny :)
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

On 5/24/07, Henning Brauer <lists-openbsd@bsws.de> wrote:
Perhaps that's why the current internet-draft for the revision of RFC

2821 has this in the table:
----

      DATA
         I: 354 -> data -> S: 250

                           E: 552, 554, 451, 452

                           E: 450, 550 (rejections for policy reasons)

----
(In a fixed width font, the E: lines line up with the S: above them)
That's from http://www.ietf.org/internet-drafts/draft-klensin-rfc2821bis-04.txt
The discussion of the revising is taking place on the

ietf-smtp@imc.org mailing list; information on subscribing and the

archive can be found at

        http://www.imc.org/ietf-smtp/
Philip Guenther

I agree with you Henning that per that paragraph a 4xy should not be

sent as a reply to the data command itself.  Instead a 4xy should

only be sent after a 354 has been sent and all the data received.

Which of course would undermine a lot of the benefit of spamd.  I

think one of the points is to reject the mail before the data is sent

down the pipe, allowing the data wastes the receivers bandwidth.
I went looking in other places within these two RFCs for indications

that a 4xy is legal in response to the DATA command.  I think I've

found points in both RFCs that make it legal to send a 4xy in response.
 From 821
4.3.  SEQUENCING OF COMMANDS AND REPLIES
.

.

.
     DATA

                I: 354 -> data -> S: 250

                                  F: 552, 554, 451, 452

                F: 451, 554

                E: 500, 501, 503, 421
 From 2821
4.3.2 Command-Reply Sequences
.

.

.
     DATA

       I: 354 -> data -> S: 250

                         E: 552, 554, 451, 452

       E: 451, 554, 503
Thus I think spamd is within the RFCs when it issues a 451 in  
Yup.
-Chad

Not really.
<quote>

   -  If the verb is initially accepted and the 354 reply issued, the

      DATA command should fail only if the mail transaction was

      incomplete ~snip~ or if the server determines that the

      message should be rejected for policy or other reasons.

</quote>
Policy reasons are accepted.
--

010100100110010101101110011000010111010101100100

010000010110110001101100011000010111001001100100
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

yeah yeah I was talking about 4xx to DATA
--

Henning Brauer, hb@bsws.de, henning@openbsd.org

BS Web Services, http://bsws.de

Full-Service ISP - Secure Hosting, Mail and DNS Services

Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

On 5/23/07, Henning Brauer <henning@openbsd.org> wrote:
At least for 451 and 421 replies, it sure seems legal to me.  To quote RFC 2821:
4.3.2 Command-Reply Sequences

...

   In addition to the codes listed below, any SMTP command can return

   any of the following codes if the corresponding unusual circumstances

   are encountered:

...

   421  Service shutting down and closing transmission channel
   Specific sequences are:

   DATA

      I: 354 -> data -> S: 250

                        E: 552, 554, 451, 452

      E: 451, 554, 503
Philip Guenther

The response to the DATA command is 354 as it should. But at the end of

the DATA phase, a 451 is returned.
--

010100100110010101101110011000010111010101100100

010000010110110001101100011000010111001001100100
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

With Mdaemon, the problem is fixed in version 9.02 and onwards

(http://tweakers.net/meuktracker/12778/MDaemon-9.0.4.html search for 4xx)

I got issues with both Mdaemon (multiple versions treating 4xx errors at

DATA as permanent errors) and with 3 servers running MS exchange 2003

(hiding messages from the queue and not retrying them until the service

was restarted).

I must admit it is quite hard to prove it as it is very hard to notice,

especially in the MS case as mails are not shown anymore in the queue

and exchange is not known for having some kind of useful logs. Also, it

is not always easy to get someone on the other side of the phone and ask

them to do some tests on their server when you are not managing it and

while they think *you* have problems, not them as they don't have

anything in their queue.

If you really wish some hard proof, I will have to install an MS

exchange server, although, as I said, exchange hides the mail in the

queue and doesn't log the failure, so I don't know what you would be

able to see.
I manage about 30 mail servers, all using greylisting for years (not

OpenBSD spamd, but a version running in the MTA). But as I greylist at

RCPT TO, I only noticed the problem it when clamav did go down and the

server was producing a 4xx error at DATA when it should have scanned the

mail.
Also, as an idea, I found it quite useful to whitelist only with a

triplet (from, to, IP), and not just the IP. Why? Because some people

are behind a firewall which allows them to go out with the same IP as

their mail server (yes, IPs are expensive in Europe), so windows

spamware is going out with the same IP than their mailserver and so

bypasses the filter.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

I have definately seen issues here with other implemntations,

because the 4XX code given, the XX's matter...  Have you seen
	I find this exceedingly unhelpful. as it makes the database

huge and does unnecessarily delay mail. Generally either a service

is reasonably well run, or it isn't. This also prevents the ease of

spamlogd pre-whitelisting stuff going out. 
	It sounds like you're speaking on this topic without

any actual experience with OpenBSD spamd, but rather something

like postfix or the sendmail-milter implementation.
	-Bob

I have seen this with 451 errors, not on spamd but with the exact same

error code as the one used for spamd.
spamd error: 451 Temporary failure, please try again later.
Indeed, but the error code is the same at the same time during the

transaction, so I don't see any reason why the behavior would be different.

For Mdaemon, you can check the changelogs from version 9.0.2 as they

acknowledge the problem.
[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]