Diagnosing DNS-sourced SPF problems

So, your outgoing messages are failing SPF, or your own SPF verifier is failing message for no good reason, and you want to figure out why? It might be the DNS of the sending domain.

Example domain here is myatb.com, which once had DNS troubles. Domains like blah.myatb.com will not be covered here, but the process is similar (if a bit more complicated.)

If DNS lookups fail or timeout or otherwise get weird, your software might map the SPF results to "fail" (this behavior may be correct if there is a "ptr" mechanism being used, more on that below). A good verifier knows better, but it's up to the implementer to make this distinction (and up to you to utilize the information which your verification code provides.)

Before suspecting DNS problems, confirm that they have a valid SPF record.

You can retrieve the SPF record with:

But it's probably easier to use the first form at http://www.kitterman.com/spf/validate.html

(fill in the domain only and hit "Get SPF record"... it shouldn't give any errors. (If there is no SPF record, you can skip down to the DNS section.)

Now use the second form at that link. In the fields, fill in "myatb.com" and the contents of the quoted string above:

The output includes:

...their SPF record is strange: the two spaces before "~all" aren't a problem, but they should probably remove mx:mail.myatb.com, and replace mx:martechservices.com for mx:mail.martechservices.com. If martechservices.com was actually a different IP, their record as-is could cause mail from "blah@myatb.com" sent from a martechservices.com IP to fail SPF, since it's IP is not properly included. However, it turns out that martechservices.com has the same IP as myatb.com, so the whole mx:martechservices.com entry is redundant as well. Additionally, the IP specified is identical to their mx, and in fact their A record as well. Their record should probably just be:

...or better, because it will save a lookup:

Likely a case of someone not knowing what they were doing filling in a lot of stuff to be sure, or adding it all in just in case it changes in the future. In the latter case, this would suffice:

...although it wouldn't be as efficient for resolvers on the other end because it's saying the same thing three times (a==mx==207.173.6.116).

Trying the third form at kitterman.com, we can put in actual values from the email that failed, or we can make up our own. I will make up our own for this example:

Submit the form and see what happens. In this example, it would pass, despite their redundant and misconfigured SPF record.

Most mail servers also require a "FCRDNS" -- a validating PTR record for the IP: take the IP of the sending mail server (from the Received header of the message) and do a reverse lookup. It will return one or more host names. One of those host names must resolve back to the IP. You may as well let the form above take care of that checking for you.

If there is a "ptr" mechanism in the SPF record (they are almost always advised against) then the PTR records discovered by the DNS lookup must additionally match or end with the domain in question in order for the "ptr" mechanism to match. Note: a DNS error when looking up the PTR records will make the "ptr" mechanism not match. A DNS error when looking up the A records for any domains discovered in the PTR lookup simply means those domains will not be able to be used as possible matches.

If SPF all checks out on the form, but not in your system, there may still be problems with DNS causing failures or intermittent failures in one place (your system) but not another (the SPF checking form website).

If you want to see if the domain lookups are failing on your system, you can check whatever logs you keep for timeouts. You might also scan /var/log/messages on your nameserver machine(s) for this kind of output (obviously, your logs may look different):

In these logs, the IP at the end is the nameserver (usually one of the TLD zone nameservers) that "named" was working with, but it doesn't tell you the IP of the server that is "lame".

Note that depending on your logging and what your MX is handling, there could be lots of these messages in /var/log/messages, and you may see lots of timeouts in whatever logs you keep. This is usually normal (spammer domains, mostly.)

Let's check out the DNS records:

Note the first two ns*.martechservices.com lines: these are the nameservers that the "com" nameservers lists for the myatb.com zone. As it happens with this particular domain, ns.martechservices.com isn't responding, and if you ran the dig command and the resolver happened to chose "ns" instead of "ns2" (the nameserver in parenthesis on the last line was the nameserver responding with the IP) the end of the above output would list a failure instead of listing the IP. Anyway, let's check each nameserver individually to see if it responds (there may be a lot in the list; in this case there are only two. Watch out for unexpected but common ns1.easydns.org vs. ns2.easydns.net discrepancies, etc.)

Aha, ns.martechservices.com is not responding. This will cause DNS failures (and possibly inappropriate SPF failures as a result) when checking SPF for this domain.

Let's also see if there are any SOA issues ("Start of Authority", a type of DNS record). Valid SOA is not a requirement of SPF, but without it the domain will likely have problems in general.

...looks like a problem. Normally, something like this comes back:

This is likely the culprit:

Compare to:

...I don't know enough about DNS to really call that out as "wrong", but I'm pretty sure it is.