Actually, DMARC works fine with mailing lists

There’s a common belief online that DMARC, an anti-phishing technology, prevents the proper operation of electronic mailing lists. To get around the perceived problem, many articles advise reconfiguring listserv software in ways that break the norms of SMTP.

However there is a configuration that allows mailing lists to work perfectly and still be able to relay messages in the presence of DMARC. Before describing it, let’s get some terminology straight.

Terminology and Basics

RFC5321.MailFrom (A.k.a. return-path.) A record in an SMTP message’s “envelope,” saying whom to contact in the event of delivery failure. Emails can travel in multiple hops, being relayed from server to server, before reaching their destination. During transit, the RFC5321.MailFrom header contains an email address for the server that sent the message along the current hop. RFC5322.From A header inside the message itself, hopefully preserved across hops. This is the “From” field that email clients display, and it is supposed to be the email address of the person responsible for the email. Sender A less commonly known email header which indicates an intermediary who writes or sends a message on behalf of someone else. RFC 5322 gives the example of a secretary (Sender) who writes and sends a message on behalf of the boss (From). SPF Sender Policy Framework. A TXT DNS record which publishes the addresses of mail servers who are allowed to send or relay messages for a (sub)domain. Receiving mail servers can inspect RFC5321.MailFrom and bounce the message if it doesn’t match the servers listed in the SPF record. DKIM DomainKeys Identified Mail. DKIM are TXT DNS records which each contain a public key that is permitted to cryptographically sign the body, and a list of which email headers should be covered in the signature. Sending servers must know the private key in order to sign outgoing mail, and receiving mail servers can check the email against the public key to make sure it passes. This ensures that only authorized servers can originate mail for a domain, and also that relay servers cannot modify the message. DMARC Domain-based Message Authentication, Reporting and Conformance. A TXT DNS record which publishes a domain’s policy for how receiving mail servers should act on SPF or DKIM failure. Actions include ignoring the problem, sending the message to the spam folder, or outright bouncing the message. The DMARC record contains some other settings such as an email address for mail servers to report any non-conforming mail they receive. Mailing list A relay mail server which hosts a special address called a reflector . The mailing list relays any message sent to its reflector to the addresses of list subscribers. Mailing lists are used for group discussions, including collaborative software development.

While earlier methods such as PGP have existed to unmask mail forgery, DMARC has become the popular choice and is honored today on all major mail servers. However people whose domains employ DMARC are unable to send messages through traditionally-configured mailing lists. During DMARC adoption on major domains such as gmail and yahoo, mailing lists started having widespread problems.

Some people called DMARC a broken standard. They failed to implement elegant adjustments to mailing list behavior, and used hacky workarounds to fix the problem instead.

Let’s examine how a message passes or fails DMARC, and then learn how to fix lists. A message passes if either SPF or DKIM passes, and only fails if both SPF and DKIM fail. This way SPF-only and DKIM-only messages can pass DMARC, but messages without either SPF/DKIM will always fail.

Secondly, the message must pass “DMARC alignment.” Alignment means two different things:

1. For SPF, it means that RFC5321.MailFrom == RFC5322.From
2. For DKIM, it means that the message has one valid DKIM signature with “d=RFC5322.From”.

Fixing list behavior

If DKIM is not used

A mailing list is going to have a hard time relaying messages for any domain that uses DMARC and SPF but not DKIM. Consider this message:

Return-Path: <[email protected]>
From: "Sally Sender" <[email protected]>
To: "Fun List" <[email protected]>
Subject: Hi

Hi everyone

This would ordinarily be a proper reflection to a subscriber:

Return-Path: <[email protected]>
From: "Sally Sender" <[email protected]>
Sender: "Fun List" <[email protected]>
To: "Roger Reader" <[email protected]>
Subject: Hi

Hi everyone

However destination.com will reject this message if mailinglist.org is not in the SPF record for dmarcdomain.com . Furthermore a good SPF record is not enough, since the Return-Path and From are out of alignment (they specify different domains).

Dealing with this situation is what causes the hacky workarounds. Lists have to use techniques like From-munging or MIME message wrapping to get the mail through.

Here is what From-munging looks like:

Return-Path: <[email protected]>
From: "Sally Sender via Fun List" <[email protected]>
Reply-To: "Sally Sender" <[email protected]>
To: "Roger Reader" <[email protected]>
Subject: Hi

Hi everyone

The recipient would only have to check DMARC for mailinglist.org which would pass. However this is a poor use of email, since it’s misrepresenting who originated the message. Also email clients often have a degraded interface with respect to the Reply-To header. It’s usually not visible in the message list, not used for sorting, and not added to the address book.

If DKIM is used

If the sending domain uses DKIM, it avoids the need for From-munging or other hacks. It works under the condition that the list does not modify the message.

Consider the properly reflected message again:

Return-Path: <[email protected]>
From: "Sally Sender" <[email protected]>
Sender: "Fun List" <[email protected]>
To: "Roger Reader" <[email protected]>
Subject: Hi

Hi everyone

SPF will fail, as we saw earlier, so DMARC will try a DKIM check. The From, Subject and body were not modified so they will be properly signed. DKIM checks alignment between RFC5322.From and the signature’s domain, which will also match. DKIM passes, and message gets delivered.

Should be perfect, right? Well it could be, except lists traditionally add extra information in the subject and body of relayed messages, and the modified fields don’t pass a DKIM check. Messages from traditional mailing lists typically look more like this:

Return-Path: <[email protected]>
From: "Sally Sender" <[email protected]>
Sender: "Fun List" <[email protected]>
To: "Roger Reader" <[email protected]>
Subject: [fun] Hi

Hi everyone

--
You are subscribed to the fun list, to unsubscribe
visit https://mailinglist.org/unsub/123

The subject line tag is typically used to sort the messages into a separate mailbox following user-defined rules in the email client. The unsubscribe link in the body is a convenience (and avoids government fines from violating the CAN-SPAM act.)

We need to give the client a way to sort mail, unsubscribe, etc without modifying the parts of the message signed by DKIM. Fortunately, there are RFCs for this. RFC2369 from 1998 and RFC2919 from 2001 both predate the DMARC machinery. The first introduces header fields for list information and control.

For our example RFC2369 allows us to add List-Unsubscribe: <https://mailinglist.org/unsub/123> . It also introduces headers like List-Help, List-Subscribe, List-Post, List-Owner, and List-Archive. What’s more, many mail clients understand these headers. Gmail adds an unsubscribe button to the web interface when it detects List-Unsubscribe.

The second RFC offers a way to identify a list with another header, like List-Id: The Fun List <fun.mailinglist.org> . Mail client rules can query this header rather than checking whether the subject field contains [fun] .

Recommendation

It is reasonable nowadays to require that mailing list users whose domains use DMARC also enable DKIM. In fact the list software could check the sender’s domain at subscription time and raise an error if their domain uses DMARC but not DKIM.

Lists should keep the From address, the Subject, and the Message totally unchanged. They should add a Sender header to indicate their relay role, and set at least the List-Id and List-Unsubscribe headers for mailbox rules and subscription management.

This configuration will allow mailing lists to function as proper SMTP citizens in the age of DMARC.