An SMTP server acting as an open relay is a security vulnerability that allows any internet user to send email through your server, regardless of whether the email is destined for a local recipient. This is a critical security flaw because it can turn your server into a spam distribution hub, leading to IP blacklisting and reputational damage.
Let’s look at how this happens with a common setup, Postfix, and how to lock it down.
The Problem: An Open Relay Attack Scenario
Imagine an attacker on the internet wants to send spam emails. They discover your mail server, mail.example.com, is configured to accept mail from anyone and relay it to any destination.
Here’s a simplified interaction they might have with your server:
<-- 220 mail.example.com ESMTP Postfix
--> EHLO attacker.com
<-- 250-mail.example.com
<-- 250-PIPELINING
<-- 250-SIZE 10240000
<-- 250-VRFY
<-- 250-ETRN
<-- 250-ENHANCEDSTATUSCODES
<-- 250-8BITMIME
<-- 250 DSN
--> MAIL FROM:<victim@external.com>
<-- 250 2.1.0 Ok
--> RCPT TO:<randomuser@hacker.net>
<-- 250 2.1.5 Ok
--> DATA
<-- 354 Start mail input; end with <CRLF>.<CRLF>
Subject: SPAM!
This is a spam message.
.
<-- 250 2.0.0 Ok: queued as 1234567890
Notice how the server accepted MAIL FROM:<victim@external.com> and RCPT TO:<randomuser@hacker.net> without any authentication or checks to see if attacker.com was authorized to send mail through mail.example.com. This is the essence of an open relay.
The Core Issue: Trusting the HELO/EHLO Name
Historically, SMTP servers relied heavily on the HELO or EHLO command to identify the sending host. If a server was configured to accept mail for any recipient, and didn’t have strong restrictions on who could send mail through it, it would become an open relay. The server essentially trusted that any client connecting and identifying itself was legitimate.
Common Causes and Fixes
The most common reason for an SMTP open relay is misconfiguration of access control lists (ACLs) or smtpd_recipient_restrictions in Postfix (a very popular Mail Transfer Agent).
-
Incorrect
mynetworksConfiguration:- What broke: The
mynetworksparameter in Postfix defines which IP addresses and networks are considered "trusted" and allowed to relay mail without further checks. If this list is too broad, it can inadvertently include external, untrusted networks. - Diagnosis: Examine your Postfix
main.cffile for themynetworkssetting.grep "mynetworks" /etc/postfix/main.cf - Fix: Ensure
mynetworksonly includes your trusted internal IP addresses and localhost. For example, if your internal network is192.168.1.0/24and your server’s IP is192.168.1.10, you’d set it like this:
Then reload Postfix:mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.1.0/24postfix reload - Why it works: This explicitly tells Postfix which networks it should implicitly trust to relay mail, preventing unknown external IPs from doing so.
- What broke: The
-
Overly Permissive
smtpd_recipient_restrictions:- What broke: This is the primary mechanism for controlling who can send mail through your server. If restrictions are missing or too permissive, mail can be relayed.
- Diagnosis: Check your
main.cfforsmtpd_recipient_restrictions.
A common problematic configuration might look like this, omitting critical checks:grep "smtpd_recipient_restrictions" /etc/postfix/main.cf
The issue here issmtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destinationreject_unauth_destinationis afterpermit_sasl_authenticatedandpermit_mynetworks. If you have any external IP inmynetworksor an improperly configured SASL, it can be exploited. - Fix: The most secure and standard configuration is:
This order is crucial. It first allows mail from your trusted networks (smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destinationpermit_mynetworks), then from authenticated users (permit_sasl_authenticated), and then it rejects any attempt to relay mail to a destination that is not for a local recipient.postfix reload - Why it works: It enforces a strict policy: mail is only accepted if it’s from a trusted source or authenticated user, and the destination is actually handled by this mail server. Anything else is rejected.
-
Missing
smtpd_sender_restrictions:- What broke: While
smtpd_recipient_restrictionshandles the destination,smtpd_sender_restrictionscan add an extra layer to reject mail based on the sender’s origin before Postfix even considers the recipient. - Diagnosis: Check
main.cf.
If it’s not present, or if it’s configured to permit everything, it’s a weakness.grep "smtpd_sender_restrictions" /etc/postfix/main.cf - Fix: Add checks to
smtpd_sender_restrictionsto reject mail from known bad sources or those that fail basic checks. A simple addition to yourmain.cfmight look like this:
And ensure yoursmtpd_sender_restrictions = reject_unknown_sender_domain, permit_sasl_authenticated, permit_mynetworkssmtpd_recipient_restrictionsis still correctly configured as above.postfix reload - Why it works: This adds a pre-check. If the sender’s domain doesn’t exist or is clearly malicious, the connection can be dropped early, reducing load and preventing further processing that could lead to relaying.
- What broke: While
-
Allowing Anonymous Access to
smtpd_relay_restrictions:- What broke:
smtpd_relay_restrictionsis a newer, more granular control that defines which clients are allowed to relay mail. If it’s misconfigured to allow anonymous clients to relay, it bypasses other checks. - Diagnosis: Check
main.cf.
A common mistake is setting it togrep "smtpd_relay_restrictions" /etc/postfix/main.cfpermit_mynetworks. - Fix: The recommended setting is:
This ensures that only trusted networks or authenticated users can relay, and it still enforces thesmtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destinationreject_unauth_destinationrule.postfix reload - Why it works: This parameter acts as a gatekeeper specifically for relaying, ensuring that only authorized clients can perform this action.
- What broke:
-
Missing or Incorrect SASL Authentication:
- What broke: If you intend for external users to send mail through your server (e.g., for remote employees), they must authenticate using SASL (Simple Authentication and Security Layer). If SASL is not enforced or improperly configured, attackers might exploit it.
- Diagnosis: Check your
main.cffor SASL related settings and ensuresmtpd_recipient_restrictionsandsmtpd_relay_restrictionsincludepermit_sasl_authenticated.grep "smtpd_sasl_auth_enable" /etc/postfix/main.cf grep "smtpd_recipient_restrictions" /etc/postfix/main.cf - Fix: Ensure SASL is enabled and properly configured with a robust authentication mechanism (e.g., Dovecot SASL). Then, make sure
permit_sasl_authenticatedis present in yoursmtpd_recipient_restrictionsandsmtpd_relay_restrictionsdirectives.postfix reload - Why it works: This forces any sender that isn’t part of your trusted network to prove their identity via username and password before being allowed to send mail, preventing unauthorized relaying.
-
Allowing
smtpd_client_restrictionsto Permit Relaying:- What broke:
smtpd_client_restrictionscan also control relaying. If a rule here allows unknown clients to relay, it can bypasssmtpd_recipient_restrictions. - Diagnosis: Check
main.cf.grep "smtpd_client_restrictions" /etc/postfix/main.cf - Fix: Ensure
smtpd_client_restrictionsdoes not contain rules that permit relaying for unknown clients. A common safe configuration might be:
The key is that it doesn’t have a broadsmtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.orgpermitfor unauthenticated clients.postfix reload - Why it works: This acts as an early filter, blocking known bad IP addresses or clients that don’t meet basic criteria before they even attempt to send mail, reducing the chances of relaying.
- What broke:
After applying these fixes, you should also check your server’s public IP address against RBLs (Real-time Blackhole Lists) to ensure it hasn’t already been blacklisted.
The next error you’ll likely encounter if you’ve fixed open relay is a 550 5.7.1 Relaying denied message when an unauthorized client attempts to send mail.