An SMTP server acting as an open relay is a security vulnerability that allows any internet user to send email through your server, regardless of whether the email is destined for a local recipient. This is a critical security flaw because it can turn your server into a spam distribution hub, leading to IP blacklisting and reputational damage.

Let’s look at how this happens with a common setup, Postfix, and how to lock it down.

The Problem: An Open Relay Attack Scenario

Imagine an attacker on the internet wants to send spam emails. They discover your mail server, mail.example.com, is configured to accept mail from anyone and relay it to any destination.

Here’s a simplified interaction they might have with your server:

<-- 220 mail.example.com ESMTP Postfix
--> EHLO attacker.com
<-- 250-mail.example.com
<-- 250-PIPELINING
<-- 250-SIZE 10240000
<-- 250-VRFY
<-- 250-ETRN
<-- 250-ENHANCEDSTATUSCODES
<-- 250-8BITMIME
<-- 250 DSN
--> MAIL FROM:<victim@external.com>
<-- 250 2.1.0 Ok
--> RCPT TO:<randomuser@hacker.net>
<-- 250 2.1.5 Ok
--> DATA
<-- 354 Start mail input; end with <CRLF>.<CRLF>
Subject: SPAM!

This is a spam message.
.
<-- 250 2.0.0 Ok: queued as 1234567890

Notice how the server accepted MAIL FROM:<victim@external.com> and RCPT TO:<randomuser@hacker.net> without any authentication or checks to see if attacker.com was authorized to send mail through mail.example.com. This is the essence of an open relay.

The Core Issue: Trusting the HELO/EHLO Name

Historically, SMTP servers relied heavily on the HELO or EHLO command to identify the sending host. If a server was configured to accept mail for any recipient, and didn’t have strong restrictions on who could send mail through it, it would become an open relay. The server essentially trusted that any client connecting and identifying itself was legitimate.

Common Causes and Fixes

The most common reason for an SMTP open relay is misconfiguration of access control lists (ACLs) or smtpd_recipient_restrictions in Postfix (a very popular Mail Transfer Agent).

  1. Incorrect mynetworks Configuration:

    • What broke: The mynetworks parameter in Postfix defines which IP addresses and networks are considered "trusted" and allowed to relay mail without further checks. If this list is too broad, it can inadvertently include external, untrusted networks.
    • Diagnosis: Examine your Postfix main.cf file for the mynetworks setting.
      grep "mynetworks" /etc/postfix/main.cf
      
    • Fix: Ensure mynetworks only includes your trusted internal IP addresses and localhost. For example, if your internal network is 192.168.1.0/24 and your server’s IP is 192.168.1.10, you’d set it like this:
      mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.1.0/24
      
      Then reload Postfix:
      postfix reload
      
    • Why it works: This explicitly tells Postfix which networks it should implicitly trust to relay mail, preventing unknown external IPs from doing so.
  2. Overly Permissive smtpd_recipient_restrictions:

    • What broke: This is the primary mechanism for controlling who can send mail through your server. If restrictions are missing or too permissive, mail can be relayed.
    • Diagnosis: Check your main.cf for smtpd_recipient_restrictions.
      grep "smtpd_recipient_restrictions" /etc/postfix/main.cf
      
      A common problematic configuration might look like this, omitting critical checks:
      smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
      
      The issue here is reject_unauth_destination is after permit_sasl_authenticated and permit_mynetworks. If you have any external IP in mynetworks or an improperly configured SASL, it can be exploited.
    • Fix: The most secure and standard configuration is:
      smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
      
      This order is crucial. It first allows mail from your trusted networks (permit_mynetworks), then from authenticated users (permit_sasl_authenticated), and then it rejects any attempt to relay mail to a destination that is not for a local recipient.
      postfix reload
      
    • Why it works: It enforces a strict policy: mail is only accepted if it’s from a trusted source or authenticated user, and the destination is actually handled by this mail server. Anything else is rejected.
  3. Missing smtpd_sender_restrictions:

    • What broke: While smtpd_recipient_restrictions handles the destination, smtpd_sender_restrictions can add an extra layer to reject mail based on the sender’s origin before Postfix even considers the recipient.
    • Diagnosis: Check main.cf.
      grep "smtpd_sender_restrictions" /etc/postfix/main.cf
      
      If it’s not present, or if it’s configured to permit everything, it’s a weakness.
    • Fix: Add checks to smtpd_sender_restrictions to reject mail from known bad sources or those that fail basic checks. A simple addition to your main.cf might look like this:
      smtpd_sender_restrictions = reject_unknown_sender_domain, permit_sasl_authenticated, permit_mynetworks
      
      And ensure your smtpd_recipient_restrictions is still correctly configured as above.
      postfix reload
      
    • Why it works: This adds a pre-check. If the sender’s domain doesn’t exist or is clearly malicious, the connection can be dropped early, reducing load and preventing further processing that could lead to relaying.
  4. Allowing Anonymous Access to smtpd_relay_restrictions:

    • What broke: smtpd_relay_restrictions is a newer, more granular control that defines which clients are allowed to relay mail. If it’s misconfigured to allow anonymous clients to relay, it bypasses other checks.
    • Diagnosis: Check main.cf.
      grep "smtpd_relay_restrictions" /etc/postfix/main.cf
      
      A common mistake is setting it to permit_mynetworks.
    • Fix: The recommended setting is:
      smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
      
      This ensures that only trusted networks or authenticated users can relay, and it still enforces the reject_unauth_destination rule.
      postfix reload
      
    • Why it works: This parameter acts as a gatekeeper specifically for relaying, ensuring that only authorized clients can perform this action.
  5. Missing or Incorrect SASL Authentication:

    • What broke: If you intend for external users to send mail through your server (e.g., for remote employees), they must authenticate using SASL (Simple Authentication and Security Layer). If SASL is not enforced or improperly configured, attackers might exploit it.
    • Diagnosis: Check your main.cf for SASL related settings and ensure smtpd_recipient_restrictions and smtpd_relay_restrictions include permit_sasl_authenticated.
      grep "smtpd_sasl_auth_enable" /etc/postfix/main.cf
      grep "smtpd_recipient_restrictions" /etc/postfix/main.cf
      
    • Fix: Ensure SASL is enabled and properly configured with a robust authentication mechanism (e.g., Dovecot SASL). Then, make sure permit_sasl_authenticated is present in your smtpd_recipient_restrictions and smtpd_relay_restrictions directives.
      postfix reload
      
    • Why it works: This forces any sender that isn’t part of your trusted network to prove their identity via username and password before being allowed to send mail, preventing unauthorized relaying.
  6. Allowing smtpd_client_restrictions to Permit Relaying:

    • What broke: smtpd_client_restrictions can also control relaying. If a rule here allows unknown clients to relay, it can bypass smtpd_recipient_restrictions.
    • Diagnosis: Check main.cf.
      grep "smtpd_client_restrictions" /etc/postfix/main.cf
      
    • Fix: Ensure smtpd_client_restrictions does not contain rules that permit relaying for unknown clients. A common safe configuration might be:
      smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_rbl_client zen.spamhaus.org
      
      The key is that it doesn’t have a broad permit for unauthenticated clients.
      postfix reload
      
    • Why it works: This acts as an early filter, blocking known bad IP addresses or clients that don’t meet basic criteria before they even attempt to send mail, reducing the chances of relaying.

After applying these fixes, you should also check your server’s public IP address against RBLs (Real-time Blackhole Lists) to ensure it hasn’t already been blacklisted.

The next error you’ll likely encounter if you’ve fixed open relay is a 550 5.7.1 Relaying denied message when an unauthorized client attempts to send mail.

Want structured learning?

Take the full Smtp course →