nginx | 05/12/2026 | By Thijs Eilander

Postfix + Dovecot Mail Server Setup on Debian 12 and 13 (2026 Guide)

Running your own mail server in 2026 is not the nightmare people say it is — if you use the right packages and follow a methodical setup. The horror stories you’ve read online are mostly about people who skipped DNS records, used outdated TLS configs, or ran ancient software with no spam filtering. Do it right and you’ll have a server that delivers reliably, stays off blacklists, and handles a few hundred mailboxes without breaking a sweat.

Postfix + Dovecot + Rspamd is the production standard combination in 2026. Postfix handles SMTP (sending and receiving), Dovecot handles IMAP (what your mail client connects to), and Rspamd filters spam, signs outgoing mail with DKIM, and plugs into Postfix as a milter. All three are available from the myguard APT repository — updated within hours of upstream releases, so you’re never stuck on a version Debian stable froze two years ago.

This guide is thorough by design. You’ll finish with a fully working mail server on Debian 12 (Bookworm) or Debian 13 (Trixie), including TLS everywhere, SPF, DKIM, DMARC, spam filtering, and a 10/10 score on mail-tester.com. No shortcuts that come back to bite you at 2am.

What you’re building

A complete inbound + outbound mail server with:

  • Postfix — receives mail on port 25 from the internet, accepts submissions from your users on port 587 (STARTTLS) and 465 (SMTPS)
  • Dovecot — serves mail to your IMAP clients on port 993 (IMAPS), delivers mail from Postfix to Maildir via LMTP
  • Rspamd — scans inbound mail for spam, signs outbound mail with DKIM, integrates with Postfix as a milter
  • Redis — Rspamd’s backend for Bayes learning and rate limiting
  • Let’s Encrypt TLS — proper certificates so mail clients don’t complain
  • SPF, DKIM, DMARC — the three DNS records that prevent your mail from being treated as spam

Before you start — prerequisites

  • A VPS or dedicated server with a static IP. Dynamic IPs are blacklisted by virtually every major mail provider. Hetzner, OVH, Contabo, and DigitalOcean all work well.
  • Port 25 unblocked. Many providers block outbound port 25 by default. Contact support and ask them to unblock it. Hetzner does it within minutes. DigitalOcean requires account verification.
  • A domain you control. You need to be able to edit DNS records — MX, TXT, and PTR.
  • Reverse DNS (PTR record). Your server’s IP must resolve back to your mail hostname. Log in to your provider’s control panel and set the PTR record for your IP to mail.example.com. This is separate from your regular DNS — it’s set at the IP level.
  • A valid hostname. Set your server’s hostname to match your PTR record: hostnamectl set-hostname mail.example.com.
  • Firewall rules. Open ports 25 (SMTP), 465 (SMTPS), 587 (SMTP submission), 993 (IMAPS). Block 110 (POP3) and 143 (plain IMAP) — there’s no reason to offer unencrypted access.

Step 1 — Add the repository and install packages

wget https://deb.myguard.nl/pool/myguard.deb
dpkg -i myguard.deb
apt-get update
apt-get install postfix postfix-pcre dovecot-core dovecot-imapd dovecot-lmtpd rspamd redis-server

The installer will ask two questions during Postfix installation:

  • Configuration type: select Internet Site
  • System mail name: enter your domain — example.com, not mail.example.com

After installation, get a TLS certificate before touching any config. Everything breaks without one:

apt-get install certbot
certbot certonly --standalone -d mail.example.com

Step 2 — Configure Postfix

Postfix is configured through two files: /etc/postfix/main.cf (settings) and /etc/postfix/master.cf (service definitions). Replace your main.cf with this production configuration:

# Identity
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain

# Network
inet_interfaces = all
inet_protocols = ipv4
mynetworks = 127.0.0.0/8

# Mailbox delivery via Dovecot LMTP
mailbox_transport = lmtp:unix:private/dovecot-lmtp
virtual_transport = lmtp:unix:private/dovecot-lmtp

# TLS inbound (from other mail servers)
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_protocols = !SSLv2,!SSLv3,!TLSv1,!TLSv1.1
smtpd_tls_mandatory_protocols = !SSLv2,!SSLv3,!TLSv1,!TLSv1.1
smtpd_tls_mandatory_ciphers = high
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_loglevel = 1

# TLS outbound (to other mail servers)
smtp_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtp_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtp_tls_security_level = may
smtp_tls_protocols = !SSLv2,!SSLv3,!TLSv1,!TLSv1.1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_loglevel = 1

# SASL authentication (Dovecot handles this)
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous

# Anti-spam restrictions
smtpd_recipient_restrictions =
  permit_mynetworks,
  permit_sasl_authenticated,
  reject_unauth_destination,
  reject_invalid_hostname,
  reject_non_fqdn_hostname,
  reject_non_fqdn_sender,
  reject_non_fqdn_recipient,
  reject_unknown_sender_domain,
  reject_unknown_recipient_domain,
  reject_rbl_client zen.spamhaus.org

# Rspamd milter
smtpd_milters = inet:127.0.0.1:11332
non_smtpd_milters = inet:127.0.0.1:11332
milter_protocol = 6
milter_default_action = accept

# Limits
message_size_limit = 52428800
mailbox_size_limit = 0

Now enable the submission ports in /etc/postfix/master.cf. Find (or uncomment) these lines:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Step 3 — Configure Dovecot

Dovecot’s configuration lives in /etc/dovecot/dovecot.conf and the /etc/dovecot/conf.d/ directory. The conf.d approach is clean — each file handles one concern. Edit these key files:

/etc/dovecot/dovecot.conf

protocols = imap lmtp

/etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/Maildir
namespace inbox {
  inbox = yes
}

/etc/dovecot/conf.d/10-ssl.conf

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem
ssl_min_protocol = TLSv1.2
ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256
ssl_prefer_server_ciphers = yes

/etc/dovecot/conf.d/10-auth.conf

disable_plaintext_auth = yes
auth_mechanisms = plain login

!include auth-system.conf.ext

/etc/dovecot/conf.d/10-master.conf

This is the most important file — it wires Dovecot to Postfix via LMTP and auth sockets:

service imap-login {
  inet_listener imap {
    port = 0   # disable plain IMAP
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }
  unix_listener auth-userdb {
    mode = 0600
    user = dovecot
  }
}

Create mail users

For a simple setup, use system users. Create a mail user for each mailbox:

useradd -m -s /sbin/nologin alice
useradd -m -s /sbin/nologin bob
passwd alice   # set a password for IMAP login

Step 4 — Configure Rspamd

Rspamd is a modern spam filter that replaces SpamAssassin — it’s significantly faster and has better default rules. It integrates with Postfix as a milter (mail filter), meaning Postfix runs every message through Rspamd before accepting it.

Enable Redis backend

cat > /etc/rspamd/local.d/redis.conf <<'EOF'
servers = "127.0.0.1";
EOF

Generate DKIM keys

DKIM cryptographically signs your outgoing mail. Every major mail provider (Gmail, Outlook, Yahoo) checks DKIM before deciding whether your mail is legitimate. Without it, you’ll land in spam.

mkdir -p /var/lib/rspamd/dkim
rspamadm dkim_keygen -s mail -d example.com -k /var/lib/rspamd/dkim/mail.key > /tmp/dkim-dns.txt
cat /tmp/dkim-dns.txt   # you'll need this DNS record shortly
chmod 640 /var/lib/rspamd/dkim/mail.key
chown rspamd:rspamd /var/lib/rspamd/dkim/mail.key

Configure DKIM signing

cat > /etc/rspamd/local.d/dkim_signing.conf <<'EOF'
path = "/var/lib/rspamd/dkim/$domain.$selector.key";
selector = "mail";
EOF

Set spam action thresholds

cat > /etc/rspamd/local.d/actions.conf <<'EOF'
reject = 15;      # reject outright (clear spam)
greylist = 4;     # greylist suspicious mail
add_header = 6;   # add X-Spam header
EOF

Enable the Rspamd web UI (optional)

cat > /etc/rspamd/local.d/worker-controller.inc <<'EOF'
bind_socket = "127.0.0.1:11334";
password = "$(rspamadm pw -p YOUR_PASSWORD_HERE)";
EOF

Access the UI through an SSH tunnel: ssh -L 11334:127.0.0.1:11334 yourserver, then open http://localhost:11334.

Step 5 — DNS records

These are the records that determine whether your mail gets delivered or dumped in spam. Every single one matters.

MX record — where to deliver mail to your domain

example.com.    IN  MX  10  mail.example.com.

A record — your mail server’s IP

mail.example.com.  IN  A  YOUR_SERVER_IP

PTR record (reverse DNS) — set at your provider, not in your DNS panel

Log into your VPS provider control panel. Find the IP management section. Set the PTR/reverse DNS for your server IP to mail.example.com. This is checked by most receiving servers. If it’s wrong, your mail will be rejected or flagged.

SPF record — which servers are allowed to send mail for your domain

example.com.  IN  TXT  "v=spf1 mx -all"

The mx means your MX server is allowed to send. The -all means everyone else is a hard fail. If you also send from a third-party service (Mailchimp, Sendgrid), add their include: clause before the -all.

DKIM record — the public key that validates your DKIM signatures

The public key was output in /tmp/dkim-dns.txt in the previous step. It looks like:

mail._domainkey.example.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA..."

Copy the full string from your file. The key is long — make sure you get all of it.

DMARC record — policy for failed SPF/DKIM

_dmarc.example.com.  IN  TXT  "v=DMARC1; p=quarantine; rua=mailto:dmarc@example.com; ruf=mailto:dmarc@example.com; fo=1"

Start with p=quarantine (send failures to spam) rather than p=reject (block them). Once you’ve monitored DMARC reports for a month and confirmed everything is aligned, switch to p=reject.

MTA-STS and TLSRPT (optional but recommended)

MTA-STS tells sending servers they must use TLS when delivering to you. Add:

_mta-sts.example.com.  IN  TXT  "v=STSv1; id=20260512"
_smtp._tls.example.com.  IN  TXT  "v=TLSRPTv1; rua=mailto:tlsrpt@example.com"

Then create a policy file at https://mta-sts.example.com/.well-known/mta-sts.txt:

version: STSv1
mode: enforce
mx: mail.example.com
max_age: 86400

Step 6 — Start and verify

systemctl enable --now redis-server rspamd dovecot postfix
systemctl restart redis-server rspamd dovecot postfix

Check each service is up

systemctl status postfix dovecot rspamd redis-server

Check ports are listening

ss -tlnp | grep -E '25|465|587|993|11332'

You should see Postfix on 25, 465, 587; Dovecot on 993; Rspamd milter on 11332.

Test SMTP inbound

telnet mail.example.com 25
# Should see: 220 mail.example.com ESMTP Postfix

Test IMAP

openssl s_client -connect mail.example.com:993
# Should see: * OK Dovecot ready.

Send a test email

echo "Test mail" | mail -s "Test" you@gmail.com
tail -f /var/log/mail.log

Watch the log for errors. A clean delivery looks like:

postfix/smtp[12345]: ... status=sent (250 2.0.0 OK)

Step 7 — Test your score at mail-tester.com

Go to mail-tester.com. It gives you a unique address to send a test email to, then scores your setup from 1–10 across deliverability, authentication, blacklists, and content.

echo "Testing my mail server" | mail -s "Mail tester" [your-unique-address]@mail-tester.com

Common issues and fixes:

  • SPF fails: Check your TXT record syntax. Wait up to 10 minutes for DNS propagation.
  • DKIM fails: Check the DKIM public key matches the private key. Re-run rspamadm dkim_keygen if needed.
  • DMARC fails: Both SPF and DKIM must pass (or at least one aligned with the From domain).
  • Listed on a blacklist: Check MXToolbox. Fresh IPs can be pre-listed on some blocklists — most accept a quick request for removal.
  • PTR mismatch: Verify dig -x YOUR_IP returns mail.example.com. If not, fix the PTR at your provider.

Step 8 — Ongoing maintenance

Let’s Encrypt certificate renewal

Add a post-renewal hook to restart mail services after certificate renewal:

cat > /etc/letsencrypt/renewal-hooks/post/mail.sh <<'EOF'
#!/bin/bash
systemctl reload postfix
systemctl reload dovecot
EOF
chmod +x /etc/letsencrypt/renewal-hooks/post/mail.sh

Monitor the mail log

tail -f /var/log/mail.log

Common things to watch for: repeated delivery failures to a domain (their mail server may be down), spikes in rejected connections (potential attack), and authentication failures (someone testing your credentials).

Train Rspamd’s Bayes filter

After a week or two, train Rspamd with actual spam and ham to improve accuracy:

# Train as spam
rspamc learn_spam < /path/to/spam-message.eml

# Train as ham (legitimate mail)
rspamc learn_ham < /path/to/good-message.eml

DMARC reports

Once you’ve set up the rua address in your DMARC record, you’ll receive aggregate reports from major mail providers. These show which IPs are sending mail claiming to be from your domain. If you see IPs you don’t recognise, someone is spoofing you — tighten your DMARC policy from quarantine to reject.

Adding virtual mailboxes (multiple domains)

The system user approach works for small setups. For multiple domains or many users, virtual mailboxes are cleaner — users exist only in a database, not as Unix accounts.

Install a simple flat-file virtual setup

# /etc/postfix/main.cf additions:
virtual_mailbox_domains = example.com otherdomain.com
virtual_mailbox_base = /var/mail/virtual
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_minimum_uid = 100
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000

# Create the user that owns all virtual mail
groupadd -g 5000 vmail
useradd -u 5000 -g 5000 -s /sbin/nologin -d /var/mail/virtual vmail
mkdir -p /var/mail/virtual
chown vmail:vmail /var/mail/virtual
# /etc/postfix/vmailbox
alice@example.com     example.com/alice/
bob@example.com       example.com/bob/
info@otherdomain.com  otherdomain.com/info/
postmap /etc/postfix/vmailbox
postfix reload

For large deployments (100+ users), store virtual mailboxes in MySQL or PostgreSQL using postfix-mysql and dovecot-mysql.

Security hardening

A mail server open to the internet is a constant target. These settings close common attack vectors.

Postfix hardening

# Add to /etc/postfix/main.cf

# Prevent open relay
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject

# Rate limiting
smtpd_client_connection_rate_limit = 20
smtpd_client_message_rate_limit = 30
anvil_rate_time_unit = 60s

# Disable VRFY (stops address enumeration)
disable_vrfy_command = yes

# Don't expose version info
smtpd_banner = $myhostname ESMTP

Fail2ban for mail

apt-get install fail2ban

# /etc/fail2ban/jail.d/postfix.conf
[postfix]
enabled = true
port = smtp,465,587
logpath = /var/log/mail.log
maxretry = 5
bantime = 3600

[dovecot]
enabled = true
port = imaps
logpath = /var/log/mail.log
maxretry = 5
bantime = 3600

Rspamd DMARC enforcement

# /etc/rspamd/local.d/dmarc.conf
reporting = false;   # disable reporting (handle manually via rua address)
enforce_blackhole = true;
enforce_reject = true;

Frequently asked questions

Will my emails end up in spam?
Not if you set up SPF, DKIM, DMARC, and reverse DNS correctly. New IP addresses may have a brief warm-up period with some providers — start by sending small volumes to your own accounts at Gmail and Outlook before sending to customers. A score of 10/10 on mail-tester.com means you’re configured correctly.
What is Rspamd and why use it instead of SpamAssassin?
Rspamd is a modern spam filter written in C, designed to be 10-100x faster than SpamAssassin which is written in Perl. It has a built-in DKIM signing module, Redis-based Bayes learning, native milter support for Postfix, and a web UI. For new setups in 2026, Rspamd is the right choice.
How do I add more email addresses?
For system user mailboxes: create a new Unix user with useradd. For virtual mailboxes: add the address to /etc/postfix/vmailbox and run postmap. For aliases (forwarding): add to /etc/aliases and run newaliases.
Does myguard ship the latest Postfix and Dovecot versions?
Yes. The myguard repository tracks upstream releases and publishes updated packages within hours of new versions. Postfix 3.9.x, Dovecot 2.4.x, and Rspamd 3.x are available — compared to the older versions frozen in Debian stable.
Can I run this mail server alongside NGINX on the same machine?
Yes. Postfix and Dovecot use completely different ports from NGINX (25, 465, 587, 993 vs 80, 443). They coexist without conflict. A typical setup runs NGINX as the web server and Let’s Encrypt provider, with Postfix and Dovecot using the same certificates.
How do I back up email?
Maildir stores each message as a separate file under ~/Maildir/. Backing up is as simple as rsync: rsync -av /home/alice/Maildir/ backup-server:/backups/alice/. Run this daily via cron. Incremental backups work naturally because new files are just new messages.
My IP is on a spam blacklist — how do I get off?
Check at MXToolbox blacklist checker. Most blacklists have a self-service removal form. Spamhaus ZEN is the most important — their form is at spamhaus.org/removal. Confirm you’ve fixed the underlying issue first (open relay, malware, etc.) before requesting removal.

Related posts