nginx | 05/12/2026 | By Thijs Eilander

NGINX ModSecurity Setup on Debian and Ubuntu: WAF with OWASP Core Rule Set

ModSecurity is a Web Application Firewall (WAF) — a layer that sits in front of your application and blocks attacks before they reach your PHP or your database. SQL injection, XSS, file inclusion, shell injection, scanner traffic, bad bots — all of these have well-known HTTP patterns, and ModSecurity plus the OWASP Core Rule Set (CRS) blocks the vast majority of them automatically.

The myguard APT repository ships a pre-built ModSecurity v3 NGINX connector module as a dynamic loadable package. No compiling from source, no dependency hell. Install, configure, done.

Why You Actually Need a WAF

Here’s the uncomfortable truth: your application has vulnerabilities you don’t know about. Every PHP CMS, every plugin ecosystem, every third-party library — they all have bugs that get discovered, exploited, and published faster than you can patch them. On any given day, there are probably active exploit attempts running against your server right now, probing for known CVEs.

A WAF doesn’t replace keeping software updated. But it buys time. When a new WordPress plugin CVE drops on a Monday and the patch lands on Wednesday, a properly configured WAF can block the exploit attempts during those 48 hours — often because the attack pattern matches a general rule, not a specific one.

The OWASP Core Rule Set (CRS) is maintained by a team of security researchers and covers the OWASP Top 10 attack categories. It’s updated regularly and is the industry standard baseline for HTTP-layer WAF protection.

ModSecurity v3 vs v2: What Changed

ModSecurity v2 was an Apache module. ModSecurity v3 (libmodsecurity3) is a standalone library with a connector model: the WAF logic lives in the library, and thin connector modules bridge it to your web server. This means the same detection engine works with NGINX, Apache, IIS, and others — one library, multiple server plugins.

Key differences from v2:

  • Better performance: v3 processes rules more efficiently, with lower per-request overhead
  • Dynamic rule loading: Rules can be updated without restarting NGINX (with the right config)
  • More complete request body inspection: JSON, XML, multipart — all properly parsed
  • Some v2 rules don’t work in v3: If you migrate from v2, audit your custom rules

Step 1 — Install ModSecurity

# Add the myguard repository if not already done
wget https://deb.myguard.nl/pool/myguard.deb
dpkg -i myguard.deb
apt-get update

# Install ModSecurity v3 library and NGINX connector
apt-get install libmodsecurity3 libnginx-mod-http-modsecurity

# Verify the module loaded
nginx -V 2>&1 | grep modsecurity

Step 2 — Enable the Module

Load the module in NGINX’s main config. The myguard package installs a conf.d snippet automatically, but verify:

cat /etc/nginx/modules-enabled/50-mod-http-modsecurity.conf
# Should contain: load_module modules/ngx_http_modsecurity_module.so;

Step 3 — Configure ModSecurity

# Create the ModSecurity config directory
mkdir -p /etc/nginx/modsec

# Copy the default config
cp /usr/share/modsecurity-crs/modsecurity.conf-recommended /etc/nginx/modsec/modsecurity.conf

# Enable detection mode (NOT blocking yet)
ssed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /etc/nginx/modsec/modsecurity.conf

# But actually: start in DetectionOnly for the first week
# Change to 'On' only after reviewing the audit log

Important: Start with SecRuleEngine DetectionOnly. Log what would be blocked, tune out false positives, then switch to SecRuleEngine On (blocking mode). Jumping straight to blocking on a production site will break things.

Step 4 — Download and Configure the OWASP CRS

# Install CRS from the myguard repository (or download from OWASP)
apt-get install modsecurity-crs

# Or manually:
git clone https://github.com/coreruleset/coreruleset /etc/modsecurity-crs
cp /etc/modsecurity-crs/crs-setup.conf.example /etc/modsecurity-crs/crs-setup.conf

# Create the main ModSecurity include file
cat > /etc/nginx/modsec/main.conf << 'EOF'
Include /etc/nginx/modsec/modsecurity.conf
Include /etc/modsecurity-crs/crs-setup.conf
Include /etc/modsecurity-crs/rules/*.conf
EOF

Step 5 — Enable ModSecurity in Your Server Block

server {
    listen 443 ssl;
    http2 on;
    server_name example.com;

    # Enable ModSecurity for this server block
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/main.conf;

    # ... rest of your config
}

Test and reload:

nginx -t && systemctl reload nginx

Step 6 — CRS Paranoia Levels Explained

The OWASP CRS has four paranoia levels that control how aggressively it blocks. This is the most important tuning decision you'll make:

  • Level 1 (default): Blocks only obvious attacks. Very few false positives. Good for most sites.
  • Level 2: More comprehensive blocking. Some false positives on complex web apps. Recommended for public-facing APIs and WordPress.
  • Level 3: Aggressive. Significant false positives. Only for high-security environments with time to tune.
  • Level 4: Very aggressive. Unsuitable for most production use without extensive tuning.

Set paranoia level in /etc/modsecurity-crs/crs-setup.conf:

# Default is level 1. For WordPress, level 2 is practical:
SecAction 
  "id:900000,
   phase:1,
   nolog,
   pass,
   t:none,
   setvar:tx.paranoia_level=2"

WordPress-Specific CRS Tuning

WordPress has some HTTP patterns that CRS level 2 flags as suspicious: large POST bodies (uploading media), base64-encoded content in post bodies (Gutenberg serializes block data), and admin pages that send complex query parameters. These generate false positives you'll need to handle.

The OWASP CRS project maintains an official WordPress exclusion ruleset:

# In your main.conf, include the WordPress exclusions BEFORE the CRS rules
cat > /etc/nginx/modsec/main.conf << 'EOF'
Include /etc/nginx/modsec/modsecurity.conf
Include /etc/modsecurity-crs/crs-setup.conf
# WordPress exclusions - must come BEFORE the CRS rules
Include /etc/modsecurity-crs/plugins/wordpress-exclusions-before.conf
Include /etc/modsecurity-crs/rules/*.conf
Include /etc/modsecurity-crs/plugins/wordpress-exclusions-after.conf
EOF

If the exclusion files don't exist yet, check the CRS plugins directory or download from the OWASP CRS GitHub repository.

Reading the Audit Log

ModSecurity logs blocked requests and detection-mode hits to the audit log:

# Watch the audit log in real time
tail -f /var/log/modsec_audit.log

# Or check the NGINX error log for ModSecurity messages
tail -f /var/log/nginx/error.log | grep ModSecurity

# Count rule triggers by rule ID
grep 'id "' /var/log/modsec_audit.log | grep -oP 'id "K[^"]+' | sort | uniq -c | sort -rn | head -20

Focus on the top triggers first. If rule 920350 (Host header with IP address) is triggering on your legitimate monitoring tool, whitelist it. If rule 942100 (SQL injection via libinjection) fires on a legitimate API call, investigate whether it's actually a false positive or if you have a real injection vulnerability.

Whitelisting False Positives

When you identify a legitimate request being blocked, whitelist it precisely. Don't disable entire rule groups; whitelist specific rule IDs for specific URIs or IP ranges:

# Create a custom exceptions file
cat > /etc/nginx/modsec/exceptions.conf << 'EOF'
# Whitelist rule 920350 for monitoring tool IPs
SecRule REMOTE_ADDR "@ipMatch 192.168.1.10,10.0.0.5" 
    "id:1000,phase:1,pass,nolog,
    ctl:ruleRemoveById=920350"

# Whitelist rule 942100 for the /api/data endpoint only
SecRule REQUEST_URI "@beginsWith /api/data" 
    "id:1001,phase:1,pass,nolog,
    ctl:ruleRemoveById=942100"
EOF

# Include BEFORE the CRS rules in main.conf

Per-Location Rules

You can disable ModSecurity entirely for specific locations (e.g., your health check endpoint) or apply different rule sets:

server {
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsec/main.conf;

    # Disable WAF for health checks (no user input, no attack surface)
    location = /health {
        modsecurity off;
        return 200 'OK';
    }

    # Stricter rules for login endpoint
    location /wp-login.php {
        modsecurity on;
        modsecurity_rules_file /etc/nginx/modsec/strict.conf;
    }
}

Updating CRS Rules

The OWASP CRS team releases updates regularly. If you installed via apt:

apt-get update && apt-get upgrade modsecurity-crs
nginx -t && systemctl reload nginx

If you installed manually via git:

cd /etc/modsecurity-crs
git pull
nginx -t && systemctl reload nginx

Performance Impact

ModSecurity v3 with CRS level 1-2 adds approximately 1–3ms per request on a modern server. This is the rule evaluation overhead for every HTTP request passing through NGINX. On a server handling 1000 req/s, this translates to about 3% additional CPU load at level 2.

At level 3-4 the overhead increases significantly because more rules run per request. For most sites, level 2 is the practical maximum without dedicated WAF hardware.

To minimize overhead:

  • Apply modsecurity on selectively — you don't need WAF on static assets (images, CSS, JS)
  • Use the SecRequestBodyAccess Off option for static file locations
  • Disable ModSecurity for trusted internal IPs (monitoring, load balancer health checks)

ModSecurity + PHP-Snuffleupagus: Defence in Depth

ModSecurity lives in NGINX and filters at the HTTP layer. PHP-Snuffleupagus lives inside PHP-FPM and controls what PHP is allowed to do. They complement each other: ModSecurity blocks malicious HTTP before it reaches PHP; Snuffleupagus prevents PHP from executing dangerous functions even if something gets through. Defence in depth — run both.

Frequently Asked Questions

What is the difference between detection mode and blocking mode?
In detection mode (SecRuleEngine DetectionOnly), ModSecurity logs every rule match but passes all requests through unchanged. In blocking mode (SecRuleEngine On), rule matches result in a 403 response. Always start in detection mode, review the logs for a week to tune out false positives, then switch to blocking.
Will ModSecurity break my WordPress site?
At CRS level 1, almost certainly not. At level 2, the WordPress-specific exclusion rules handle the most common false positives. The safest approach: install in detection mode, let it run for a week while you review the logs, fix any false positives, then switch to blocking.
Does ModSecurity protect against zero-day vulnerabilities?
Often yes — because most exploits for PHP application vulnerabilities have HTTP patterns that match existing CRS generic rules. A SQLi attempt for a new WordPress plugin CVE will usually match the generic SQL injection rules even if there's no specific rule for that CVE yet. This is one of ModSecurity's main practical advantages.
How do I check what rule blocked a request?
The audit log (/var/log/modsec_audit.log) shows the full details of each transaction, including which rule triggered, the request data, and the anomaly score. You can also check the NGINX error log: tail -f /var/log/nginx/error.log | grep ModSecurity
Can I use ModSecurity with Angie instead of NGINX?
Yes. The myguard repository ships an Angie ModSecurity connector module alongside the NGINX one. Install angie-module-http-modsecurity instead of libnginx-mod-http-modsecurity. Configuration is identical.
Is ModSecurity v3 compatible with all v2 rules?
Mostly, but not fully. The OWASP CRS rules work in both versions. However, some v2 custom rules use directives that were removed in v3 (like @inspectFile and some transformation functions). If you're migrating from a v2 setup with custom rules, audit each custom rule before porting to v3.
How often should I update the CRS rules?
The OWASP CRS project releases updates every few months. If you installed via apt, running apt-get upgrade will get you updates. Subscribe to the OWASP CRS project's release announcements for notification of security-critical updates that you should apply immediately.

Related Posts