Skip to main content

Content Scanning

Public beta

Content scanning is currently in public beta. This feature is under active development and may have limited availability or functionality.

Moat provides real-time content scanning for HTTP request bodies using ClamAV antivirus engine, protecting your application from malware and malicious uploads.

Overview​

Moat's content scanning provides:

  • Real-time malware detection using ClamAV antivirus engine
  • Selective scanning with Wirefilter expressions
  • Content-type filtering to scan only relevant requests
  • File size limits to optimize performance
  • Automatic blocking of infected content

How It Works​

Request Flow​

  1. Request arrives - HTTP request reaches Moat reverse proxy
  2. Expression evaluation - Wirefilter checks if scanning is needed
  3. ClamAV scan - Request body is scanned for malware
  4. Decision - Clean requests proceed, infected ones are blocked
  5. Upstream forwarding - Only clean requests reach your application

ClamAV Integration​

Moat connects to a ClamAV daemon (clamd) for virus scanning:

  • TCP socket connection to ClamAV daemon
  • Streaming scan for efficient memory usage
  • Signature-based detection of known malware patterns
  • Multi-format support - Archives, executables, documents, etc.

Configuration​

Basic Setup​

Enable content scanning in your Moat configuration:

content_scanning:
  # Enable or disable content scanning
  enabled: true

  # ClamAV server address (host:port)
  clamav_server: "localhost:3310"

  # Maximum file size to scan (10MB default)
  max_file_size: 10485760

  # Content types to scan (empty = scan all)
  scan_content_types:
    - "text/html"
    - "application/x-www-form-urlencoded"
    - "multipart/form-data"
    - "application/json"
    - "text/plain"
    - "application/octet-stream"

  # Skip scanning for specific file extensions
  skip_extensions: []

  # Wirefilter expression to determine when to scan
  scan_expression: 'http.request.method == "POST" or http.request.method == "PUT"'

ClamAV Server Setup​

Moat requires a running ClamAV daemon (clamd):

Docker Compose Example​

services:
  clamav:
    image: clamav/clamav:latest
    ports:
      - "3310:3310"
    volumes:
      - clamav-data:/var/lib/clamav

  moat:
    image: moat:latest
    depends_on:
      - clamav
    environment:
      - AX_CONTENT_SCANNING_ENABLED=true
      - AX_CONTENT_SCANNING_CLAMAV_SERVER=clamav:3310
    # ... other moat config

volumes:
  clamav-data:

Kubernetes Example​

apiVersion: v1
kind: Service
metadata:
  name: clamav
spec:
  selector:
    app: clamav
  ports:
  - port: 3310
    targetPort: 3310
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: clamav
spec:
  replicas: 1
  selector:
    matchLabels:
      app: clamav
  template:
    metadata:
      labels:
        app: clamav
    spec:
      containers:
      - name: clamav
        image: clamav/clamav:latest
        ports:
        - containerPort: 3310

Scan Expression​

Control which requests are scanned using Wirefilter expressions:

# Scan only POST and PUT requests
scan_expression: 'http.request.method == "POST" or http.request.method == "PUT"'

# Scan only file uploads
scan_expression: 'http.request.content_type contains "multipart/form-data"'

# Scan specific paths
scan_expression: 'http.request.path starts_with "/upload"'

# Combine multiple conditions
scan_expression: '(http.request.method == "POST") and (http.request.path starts_with "/upload" or http.request.path starts_with "/api/files")'

Supported File Types​

ClamAV can scan virtually any file type:

  • Documents: PDF, DOC/DOCX, XLS/XLSX, PPT/PPTX, ODT, RTF
  • Images: JPEG, PNG, GIF, BMP, TIFF, SVG
  • Archives: ZIP, RAR, 7Z, TAR, GZ, BZ2
  • Executables: EXE, MSI, DMG, DEB, RPM, APK
  • Scripts: JS, PHP, Python, Shell scripts
  • Web: HTML, XML, JSON, CSS

Behavior​

When Malware is Detected​

If ClamAV detects malware in a request:

  1. Request is blocked - Returns 403 Forbidden
  2. Client receives error - Clear message about blocked content
  3. Log entry created - Event logged for security monitoring
  4. Upstream not called - Infected content never reaches your application

When Scan Fails​

If the ClamAV connection fails or scan errors occur:

  • Fail-open behavior - Request proceeds by default (configurable)
  • Error logged - Scan failure recorded for monitoring
  • Alert triggered - Operators notified of ClamAV issues

Performance Considerations​

Latency Impact​

Content scanning adds latency to requests:

  • Small files (< 1MB): 10-50ms
  • Medium files (1-10MB): 50-200ms
  • Large files (10MB+): 200ms+

Optimization Tips​

  • Limit max file size to avoid scanning very large files
  • Use content-type filtering to scan only relevant requests
  • Deploy ClamAV locally to minimize network latency
  • Use scan expressions to skip unnecessary scans

Resource Usage​

ClamAV resource requirements:

  • Memory: 1-2GB for signature database
  • CPU: Varies with scan volume
  • Disk: ~500MB for signatures
  • Network: Minimal (Moat to ClamAV only)

Monitoring​

ClamAV Health​

Monitor ClamAV daemon health:

# Check ClamAV status
echo "PING" | nc localhost 3310
# Response: PONG

# Check version
echo "VERSION" | nc localhost 3310
# Response: ClamAV 1.0.x/...

# Check signature database
echo "STATS" | nc localhost 3310

Logs​

Moat logs content scanning events:

{
  "level": "warn",
  "msg": "Content scan detected malware",
  "ip": "203.0.113.45",
  "method": "POST",
  "path": "/upload",
  "virus_name": "Win.Trojan.Generic",
  "action": "blocked"
}

Best Practices​

Production Deployment​

  • Run ClamAV close to Moat - Same host or same datacenter
  • Keep signatures updated - Enable automatic freshclam updates
  • Set reasonable file size limits - Balance security vs performance
  • Monitor ClamAV health - Alert on connection failures
  • Use fail-open cautiously - Consider fail-closed for high-security environments

Security Considerations​

  • Signature updates are critical - Outdated signatures miss new threats
  • Scan compressed archives - ClamAV automatically extracts and scans
  • Log all scan results - Track blocked content for analysis
  • Combine with WAF rules - Use both content scanning and WAF together

Performance Tuning​

  • Adjust max_file_size based on your use case
  • Filter by content-type to reduce unnecessary scans
  • Use scan expressions to target specific endpoints
  • Scale ClamAV horizontally if needed for high volume

Troubleshooting​

ClamAV Connection Failed​

Error: Failed to connect to ClamAV at localhost:3310

Solutions:

  1. Verify ClamAV is running: docker ps | grep clamav
  2. Check network connectivity: telnet localhost 3310
  3. Review ClamAV logs: docker logs clamav
  4. Ensure correct server address in config

Signatures Not Updated​

ClamAV requires regular signature updates:

# Manual update (in ClamAV container)
freshclam

# Check last update
grep "Database updated" /var/log/clamav/freshclam.log

High Memory Usage​

ClamAV signature database requires significant memory:

  • Ensure at least 2GB RAM for ClamAV container
  • Monitor memory usage and adjust limits
  • Consider using ClamAV's on-access scanning options

Limitations​

Current Limitations
  • Single scanner engine - Only ClamAV (no multi-engine support)
  • Synchronous scanning - Adds latency to request path
  • No file quarantine - Infected files are blocked but not stored
  • Signature-based only - No heuristic or behavioral analysis
  • ClamAV dependency - Requires external ClamAV daemon