# RESTler - API Fuzzing

Comprehensive guide to stateful REST API fuzzing with RESTler for automated security testing and bug discovery.

## Overview

**Container**: `mcr.microsoft.com/restlerfuzzer/restler:latest`
**Category**: API Security Testing
**Port**: N/A (CLI tool)

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs.

## Quick Start

```bash
# Compile OpenAPI spec
docker exec restler restler compile --api_spec /specs/openapi.json

# Run quick fuzz test
docker exec restler restler fuzz-lean --grammar_file /Compile/grammar.py --dictionary_file /Compile/dict.json

# Run full fuzz test
docker exec restler restler fuzz --grammar_file /Compile/grammar.py --dictionary_file /Compile/dict.json --time_budget 1
```

## Workflow

RESTler uses a four-phase approach:

```
1. Compile → 2. Test → 3. Fuzz-Lean → 4. Fuzz
```

| Phase | Purpose | Duration |
|-------|---------|----------|
| **Compile** | Parse OpenAPI spec to grammar | Seconds |
| **Test** | Validate API reachability | Minutes |
| **Fuzz-Lean** | Quick security check | Minutes |
| **Fuzz** | Deep stateful fuzzing | Hours |

## Compile Phase

```bash
# Basic compile
docker exec restler restler compile --api_spec /specs/openapi.json

# With custom settings
docker exec restler restler compile \
  --api_spec /specs/openapi.json \
  --custom_mutations /mutations/custom.json

# Multiple specs
docker exec restler restler compile \
  --api_spec /specs/users.json \
  --api_spec /specs/orders.json
```

### Compile Output

```
Compile/
├── grammar.py           # Request grammar
├── dict.json           # Fuzzing dictionary
├── engine_settings.json # Engine configuration
├── grammar.json        # Parsed grammar (JSON)
└── dependencies.json   # Request dependencies
```

## Test Phase

Validates API connectivity and authentication:

```bash
# Run test mode
docker exec restler restler test \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --target_ip api.example.com \
  --target_port 443 \
  --host api.example.com \
  --use_ssl
```

### With Authentication

```bash
# Token-based auth
docker exec restler restler test \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --token_refresh_cmd "python /auth/get_token.py" \
  --token_refresh_interval 3600
```

## Fuzz-Lean Phase

Quick security scan for common vulnerabilities:

```bash
# Basic fuzz-lean
docker exec restler restler fuzz-lean \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --target_ip localhost \
  --target_port 8080

# With settings file
docker exec restler restler fuzz-lean \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --settings /config/engine_settings.json
```

### Fuzz-Lean Checkers

| Checker | Description |
|---------|-------------|
| UseAfterFree | Tests for use-after-free bugs |
| ResourceHierarchy | Validates resource access control |
| LeakageRule | Checks for information leakage |
| InvalidDynamicObject | Tests invalid object references |
| PayloadBody | Fuzzes request body content |
| Examples | Tests with spec examples |

## Fuzz Phase

Full stateful fuzzing for comprehensive coverage:

```bash
# Time-based fuzzing (1 hour)
docker exec restler restler fuzz \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --time_budget 1 \
  --target_ip localhost \
  --target_port 8080

# With all checkers
docker exec restler restler fuzz \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --time_budget 2 \
  --enable_checkers '*'
```

## Configuration

### Engine Settings

```json
// engine_settings.json
{
  "max_combinations": 20,
  "max_request_execution_time": 120,
  "no_ssl": true,
  "host": "localhost",
  "target_port": 8080,
  "include_user_agent": true,
  "garbage_collection_interval": 30,
  "reconnect_on_every_request": false,
  "checkers": {
    "useafterfree": {
      "mode": "exhaustive"
    },
    "leakagerule": {
      "mode": "normal"
    },
    "resourcehierarchy": {
      "mode": "normal"
    },
    "invaliddynamicobject": {
      "mode": "normal"
    }
  }
}
```

### Custom Dictionary

```json
// dict.json
{
  "restler_fuzzable_string": ["test", "admin", "user", "<script>alert(1)</script>"],
  "restler_fuzzable_int": ["0", "-1", "999999", "2147483647"],
  "restler_fuzzable_bool": ["true", "false"],
  "restler_fuzzable_datetime": ["2024-01-01T00:00:00Z", "1970-01-01T00:00:00Z"],
  "restler_fuzzable_uuid4": ["00000000-0000-0000-0000-000000000000"],
  "restler_custom_payload": {
    "userId": ["1", "2", "admin"],
    "role": ["user", "admin", "superadmin"]
  },
  "restler_custom_payload_uuid4_suffix": {
    "sessionId": ["test-session"]
  }
}
```

### Custom Mutations

```json
// custom_mutations.json
{
  "mutations": [
    {
      "name": "sql_injection",
      "type": "string",
      "values": [
        "' OR '1'='1",
        "1; DROP TABLE users--",
        "admin'--"
      ]
    },
    {
      "name": "xss_payload",
      "type": "string",
      "values": [
        "<script>alert('XSS')</script>",
        "javascript:alert(1)",
        "<img src=x onerror=alert(1)>"
      ]
    },
    {
      "name": "path_traversal",
      "type": "string",
      "values": [
        "../../../etc/passwd",
        "..\\..\\..\\windows\\system32\\config\\sam"
      ]
    }
  ]
}
```

## Authentication

### Token Refresh Script

```python
# get_token.py
import requests
import json

def get_token():
    response = requests.post(
        'http://localhost:8080/auth/login',
        json={
            'username': 'test@example.com',
            'password': 'password123'
        }
    )
    token = response.json()['token']
    return {'Authorization': f'Bearer {token}'}

if __name__ == '__main__':
    print(json.dumps(get_token()))
```

### Using Token Refresh

```bash
docker exec restler restler fuzz-lean \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --token_refresh_cmd "python /auth/get_token.py" \
  --token_refresh_interval 1800
```

## Bug Buckets

RESTler categorizes findings into bug buckets:

| Bucket | Description | Severity |
|--------|-------------|----------|
| 500 | Internal Server Error | High |
| InvalidDynamicObjectChecker | Unauthorized resource access | Critical |
| UseAfterFreeChecker | Use-after-free vulnerability | Critical |
| ResourceHierarchyChecker | Access control bypass | Critical |
| PayloadBodyChecker | Body parsing issues | Medium |
| LeakageRuleChecker | Information disclosure | Medium |

## Results Analysis

### Bug Report Structure

```
FuzzLean/
├── bug_buckets/
│   ├── 500/
│   │   └── bug_0001.txt
│   └── InvalidDynamicObjectChecker/
│       └── bug_0001.txt
├── testing_summary.json
├── network.testing.log
└── speccov.json
```

### Sample Bug Report

```json
{
  "reproducible": true,
  "checker_name": "InvalidDynamicObjectChecker",
  "request_sequence": [
    {
      "method": "POST",
      "endpoint": "/api/users",
      "body": {"name": "test"}
    },
    {
      "method": "GET",
      "endpoint": "/api/users/2/orders",
      "response_code": 200,
      "bug": "Accessed user 2's orders without authorization"
    }
  ]
}
```

### Coverage Report

```bash
# View spec coverage
docker exec restler cat /Fuzz/speccov.json | jq '.'

# Coverage includes:
# - Endpoints tested
# - Methods covered
# - Status codes received
# - Request sequences
```

## CI/CD Integration

### GitHub Actions

```yaml
name: API Security Fuzzing
on:
  schedule:
    - cron: '0 2 * * *'  # Nightly
  workflow_dispatch:

jobs:
  restler-fuzz:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Start API
        run: docker-compose up -d api

      - name: Wait for API
        run: sleep 30

      - name: Compile Grammar
        run: |
          docker run --rm -v ${PWD}:/workspace \
            mcr.microsoft.com/restlerfuzzer/restler:latest \
            restler compile --api_spec /workspace/openapi.json

      - name: Run Fuzz-Lean
        run: |
          docker run --rm -v ${PWD}:/workspace --network host \
            mcr.microsoft.com/restlerfuzzer/restler:latest \
            restler fuzz-lean \
            --grammar_file /workspace/Compile/grammar.py \
            --dictionary_file /workspace/Compile/dict.json \
            --target_ip localhost \
            --target_port 8080

      - name: Check for Bugs
        run: |
          if [ -d "FuzzLean/bug_buckets" ] && [ "$(ls -A FuzzLean/bug_buckets)" ]; then
            echo "BUGS FOUND!"
            find FuzzLean/bug_buckets -name "*.txt" -exec cat {} \;
            exit 1
          fi

      - name: Upload Results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: restler-results
          path: FuzzLean/
```

### Pipeline Script

```bash
#!/bin/bash
# run-api-fuzzing.sh

set -e

API_SPEC=$1
TARGET=${2:-"localhost:8080"}
TIME_BUDGET=${3:-1}

echo "=== RESTler API Fuzzing ==="

# Compile
echo "Compiling OpenAPI spec..."
docker exec restler restler compile --api_spec $API_SPEC

# Test connectivity
echo "Testing API connectivity..."
docker exec restler restler test \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --target_ip ${TARGET%:*} \
  --target_port ${TARGET#*:}

# Fuzz-lean (quick security check)
echo "Running fuzz-lean..."
docker exec restler restler fuzz-lean \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --target_ip ${TARGET%:*} \
  --target_port ${TARGET#*:}

# Check for bugs
BUG_COUNT=$(find /results/FuzzLean/bug_buckets -name "*.txt" 2>/dev/null | wc -l)
if [ "$BUG_COUNT" -gt 0 ]; then
  echo "FAIL: $BUG_COUNT bugs found!"
  exit 1
fi

# Full fuzz if time permits
if [ "$TIME_BUDGET" -gt 0 ]; then
  echo "Running full fuzz for $TIME_BUDGET hour(s)..."
  docker exec restler restler fuzz \
    --grammar_file /Compile/grammar.py \
    --dictionary_file /Compile/dict.json \
    --target_ip ${TARGET%:*} \
    --target_port ${TARGET#*:} \
    --time_budget $TIME_BUDGET
fi

echo "API fuzzing complete!"
```

## Best Practices

1. **Start with Test Mode** - Validate connectivity first
2. **Use Fuzz-Lean** - Quick security check in CI
3. **Full Fuzz Nightly** - Deep testing on schedule
4. **Custom Dictionary** - Add domain-specific payloads
5. **Authentication** - Ensure proper token refresh
6. **Coverage Tracking** - Monitor spec coverage
7. **Bug Triage** - Review all findings promptly
8. **Reproducibility** - Document reproduction steps

## Security Payloads

### SQL Injection

```json
{
  "restler_fuzzable_string": [
    "' OR 1=1--",
    "'; DROP TABLE users;--",
    "1' UNION SELECT null,username,password FROM users--"
  ]
}
```

### Command Injection

```json
{
  "restler_fuzzable_string": [
    "; ls -la",
    "| cat /etc/passwd",
    "$(whoami)"
  ]
}
```

### XXE Payloads

```json
{
  "restler_fuzzable_string": [
    "<?xml version=\"1.0\"?><!DOCTYPE foo [<!ENTITY xxe SYSTEM \"file:///etc/passwd\">]><foo>&xxe;</foo>"
  ]
}
```

## Integration with Stack

- Complements OWASP ZAP DAST scanning
- Deeper API coverage than Nuclei
- Works with Pact contract tests
- Findings tracked in DefectDojo
- Results included in Allure reports
- Run after Newman API tests pass

## Troubleshooting

### Common Issues

**Issue**: Compile fails
```bash
# Validate OpenAPI spec
docker exec restler restler compile --api_spec /specs/openapi.json --debug

# Check spec is valid JSON/YAML
jq . /specs/openapi.json
```

**Issue**: Authentication fails
```bash
# Test token refresh manually
python get_token.py

# Check token format
docker exec restler restler test \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --debug
```

**Issue**: Low coverage
```bash
# Check dependencies resolved
cat /Compile/dependencies.json | jq '.'

# Enable verbose logging
docker exec restler restler fuzz-lean \
  --grammar_file /Compile/grammar.py \
  --dictionary_file /Compile/dict.json \
  --debug
```
