# BackstopJS - Visual Regression Testing

Comprehensive guide to visual regression testing with BackstopJS for catching unintended UI changes.

## Overview

**Container**: `backstopjs/backstopjs:latest`
**Category**: Visual Regression Testing
**Port**: N/A (CLI tool)

BackstopJS automates visual regression testing by comparing screenshots of web pages over time, catching CSS regressions and unintended visual changes.

## Quick Start

```bash
# Initialize configuration
docker exec backstopjs backstop init

# Create reference screenshots
docker exec backstopjs backstop reference

# Run visual regression test
docker exec backstopjs backstop test

# Approve changes (update references)
docker exec backstopjs backstop approve

# Open report
open backstop_data/html_report/index.html
```

## Configuration

```json
// backstop.json
{
  "id": "myapp",
  "viewports": [
    {
      "label": "phone",
      "width": 320,
      "height": 480
    },
    {
      "label": "tablet",
      "width": 1024,
      "height": 768
    },
    {
      "label": "desktop",
      "width": 1920,
      "height": 1080
    }
  ],
  "scenarios": [
    {
      "label": "Homepage",
      "url": "http://target:8080/",
      "delay": 500,
      "misMatchThreshold": 0.1
    },
    {
      "label": "Login Page",
      "url": "http://target:8080/login",
      "delay": 500
    },
    {
      "label": "Dashboard",
      "url": "http://target:8080/dashboard",
      "delay": 1000,
      "cookiePath": "cookies.json"
    }
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "engine_scripts": "backstop_data/engine_scripts",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  "engine": "puppeteer",
  "engineOptions": {
    "args": ["--no-sandbox"]
  },
  "asyncCaptureLimit": 5,
  "asyncCompareLimit": 50,
  "debug": false,
  "debugWindow": false
}
```

## Scenario Options

| Option | Description | Default |
|--------|-------------|---------|
| `label` | Scenario name (required) | - |
| `url` | URL to test (required) | - |
| `delay` | Wait before screenshot (ms) | 0 |
| `misMatchThreshold` | Allowed difference % | 0.1 |
| `selectors` | CSS selectors to capture | ["document"] |
| `removeSelectors` | Elements to remove | [] |
| `hideSelectors` | Elements to hide | [] |
| `clickSelector` | Element to click | - |
| `hoverSelector` | Element to hover | - |
| `scrollToSelector` | Scroll to element | - |
| `cookiePath` | Path to cookies JSON | - |
| `postInteractionWait` | Wait after interaction | 0 |
| `readySelector` | Wait for element | - |
| `readyEvent` | Wait for custom event | - |

## Advanced Scenarios

### Login Flow

```json
{
  "label": "Dashboard After Login",
  "url": "http://target:8080/login",
  "delay": 500,
  "onReadyScript": "login.js",
  "postInteractionWait": 2000,
  "selectors": [".dashboard-content"]
}
```

```javascript
// engine_scripts/login.js
module.exports = async (page, scenario) => {
  await page.type('#email', 'test@example.com');
  await page.type('#password', 'password123');
  await page.click('#login-button');
  await page.waitForNavigation();
};
```

### Hover State

```json
{
  "label": "Button Hover State",
  "url": "http://target:8080/",
  "hoverSelector": ".btn-primary",
  "postInteractionWait": 500,
  "selectors": [".btn-primary"]
}
```

### Hide Dynamic Content

```json
{
  "label": "Dashboard (No Timestamps)",
  "url": "http://target:8080/dashboard",
  "hideSelectors": [".timestamp", ".dynamic-chart", ".user-avatar"],
  "removeSelectors": [".ads", ".cookie-banner", ".chat-widget"]
}
```

### Multiple Selectors

```json
{
  "label": "Key Components",
  "url": "http://target:8080/",
  "selectors": [
    "header",
    "nav.main-nav",
    ".hero-section",
    ".feature-cards",
    "footer"
  ],
  "selectorExpansion": true
}
```

### Click Interactions

```json
{
  "label": "Dropdown Menu Open",
  "url": "http://target:8080/",
  "clickSelector": ".dropdown-toggle",
  "postInteractionWait": 500,
  "selectors": [".dropdown-menu"]
}
```

### Scroll Testing

```json
{
  "label": "Footer Section",
  "url": "http://target:8080/",
  "scrollToSelector": "footer",
  "delay": 500,
  "selectors": ["footer"]
}
```

## Custom Scripts

### onBefore Script

```javascript
// engine_scripts/onBefore.js
module.exports = async (page, scenario, viewport, isReference, browserContext) => {
  // Set cookies before page load
  await page.setCookie({
    name: 'session',
    value: 'abc123',
    domain: 'localhost'
  });

  // Set viewport
  await page.setViewport({
    width: viewport.width,
    height: viewport.height
  });
};
```

### onReady Script

```javascript
// engine_scripts/onReady.js
module.exports = async (page, scenario, viewport, isReference, browserContext) => {
  // Wait for animations to complete
  await page.waitForTimeout(1000);

  // Remove dynamic content
  await page.evaluate(() => {
    document.querySelectorAll('.animated').forEach(el => el.remove());
  });

  // Set consistent date
  await page.evaluate(() => {
    document.querySelectorAll('.date').forEach(el => {
      el.textContent = '2024-01-15';
    });
  });
};
```

### Interaction Script

```javascript
// engine_scripts/modal.js
module.exports = async (page, scenario) => {
  // Open modal
  await page.click('[data-modal="settings"]');
  await page.waitForSelector('.modal-content', { visible: true });

  // Fill form
  await page.type('#setting-name', 'Test Setting');
  await page.select('#setting-type', 'option2');

  // Wait for animation
  await page.waitForTimeout(500);
};
```

## CLI Commands

```bash
# Initialize project
docker exec backstopjs backstop init

# Create reference images
docker exec backstopjs backstop reference

# Run comparison test
docker exec backstopjs backstop test

# Approve current test as new reference
docker exec backstopjs backstop approve

# Run specific scenario
docker exec backstopjs backstop test --filter="Homepage"

# Use custom config
docker exec backstopjs backstop test --config=custom-backstop.json

# Generate CI-friendly report
docker exec backstopjs backstop test --reporter=ci

# Run in Docker
docker exec backstopjs backstop test --docker
```

## Report Output

```
backstop_data/
├── bitmaps_reference/        # Reference screenshots
│   ├── myapp_Homepage_0_document_0_phone.png
│   ├── myapp_Homepage_0_document_0_tablet.png
│   └── myapp_Homepage_0_document_0_desktop.png
├── bitmaps_test/            # Test screenshots
│   └── YYYYMMDD-HHMMSS/
│       ├── myapp_Homepage_0_document_0_phone.png
│       └── diff/
│           └── myapp_Homepage_0_document_0_phone.png
├── html_report/             # Visual HTML report
│   └── index.html
├── ci_report/               # JSON for CI
│   └── xunit.xml
└── engine_scripts/          # Custom scripts
    ├── onBefore.js
    ├── onReady.js
    └── login.js
```

## Mismatch Threshold

```json
{
  "label": "Allow Small Differences",
  "url": "http://target:8080/",
  "misMatchThreshold": 0.5
}
```

| Threshold | Use Case |
|-----------|----------|
| 0 | Pixel-perfect matching |
| 0.1 | Default, catches major changes |
| 0.5 | Tolerates anti-aliasing differences |
| 1.0+ | Major layout changes only |

## CI/CD Integration

### GitHub Actions

```yaml
name: Visual Regression
on: [pull_request]

jobs:
  backstop:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Start Application
        run: docker-compose up -d app

      - name: Download Reference Images
        uses: actions/download-artifact@v3
        with:
          name: backstop-reference
          path: backstop_data/bitmaps_reference/
        continue-on-error: true

      - name: Run BackstopJS
        run: |
          docker run --rm --network host \
            -v ${PWD}:/src \
            backstopjs/backstopjs test

      - name: Upload Report
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: backstop-report
          path: backstop_data/html_report/

      - name: Upload Test Images
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: backstop-test-images
          path: backstop_data/bitmaps_test/

      - name: Save Reference (on main)
        if: github.ref == 'refs/heads/main'
        uses: actions/upload-artifact@v3
        with:
          name: backstop-reference
          path: backstop_data/bitmaps_reference/
```

### Pipeline Script

```bash
#!/bin/bash
# visual-regression.sh

set -e

TARGET_URL=${1:-"http://localhost:8080"}

# Create/update backstop config
cat > backstop.json <<EOF
{
  "id": "visual-test",
  "viewports": [
    {"label": "mobile", "width": 375, "height": 667},
    {"label": "desktop", "width": 1920, "height": 1080}
  ],
  "scenarios": [
    {"label": "Homepage", "url": "${TARGET_URL}/"},
    {"label": "Products", "url": "${TARGET_URL}/products"},
    {"label": "Contact", "url": "${TARGET_URL}/contact"}
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  "engine": "puppeteer",
  "engineOptions": {"args": ["--no-sandbox"]}
}
EOF

# Check if reference exists
if [ ! -d "backstop_data/bitmaps_reference" ]; then
  echo "Creating reference screenshots..."
  docker exec backstopjs backstop reference
else
  echo "Running visual regression test..."
  docker exec backstopjs backstop test
fi

echo "Report available at backstop_data/html_report/index.html"
```

## Component Testing

### Test UI Components

```json
{
  "scenarios": [
    {
      "label": "Button - Default",
      "url": "http://localhost:6006/iframe.html?id=button--default",
      "selectors": ["#root"]
    },
    {
      "label": "Button - Primary",
      "url": "http://localhost:6006/iframe.html?id=button--primary",
      "selectors": ["#root"]
    },
    {
      "label": "Button - Disabled",
      "url": "http://localhost:6006/iframe.html?id=button--disabled",
      "selectors": ["#root"]
    },
    {
      "label": "Card - Default",
      "url": "http://localhost:6006/iframe.html?id=card--default",
      "selectors": ["#root"]
    }
  ]
}
```

## Best Practices

### 1. Hide Dynamic Content

```json
{
  "hideSelectors": [
    ".timestamp",
    ".current-date",
    ".random-ad",
    ".chat-widget",
    ".notification-badge"
  ]
}
```

### 2. Wait for Loading

```json
{
  "delay": 2000,
  "readySelector": ".content-loaded",
  "readyEvent": "page-ready"
}
```

### 3. Test Multiple Viewports

```json
{
  "viewports": [
    {"label": "iPhone SE", "width": 375, "height": 667},
    {"label": "iPhone 12 Pro", "width": 390, "height": 844},
    {"label": "iPad", "width": 768, "height": 1024},
    {"label": "Desktop", "width": 1440, "height": 900},
    {"label": "4K", "width": 2560, "height": 1440}
  ]
}
```

### 4. Use Selectors Wisely

```json
{
  "selectors": [
    "header",
    ".main-content",
    ".sidebar",
    "footer"
  ],
  "selectorExpansion": true
}
```

### 5. Handle Animations

```javascript
// onReady script
await page.evaluate(() => {
  // Disable CSS animations
  const style = document.createElement('style');
  style.innerHTML = '*, *::before, *::after { animation: none !important; transition: none !important; }';
  document.head.appendChild(style);
});
```

## Integration with Stack

- Run after Playwright E2E tests pass
- Complements Pa11y accessibility testing
- Part of UI component validation pipeline
- Reference images in version control (git-lfs)
- Reports linkable from Allure dashboard
- Track visual bugs in DefectDojo

## Troubleshooting

### Common Issues

**Issue**: Flaky tests from dynamic content
```json
{
  "hideSelectors": [".dynamic-content"],
  "delay": 2000,
  "misMatchThreshold": 0.5
}
```

**Issue**: Chrome crashes
```json
{
  "engineOptions": {
    "args": ["--no-sandbox", "--disable-setuid-sandbox", "--disable-dev-shm-usage"]
  }
}
```

**Issue**: Large diff images
```bash
# Reduce viewport size for faster tests
# Use selector-based testing
# Increase mismatch threshold for non-critical pages
```

**Issue**: Test timeout
```json
{
  "asyncCaptureLimit": 2,
  "asyncCompareLimit": 10,
  "delay": 5000
}
```
