# Pa11y - Accessibility Testing

Comprehensive guide to automated accessibility testing with Pa11y for WCAG compliance and inclusive web development.

## Overview

**Container**: `pa11y/pa11y:latest`
**Category**: Accessibility Testing
**Port**: N/A (CLI tool)

Pa11y is an automated accessibility testing tool that checks web pages against WCAG (Web Content Accessibility Guidelines) standards.

## Quick Start

```bash
# Basic accessibility test
docker exec pa11y pa11y http://example.com

# Test with specific standard
docker exec pa11y pa11y --standard WCAG2AA http://example.com

# JSON output
docker exec pa11y pa11y --reporter json http://example.com

# Test local file
docker exec pa11y pa11y file:///code/index.html
```

## Accessibility Standards

| Standard | Description | Compliance Level |
|----------|-------------|------------------|
| WCAG2A | Basic accessibility | Minimum |
| WCAG2AA | Enhanced accessibility | Recommended |
| WCAG2AAA | Highest accessibility | Aspirational |
| Section508 | US Federal standard | Government |

## CLI Commands

```bash
# Basic test
docker exec pa11y pa11y http://example.com

# Specify standard
docker exec pa11y pa11y --standard WCAG2AA http://example.com

# Include notices (info level)
docker exec pa11y pa11y --include-notices http://example.com

# Include warnings
docker exec pa11y pa11y --include-warnings http://example.com

# Ignore specific rules
docker exec pa11y pa11y --ignore "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail" http://example.com

# Set timeout
docker exec pa11y pa11y --timeout 60000 http://example.com

# Specify screen size
docker exec pa11y pa11y --screen-size 1920x1080 http://example.com

# Run actions before testing
docker exec pa11y pa11y --actions "wait for element #main" http://example.com

# Wait for element before testing
docker exec pa11y pa11y --wait 5000 http://example.com
```

## Output Formats

```bash
# CLI output (default)
docker exec pa11y pa11y http://example.com

# JSON output
docker exec pa11y pa11y --reporter json http://example.com > results.json

# CSV output
docker exec pa11y pa11y --reporter csv http://example.com > results.csv

# HTML output
docker exec pa11y pa11y --reporter html http://example.com > report.html

# TSV output
docker exec pa11y pa11y --reporter tsv http://example.com > results.tsv
```

## Configuration File

```json
// .pa11yci.json
{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 60000,
    "wait": 1000,
    "includeNotices": false,
    "includeWarnings": true,
    "chromeLaunchConfig": {
      "args": ["--no-sandbox"]
    }
  },
  "urls": [
    "http://example.com/",
    "http://example.com/about",
    "http://example.com/contact",
    {
      "url": "http://example.com/login",
      "actions": [
        "set field #username to test@example.com",
        "set field #password to password123",
        "click element #submit",
        "wait for path to be /dashboard"
      ]
    }
  ]
}
```

## Actions

Pa11y supports actions to interact with pages before testing:

| Action | Syntax | Description |
|--------|--------|-------------|
| click element | `click element #id` | Click an element |
| set field value | `set field #id to value` | Fill form field |
| check field | `check field #id` | Check checkbox |
| uncheck field | `uncheck field #id` | Uncheck checkbox |
| screen capture | `screen capture path.png` | Take screenshot |
| wait for | `wait for element #id` | Wait for element |
| navigate to | `navigate to url` | Go to URL |

### Actions Example

```json
{
  "url": "http://example.com/login",
  "actions": [
    "set field #email to test@example.com",
    "set field #password to SecurePass123",
    "click element button[type=submit]",
    "wait for url to be http://example.com/dashboard",
    "screen capture /reports/dashboard.png"
  ]
}
```

## Pa11y CI

For testing multiple URLs in CI/CD:

```bash
# Run Pa11y CI with config
docker exec pa11y pa11y-ci

# Specify config file
docker exec pa11y pa11y-ci --config .pa11yci.json

# Set concurrency
docker exec pa11y pa11y-ci --concurrency 5

# Threshold for failures
docker exec pa11y pa11y-ci --threshold 10
```

### Pa11y CI Configuration

```json
// .pa11yci.json
{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 30000,
    "chromeLaunchConfig": {
      "args": ["--no-sandbox", "--disable-setuid-sandbox"]
    }
  },
  "urls": [
    "http://localhost:8080/",
    "http://localhost:8080/products",
    "http://localhost:8080/cart",
    {
      "url": "http://localhost:8080/checkout",
      "threshold": 5,
      "ignore": [
        "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
      ]
    }
  ],
  "concurrency": 3,
  "threshold": 0
}
```

## Common WCAG Issues

### Level A (Critical)

| Rule | Issue | Solution |
|------|-------|----------|
| 1.1.1 | Missing alt text | Add alt attribute to images |
| 1.3.1 | Missing form labels | Associate labels with inputs |
| 2.1.1 | Not keyboard accessible | Ensure all interactive elements work with keyboard |
| 2.4.1 | No skip navigation | Add skip links |
| 4.1.1 | Duplicate IDs | Ensure unique IDs |
| 4.1.2 | Missing ARIA attributes | Add proper ARIA roles |

### Level AA (Standard)

| Rule | Issue | Solution |
|------|-------|----------|
| 1.4.3 | Low color contrast | Increase contrast ratio (4.5:1 for text) |
| 1.4.4 | Text not resizable | Use relative units |
| 2.4.4 | Unclear link text | Use descriptive link text |
| 2.4.6 | Missing headings | Add proper heading structure |
| 3.3.1 | No error identification | Clearly identify form errors |
| 3.3.2 | Missing input instructions | Provide clear labels and hints |

## Programmatic Usage (Node.js)

```javascript
// accessibility-test.js
const pa11y = require('pa11y');
const fs = require('fs');

async function runAccessibilityTests() {
  const urls = [
    'http://localhost:8080/',
    'http://localhost:8080/about',
    'http://localhost:8080/contact'
  ];

  const results = [];

  for (const url of urls) {
    const result = await pa11y(url, {
      standard: 'WCAG2AA',
      includeWarnings: true,
      chromeLaunchConfig: {
        args: ['--no-sandbox']
      }
    });

    results.push({
      url,
      issues: result.issues.length,
      errors: result.issues.filter(i => i.type === 'error').length,
      warnings: result.issues.filter(i => i.type === 'warning').length,
      details: result.issues
    });

    console.log(`${url}: ${result.issues.length} issues`);
  }

  // Write summary report
  fs.writeFileSync(
    'accessibility-report.json',
    JSON.stringify(results, null, 2)
  );

  // Check for critical failures
  const totalErrors = results.reduce((sum, r) => sum + r.errors, 0);
  if (totalErrors > 0) {
    console.error(`FAIL: ${totalErrors} accessibility errors found`);
    process.exit(1);
  }

  console.log('PASS: No accessibility errors');
}

runAccessibilityTests();
```

## CI/CD Integration

### GitHub Actions

```yaml
name: Accessibility Testing
on: [push, pull_request]

jobs:
  accessibility:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Start application
        run: docker-compose up -d app

      - name: Wait for app
        run: sleep 10

      - name: Run Pa11y CI
        run: |
          docker run --rm --network host \
            -v ${PWD}/.pa11yci.json:/.pa11yci.json \
            pa11y/pa11y-ci \
            pa11y-ci --config /.pa11yci.json

      - name: Generate HTML Report
        if: always()
        run: |
          docker run --rm --network host \
            pa11y/pa11y \
            pa11y --reporter html http://localhost:8080 > accessibility-report.html

      - name: Upload Report
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: accessibility-report
          path: accessibility-report.html
```

### Pipeline Script

```bash
#!/bin/bash
# test-accessibility.sh

set -e

TARGET_URL=${1:-"http://localhost:8080"}
REPORT_DIR=${2:-"./reports/a11y"}

mkdir -p $REPORT_DIR

# Run Pa11y with JSON output
docker exec pa11y pa11y \
  --standard WCAG2AA \
  --reporter json \
  --include-warnings \
  $TARGET_URL > "$REPORT_DIR/pa11y.json"

# Count issues by type
ERRORS=$(jq '[.issues[] | select(.type == "error")] | length' "$REPORT_DIR/pa11y.json")
WARNINGS=$(jq '[.issues[] | select(.type == "warning")] | length' "$REPORT_DIR/pa11y.json")
NOTICES=$(jq '[.issues[] | select(.type == "notice")] | length' "$REPORT_DIR/pa11y.json")

echo "Accessibility Report for $TARGET_URL"
echo "=================================="
echo "Errors: $ERRORS"
echo "Warnings: $WARNINGS"
echo "Notices: $NOTICES"

# Generate HTML report
docker exec pa11y pa11y \
  --standard WCAG2AA \
  --reporter html \
  $TARGET_URL > "$REPORT_DIR/report.html"

# Fail on errors
if [ "$ERRORS" -gt 0 ]; then
  echo "FAIL: $ERRORS accessibility errors found"
  exit 1
fi

echo "PASS: No critical accessibility errors"
```

## Quality Gates

### Threshold-Based Gating

```json
// .pa11yci.json with thresholds
{
  "defaults": {
    "standard": "WCAG2AA",
    "threshold": 0
  },
  "urls": [
    {
      "url": "http://localhost:8080/",
      "threshold": 0
    },
    {
      "url": "http://localhost:8080/legacy",
      "threshold": 10,
      "ignore": [
        "WCAG2AA.Principle1.Guideline1_4.1_4_3"
      ]
    }
  ]
}
```

### Custom Quality Gate

```bash
#!/bin/bash
# a11y-gate.sh

RESULTS="pa11y-results.json"

# Parse results
ERRORS=$(jq '[.issues[] | select(.type == "error")] | length' $RESULTS)
CONTRAST_ERRORS=$(jq '[.issues[] | select(.code | contains("1_4_3"))] | length' $RESULTS)
LABEL_ERRORS=$(jq '[.issues[] | select(.code | contains("1_3_1"))] | length' $RESULTS)

# Quality gate rules
FAIL=false

if [ "$ERRORS" -gt 0 ]; then
  echo "CRITICAL: $ERRORS accessibility errors"
  FAIL=true
fi

if [ "$CONTRAST_ERRORS" -gt 5 ]; then
  echo "WARNING: $CONTRAST_ERRORS contrast issues exceed threshold"
fi

if [ "$LABEL_ERRORS" -gt 0 ]; then
  echo "CRITICAL: $LABEL_ERRORS form labeling errors"
  FAIL=true
fi

if [ "$FAIL" = true ]; then
  exit 1
fi

echo "Accessibility quality gate passed"
```

## Integration with Playwright

```javascript
// playwright-a11y.spec.js
const { test, expect } = require('@playwright/test');
const pa11y = require('pa11y');

test.describe('Accessibility', () => {
  test('homepage meets WCAG 2.1 AA', async ({ page }) => {
    await page.goto('/');

    const results = await pa11y(page.url(), {
      standard: 'WCAG2AA',
      runners: ['axe', 'htmlcs']
    });

    const errors = results.issues.filter(i => i.type === 'error');
    expect(errors).toHaveLength(0);
  });

  test('forms are accessible', async ({ page }) => {
    await page.goto('/contact');

    const results = await pa11y(page.url(), {
      standard: 'WCAG2AA'
    });

    const labelErrors = results.issues.filter(
      i => i.code.includes('1_3_1')
    );
    expect(labelErrors).toHaveLength(0);
  });
});
```

## Best Practices

1. **Test Early** - Include accessibility testing in development
2. **WCAG 2.1 AA** - Target at minimum AA compliance
3. **Automated + Manual** - Pa11y catches ~30% of issues
4. **Fix Errors First** - Address errors before warnings
5. **Test User Flows** - Use actions for authenticated pages
6. **Monitor Trends** - Track accessibility over time
7. **Document Exceptions** - Record accepted warnings
8. **Screen Reader Testing** - Manual testing for completeness

## Integration with Stack

- Run alongside Playwright E2E tests
- Part of visual regression with BackstopJS
- Results trackable in DefectDojo
- Reports included in Allure
- Complements security testing (OWASP includes accessibility)

## Troubleshooting

### Common Issues

**Issue**: Timeout errors
```bash
# Increase timeout
docker exec pa11y pa11y --timeout 120000 http://example.com

# Add wait time
docker exec pa11y pa11y --wait 5000 http://example.com
```

**Issue**: Chrome crashes
```bash
# Add sandbox flags
docker exec pa11y pa11y \
  --chrome-launch-config '{"args":["--no-sandbox","--disable-setuid-sandbox"]}' \
  http://example.com
```

**Issue**: Dynamic content not tested
```bash
# Wait for element
docker exec pa11y pa11y --actions "wait for element #content" http://example.com
```

## Resources

- **WCAG Guidelines**: https://www.w3.org/WAI/WCAG21/quickref/
- **Pa11y Documentation**: https://pa11y.org/
- **Color Contrast Checker**: https://webaim.org/resources/contrastchecker/
- **ARIA Authoring Practices**: https://www.w3.org/WAI/ARIA/apg/
