Testing & Security Stack

Stack Overview

Category	Services	Purpose
Code Quality	SonarQube, Semgrep	Static analysis, code quality
Security Testing	OWASP ZAP, Nuclei, Gitleaks, Checkov, Falco	Dynamic security, secret detection, IaC, runtime
Container Security	Trivy, Syft	Vulnerability scanning, SBOM generation
AI Security	CAI	AI-powered offensive/defensive security automation
API Testing	Newman, WireMock, Pact, RESTler	API testing, mocking, contracts, fuzzing
Performance Testing	K6, Locust, Artillery, Toxiproxy	Load testing, stress testing, chaos engineering
Browser Testing	Playwright, BackstopJS, Pa11y	E2E, visual regression, accessibility
Database Testing	Flyway	Migration testing, schema validation
Vulnerability Mgmt	DefectDojo	Finding aggregation, tracking, reporting
Reporting	Allure	Test result aggregation and visualization

Quick Start Commands

# Start all services
make up

# Run all scans
make scan-all

# Run security scans only
make security-scan

# Run performance tests
make perf-test

# Run API tests
make api-test

# Stop services
make down

Service Access Points

Service	URL	Credentials
SonarQube	http://localhost:9000	admin/admin
OWASP ZAP	http://localhost:8082	-
WireMock	http://localhost:8081	-
Allure UI	http://localhost:5252	-
Dashboard	http://localhost:10000	-
Locust	http://localhost:8089	-
DefectDojo	http://localhost:8083	admin/defectdojo
Pact Broker	http://localhost:9292	-

Key Workflows

Full Security Scan Pipeline

SAST → DAST → Secret Detection → Results Aggregation

# Run static analysis (SAST)
docker exec semgrep semgrep --config auto --json /src

# Run dynamic analysis (DAST)
docker exec owasp-zap zap-baseline.py -t http://target:8080

# Check for secrets
docker exec gitleaks gitleaks detect --source /src

Performance Testing Pipeline

Smoke Test → Load Test → Stress Test → Analysis

# K6 load test
docker exec k6 k6 run /scripts/load-test.js

# Locust distributed testing
docker exec locust locust -f /locust/locustfile.py --headless

# Artillery scenario test
docker exec artillery artillery run /scripts/spike-test.yml

API Testing Pipeline

# Newman collection tests
docker exec newman newman run /etc/newman/collection.json

# Contract tests with Pact
docker exec pact npm test -- --testPathPattern=pact

# API fuzzing with RESTler
docker exec restler restler fuzz-lean --grammar_file /Compile/grammar.py

Tool Decision Trees

Choosing Security Scanning Tools

Static Analysis (SAST):
├── Code quality + coverage → SonarQube
├── Security patterns + rules → Semgrep
├── Container/FS vulnerabilities → Trivy
├── Secret detection → Gitleaks
├── IaC security (Terraform/K8s) → Checkov
└── Supply chain/SBOM → Syft + Grype

Dynamic Analysis (DAST):
├── Web app scanning → OWASP ZAP
├── Template-based scanning → Nuclei
├── Stateful API fuzzing → RESTler
├── Runtime monitoring → Falco
└── AI-powered analysis → CAI

Choosing Performance Testing Tools

Performance Testing:
├── Developer-friendly (JS) → K6
│   └── Best for: CI/CD integration, scripted tests
├── Distributed testing (Python) → Locust
│   └── Best for: Large scale, real-time dashboard
├── Scenario-based (YAML) → Artillery
│   └── Best for: Complex scenarios, protocol variety
└── Chaos engineering → Toxiproxy
    └── Best for: Failure injection, resilience testing

Quality Gates & Thresholds

Security Thresholds

Severity	Action	Threshold
Critical	Block deployment	0 findings
High	Block deployment	0 findings
Medium	Review required	< 5 findings
Low	Track in backlog	Unlimited

Performance Thresholds

Metric	Target	Fail Threshold
Response Time (p95)	< 200ms	> 500ms
Error Rate	0%	> 1%
Throughput	> 1000 req/s	< 500 req/s
CPU Usage	< 70%	> 90%

Best Practices

Security Testing

• Run SAST before DAST (faster feedback)
• Use quality gates to block deployments
• Scan containers and dependencies
• Check for secrets in every commit
• Aggregate findings in single dashboard (DefectDojo)

Performance Testing

• Start with smoke tests (quick feedback)
• Establish baselines before changes
• Test with realistic data volumes
• Include think time in scripts
• Monitor resource usage during tests

API Testing

• Organize collections by feature
• Use environments for different stages
• Include negative test cases
• Validate contracts not just status codes
• Test authentication and authorization