DevSecOps

Shift-Left Security

Shift-left means moving security testing and validation earlier in the Software Development Lifecycle (SDLC). Vulnerabilities found at the code stage cost ~$80 to fix; the same vulnerability found in production can cost $7,600 or more (NIST data). The earlier you catch it, the cheaper and faster the fix.

SAST — Static Application Security Testing

SAST analyzes source code, bytecode, or binaries for security vulnerabilities without executing the code. It catches issues like SQL injection, hardcoded secrets, insecure deserialization, and buffer overflows at the code level.

Key SAST Tools

• Semgrep: Fast, customizable static analysis supporting 30+ languages. Write custom rules in YAML. Free tier covers most use cases.
• SonarQube/SonarCloud: Comprehensive code quality and security platform. Tracks code coverage, complexity, and security hotspots over time.
• Bandit: Python-specific SAST tool that checks for common security issues in Python code.
• gosec: Go-specific security scanner that inspects source code for security problems.
• CodeQL: GitHub's semantic code analysis engine — powerful for detecting complex vulnerability patterns.

Semgrep in GitHub Actions

name: SAST - Semgrep
on:
  pull_request:
  push:
    branches: [main]

jobs:
  semgrep:
    runs-on: ubuntu-latest
    container:
      image: semgrep/semgrep
    steps:
      - uses: actions/checkout@v4

      - name: Run Semgrep SAST
        run: |
          semgrep ci \
            --config=p/security-audit \
            --config=p/owasp-top-ten \
            --config=p/secrets \
            --sarif \
            --output=semgrep-results.sarif
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

      - name: Upload SARIF results
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: semgrep-results.sarif

      - name: Fail on high severity findings
        run: |
          semgrep ci \
            --config=p/security-audit \
            --severity ERROR \
            --error
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

Bandit for Python

# Run Bandit locally
pip install bandit
bandit -r ./src -ll -ii -f json -o bandit-report.json

# GitHub Actions integration
- name: Run Bandit (Python SAST)
  run: |
    pip install bandit
    bandit -r ./src \
      --level HIGH \
      --confidence HIGH \
      --format json \
      --output bandit-report.json || true

- name: Check Bandit results
  run: |
    HIGH_ISSUES=$(jq '.results | map(select(.issue_severity == "HIGH")) | length' bandit-report.json)
    if [ "$HIGH_ISSUES" -gt 0 ]; then
      echo "Found $HIGH_ISSUES HIGH severity issues"
      jq '.results[] | select(.issue_severity == "HIGH")' bandit-report.json
      exit 1
    fi

DAST — Dynamic Application Security Testing

DAST tests running applications from the outside, simulating an attacker. It finds vulnerabilities that only manifest at runtime: XSS, CSRF, authentication flaws, insecure headers, injection vulnerabilities reachable through the API surface.

OWASP ZAP in CI/CD

name: DAST - OWASP ZAP Scan
on:
  schedule:
    - cron: '0 2 * * *'  # Nightly
  workflow_dispatch:

jobs:
  zap-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Deploy to staging
        run: echo "Deploy step here"

      - name: Run OWASP ZAP Baseline Scan
        uses: zaproxy/[email protected]
        with:
          target: 'https://staging.myapp.example.com'
          rules_file_name: '.zap/rules.tsv'
          cmd_options: '-a -j -l WARN'

      - name: Run OWASP ZAP Full Scan (for scheduled runs)
        if: github.event_name == 'schedule'
        uses: zaproxy/[email protected]
        with:
          target: 'https://staging.myapp.example.com'
          cmd_options: '-a -j'

      - name: Upload ZAP Report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: zap-scan-report
          path: report_html.html

Container Security

Trivy Container Scanning in CI

name: Container Security - Trivy
on:
  push:
    branches: [main]
  pull_request:

jobs:
  trivy-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build Docker image
        run: docker build -t myapp:${{ github.sha }} .

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'myapp:${{ github.sha }}'
          format: 'sarif'
          output: 'trivy-results.sarif'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'
          ignore-unfixed: true
          vuln-type: 'os,library'

      - name: Upload Trivy SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: 'trivy-results.sarif'

      - name: Scan filesystem for misconfigurations
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          format: 'table'
          severity: 'CRITICAL,HIGH'
          scanners: 'misconfig,secret'

Dockerfile Security Best Practices

# GOOD: Secure Dockerfile example
FROM python:3.12-slim AS builder
WORKDIR /app

# Install dependencies as root in builder stage
COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

# Final stage: minimal runtime image
FROM gcr.io/distroless/python3-debian12

# Copy only what's needed
COPY --from=builder /root/.local /root/.local
COPY --from=builder /app /app
COPY src/ /app/src/

# Run as non-root user
USER nonroot:nonroot

# No secrets in ENV, ARGs, or COPY
# Use runtime secret injection instead

# Read-only filesystem hints
VOLUME ["/tmp"]

WORKDIR /app
EXPOSE 8080
ENTRYPOINT ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080"]

# --- AVOID ---
# FROM ubuntu:latest          # Too large, outdated
# RUN apt-get install ...     # Without pinning versions
# COPY . .                    # Copies everything including .git, secrets
# ENV SECRET_KEY=abc123       # Never hardcode secrets
# USER root                   # Don't run as root in production
# RUN pip install -r req.txt  # Without --no-cache-dir

Image Signing with Cosign

# Generate a key pair for signing (or use keyless with OIDC)
cosign generate-key-pair

# Sign an image after building and pushing
cosign sign \
  --key cosign.key \
  myregistry.io/myapp:v1.2.3@sha256:abc123...

# Keyless signing using GitHub Actions OIDC (Sigstore)
- name: Sign container image (keyless)
  run: |
    cosign sign \
      --yes \
      --rekor-url=https://rekor.sigstore.dev \
      ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.push.outputs.digest }}
  env:
    COSIGN_EXPERIMENTAL: 1

# Verify image signature before deployment
cosign verify \
  --key cosign.pub \
  myregistry.io/myapp:v1.2.3

# Kyverno policy to enforce signed images
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-image-signature
      match:
        any:
          - resources:
              kinds: [Pod]
      verifyImages:
        - imageReferences:
            - "myregistry.io/*"
          attestors:
            - count: 1
              entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
                      -----END PUBLIC KEY-----

Software Supply Chain Security

SBOM Generation with Syft and Grype

# Generate SBOM in SPDX format using Syft
syft myapp:latest -o spdx-json=sbom.spdx.json

# Generate SBOM in CycloneDX format
syft myapp:latest -o cyclonedx-json=sbom.cyclonedx.json

# Scan SBOM for vulnerabilities with Grype
grype sbom:./sbom.spdx.json --fail-on high

# GitHub Actions: Generate and attest SBOM
- name: Generate SBOM
  uses: anchore/sbom-action@v0
  with:
    image: myregistry.io/myapp:${{ github.sha }}
    format: spdx-json
    output-file: sbom.spdx.json

- name: Scan SBOM for vulnerabilities
  uses: anchore/scan-action@v3
  with:
    sbom: sbom.spdx.json
    fail-build: true
    severity-cutoff: high

- name: Attach SBOM as release artifact
  uses: actions/upload-artifact@v4
  with:
    name: sbom
    path: sbom.spdx.json

Dependency Scanning with Snyk

- name: Snyk dependency scan
  uses: snyk/actions/python@master
  env:
    SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
  with:
    command: test
    args: --severity-threshold=high --fail-on=all

# npm audit for Node.js projects
- name: npm audit
  run: |
    npm audit --audit-level=high --json | tee npm-audit.json
    CRITICAL=$(jq '.metadata.vulnerabilities.critical' npm-audit.json)
    HIGH=$(jq '.metadata.vulnerabilities.high' npm-audit.json)
    if [ "$CRITICAL" -gt 0 ] || [ "$HIGH" -gt 0 ]; then
      echo "Found $CRITICAL critical and $HIGH high vulnerabilities"
      exit 1
    fi

Infrastructure as Code Security Scanning

Checkov for Terraform

# Run Checkov on Terraform code
pip install checkov
checkov -d ./terraform \
  --framework terraform \
  --output sarif \
  --output-file checkov-results.sarif \
  --soft-fail-on LOW,MEDIUM

# GitHub Actions
- name: Checkov IaC Scan
  uses: bridgecrewio/checkov-action@v12
  with:
    directory: terraform/
    framework: terraform
    output_format: sarif
    output_file_path: reports/results.sarif
    soft_fail_on: LOW,MEDIUM
    hard_fail_on: HIGH,CRITICAL

- name: Upload Checkov SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: reports/results.sarif

tfsec for Terraform

# Install and run tfsec
brew install tfsec
tfsec ./terraform \
  --format sarif \
  --out tfsec-report.sarif \
  --minimum-severity HIGH

# Custom tfsec configuration (.tfsec/config.yml)
exclude:
  - AVD-AWS-0089  # CloudTrail encryption (handled by org policy)
minimum_severity: HIGH
include_passed: false

kube-score for Kubernetes Manifests

# Install kube-score
brew install kube-score

# Score all manifests
kube-score score ./k8s/**/*.yaml \
  --output-format ci \
  --ignore-test container-security-context-readonlyrootfilesystem

# In CI pipeline
- name: kube-score
  run: |
    curl -L https://github.com/zegl/kube-score/releases/latest/download/kube-score_linux_amd64 -o kube-score
    chmod +x kube-score
    find ./k8s -name "*.yaml" | xargs ./kube-score score \
      --output-format ci \
      --exit-one-on-warning

Pre-commit Hooks for IaC

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.92.0
    hooks:
      - id: terraform_fmt
      - id: terraform_validate
      - id: terraform_tflint
      - id: terraform_checkov
        args:
          - --args=--config-file .checkov.yaml
      - id: terraform_tfsec
        args:
          - --args=--minimum-severity HIGH

  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: detect-private-key
      - id: check-merge-conflict
      - id: end-of-file-fixer
      - id: trailing-whitespace

Policy as Code

OPA (Open Policy Agent) for Kubernetes Admission

# Rego policy: Require non-root containers
package kubernetes.admission

import future.keywords.in

deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    not container.securityContext.runAsNonRoot
    msg := sprintf("Container '%v' must run as non-root", [container.name])
}

deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    container.securityContext.runAsUser == 0
    msg := sprintf("Container '%v' cannot run as root (UID 0)", [container.name])
}

# Rego policy: Require resource limits
deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    not container.resources.limits.memory
    msg := sprintf("Container '%v' must have memory limits defined", [container.name])
}

deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    not container.resources.limits.cpu
    msg := sprintf("Container '%v' must have CPU limits defined", [container.name])
}

Kyverno ClusterPolicy Examples

# Kyverno: Disallow privileged containers
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: disallow-privileged-containers
  annotations:
    policies.kyverno.io/title: Disallow Privileged Containers
    policies.kyverno.io/severity: high
spec:
  validationFailureAction: Enforce
  background: true
  rules:
    - name: privileged-containers
      match:
        any:
          - resources:
              kinds: [Pod]
      validate:
        message: "Privileged mode is disallowed."
        pattern:
          spec:
            =(initContainers):
              - =(securityContext):
                  =(privileged): "false"
            containers:
              - =(securityContext):
                  =(privileged): "false"
---
# Kyverno: Require specific labels on all Pods
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-pod-labels
spec:
  validationFailureAction: Enforce
  rules:
    - name: check-team-label
      match:
        any:
          - resources:
              kinds: [Pod]
      validate:
        message: "Pod must have label 'app.kubernetes.io/name' and 'team'"
        pattern:
          metadata:
            labels:
              app.kubernetes.io/name: "?*"
              team: "?*"

Secret Scanning

Gitleaks Pre-commit Hook

# .pre-commit-config.yaml (add to existing config)
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.4
    hooks:
      - id: gitleaks
        name: Detect hardcoded secrets
        language: golang
        entry: gitleaks protect --verbose --redact --staged

# .gitleaks.toml configuration
[extend]
useDefault = true

[[rules]]
id = "custom-api-key"
description = "Custom API Key pattern"
regex = '''(?i)(api[_-]?key|apikey)\s*[=:]\s*['"]?([a-zA-Z0-9]{32,64})['"]?'''
tags = ["secret", "api-key"]

[allowlist]
commits = ["abcdef1234567890"]  # Known false positive commits
paths = [".gitleaks.toml", "tests/fixtures/"]
regexes = ['''EXAMPLE_KEY''', '''PLACEHOLDER''']

detect-secrets

# Install and create baseline
pip install detect-secrets
detect-secrets scan > .secrets.baseline

# Add to pre-commit
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.4.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']
        exclude: package.lock.json

# Audit the baseline (mark false positives)
detect-secrets audit .secrets.baseline

Complete DevSecOps Pipeline

name: DevSecOps Pipeline
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

permissions:
  contents: read
  security-events: write
  id-token: write  # For OIDC

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # ── GATE 1: Secret Scanning ──────────────────────────────────────
  secret-scan:
    name: Secret Detection
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for Gitleaks

      - name: Gitleaks secret scan
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  # ── GATE 2: SAST ─────────────────────────────────────────────────
  sast:
    name: Static Analysis (SAST)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Semgrep SAST
        uses: semgrep/semgrep-action@v1
        with:
          config: >-
            p/security-audit
            p/owasp-top-ten
            p/secrets
        env:
          SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}

  # ── GATE 3: Dependency Scanning ──────────────────────────────────
  dependency-scan:
    name: Dependency Vulnerabilities
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Snyk dependency scan
        uses: snyk/actions/python@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: --severity-threshold=high

  # ── GATE 4: IaC Scanning ─────────────────────────────────────────
  iac-scan:
    name: Infrastructure as Code Scan
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Checkov IaC scan
        uses: bridgecrewio/checkov-action@v12
        with:
          directory: terraform/
          framework: terraform,kubernetes,dockerfile
          soft_fail_on: LOW,MEDIUM
          hard_fail_on: HIGH,CRITICAL

  # ── GATE 5: Build & Container Scan ───────────────────────────────
  build-and-scan:
    name: Build Image & Scan
    runs-on: ubuntu-latest
    needs: [secret-scan, sast, dependency-scan]
    outputs:
      image-digest: ${{ steps.push.outputs.digest }}
    steps:
      - uses: actions/checkout@v4

      - name: Login to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push image
        id: push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}

      - name: Trivy container scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
          format: sarif
          output: trivy-results.sarif
          severity: CRITICAL,HIGH
          exit-code: 1

      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
          format: spdx-json
          output-file: sbom.spdx.json

      - name: Sign image with Cosign
        run: |
          cosign sign --yes \
            ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.push.outputs.digest }}
        env:
          COSIGN_EXPERIMENTAL: 1

  # ── GATE 6: Policy Check ─────────────────────────────────────────
  policy-check:
    name: Policy as Code (OPA/Conftest)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Conftest policy check
        run: |
          wget https://github.com/open-policy-agent/conftest/releases/latest/download/conftest_Linux_x86_64.tar.gz
          tar xzf conftest_Linux_x86_64.tar.gz
          ./conftest test ./k8s/ --policy ./policy/