Preview vs Production Workflow

The gold standard for modern visual regression: Catching bugs before they merge by directly comparing temporary Preview environments to the Live Production site.

The Strategy

Traditional visual testing requires you to manage a "golden image" baseline in your git repository. This is painful—it bloats your repo, creates merge conflicts, and the baselines are often out of date.

RegressionBot recommendation: Use your live Production site as the baseline. When a developer opens a PR, compare their Preview URL against the Live URL. No images in git, zero maintenance.

Zero Baseline Bloat

Stop storing thousands of PNGs in your Git history. Let the cloud handle it.

Self-Healing

Baselines naturally update automatically as your production site evolves.

The 3-Step Lifecycle

Automated Check CI/CD

On every push, GitHub Actions waits for your preview deployment to finish and then triggers a RegressionBot job. It renders screenshots of both the preview URL and production URL to compare them.

Visual Review Developer

If regressions are detected, the PR check fails. The developer follows the link in the PR comment to inspect the diff images.

Approval ChatOps

If the change is intentional (e.g., a new UI feature), the developer comments /approve-visual on the PR. RegressionBot approves the job, and the PR check turns green.

GitHub Actions Configuration

The simplest way to integrate RegressionBot is by using our official shared GitHub Action, which automatically triggers runs and generates formatted markdown reports.

name: Visual Check
on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write # Required to post/update comments on your PR
    steps:
      - uses: actions/checkout@v4

      # Replace this with your preview provider's output or dynamic setup.
      - name: Set preview URL
        run: echo "PREVIEW_URL=https://preview.myapp.com" >> "$GITHUB_ENV"

      - name: Run Visual Check
        id: vrs
        uses: RegressionBot/regressionbot-action@v0
        with:
          api-key: ${{ secrets.REGRESSIONBOT_API_KEY }}
          test-origin: ${{ env.PREVIEW_URL }}
          base-origin: 'https://myapp.com'
          sitemap-url: '${{ env.PREVIEW_URL }}/sitemap.xml'
          scan: '**'
          project: 'my-web-app'
          devices: 'Desktop Chrome, iPhone 12'
          github-token: ${{ secrets.GITHUB_TOKEN }}
          run-context: |
            prTitle: ${{ github.event.pull_request.title }}
            prDescription: ${{ github.event.pull_request.body }}

⚠️ Forked Repositories & PR PermissionsBy default, GitHub Actions runs triggered by pull requests from forked repositories have a read-only GITHUB_TOKEN for security. This will cause comment creation/update steps to fail. If you accept community contributions from forks, you should run the workflow on the pull_request_target event (with appropriate checkouts) or configure the comment steps to fail gracefully using continue-on-error: true.

For advanced custom integrations where you want to write your own reporting logic or parse raw job outputs, you can also use our Node.js SDK:

name: Visual Check
on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write # Required to post/update comments on your PR
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20

      # Replace this with your preview provider's output.
      - name: Set preview URL
        run: echo "PREVIEW_URL=https://preview.myapp.com" >> "$GITHUB_ENV"

      - name: Install RegressionBot SDK
        run: npm install @regressionbot/sdk

      - name: Run RegressionBot and write report
        run: |
          node --input-type=module <<'EOF'
          import fs from 'node:fs';
          import { RegressionBot } from '@regressionbot/sdk';

          const client = new RegressionBot(process.env.REGRESSIONBOT_API_KEY);
          const job = await client
            .test(process.env.PREVIEW_URL)
            .against('https://myapp.com')
            .forProject('my-web-app')
            .on(['Desktop Chrome', 'iPhone 12'])
            .sitemap(process.env.PREVIEW_URL + '/sitemap.xml')
            .scan('/pages/**')
            .withContext({
              prTitle: process.env.PR_TITLE,
              prDescription: process.env.PR_BODY
            })
            .run();

          await job.waitForCompletion(5000, undefined, { waitForSummaries: true });
          const summary = await job.getSummary();

          const lines = [
            '## 🚀 RegressionBot Visual Test Results',
            '',
            '**Job ID:** `' + job.jobId + '`',
            '**Score:** ' + summary.overallScore + '/100',
            '**Regressions:** ' + summary.regressionCount,
            ''
          ];

          for (const regression of summary.regressions || []) {
            const diff = (regression.diffPercentage ?? 0).toFixed(3);
            lines.push(
              '### ' + regression.url,
              '',
              '**Variant:** ' + regression.variantName,
              '**Diff:** ' + diff + '%',
              '**Visual Match:** ' + regression.visualMatchScore + '/100',
              ''
            );

            if (regression.regressionbotSummary) {
              const summaryLines = Array.isArray(regression.regressionbotSummary)
                ? regression.regressionbotSummary.map((item) =>
                    item.label ? '**' + item.label + '**: ' + item.text : item.text)
                : [regression.regressionbotSummary];
              lines.push(
                '> **RegressionBot Summary:**',
                ...summaryLines.map((l) => '> ' + l),
                ''
              );
            }

            if (regression.diffUrl) {
              lines.push('[Open diff image](' + regression.diffUrl + ')', '');
            }
          }

          fs.writeFileSync('regressionbot-report.md', lines.join('
'));
          if (summary.regressionCount > 0) process.exit(1);
          EOF
        env:
          REGRESSIONBOT_API_KEY: ${{ secrets.REGRESSIONBOT_API_KEY }}
          PR_TITLE: ${{ github.event.pull_request.title }}
          PR_BODY: ${{ github.event.pull_request.body }}

      - name: Comment visual summary on PR
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const body = fs.existsSync('regressionbot-report.md')
              ? fs.readFileSync('regressionbot-report.md', 'utf8')
              : 'RegressionBot did not produce a report. Check the workflow logs.';

            const { data: comments } = await github.rest.issues.listComments({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
            });

            const botComment = comments.find(comment => 
              comment.body.includes('🚀 RegressionBot Visual Test Results')
            );

            if (botComment) {
              await github.rest.issues.updateComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                comment_id: botComment.id,
                body
              });
            } else {
              await github.rest.issues.createComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: context.issue.number,
                body
              });
            }

Enable ChatOps Approval

To allow developers to approve changes directly from comments, add this "Approval" workflow. This will also update the commit status check on GitHub to green.

on:
  issue_comment:
    types: [created]

jobs:
  approve:
    if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/approve-visual')
    runs-on: ubuntu-latest
    permissions:
      statuses: write # Required to update commit status check
      pull-requests: read
    steps:
      - name: Approve RegressionBot Job
        run: |
          JOB_ID=$(gh api repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/comments --jq '.[] | select(.body | contains("🚀 RegressionBot Visual Test Results")) | .body' | grep "Job ID:" | awk -F'`' '{print $2}' | tail -n 1)
          if [ -z "$JOB_ID" ]; then
            echo "Error: Could not find a RegressionBot job ID for this PR."
            exit 1
          fi
          npx regressionbot approve "$JOB_ID"
        env:
          REGRESSIONBOT_API_KEY: ${{ secrets.REGRESSIONBOT_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Update GitHub status check to success
        run: |
          # Fetch PR head SHA
          SHA=$(gh api repos/${{ github.repository }}/pulls/${{ github.event.issue.number }} --jq '.head.sha')
          
          # Mark visual regression status check as green/success
          gh api \
            --method POST \
            -H "Accept: application/vnd.github+json" \
            /repos/${{ github.repository }}/statuses/$SHA \
            -f state='success'             -f target_url="${{ github.event.comment.html_url }}"             -f description='Visual regressions approved by developer'             -f context='RegressionBot Visual Check'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}