Back to Guides

Verify UI changes inside the agent loop, before the PR

Published: October 2026•6 min read

Coding agents write code faster than CI can check it. The fix is not only a faster pipeline. It is checking earlier, inside the agent's own loop, against the thing users actually see.

The problem

A recent piece in The New Stack (sponsored by Signadot) gathers the numbers: Anthropic reported CI job volume up 25x in six months, and Linear wrote that AI coding has made CI a bottleneck. Its argument is that faster CI answers the wrong question. CI checks a repository. Users meet a system.

Two points from it apply directly to frontend work:

  • Placement. CI runs after the PR exists. An agent that waits 20 minutes for a red check has lost its context by the time the answer arrives.
  • Repo, not system. The agent's sandbox holds the branch and its mocks. A test with a mocked API passes while the real page renders the wrong price, a missing nav link or a broken layout.

The article is about backend services. For a website, the "system" is the rendered page: the place where your code, your CMS, your APIs, your CSS and third-party scripts all land at once. That is what RegressionBot checks.

The loop

  1. The agent makes a change and serves it: a local dev server behind a tunnel, or a preview deploy.
  2. The agent asks RegressionBot to compare that origin against production, page by page, over MCP.
  3. RegressionBot returns the exact edits it found, read from both pages' DOM.
  4. The agent fixes anything it did not intend, and reruns. Only then does it open the PR.

No test code is written for this. The comparison is live-vs-live: both sides are captured on every run, so there is no baseline to set up first.

Step 1: Connect the agent

claude mcp add --transport http regressionbot https://mcp.regressionbot.com/ --header "Authorization: Bearer YOUR_API_KEY"

Cursor and other MCP clients take the same server. See MCP Server.

Step 2: Give the agent something to compare

The agent's change has to be reachable by URL. A preview deploy works. For a dev server on your laptop, put a temporary tunnel in front of it (Internal Environments). If the preview sits behind a password or Cloudflare Access, see Step 3. Without the credential, a password gate fails every page with “Page refused”, and a sign-in page or SSO redirect is captured as the page, so every page reads as changed.

Step 3: Write the check once

The article's best point is that agents should not invent their own checks. If each agent improvises, no two runs are comparable. Someone writes the steps once, and every agent calls them.

Put the fixed parts of the check in a file the agent reads: the production origin, which pages, which devices, which dynamic widgets to mask. The agent fills in only its own URL and what it meant to change. For Claude Code that is a skill at .claude/skills/check-ui/SKILL.md:

---
name: check-ui
description: Before opening a PR that touches pages, templates, styles or copy, compare the change against production with RegressionBot.
---

1. Make sure the change is served at a public URL (preview deploy or tunnel).
2. Call run_regression_job with:
     testOrigin:  the preview URL
     baseOrigin:  "https://myapp.com"
     sitemapUrl:  "https://myapp.com/sitemap.xml"
     devices:     ["Desktop Chrome", "iPhone 15"]
     masks:       ["#chat-widget", ".cookie-banner"]
     runContext:  { prDescription: one or two sentences saying what you
                    changed and why }
     project:     "myapp-preview"  (only if the preview needs a password:
                    the project that holds its credential)
3. Call get_job_status with waitSeconds 25 until readyForSummary is true,
   then get_job_summary with regressionsOnly true.
4. For each changed page, read its changes entries. Fix any edit you did not
   intend and run again. A page in errors that says "Page is new" is one you
   added: there is nothing in production to compare it with, so that is expected.
5. Read intentAssessment.decision:
     "pass":   open the PR.
     "fail":   fix the page whose verdict is "bug" and run again.
     "review": read each page whose verdict is "needs_review", and its
               reasoning. If every edit on it is one you meant, open the PR
               and list those pages under "Needs a human look". Running
               again will not change them.
   In the PR description, list each changed page with its changes entries.

Every agent now runs the same check, so two runs can be compared. Cursor rules and other agents' instruction files work the same way. If the preview needs a password, the credential cannot be passed on the run: store it once on a project (update_project with testAuth) and add that project's name to the run as project, as the skill above does. The run uses the stored credential, and the agent never sees it. Because the run has a baseOrigin, it leaves the stored project unchanged. Leave project out of the skill if the preview is public.

Step 4: What the agent reads back

Each changed page carries changes: the exact edits, each naming the element and quoting both versions. The agent does not have to interpret a picture.

{
  "url": "https://preview-123.myapp.com/pricing",
  "variantName": "Desktop Chrome",
  "changed": true,
  "changes": [
    { "type": "text-edit", "element": "h2", "before": "Pro", "after": "Professional" },
    { "type": "delete", "element": "a", "before": "Compare plans" }
  ],
  "diffUrl": "https://..."
}

If the agent was asked to rename the plan, the first edit is expected and the second is not: a link disappeared. The agent can fix it before anyone reviews the PR. diffUrl is the link for a person: it opens both captures with every change drawn on them.

When the run carries an intent (runContext), the job also returns intentAssessment.decision: pass, review, fail, or not_judged when no intent was given. That one field is what an agent or a merge gate reads. A vague intent ("fixes") earns a review, not a pass. So can a precise one: a page with more edits than the verdict is shown, or one it could not tie to the intent word for word, comes back as review, which is why step 5 says what to do with it. See Reading results.

A faster check for copy and structure

For a change to words, links or page structure, a quick check (fidelity: "low") takes no screenshots and runs in a few seconds a page. It says so in its scopeNote: layout, color and images were not checked. If the change touches CSS, run the full check.

Then CI confirms

The same check can run again on the PR (PR Integration), now as a confirmation rather than the first time anyone looks. Two limits to keep in mind:

  • Scope. A run over the pages the agent changed previews that change. Only a run over the whole site can say nothing else broke.
  • What it sees. RegressionBot checks what the browser renders. It does not test a backend call that never reaches the page; service-level checks like the ones the article describes cover that.

The order changes from write, PR, wait, fix to write, check, fix, PR. The agent still has its context when the answer comes back, and the reviewer gets a PR that already shows what changed on the page.

Ready to ship with confidence?

Start catching visual changes before they reach production. Start your 7-day free trial today. No credit card required.