Troubleshooting
Find answers to common issues. These include problems encountered during setup, job execution, and CI/CD pipeline integration.
Common API Errors
The API gateway intercepts unauthorized or invalid requests before they execute.
Why this happens: The request lacked the required credentials. MCP clients authenticate using the standard Authorization: Bearer <token> header. Direct API callers send Authorization: Bearer <key> (x-api-key is still accepted); SDK callers set the REGRESSIONBOT_API_KEY environment variable.
Solution: Retrieve your API key from the Dashboard. We recommend configuring your client to use the standard Authorization: Bearer <token> header. For MCP integrations using environment variables (such as Codex's bearer_token_env_var), set the configuration parameter to the name of the environment variable (e.g. "REGRESSIONBOT_API_KEY") and export that variable in your shell (export REGRESSIONBOT_API_KEY="your_key"). Do not put the literal API key string directly into the config parameter. The legacy x-api-key header remains supported for backwards compatibility.
Why this happens: Your organization's free trial has expired. The free trial includes 500 comparisons or 7 days of access. Alternatively, your paid subscription is no longer active.
Solution: Log into the Dashboard. Navigate to the Billing tab. Subscribe to a paid plan (Hobby, Starter, or Pro) to resume triggering jobs.
Why this happens: You queried the status of a Job ID that does not exist or has expired. Visual regression jobs are archived after 30 days.
Solution: Double-check the Job ID UUID string returned from your start request. Verify you are using the correct target organization namespace.
Why this happens: You triggered a job for an existing project. The parameters in your request body did not match the configuration stored in the project's record. This includes parameters like testOrigin, devices, or paths.
Solution: Omit these parameters from your request to use the saved configuration automatically. Alternatively, trigger the job via POST /project/{name}/run. This endpoint always loads the saved configuration. To update the configuration, use the MCP update_project tool. Note that updating the configuration will invalidate existing baselines.
Why this happens: There is no separate AI allowance to run out of — summaries are not metered, so this is never a quota problem. A page whose only change is in its metadata (a title, a canonical link, a JSON-LD block) has no summary, because nothing on the page looks different. On results from before September 28, 2026, a page could also be left undescribed when its DOM could not be read; those carry a domAssistSkipReason naming why.
Solution: Read the result's changes array. It lists the changes measured on the page, whether or not a summary was written.
Why this happens: Another job has approved and updated the project baseline since this test run executed. Approving the outdated job would overwrite the newer baseline changes.
Solution: Re-run the visual check to compare the branch changes against the latest updated baseline.
Handling Failed Jobs
If a visual regression job enters the FAILED state, the worker could not capture one or more pages. Common reasons include:
- DNS Failures: The host could not be resolved from inside the cloud network. Examples include
localhostor local company development URLs. - SSL Errors: Self-signed certificates are used on staging servers. Use a publicly trusted certificate for environments you want the cloud workers to capture.
- Authentication Walls: Basic auth, OAuth redirect screens, or captcha verification pages prevent headless browser crawls.
A single page can also end with status: "ERROR" while the rest of the job completes. If the error says "This capture has no DOM tree", the page's DOM could not be read, and nothing is compared without it. Run the page again. A page uploaded by an SDK must include its DOM snapshot.
Timeouts & Page Height Limits
Headless visual workers allocate a standard 30-second budget. This budget is used for screenshot capture and lazy-load scroll execution.
Tips to fix timeouts:
- Exclude large media or infinite-scrolling pages from your sitemaps. Examples include product lists or social feeds.
- Provide dynamic element masking using the
data-vr-maskattribute. This bypasses dynamic elements. - Limit comparison depths when invoking sitemap crawler endpoints.
Additionally, visual workers enforce a safe page height limit of 36,000 physical pixels (divided by the Device Pixel Ratio, or DPR). This prevents high-memory crashes on extremely tall pages. For example, at a 1x device scale, the limit is 36,000px; at 2x scale, it is 18,000px; and at 3x scale, it is 12,000px.
Capped Screenshot Fallback Behavior:
If a page exceeds this height limit or causes a browser capture timeout, the system does not fail the job. Instead, it automatically captures a **capped full-page screenshot** up to the maximum safe height. The cap applies to the screenshot, which is a picture for you to look at. The comparison reads the page's DOM.
If a long page's screenshot shows only its top portion, verify that the page height fits within the safe limits, or use viewports or device profiles with lower scale factors.
Change Metrics & Noise
RegressionBot compares the two pages' DOM trees, not their pixels. Anti-aliasing and other rendering differences in the screenshots are therefore never reported as changes.
Diff Percentage
diffPercentage is the share of the page covered by the elements that changed (the union of their boxes). It tells you how much of the page changed, not whether the change matters. A small text edit can score under 1% and is still a change.
Visual Match Score
visualMatchScore is deprecated: it is 100 minus diffPercentage, not a measure of how alike two pictures look, and the summary's overallScore is its average. Both are kept for existing callers. Read changed and changes instead.
What is already left out:
- Parts of the page that changed on their own while it was captured, such as carousels and counters. The page is read twice, 300ms apart, and anything that differs between the two reads is marked
dynamicand left out ofchanges. - Edits where only a date or time changed. They are listed with
valueOnly: "date"but do not make the page changed. - Content that was only pushed down by a change above it.
Gate on changed, not on a threshold, and read changes to see what happened:
await job.waitForCompletion();
const summary = await job.getSummary();
for (const regression of summary.regressions) {
if (!regression.changed) continue;
for (const c of regression.changes ?? []) {
console.log(regression.url, c.type, c.element, c.before ?? '', c.after ?? '');
}
}A change you can see but that is not reported:
A change that exists only in how the page is painted can go unreported. One example is an image that failed to decode while its element and source stayed the same. Many such cases are still caught: broken images, canvas content, and web fonts that failed to load (reported as a font-failed change). If you spot one in the screenshots, open the page in the inspect view to compare them.
Results created before September 28, 2026 (kept for 30 days) were compared by pixels. On those, diffPercentage is the share of pixels that differed and visualMatchScore is a pixel similarity score.
Unexpected Sticky/Fixed Element Diffs
Seeing a change reported on a header, nav, or cookie bar — either on a project you didn't touch, or a position change on an element that looks the same in both screenshots?
Why this happens: RegressionBot captures the full page from the top, so position: fixed and position: sticky elements render once at their at-rest position rather than repeating down the page. Three different situations put a change on them, and they have different answers.
- A baseline older than September 5, 2026. A baseline captured before v2.7.1 recorded those elements through an older rewriting step, so the element reads
relativeon the baseline side andstickyon the current side. The first managed run after that reports apositionchange on the header, nav or cookie bar. This is a one-time diff: approve it, and the run after that compares clean. As of v2.9.1 (September 12, 2026) this is reported rather than filtered out — earlier releases said such a change was usually classified as a reposition and flagged nothing, which is no longer how it behaves. - Someone actually pinned the element. Changing an element from
statictosticky— making a header stay put instead of scrolling away — moves no pixels while the page sits at the top. The two screenshots look identical, and the captured document is the only witness to it. RegressionBot reports this as apositionchange even though the screenshots look the same. This is a real change to the page, not noise. Approve it only if you meant to make it. - An element that renders badly at rest on every run. A fixed element may sit somewhere unhelpful at rest, such as a bottom-anchored banner landing over the first screen. That is not a one-time diff, and approving it will not make it go away.
Telling them apart: if the project's baselines were captured or last approved before September 5, 2026, start with case 1. If the same element is flagged again on the run after you approve, you are in case 3. If the baselines are newer than that date and the change is a position value on an element that looks the same in both screenshots, check whether someone changed that element's CSS — that is case 2, and the run is telling you something true.
Solution: Open the run in the dashboard, or read changes and regressionbotSummary, and confirm the change is confined to the header, nav or banner and is a position value. For case 1, approve the affected pages to update your baselines. For case 2, treat it as any other change to the page: approve it if the pinning was intended, reject it if it was not. Live-vs-live comparisons are unaffected by case 1; both sides are captured the same way. For an element that renders somewhere unhelpful on every run, use customCss to reposition or hide it — it is applied last, so it overrides the site's own rules. See Custom CSS Injection.
Before you change it: customCss changes what a capture looks like, so changing it invalidates every baseline for the project. The next run re-captures everything and every result shows isNewBaseline: true; the run after that compares. See Baseline Invalidation.
# Reposition or hide a fixed element that renders somewhere unhelpful at rest
curl -X PUT https://api.regressionbot.com/project/my-site \
-H "Authorization: Bearer $REGRESSIONBOT_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "customCss": "#sticky-banner { display: none !important; }" }'Initial Job (Empty Baseline)
Initial jobs run without prior baselines. RegressionBot treats this as a successful execution. It marks each page result status with isNewBaseline: true. Subsequent jobs will automatically compare against this newly established baseline.
Short Screenshot URLs
Troubleshoot common issues when using shortened, clickable screenshot redirect URLs.
Why this happens: The short screenshot URL token has expired. Redirection tokens are only valid for one hour.
Solution: Request the job summary again using the API. This generates fresh short URLs and new tokens in the database.
Why this happens: You requested the job summary before the job finished processing. The status is still PROCESSING or SUMMARIZING.
Solution: Poll the job status endpoint (/job/{jobId}). Wait for the status to become COMPLETED, APPROVED, or FAILED before fetching the summary.
Why this happens: The backend fell back to standard pre-signed URLs. This is caused by a database write error when saving short-lived tokens.
Solution: Check the RegressionBot system status page. Alternatively, contact support. The client received the response successfully. Your network connection is functioning correctly.