Changelog
Keep track of latest features, stability fixes, and visual comparison upgrades built for the RegressionBot SaaS platform.
v2.10.1 — The Picture Is the Page
Every page with changed pixels used to get a baked three-panel picture: baseline, current, and a diff with boxes drawn on it. It took one to two seconds a page to draw, and nothing read it. The inspect view draws its own overlay from the findings, the email carries the crop, and the share link was fetched as a raw image three times in three days.
- No more baked diff image. Changed pages finish sooner. The inspect view, the email and the summaries are unchanged.
- The share link fetched directly returns the current capture. Opened in a browser it still goes to the inspect view. Agents should read
changes, whose entries carry a box in capture pixels, and cropcurrentUrlwhen they want to look. - The Export button is gone from the inspect view. It opened the retired image.
v2.10.0 — Schedules in the Dashboard
Scheduled checks could only be set through the API or the MCP tools. The dashboard now has a Schedules tab that lists every project running unattended and lets you create or remove one without touching a config.
- Create from an existing project, or from a URL. Pick a project, or enter a name and site URL to create one and schedule it in a single step. Choose daily or weekly, an optional UTC hour, and the API key the runs bill to. A managed project is put on a rolling baseline when it is scheduled.
- The key is required. A schedule saved from a browser session used to record no key, and the runs it started skipped their summaries — the alert listed changed pages and said nothing about them. The dashboard now asks which key, and
PUT /project/{name}rejects a schedule set without one by a caller holding no key. A key named in the body must belong to your organisation. - Removing keeps everything else. Remove stops the unattended runs; baselines, history and the rolling policy stay as they are.
- Scheduled runs get a verdict. Every scheduled run used to say “Not judged: this run gave no intent”, and the help it linked to described
runContexton a triggered run — which a scheduled check cannot use. A project now carries a standingintent: what its pages are expected to do between checks and what must not change. Every scheduled run is judged against it; runs you start yourself are not. Set it on the Schedules tab, or withintentonupdate_project/PUT /project/{name}.
Schedules in the dashboard docs covers the tab; Scheduled checks covers the fields underneath it.
v2.9.4 — One Page Per Headline
A group card could quote one page and picture another. “Removed: ‘Citi Diamond Preferred Card’ — removed from /citi-strata-credit-card” showed the Strata rewards strip; the words had been removed from /credit-cards/. The email's subject, its “biggest change” line and its first bullet could likewise name three different changes of one run.
- The quote now owns the page. The change a headline quotes is chosen by whether it says something; the picture and the page name were chosen separately, by box size. They follow the quote now: the card's picture is the quoted change, cropped on the page it came from, and the caption names that page.
- Subject, lead and first bullet agree. The email's “The biggest change:” line leads with the page whose words the subject quotes.
- An edit is captioned with a page that carries it. The most common edit on a card of wording changes was captioned with whichever page ranked first, whether or not it contained that edit.
v2.9.1 — A Pinned Element Is a Change
Two engine changes about position: sticky and position: fixed, and they are one story. Baselines captured before v2.7.1 recorded a sticky element through the old rewriting pass, so it read relative on one side and sticky on the other. The engine carried a rule that dismissed exactly that pairing. The rule was papering over a bug, and it was also hiding a change worth seeing.
- Fixed: an invented header change. On a run whose baseline predated v2.7.1, one job flagged 32 pages when a single date had changed. The comparison manufactured a header region with no differing pixels in it, then described that empty region as a new header with a different logo and navigation links. Nothing about the header had changed. Runs no longer produce that region, and the descriptions that came with it are gone.
- A page that pins an element is now reported. Changing an element from
statictosticky— pinning a header that used to scroll away — moves no pixels while the page sits at the top, so a screenshot of the two states looks identical. The captured document is the only witness to it, and the engine had been declining to look. It now reports the change. Apositionchange betweenrelativeandsticky, or betweenabsoluteandfixed, is reported for the same reason. - What you will see. If a project's baseline was captured before September 5, 2026, the next managed run may report a one-time
positionchange on a sticky or fixed header, nav or banner. Approve it, and the run after that compares clean. This supersedes what v2.7.1 said — that such a change was classified as a reposition and in most cases flagged nothing. It is now reported. Baselines captured after that date are unaffected, as are live-vs-live runs, where both sides are captured the same way.
Unexpected sticky and fixed element diffs covers how to tell a one-time baseline artifact from a deliberate pinning change and from an element that renders badly on every run.
v2.9.0 — The Verdict, Where You Act
A run that carries an intent gets a verdict on every changed page. Until now that verdict was computed and then buried: a ten-pixel label in the inspect view, unrelated to the Approve and Reject buttons beside it, and a set of counts on the API. This release puts it where a person or an agent acts, without adding an endpoint.
- Inspect view. The verdict sits between the header and the capture: the decision, the confidence and its basis (hover the label for what measured and described mean), the words of the intent it cites, and one sentence of reasoning. The button the verdict recommends is outlined. Disagreeing with it asks, once and optionally, what the verdict missed. A run with no intent says “Not judged: this run gave no intent” and links to how to give one.
intentAssessment.decisionon the job summary:pass,review,failornot_judged. One field for a CI step, derived from the counts that were already there.verdictnow carriesreasoning,coveredBy(forintentional, the words of the intent quoted verbatim),basisandjudgedAt.avgConfidenceandregionsare deprecated: they are kept only so older clients keep parsing, and carry nothing you can use — do not read them.avgConfidencealways equalsminConfidence, andregionsis always empty.POST /approveandapprove_jobacceptwhere: { decision: […] }to approve only the pages whose verdict is in the list, and document the per-page form (url,variantName,action, optionalnote) that the dashboard has always used. Every approve or reject is recorded on the page astriageStatus,triagedAt,triagedByandtriagedVerdict, so a disagreement between a person and the verdict is a fact on the record rather than a memory.- Job status responses carry
intentProvided, so a client can tell “no verdict yet” from “no verdict, ever”. - Project history. Every project card on the Sites tab opens a page of that project's runs, newest first and dated, with the verdict or change count on each row, a “Changed only” toggle and a Load more. A run opened from it shows a back link at the top of the run, labelled with the project's name, that returns you to the history. The dots on the Sites tab now show the date on hover, and a quiet project links to its last change.
- Jobs tab. Filter the run list by project, status and “Changed only”, and load more than the fifty most recent. The bare
/admin/jobsshows the list rather than the latest run. - Usage tab. A “Changes over time” chart of changed pages per day by project, and Execution Trends ticks that show the date when the runs span more than a day.
/dashboardnow redirects to the dashboard. - Fixed: a run now reads Approved or Resolved once every page with a diff has a decision. It used to wait for every page, including the unchanged ones that have nothing to decide, so a 33-page run with one approved diff sat at “Needs review” beside “1/1 reviewed” and kept offering “Approve all”.
The MCP docs have a new section on giving a run an intent and reading what comes back. The verdict now appears on the inspect view, the run page, the Sites history and the Jobs list, and in alert emails.
v2.7.1 — Fixed Elements Left in Place
v2.5.0 added an automatic pass that rewrote position: fixed and position: sticky elements before each full-page screenshot so they would paint once, at rest. v2.7.1 removes that pass. A full-page capture taken from the top already paints them that way, so what you see in a capture does not change.
- Why. On React sites the pass sometimes ran before the page had finished loading, and React then re-rendered the page. The captured document ended up with a duplicate structured-data (JSON-LD) block, and the element the pass had just rewritten came back fixed. Scheduled checks reported that duplicate as a metadata change on pages nobody had touched.
- What you will see. Your existing baselines were captured with the old pass. Your next managed run compares them against pages captured in place. In most cases nothing is flagged: a header or nav that only moved is classified as a reposition, not a change. If a header, nav or cookie bar is flagged, it is a one-time diff. Approve it. Scheduled projects on a rolling baseline advance on their own; you may get one alert for the header or nav and nothing after. Live-vs-live runs are unaffected, since both sides are captured the same way.
customCssstill applies last and still hides or repositions any element you choose. Changing it invalidates the project's baselines, as before.- Runs that name a project now fill every field you omit from the stored project, not only when you omit all of them. The SDK builder always sends
testOrigin, soclient.test(url).forProject(name).run()used to check the home page alone, on default devices, with no masks; it now runs the saved paths, devices, masks andcustomCss. Fields you do send must still match what is stored.POST /project/{name}/runalready worked this way. This applies to managed runs only: a run that sendsbaseOrigin(live-vs-live) is neither checked against the project nor filled from it, so send every field you want. - Alert emails now read the same from the subject down. A moved block that lost its words is “lost”, not “rewrote”; a page whose only change is metadata is described by the fields that changed, never by a picture; a subject says which pages a clause is about (“2 pages changed, 1 of them edited its wording”); and a duplicated block reads “a second JSON-LD SoftwareApplication block”.
v2.7.0 — Scheduled Checks at a Set UTC Hour
A schedule used to say only how often. Now it can say when:
scheduleHourUtc. A whole number from0to23that pins adailyorweeklyproject to that UTC hour —3means 03:00 UTC. Without it the slot still anchors to whenever the first sweep picked the project up, which is unchanged behaviour for every existing schedule. The first run waits for the hour too. On the REST API, in project config, and throughupdate_project. See Scheduled Checks.- Hours, and UTC. The sweep that starts scheduled runs is itself hourly, so there is no minute setting rather than one that quietly rounds. RegressionBot has no timezone setting anywhere, so there is no local time to convert from — the hour is UTC and says so in its name.
- Rejected where it would mislead. An hour on an
hourlyschedule is an error, not a silent no-op: accepted quietly it reads as "runs once a day at that hour" to whoever set it. Clearing the schedule clears the hour with it.
One behaviour worth knowing: a pinned hour does not catch up. If a run cannot start at its hour, the project waits for the next one rather than running late — where an unpinned schedule would have been picked up by the following sweep.
v2.6.0 — Scheduled Checks & Alerting
A project can now watch itself, and tell you when it stops being able to:
- Scheduled checks.
schedule(hourly,daily,weekly) withbaselinePolicy: "rolling"runs a project unattended — catching changes no deploy caused, like a third-party script updating itself or an edit in your CMS. Available on the REST API, in project config, and through theupdate_projectMCP tool. See Scheduled Checks. - Alerts when a check cannot check. Owners were already emailed when a scheduled run found changes. They are now also emailed when a run stops outright, finishes without capturing a single page, or never starts at all — the cases where silence and a healthy site look identical.
- Baseline invalidation is scoped to what it should always have been scoped to. Updating a project used to discard every baseline regardless of what changed, so turning on a schedule cost a full re-prime. Only changes that alter what a capture looks like — origin, sitemap, paths, scans, devices, masks,
customCss, gate credentials — invalidate now. Schedule, baseline policy, concurrency and AI options leave baselines intact, and re-sending an identical value is not a change. list_projectsanswers the question people actually ask. It now returns schedule and baseline state per project, includingnextRunComparesAgainstBaseline— whether a run right now would detect regressions or capture baselines.
Also in this release: the dashboard's Review tab became Jobs at /admin/jobs, where any job id from an API response can be opened directly, and alert emails now link to the run rather than to the dashboard root.
v2.5.0 — Sticky Element Handling & Custom CSS
The un-sticking pass described here was removed in v2.7.1. The outcome is unchanged: fixed and sticky elements paint once at rest, and customCss still applies.
Cleaner full-page screenshots, plus a new escape hatch for dynamic content:
- Automatic sticky/fixed element handling. Before each full-page screenshot, RegressionBot now un-sticks
position: fixedandposition: stickyelements (headers, navs, cookie bars) so they render once at their at-rest position instead of floating over — or repeating down — the page. This is automatic; there is no flag to enable. customCss— inject arbitrary CSS into the page right before capture, e.g.#chat-widget { display: none !important; }to fully remove a dynamic widget (and reflow the layout, unlike masks, which only blank the region). Because it is applied last,customCsscan also override the automatic un-sticking for a specific element. Capped at 4096 characters.
customCss is available on the REST API (POST /run, POST /project/{name}/run), in project configuration (create / update), and via the run_regression_job and update_project MCP tools.
Heads up (historical): one-time re-baseline for existing projects
Because sticky/fixed handling is now automatic, the next run of an existing managed project re-captures elements that were previously recorded “stuck” in your baselines. Pages with sticky headers, navs, or cookie bars may show a one-time wave of diffs — simply approve the affected pages to update your baselines. If a specific element renders wrong under the new handling, use customCss to reposition or hide it. Live-vs-live comparisons are unaffected, since both sides are captured the same way.
v2.1.0 — Summary Status Lifecycles
Introduced formal summaryStatus lifecycles for visual regression results. This update splits the core test completion from the AI summary generation. Teams and AI agents can now download screenshots and view pixel diff metrics immediately upon test completion, and poll summaryStatus === 'COMPLETE' to retrieve visual summaries, eliminating empty or incomplete visual description states in slow AI summarization cycles.
v2.0.0 — AI Summaries & MCP
This release introduced two major customer-facing capabilities for faster review and approval workflows:
AI-Powered Regression Summaries
Every detected regression now includes a plain-English summary (regressionbotSummary) so teams can quickly understand what changed before deciding to approve or investigate.
Remote Streamable HTTP Model Context Protocol (MCP) Server
Connect RegressionBot directly to agentic IDE sessions. Assistants can now run comparisons, check job status, review summaries, and approve baselines directly from the editor or terminal.
v1.2.0 — Better Visual Scoring
Improved scoring to better reflect real visual change severity. The comparison engine now reduces rendering-noise false positives (like antialiasing shifts), making pass/fail review decisions more reliable.
v1.0.0 — Commercial Launch
Official production release of RegressionBot SaaS. Multi-tenant visual regression capture, sitemap-based parallel testing, dashboard baseline management, and API-key authentication are all live.