Project Configurations

In RegressionBot, a project serves as both a baseline namespace and a saved execution configuration. Once created, a project locks in its testing parameters — preventing baseline drift and ensuring that every run compares against the same set of pages, devices, and masks.

Projects operate in managed mode (where screenshots are stored as baselines and compared on subsequent runs) by default. They work the same way across all interfaces: the REST API, the Node.js SDK, and MCP agentic tools. The first run against a new project name creates the project and saves its configuration. Repeat runs can omit all parameters and load the saved configuration automatically.

Project Configuration Locking

Configuration locking applies exclusively to managed mode runs (where no baseOrigin is supplied and screenshots are stored as baselines for comparison on future runs). In live-vs-live mode (where baseOrigin is present), parameters are always accepted since there are no stored baselines to protect.

First Run (Creation)

Executing a job for a new project name automatically creates the project entity. The parameters you provide (testOrigin, paths, devices, masks) are saved as the configuration for all future runs.

Subsequent Runs (Enforcement)

Subsequent runs against the same project name must either omit parameters (inheriting them from the stored configuration) or pass identical values. Providing differing parameters returns a 400 validation error.

Param Matching Rules

When verifying parameters, RegressionBot compares the following fields against the stored project configuration. Only fields you supply are checked — omitting a field is always allowed.

Field	Comparison
`testOrigin`	Exact string match
`devices`	Set equality (order-insensitive)
`paths`	Set equality on `path` values and label descriptions
`masks`	Set equality on CSS selector strings
`sitemapUrl`	Exact string match
`scans`	Set equality on `pattern` values
`concurrency`	Exact numeric match

Baseline Invalidation & Rollback

When you need to update a project's parameters (for example, adding a new page path or a new device), the existing baselines are no longer compatible with the new configuration.

To update a project's configuration via the REST API, use PUT /project/{name}. This updates the stored config and invalidates existing baselines:

curl -X PUT https://api.regressionbot.com/project/my-project \
  -H "x-api-key: $REGRESSIONBOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"testOrigin": "https://new-origin.com", "devices": ["Desktop Chrome"]}'

This triggers a baseline invalidation — because baselineVersion changes, the next run creates new baselines from scratch. All results on the next job will show isNewBaseline: true. Run the job a second time to begin detecting regressions.

What happens on invalidation:

No destructive deletes: Historical baseline screenshots are never deleted when a configuration changes.
Superseded in place: The next job run captures new screenshots and saves them as fresh baselines (isNewBaseline: true), bypassing any comparison against the old configuration.
New baselines on next run: After PUT /project/{name}, the next run re-baselines everything. Run it a second time to start detecting regressions against the updated baselines.

Sitemap Crawling & Discovery

Instead of manually specifying every single page path to test, you can configure RegressionBot to discover pages dynamically by crawling your environment's sitemap.

How it Works

When sitemap crawling is triggered, RegressionBot downloads the target sitemap XML (e.g., your staging sitemap), extracts only the pathnames from the URLs (e.g. https://staging.regressionbot.com/docs/ becomes /docs/), and then reconstructs the final target URL by prepending your current testOrigin (e.g. http://localhost:3000/docs/).

This decoupling allows you to supply a sitemap hosted on production (e.g., https://regressionbot.com/sitemap.xml) to crawl and test your local dev server running on http://localhost:3000.

Absolute URLs Only

The sitemapUrl parameter must always be a fully qualified, absolute URL. Relative paths (like /sitemap.xml) fail input validation and return a 400 Bad Request.

On-the-fly Baseline Seeding

When you deploy a new page to staging and it gets added to the sitemap, the next run automatically detects it. The system captures it and seeds a new baseline (isNewBaseline: true) for that specific path without invalidating the rest of the project's baselines.

Sitemap Defaults & Config Locking

Default Fallback: If sitemap scanning is active (either via `/crawl` or loaded from project config) but you omit sitemapUrl, it automatically falls back to {testOrigin}/sitemap.xml.
Managed Mode: The sitemapUrl is locked to prevent baseline drift. You cannot override it inline during a run; you must use PUT /project/{name} to update it (which resets the baselines).
Live-vs-Live Mode: Config locking is disabled. You are free to pass any sitemapUrl inline for each test run.

How to Configure

Projects can be configured and run across all interfaces. Use the selectors below to choose your environment:

1. Creating a Project (First Run)

Executing a job for a new project name automatically creates it. The parameters you provide (origin, paths, devices) are locked in as the configuration for all future runs.

import { RegressionBot } from '@regressionbot/sdk';

const client = new RegressionBot(process.env.REGRESSIONBOT_API_KEY!);

const job = await client
  .test('https://staging.myapp.com')
  .forProject('my-billing-app')
  .on(['Desktop Chrome', 'iPhone 12'])
  .check('/invoice')
  .check('/settings')
  .run();

2. Parameter-less Reruns

Once a project exists, you can trigger a run by providing only the project name. The stored configuration is loaded automatically.

import { RegressionBot } from '@regressionbot/sdk';

const client = new RegressionBot(process.env.REGRESSIONBOT_API_KEY!);

// Triggers a run against the saved project configuration
const job = await client.runProject('my-billing-app');

Troubleshooting

400 Bad RequestParameter mismatch

Why this happens: You triggered a job against an existing project but supplied parameters — such as testOrigin, devices, or paths — that differ from the saved config. The error message names the specific fields that conflict.

Solution: Either omit those parameters so the saved config is used automatically, or use POST /project/{name}/run with no body. To intentionally change the config (for example, to add new pages), use the MCP update_project tool or PUT /project/{name} via REST — note that updating the config means the next job will re-baseline everything.

404 Not FoundProject not found

Why this happens: You called GET /project/{name} or POST /project/{name} /run but no project with that name exists in your organisation.

Solution: Check the name for typos. Use GET /projects to list all projects in your org. To create a project, run a managed-mode job against the name via POST /crawl with the full parameter set.

400 Bad RequestMissing testOrigin on parameter-less run

Why this happens: You called POST /project/{name} /run with no body, but the project exists without a saved testOrigin. This can happen if the project was created before the config-locking feature was introduced.

Solution: Run a full job via POST /crawl with the project name and all required parameters. This saves the config to the project for future parameter-less runs.

isNewBaseline: trueAll results show isNewBaseline: true

Why this happens: This is expected on the first run for a new project — no comparison has been made yet, baselines were created. It also occurs after a PUT /project/{name} update, which resets baselineVersion and forces re-baselining.

Solution: Run the job a second time to detect regressions against the newly created baselines. Use POST /approve (or the approve_job MCP tool) after the first run if you want to lock in these screenshots as baselines immediately without a second run.