View demo evidenceStart free trial Español

Your tests stay Playwright, Selenium and pytest. We don't replace them. We make them enterprise-ready, and you own everything: code, evidence and history.

Why do my tests fail when the application is fine?

An end-to-end test goes red when the network is slow, the DNS fails, the site answers with a 403 or a 429 or shows a bot challenge, or the browser dies, and none of that is a defect in your application. Nexus Studio, from Verdict System LLC, gives these failures their own verdicts: TIMEOUT, UNREACHABLE, BLOCKED and CRASHED. None of them is reported as a bug in the application.

By the Nexus Studio team, Verdict System LLC. October 7, 2026.

What makes a test fail without a bug?

A test only reports that something did not happen. The cause can sit in four places that have nothing to do with your code:

  • the page was slow and the test ran out of time;
  • the host could not be reached, for example because the name did not resolve;
  • the site defended itself with a 403, a 429, a CAPTCHA or a bot challenge, so the request never reached your product;
  • the browser or the process died.

When every one of these reads as a plain red, the team files them as bugs, or learns to rerun reds instead of reading them. Either way the real defect hides in the noise.

How does Nexus Studio tell a false failure from a bug?

When a test fails, the engine reads the error message and the network log of the run and tests them against groups of patterns in a fixed order. The first group that matches decides the verdict. The order is CONFIG_ERROR, then CRASHED, BLOCKED, UNREACHABLE and TIMEOUT, and only then BREACHED_FLAKY or BREACHED. So BREACHED, the verdict that means a bug report, is left for a failure that no environment or harness pattern explains. The patterns cover both Playwright and Selenium WebDriver wording.

What do TIMEOUT, UNREACHABLE, BLOCKED and CRASHED mean?

  • TIMEOUT: the test never got an answer in time. That is not proof the application is broken, and it is not proof it works: the run cannot tell. A navigation that ran out of time is a TIMEOUT, because it started and the page was slow.
  • UNREACHABLE: the host could not be reached at all. DNS or the connection failed before any page was observed, as with a name that did not resolve, a refused connection or an offline machine. A certificate error is not here: that one is a finding.
  • BLOCKED: the site defended itself with a 403, a 429, a block page or a bot challenge. When a defense was on screen, metadata.json names it. A real Android phone that fell asleep or is locked is BLOCKED too, as an unmet prerequisite.
  • CRASHED: the browser or the process died, for example a closed page, a DevTools protocol error or a dead WebDriver session. It is infrastructure: look at the machine and the driver.

The verdicts guide lists the exact error patterns behind each one.

What does my CI pipeline see?

Seven of the 14 verdicts exit 0: SECURE, AUTO_RECOVER, FLAKY, CHAOS_RESILIENT, PERFORMANCE_DEGRADED, A11Y_VIOLATION and SECURITY_VIOLATION. The other seven exit 1, and TIMEOUT, UNREACHABLE, BLOCKED and CRASHED are among them. So the build still goes red, because the test did not pass, but the report and History say which cause it was. The pytest and Selenium runners exit 2 when the run could not start at all.

What evidence does each false failure keep?

Each of the four keeps report.md, metadata.json, raw-data.json and a screenshot of the final frame. Video differs:

  • TIMEOUT and BLOCKED: video is off by default. Turn it on in Settings, under Keep failure video for.
  • UNREACHABLE: no video is offered, because no page loaded and the recording would be blank.
  • CRASHED: no video is offered, because the recording of a dead browser comes out broken.

Only Playwright web and emulated mobile runs record video. A real Android phone, a Selenium run and an API test do not.

With Jira switched on, a TIMEOUT files a ticket, and a block, a crash and an unreachable site never do.

What are the limits?

  • The classification reads error text. A message you wrote yourself that contains 403, Forbidden or Access Denied reads as BLOCKED. Keep custom assertion messages specific, and open metadata.json when a verdict surprises you.
  • A TIMEOUT is not proof that the application works. A test that times out on every run deserves a look.
  • A longer timeout does not hide a broken element: it fails later.
  • These verdicts name the cause of a failed run. They do not remove the cause: a blocked site is still blocked until you change how you reach it.

Where can I see it?

The Evidence page follows one failed test and lists every file each verdict leaves on your disk. The verdicts guide explains in what order the engine decides all 14. The 30-day trial runs the same engine on your own tests.