Regression testing

Automated regression testingKnow the next release did not break what already worked

Walk through the parts of your product that matter once, saying what should be true on each screen. BitAssure checks them again after every release and on a schedule, and tells you in your own words the moment one of them stops holding.

No card needed.

Checkout, release by releaseStorefront, website
Check2.092.102.112.122.132.14
Signing in shows "Welcome back"
The total reads $86.40
Pay now is enabled
The discount code takes 10% offFound $86.40 instead of $77.76

Worked in every release since 2.09. Stopped in 2.14, and the team heard that morning.

What it catches

The things that worked yesterday and quietly stopped

A regression is rarely the new feature. It is the sign-in, the checkout or the total that nobody touched and nobody checked. A journey recorded once keeps checking them, release after release.

  • The payment confirmation is shown

    Flows that used to work

    Signing in, paying, sending a form. Each is a journey recorded once and walked again after every change, the same way each time.

  • The balance reads $4,120.00

    Figures that should not move

    A price, a balance, a reference number. When a change upstream alters one, the check that says what it should read is the one that fails.

  • Confirm is enabled

    Buttons that stopped answering

    A control greyed out because something before it failed is the most common silent break. It is checked as enabled, at the moment it should be.

  • The order id is in the response

    Responses behind the screens

    For an API, the status, a field in the JSON and how long it took, call by call, so a backend change that breaks a client is caught at the source.

How it runs

Regression tests that run themselves after every change

Nobody has to remember to test again. The journeys run when a release goes out, every night, or both, and the team reads one answer.

  • From your pipeline

    Start a run from CI with an API key when a release is built, and read every check as JUnit in GitHub Actions or any CI, by name.

  • On a schedule

    Hourly, daily, every weekday or weekly, on every plan, so a break from a dependency or a config change is caught even with no release.

  • Told when it changes

    A scheduled run mails the team when a journey starts failing, and again when it passes, never every night it stays the same.

  • Since when

    Every journey shows its last runs as marks, oldest to newest, so a one-off blip, a flaky check and a break that started on Tuesday look different.

  • A video of the failure

    Each run is recorded with every check marked at the moment it was judged, so a regression is watched, not reproduced.

  • One way across platforms

    Websites, Android and iPhone apps and APIs are recorded and read the same way, so one release is checked everywhere it ships.

Where it stops

What it does not do, so you can plan around it

  • It checks what a customer would see and do. It does not replace the unit and integration tests your engineers write; the two sit side by side.
  • It does not look inside your database, read your logs or confirm that an email arrived.
  • Websites run in Chrome only, and it does not compare screenshots pixel by pixel.
  • When a screen is redesigned on purpose, the checks on it are changed to match: a check that held on to the old layout fails until it is put right.

Pay for the runs, not the seats

A website or API run costs one credit, $1.75 on pay as you go before tax, and a phone run is charged by the minute. Every account starts with 50 free credits and as many people as you like.