Running from CI

Check your app from a pipeline after every build or release. Every check shows up by name in the pipeline's own test tab, and the PDF report is kept with the build. Any CI that can run curl and jq works.

1. Create an API key

1. Go to Settings → API keys and press Create key.
2. Name it after where it runs, for example "GitHub Actions".
3. Copy the key. It starts with ba_ and is shown once.

An API key can start runs and read results, and nothing more: it cannot add members, delete anything or create more keys. Store it as a secret in your CI, never in the repository. See The REST API.

2. Copy the step from BitAssure

Under Settings → API keys, Run from your pipeline writes the step for you:

1. Choose the app.
2. Choose what to run:
   • Everything in the app: every journey on every platform.
   • One platform: every journey on it.
   • Chosen journeys: only the ones you tick.
3. Leave Email the team when the result changes ticked if you want to be told by email.
4. Choose your CI: GitHub Actions, GitLab CI, Azure Pipelines, Jenkins, or any shell.
5. Press Copy and paste the step into your pipeline.
6. Store the key as the secret the page names (BITASSURE_KEY, or the credential bitassure-key in Jenkins).

Every id is already filled in. With Everything or One platform, BitAssure decides which journeys to run each time the step runs. A journey you record next week is included, and one you archive is left out, with no change to the CI file.

What the step does

1. Starts a pipeline run. POST /api/pipeline-runs starts one run for each journey and answers at once. If nothing can start, the step fails and prints BitAssure's reason. For example: "This pipeline run costs 3 credits and this account has 1. Top up to run it."
2. Waits for it. It asks GET /api/pipeline-runs/{id} every 15 seconds until finished is true, for up to 90 minutes.
3. Fetches the results. GET /api/pipeline-runs/{id}/junit.xml is one results file with a test suite per journey. Your CI's test tab reads it.
4. Keeps the report. GET /api/pipeline-runs/{id}/report.pdf is one PDF. It opens with a page saying what ran and how each journey did, followed by each run's own report. The step saves it with the build: as an artifact in GitHub Actions and GitLab, and in the published artifacts in Azure and Jenkins.
5. Passes or fails the build. The step fails unless every journey passed. It also prints each journey's result with a link to its run.

Being told by email

When Email the team when the result changes is on, BitAssure mails the account's members:

• once when the pipeline starts failing, saying which checks did not hold
• once when it passes again

Each email has the PDF report attached. Nothing is sent while the result stays the same, so a broken build does not mean an email on every push. The comparison is with the previous pipeline run of the same thing (the same app, platform or set of journeys). Anyone can switch these emails off under Settings → Your account.

The request, for writing your own

curl -X POST https://api.bitassure.app/api/pipeline-runs \
  -H "Authorization: Bearer $BITASSURE_KEY" -H 'Content-Type: application/json' \
  -d '{"projectId":"…","notifyOnChange":true,"triggeredBy":"CI"}'

Name exactly one of projectId, platformId or journeyIds (a list, at most 100). The answer, and GET /api/pipeline-runs/{id} afterwards, carries:

• status: PENDING, RUNNING, PASSED, FAILED or ERROR
• finished
• runs: each with its journey, status and a link to its page
• skipped: journeys that could not be started and why, for example one with nothing recorded yet
• where to fetch junitUrl and reportUrl

The results file and the PDF answer 409 until every run has finished.

The results file

• Each journey's run is a test suite, named after the app, journey and platform.
• Each check is a test case. Its class is the scenario name, and its name is the check's sentence, such as "Total should read “$64.97”".
• A failed check carries its reason as the failure message.
• A run that stopped before the end adds one failing case saying where, because the checks after it never ran.
• An errored run is a single error, not a list of passes.
• A run property links to the run's page, with its video and PDF.

One run at a time

The older single-run calls still work: POST /api/journeys/{id}/runs for one journey, then GET /api/runs/{id} and GET /api/runs/{id}/junit.xml. A journey's id is the last part of its address in BitAssure.

Costs and limits

A pipeline's runs cost the same as any other run, and come out of the same prepaid credits, so a pipeline can never spend more than the account holds. BitAssure checks that the account can pay for every journey before starting any, so you never get a half-run pipeline that looks green. Runs count towards the account's runs at once (two, or four on Monthly credits). A pipeline with more journeys than that runs them in turn.