APIs and load testing

An API has no screen to walk through, so it is written rather than recorded: the address once, then each call and what its answer should be. The result reads the same as any other platform's, as a list of sentences that held or did not, in the same report.

Setting it up

Create an API platform with its Base URL, for example https://api.example.com. Each call in a flow gives only its path, such as /orders, and the base is added when it runs.

If the API needs credentials, set them on the platform under How this API expects to be told: a bearer token, a header such as X-API-Key, or a username and password. They are stored encrypted, sent only with this platform's calls, and never shown again after saving.

Flows

A journey on an API platform is called a flow: an ordered list of calls. Each call has:

• a method and a path, such as POST /login
• optional headers and a body (JSON)
• checks on its answer
• optional values to carry forward

Try it sends one call on its own and shows the answer, so you can write checks against what really comes back.

Checks on an answer

Check	Sentence
Status is	Create order should answer 201
Succeeded (any 2xx)	Create order should succeed
Answered within	Create order should answer within 800 ms
Body contains	Create order should return “confirmed”
JSON value equals	Create order should return “paid” at $.status
JSON value exists	Create order should return something at $.id
Header equals	Create order should return “no-store” in the Cache-Control header

JSON values are found with JSONPath, such as $.order.id or $.items[0].price.

Carrying values between calls

Carry a value forward takes a value out of one answer and names it, for example $.token as token. Later calls use it as {{token}} in their path, headers or body:

POST /login          → carry $.token as token
GET  /orders         header  Authorization: Bearer {{token}}
GET  /orders/{{orderId}}

Cookies a call receives are sent with the calls after it in the same pass, as a browser would.

Generated values

Some values should be different every time, such as a new customer's email. These placeholders are filled in once per pass and then remembered, so the same placeholder written twice is the same value within a pass:

Placeholder	Becomes
{{uuid}}	a random UUID
{{timestamp}}	the time in milliseconds
{{isoTimestamp}}	the time as ISO 8601
{{randomInt(1,100)}}	a whole number in that range
{{randomString(8)}}	random letters and digits of that length
{{virtualUser}}	which virtual user this is, in a load test
{{iteration}}	which pass this is, in a load test

So "register as user-{{uuid}}@example.com, then sign in as user-{{uuid}}@example.com" works, and the next pass is a different person. A placeholder that has no value is left as written, so the call that fails shows why in its own path.

Load tests

A flow can also be run by many virtual users at once. Add a load profile to it:

• People at once: how many virtual users
• They arrive over (seconds): the ramp-up, so they do not all arrive in the same instant
• Then it runs for (seconds): how long the full population keeps going
• Each person pauses between passes (ms)

A load run reports, for every call and for the flow as a whole: how many calls were made, the error rate, median, 95th and 99th percentile response times, time to first byte, and throughput, second by second next to how many users were active. It also lists every distinct answer each call got, such as 200, 429 or "connection refused", with a count, so "2% failed" comes with what they failed with. You can watch it live and stop it early, keeping everything measured so far.

An answer counts as an error only when nothing came back or the server answered 5xx. A 4xx is the server answering, and whether that was right is what the call's checks say.

Thresholds

Load checks pass or fail like any other check:

Check	Sentence
95th percentile under	95% of calls to Create order should answer within 400 ms
Error rate under	Fewer than 1% of calls to Create order should fail
Throughput at least	Create order should sustain at least 50 calls a second
Flow 95th percentile under	95% of passes through this flow should finish within 2000 ms
Flow error rate under	Fewer than 1% of passes through this flow should fail
Flow throughput at least	This flow should complete at least 10 passes a second

On an ordinary (single pass) run, load checks are shown as not evaluated rather than failed.

Limits on load tests

• A load test may only call its platform's own Base URL. A flow that names another address is refused under load.
• The number of virtual users and the length of a load test are capped per account. One load test runs at a time per platform.
• A load test costs 1 credit per 50 virtual-user-minutes, counted at the full population. For example, 100 users for 10 minutes is 20 credits.