API Regression Testing Checklist: 30 Edge Cases Most Teams Miss

Regression failures almost never come from the “happy path” you recorded last week. They come from contract edge cases that only show up under load, partial outages, retries, or when old clients hit a newly deployed backend.

This checklist is aimed at teams doing YAML-based API regression testing in Git (reviewed in PRs, executed locally and in CI) where determinism matters more than UI clicks. If you are migrating from Postman/Newman or evaluating Bruno, the main advantage of keeping these cases in native YAML is simple: the diff is the review. No opaque collection JSON, no UI-locked state, no “someone changed a script” mystery.

If you want a reference baseline for assertions and Git-friendly structure, see:

• API Assertions in YAML: Status, JSON Paths, Headers, Timing Thresholds
• YAML API Test File Structure: Conventions for Readable Git Diffs

How to use this checklist (without creating flaky tests)

Treat each edge case below as a small, isolated flow:

• Prefer deterministic fixtures (seeded users, stable product IDs, known quotas) over random data.
• Make request chaining explicit (capture IDs and tokens, then assert on the next step).
• Avoid “assert full JSON equals” for large responses, assert shape + invariants instead.
• In CI, run edge-case flows with controlled concurrency, especially for rate-limit and idempotency checks.

For CI setup patterns, refer to the GitHub Actions guide (this article focuses on what to test, not pipeline wiring):

• API regression testing in GitHub Actions

The 30 edge cases most teams miss (categorized)

Each row is framed as: what usually breaks and what to assert in a JSON API regression test.

Auth (5)

These are the failures that only show up after deploys, key rotations, or account lifecycle changes.

Copy-paste YAML snippet: expired token, then re-auth and retry

This pattern avoids conditional logic by using a known expired token fixture (in CI, provide it via secret/env var).

# flows/auth_expired_token.yaml
# Purpose: prove the API rejects expired tokens deterministically, and that a fresh login restores access.

env:
  BASE_URL: '{{BASE_URL}}'
  EXPIRED_ACCESS_TOKEN: '{{EXPIRED_ACCESS_TOKEN}}'

flows:
  - name: ExpiredTokenFlow
    variables:
      - name: USERNAME
        value: '{{USERNAME}}'
      - name: PASSWORD
        value: '{{PASSWORD}}'
    steps:
      - request:
          name: RejectExpired
          method: GET
          url: '{{BASE_URL}}/v1/me'
          headers:
            Accept: application/json
            Authorization: Bearer {{EXPIRED_ACCESS_TOKEN}}

      - js:
          name: ValidateRejection
          code: |
            export default function(ctx) {
              const status = ctx.RejectExpired?.response?.status;
              if (status !== 401) throw new Error(`Expected 401, got ${status}`);
              return { rejected: true };
            }
          depends_on: RejectExpired

      - request:
          name: Login
          method: POST
          url: '{{BASE_URL}}/v1/login'
          headers:
            Accept: application/json
            Content-Type: application/json
          body:
            username: '{{USERNAME}}'
            password: '{{PASSWORD}}'
          depends_on: ValidateRejection

      - js:
          name: ValidateLogin
          code: |
            export default function(ctx) {
              const resp = ctx.Login?.response;
              if (resp?.status !== 200) throw new Error(`Expected 200, got ${resp?.status}`);
              if (!resp?.body?.access_token) throw new Error("access_token missing");
              return { validated: true };
            }
          depends_on: Login

      - request:
          name: AcceptFresh
          method: GET
          url: '{{BASE_URL}}/v1/me'
          headers:
            Accept: application/json
            Authorization: Bearer {{Login.response.body.access_token}}
          depends_on: ValidateLogin

      - js:
          name: ValidateFreshAccess
          code: |
            export default function(ctx) {
              const resp = ctx.AcceptFresh?.response;
              if (resp?.status !== 200) throw new Error(`Expected 200, got ${resp?.status}`);
              if (!resp?.body?.id) throw new Error("id missing from /me response");
              return { userId: resp.body.id };
            }
          depends_on: AcceptFresh

Notes for Git review:

• Keep the expired token fixture separate from “normal auth” flows so you do not accidentally depend on token timing.
• Make your error contract assertions explicit (status, content-type, and a minimal error shape).

Rate limiting (4)

The edge cases here are about honoring server guidance and keeping CI deterministic under parallelism.

Reference for semantics: RFC 6585 (429 Too Many Requests) and RFC 9110 (Retry-After header).

Practical tip: run these as a dedicated CI job with pinned concurrency so the test itself does not become the source of noise. DevTools’ Go runner speed makes it easy to generate bursts, but you still want controlled bursts.

If you need deeper CI patterns for retries and rate limits, see API Testing in GitHub Actions: Secrets, Auth, Retries, Rate Limits.

Pagination drift (4)

If your pagination tests only validate “page 1 returns 20 items”, you are not testing drift. Drift happens when the dataset changes between requests.

Request chaining tip: capture the first page’s IDs, then assert that the second page does not include them. This is an invariants-first approach that is stable in Git and stable in CI.

Null vs missing fields (4)

Teams often regress here during refactors because “nullable vs optional” is under-specified and tests are too strict or too loose.

Reference: RFC 8259 (The JSON Data Interchange Syntax).

Where this bites in Git review: Postman/Newman tests often do full-body equality or implicit JS truthiness checks. In YAML, prefer explicit JSONPath assertions for presence and type so reviewers can see exactly what changed.

Backward compatibility (4)

Backward compatibility is where over-specified tests become a liability. You want tests that catch real contract breaks, not tests that pin incidental fields.

This is one place where YAML in Git shines versus UI-driven tools: reviewers can see you changed “assert full body” into “assert required fields + invariants”, which is a compatibility decision.

Idempotency (4)

Idempotency is not just “use PUT”. It is “what happens when the network lies”, especially in CI and under retries.

Copy-paste YAML snippet: Idempotency-Key replay

This is a regression test you can run against staging without heavy fixtures. It uses request chaining to assert that the server returns the same id on replay.

# flows/idempotency_post_replay.yaml

env:
  BASE_URL: '{{BASE_URL}}'
  API_TOKEN: '{{API_TOKEN}}'

flows:
  - name: IdempotencyReplayFlow
    variables:
      - name: run_id
        value: '{{run_id}}'
    steps:
      - request:
          name: CreateOnce
          method: POST
          url: '{{BASE_URL}}/v1/orders'
          headers:
            Accept: application/json
            Authorization: Bearer {{API_TOKEN}}
            Content-Type: application/json
            Idempotency-Key: 'regression-{{run_id}}'
          body:
            currency: "USD"
            amount_cents: 1234

      - js:
          name: ValidateCreate
          code: |
            export default function(ctx) {
              const resp = ctx.CreateOnce?.response;
              if (resp?.status !== 201) throw new Error(`Expected 201, got ${resp?.status}`);
              if (!resp?.body?.id) throw new Error("id missing from create response");
              return { orderId: resp.body.id };
            }
          depends_on: CreateOnce

      - request:
          name: ReplaySameKey
          method: POST
          url: '{{BASE_URL}}/v1/orders'
          headers:
            Accept: application/json
            Authorization: Bearer {{API_TOKEN}}
            Content-Type: application/json
            Idempotency-Key: 'regression-{{run_id}}'
          body:
            currency: "USD"
            amount_cents: 1234
          depends_on: ValidateCreate

      - js:
          name: ValidateReplay
          code: |
            export default function(ctx) {
              const resp = ctx.ReplaySameKey?.response;
              if (resp?.status !== 200) throw new Error(`Expected 200, got ${resp?.status}`);
              const originalId = ctx.CreateOnce?.response?.body?.id;
              if (resp?.body?.id !== originalId) {
                throw new Error(`Replay returned different id: ${resp?.body?.id} vs ${originalId}`);
              }
              return { idempotent: true };
            }
          depends_on: ReplaySameKey

Git hygiene tip: keep the idempotency key generation deterministic per run (for example run_id), so parallel CI shards do not collide.

Partial failures (5)

Partial failure behavior is where “status code only” testing becomes meaningless. You need to validate the error envelope and per-item results.

Reference: RFC 4918 (207 Multi-Status).

Copy-paste YAML snippet: 207 Multi-Status batch response

This example asserts both the HTTP status and the per-item contract. Adjust JSONPaths to your payload shape.

# flows/batch_partial_failure_207.yaml

env:
  BASE_URL: '{{BASE_URL}}'
  API_TOKEN: '{{API_TOKEN}}'

flows:
  - name: BatchPartialFailureFlow
    steps:
      - request:
          name: BatchGet
          method: POST
          url: '{{BASE_URL}}/v1/widgets:batchGet'
          headers:
            Accept: application/json
            Authorization: Bearer {{API_TOKEN}}
            Content-Type: application/json
          body:
            ids:
              - "known-good-id"
              - "known-missing-id"

      - js:
          name: ValidateBatchResponse
          code: |
            export default function(ctx) {
              const resp = ctx.BatchGet?.response;
              if (resp?.status !== 207) throw new Error(`Expected 207, got ${resp?.status}`);
              const results = resp?.body?.results;
              if (!results) throw new Error("results missing");
              if (!results[0]?.status) throw new Error("results[0].status missing");
              if (!results[1]?.status) throw new Error("results[1].status missing");
              return { resultCount: results.length };
            }
          depends_on: BatchGet

Why YAML-first matters for these edge cases (and where Postman/Newman/Bruno often struggle)

These cases are not hard because they are conceptually advanced, they are hard because they require discipline:

• Deterministic diffs: In Postman, you tend to accumulate "test scripts" and implicit collection state. Reviewers rarely catch subtle logic changes. With YAML in Git, a change from checking status !== 200 to status !== 200 && status !== 207 in a JS validation node is explicit.
• Request chaining is visible: Node references like {{Login.response.body.token}} live next to the request that uses them, which makes drift bugs easier to diagnose in PRs.
• CI portability: Newman runs Postman’s model in CI, but you still carry Postman’s collection JSON and scripting conventions. Bruno is local-first, but still uses a custom file format. Native YAML flows are easier to audit, template, and enforce via repo policies.

If you are actively migrating, the most practical approach is: record traffic (HAR), generate a baseline flow, then add the edge-case flows above as small, reviewable YAML files. DevTools supports the HAR to YAML workflow directly (see Generate a HAR file in Chrome (Safely) and HAR to YAML: Generate API Regression Tests for CI).

The “PR gate” version of this checklist

When someone changes an endpoint, require them to answer, in the PR description, which of these categories they validated:

• Auth behavior (including failure modes)
• Rate limit behavior under concurrency
• Pagination stability under drift
• Null vs missing contract
• Backward compatibility for old clients
• Idempotency under retries
• Partial failure envelopes

That is the difference between "we have tests" and "we have regression coverage".

Many of these edge cases are most effective when tested as part of a multi-step workflow rather than in isolation. For a guide on chaining these checks into full end-to-end API test flows, see: End-to-End API Testing: The Complete Guide.