New from the DevTools team: Stresseur, the AI test engineer for APIs →
DevTools
Back to Blog
Abstract illustration of a sheet unfolding into a row of connected squares

Turn an OpenAPI Spec Into API Tests for CI

Mustafa BayramogluMustafa Bayramoglu

Short answer: import the spec into DevTools Studio, which turns each operation into a request step in one flow, then add the checks and the chaining the spec cannot supply, export the flow as YAML and run it with the CLI in CI. The only assertion the import writes is a status check taken from an operation's first 2xx response, so you supply the body assertions and the order in which calls depend on each other.

This guide walks through those four steps, then explains where a spec-driven contract check ends and a multi-step flow test begins.

What an OpenAPI spec gives a test, and what it cannot

A spec already describes the surface of an API: its paths, the methods on each path, parameters, request bodies and the response codes each operation can return. That is enough to build a request for every operation, and the guides that rank for this question lean toward contract testing: Speakeasy's guide is titled "Contract testing with OpenAPI", and Apideck's is a complete guide to OpenAPI testing that covers contract and fuzz testing.

If that is the job, dedicated tools do it directly. Schemathesis generates test cases from your OpenAPI definitions with property-based testing, and Dredd walks every defined endpoint, sends a real request and fails when the status code or body deviates from the spec. Those tools answer one question well: does the live API still match its description?

A spec cannot answer the questions a real workflow raises. It does not say that an order must be created before it can be fetched, that a token from a login call goes into the next request, or that deleting a record must come last. State, authentication flows and ordering live in your head and in your test code, not in the file. The rest of this guide covers that gap: use the spec to generate the requests, then build the flow around them with DevTools, an open-source API testing tool that chains multi-step requests into reusable YAML.

Step 1: Import the spec into Studio

Open your workspace in Studio and choose Import. The import dialog takes Swagger and OpenAPI files, or a pasted URL. The converter reads both Swagger 2.0 and OpenAPI 3.x, in JSON or YAML. The feature has shipped in Studio releases since February 2026.

Two limits are worth knowing before you start. First, a spec URL that resolves to a private or loopback network range is rejected, so a spec served from localhost cannot be imported by URL. Save the file and import it from disk instead. Second, the import is a Studio feature: the CLI's import command has subcommands for curl, Postman and HAR, and none for OpenAPI. If you want the same route for browser traffic, the guide on HAR capture to CI gate covers it.

Step 2: Review what the import generated

The import creates a single flow with one request step per operation, linked in order, with paths sorted first and methods sorted within each path. That order is alphabetical, not logical, so expect to rearrange it. Read the result as a list of requests ready to run, not as a finished test.

The converter fills in what the spec makes available:

  • Base URL. Each request URL starts from the spec's first server URL, or from the scheme, host and base path in a Swagger 2.0 file, followed by the operation's path. Replace that prefix with {{BASE_URL}} so one file runs against staging and local.
  • Path parameters. A parameter with an example in the spec gets that value. Without one, the URL carries a {{name}} variable.
  • Query parameters. Only the ones the spec marks as required are switched on.
  • Bodies. The request body comes from the spec's example. When there is none, the importer generates a placeholder from the schema's properties.

The only assertion it writes is a status check taken from the first 2xx response the operation declares, such as response.status == 200. Nothing checks the response body, headers or timing. Add those yourself on the steps that matter, using API assertions in YAML as the reference.

One limitation can change the output. The schema parser does not resolve $ref references, and a code comment in the converter says imported schemas that use them can have missing data. The same comment notes that real-world specs use $ref extensively. Open each POST and PUT step and check that the body is what the API expects. Where it is empty or wrong, paste a valid example.

Step 3: Chain the requests into a flow

A flow test earns its value when later steps use what earlier steps returned. Pick the sequence that matters, for example create an order, fetch it, then delete it, and replace the placeholder path variables with references to earlier responses. A step reads an earlier step's output with the form {{Login.response.body.token}}.

Rename the imported steps to short names first, because the name becomes part of every reference. Then wire the second step to the first:

- request:
    name: CreateOrder
    method: POST
    url: '{{BASE_URL}}/orders'
    headers:
      Content-Type: application/json
    body:
      sku: 'demo-sku'
      quantity: 1
    assertions:
      - response.status == 201
      - response.body.id != null

- request:
    name: GetOrder
    method: GET
    url: '{{BASE_URL}}/orders/{{CreateOrder.response.body.id}}'
    depends_on: CreateOrder
    assertions:
      - response.status == 200
      - response.body.id != null

The depends_on line is not optional style. Import rejects a step that reads another step's output without depending on that step, and the error names the step and the missing depends_on. For longer sequences, the API chain testing guide goes deeper.

Step 4: Export the YAML and run it in CI

To get the file into Git, right-click the flow in Studio and choose Export, then YAML. Commit the file next to your code so a change to the API and a change to its tests land in the same pull request.

The CLI runs the file with devtools flow run <yamlflow-file> [flow-name]. It exits with 0 when all flows succeed, 1 when one or more fail and 2 on a configuration or setup error, so CI can gate on the exit code. The documented GitHub Actions workflow installs the CLI with the install script, runs the file with --report junit:test-results.xml and publishes the result with a JUnit reporter:

- name: Install DevTools CLI
  run: curl -fsSL https://sh.dev.tools/install.sh | bash

- name: Run API Tests
  env:
    BASE_URL: https://api-staging.example.com
  run: |
    devtools flow run api-tests.yaml \
      --report junit:test-results.xml

- name: Publish JUnit Report
  uses: dorny/test-reporter@v1
  if: always()
  with:
    name: API Test Results
    path: test-results.xml
    reporter: java-junit

The full workflow is in API testing in GitHub Actions, and the JUnit reports guide shows how failures appear on the pull request.

Where contract testing stops and flow tests start

A contract check compares responses with the spec, and it fails when the API drifts from what the spec promises. A flow test checks behavior: whether the token from login works on the next call, and whether the record you created is the one you fetched. They fail for different reasons, so the two are complements. The posts on contract testing versus API testing and contract testing versus end-to-end API testing lay out when each earns its place.

A practical split: let a spec-driven tool such as Schemathesis or Dredd check responses against the spec, and keep a small set of imported, chained flows for the journeys whose failure would block a release. Start with the two or three that matter most, and use the REST API test cases checklist to decide what to add next.

Common questions

Does the DevTools CLI import an OpenAPI file?

No. The CLI's import command covers curl, Postman and HAR. Import the spec in Studio, export the flow as YAML, and run that YAML with the CLI.

Can I import a spec from a localhost URL?

Not by URL. The importer rejects hosts that resolve to private or loopback ranges. Save the spec to a file and import the file.

Does the import resolve $ref schemas?

Not fully. A comment in the converter says it does not resolve $ref references and that imported schemas using them can have missing data. Check request bodies after the import.

Does the import generate assertions on response bodies?

No. The generated assertion is a status check from the first 2xx response. Body, header and timing checks are yours to add.

Conclusion

A spec gives you the requests and a status check for each operation. It does not give you order, state or authentication. Import the spec, fix the bodies, chain the two or three calls that matter and let CI run the YAML on every pull request. Add a spec-driven contract tool alongside if you also want the response shapes guarded.