Newman alternative for CI: DevTools CLI

Name: DevTools
Author: DevTools

DevTools CLI runs YAML API test flows in CI with parallel execution, multiple report formats (JUnit/JSON), and clean exit codes. It replaces Newman by removing npm dependencies, storing tests in Git, and giving you production-real test runs that are easy to wire into PRs.

Quickstart (local)

Install, verify, then run a flow with both JUnit and JSON reports. See the CLI reference for all options.

$ curl -fsSL https://sh.dev.tools/install.sh | bash

$ devtools version

$ devtools flow run tests.yaml --report json:results.json --report junit:results.xml

GitHub Actions (real pipeline)

Runs on push and pull_request, passes secrets via env, and uploads artifacts even on failure. See templates for more variants: GitHub Actions and GitLab CI.

name: API tests (DevTools)
on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  api-tests:
    runs-on: ubuntu-latest
    env:
      # Inject CI secrets into the process env
      API_BASE_URL: ${{ secrets.API_BASE_URL }}
      SECRET_API_KEY: ${{ secrets.SECRET_API_KEY }}
    steps:
      - uses: actions/checkout@v4

      - name: Install DevTools CLI
        run: curl -fsSL https://sh.dev.tools/install.sh | bash

      - name: Run API tests (JUnit + JSON)
        run: devtools flow run tests.yaml --report junit:results.xml --report json:results.json

      - name: Upload JUnit results (always)
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: junit-results
          path: results.xml
          if-no-files-found: warn
          retention-days: 7

      - name: Upload JSON results (always)
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: json-results
          path: results.json
          if-no-files-found: warn
          retention-days: 7

Micro‑video: CI run passing + artifact

Passing secrets and base URLs

Reference CI environment variables using #env:. Keep secrets out of YAML. Full details in environments & variables and the CLI reference.

# tests.yaml (excerpt)

name: Smoke tests
env:
  API_KEY: '#env:SECRET_API_KEY'
  BASE_URL: '#env:API_BASE_URL'

steps:
  - name: Login
    request:
      url: '{{BASE_URL}}/login'
      method: POST
      headers:
        Authorization: 'Bearer {{API_KEY}}'

# GitHub Actions env injection

env:
  API_BASE_URL: ${{ secrets.API_BASE_URL }}
  SECRET_API_KEY: ${{ secrets.SECRET_API_KEY }}

Reports + exit codes

JUnit for CI test reporting. Add --report junit:results.xml and upload as an artifact.
JSON for programmatic parsing or dashboards. Add --report json:results.json.
Multiple reports at once are supported. Use both flags in a single run.
Exit codes: 0 = success, 1 = test failures, 2 = CLI error. CI should fail builds on non‑zero codes. See full list in the CLI reference.

Micro‑video: Local run producing JSON + JUnit

Migration path (Newman users)

If you have Postman collections, migrate/rebuild them as DevTools YAML flows. See the migration guide.
Keep flows in Git next to your code; review with PRs. See supporting pages Postman CLI alternative and Postman alternative.
Run in CI with DevTools CLI. Faster than Newman and no npm install required. Output JUnit/JSON and fail on proper exit codes.

Note: Newman is Node‑based and typically installed via npm, which adds dependency/setup overhead in CI.

Troubleshooting

Flow file not found

Check the path (e.g., tests.yaml) and working directory in CI.

Variable not resolved

Missing env var or typo. Define CI secrets and reference with #env: (example above).

Request timeout

Increase per‑request timeout in the flow or stabilize the upstream service.

Install permissions / custom install dir

Use the one‑liner installer; on restricted runners, set a custom bin path per the CLI reference.

Frequently Asked Questions

Is this a Newman replacement?

Yes. DevTools CLI replaces Newman in CI with YAML flows, multi‑format reports, and proper exit codes.

How do I get JUnit output?

Add --report junit:results.xml to your run. Upload the file as an artifact in CI.

How do I pass secrets in CI?

Map CI secrets to env vars (e.g., GitHub Secrets) and reference them in YAML with#env: (e.g., API_KEY: '#env:SECRET_API_KEY').

What exit code fails the build?

Non‑zero. 1 indicates test failures; 2 indicates a CLI error.

Can I output both JSON and JUnit?

Yes—use multiple --report flags in one run.

Ready to replace Newman?

Install the CLI, commit your flows, and wire it into CI. See templates for more examples.

Download DevTools CLI reference

Also see templates: GitHub Actions and GitLab CI.