10 · CI and scheduling

Time: 20 minutes · You need: chapter 9 finished.

Processes: recipes with names

Nobody should have to remember which tags, layers and browsers make up "the check we run on a pull request". A process names it once:

sdods processes list -p rwa-bank

processes:
  - name: pr-check
    trigger: pr
    tags: '@smoke'
    browsers: [chromium]
    gates: { minPassRate: 100 }
  - name: release-gate
    trigger: release
    tags: '@smoke or @regression'
    gates: { minPassRate: 100, maxFlaky: 0 }

A gate is the difference between a report and a decision: a run that misses minPassRate fails the process, whatever the individual scenarios say.

Sharding

Split the suite across machines and merge the reports afterwards:

sdods run -p rwa-bank -e local -t @regression --shard 1/3
sdods run -p rwa-bank -e local -t @regression --shard 2/3
sdods run -p rwa-bank -e local -t @regression --shard 3/3
sdods report merge .sdods/runs/<id-1> .sdods/runs/<id-2> .sdods/runs/<id-3>

report merge produces one HTML report and one JUnit file, so CI shows a single result rather than three partial ones.

In GitHub Actions

name: tests
on: [pull_request]
jobs:
  smoke:
    runs-on: ubuntu-latest
    strategy:
      matrix: { shard: [1, 2, 3] }
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - run: bun install --frozen-lockfile
      - run: bunx playwright install --with-deps chromium
      - run: bun run sdods run -p rwa-bank -e ci -t @smoke --shard ${{ matrix.shard }}/3
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: run-${{ matrix.shard }}, path: .sdods/runs }

Two things make this survive contact with a real repository:

• Exit codes. 0 passed, 1 a scenario failed, 2 the configuration is wrong. A 2 is your CI file's problem, not a broken test.
• HAR replay. --har-replay --strict runs the suite with no application at all, which is how you keep a merge gate from depending on somebody's staging environment being up (chapter 6).

Results can flow back into the shared database and onto the pull request:

sdods report ingest --run-id <id> --server "$SDODS_SERVER_URL" --token "$SDODS_TOKEN"
sdods integrations github notify --run <id>
sdods integrations jira sync --run <id>

Scheduling

The server runs processes on a cron schedule, with overlap and catch-up policies so a slow run does not pile up behind itself:

sdods schedule list
sdods schedule add nightly --cron "0 2 * * *" --process nightly-regression --timezone Europe/London
sdods serve --port 8080        # the scheduler runs inside the server

The web UI shows the same schedules, their last runs and their next fire time.

You have finished

Across ten chapters this suite went from an unknown repository to:

• a project generated by analysis and corrected by hand,
• scenarios at three layers, with page objects and healed locators,
• data, a user pool and captured login state,
• a recording converted into a reviewed proposal, and a HAR that runs offline,
• screenshot narratives and a visual baseline,
• a run database with flakiness insights,
• agents that propose changes — optionally on a model running on your own laptop,
• and processes, shards and a schedule to run all of it without you.

Point the same chapters at your own application by changing one path in chapter 1.

Where to go next

• Best practices — the habits that keep a suite alive
• Reference → CLI — every command and flag
• Known limitations — what is not built yet, in writing