Writing tests
Unit-test conventions, the zero-skip policy and the conditional registration that replaces it, premise tests, and the opt-in suites.
Tests live beside the code they cover, in each package's tests/ directory. Every suite in the default gate reports zero skipped, and a guard keeps it that way.
Conventions
- Write a test for every new function, bug fix, and untested code you touch - each one must catch a real regression.
- Assert behaviour and outcomes, never implementation details.
- Build fixtures with typed factory helpers using
Partial<T>from a package's exported types; never re-declare types by hand. - Put temp paths under
os.tmpdir(); never hard-code/tmp. - Build a case's premise instead of assuming it: a file that must exist, an env var, a configured host. Prefer a synthetic fixture over your own tree, and register conditionally when the premise genuinely cannot exist.
No skips - register conditionally
it.skip, it.skipIf, describe.skipIf, runIf, fixme, todo and testInfo.skip are banned in both runners. A test that does not apply to your build is not registered, so it never reaches the summary at all.
// Wrong - the summary reports a skip, and every reader has to chase what it meant.
describe.skipIf(config.payment.provider !== "stripe")("stripe webhooks", () => { /* … */ });
// Right - the block exists only in a build that has the feature.
if (config.payment.provider === "stripe") {
describe("stripe webhooks", () => { /* … */ });
}Keep the inverse case wherever one exists, so the off-build is covered too. Nest both arms inside one describe - the shared ESLint preset's test/no-identical-title rejects two top-level describes with the same name:
describe("organizations", () => {
if (config.tenancy.multiTenant) {
it("invites a member", () => { /* … */ });
} else {
it("ships no organization surface", () => { /* … */ });
}
});apps/web and apps/desktop are not on that preset, so two describes with inverse names are fine there.
Premise tests
A file whose every block is conditional registers one test in the other branch, asserting the config fact the file keys on. Both runners fail a file that registers nothing.
} else {
it("this build ships no stripe provider", () => {
expect(config.payment.provider).not.toBe("stripe");
});
}The guard
packages/e2e-support/tests/no-skips.test.ts parses every test file with the TypeScript compiler API and fails on any skip form: a call, a bare describe.skip passed around as a value, or a skip(...) destructured from the test context. It runs inside pnpm test, so it covers your tests as well as the shipped ones.
Opt-in suites
| Suite | How to run it | What gates it |
|---|---|---|
| Live vendors | pnpm test:live | OPENROUTER_API_KEY, FIRECRAWL_API_KEY, PARALLEL_API_KEY, TINYFISH_API_KEY |
| Live end-to-end | pnpm e2e:live | the keys above, and/or a coding CLI that can complete a turn |
| Adversarial agent suite | GENERATESAAS_ADVERSARIAL=1 pnpm --filter @repo/agent-core test | the flag, plus an installed CLI per case |
| Dev-server suites | GENERATESAAS_DEV_SERVER_URL=http://127.0.0.1:3000 pnpm --filter web test:dev-server | a dev server you started |
Each prints one not run: <item> — <why> line instead of a skip, and exits 0 when it finds nothing to run. An opt-in gate that finds nothing is a report, not a failure.
The adversarial suite drives real coding CLIs under your signed-in subscription and bills it. Print the line from process.stdout.write when you add one: Vitest's default reporter swallows console.info from a worker file, and only a globalSetup runs in the main process.