Which test suites exist, how to run them, and what CI requires before a merge to main.
| Reference | Value |
|---|---|
| Subject | Test suites, runners, CI workflows, required checks and test conventions; for anyone opening a PR |
| Verified source | apps/web/vite.config.ts, playwright.config.ts, .github/ and check-ssr-runtime.mjs on main at 9dc18495, with the Playwright config and .github/ re-checked at 7a71e081; CodeQL setup from the GitHub API, 2026-11-07; CI section re-checked on main at 6840b0fa after #684, #685, #687, #688 and #689; ruleset 15785309 read 2026-11-10; Observable boundaries re-checked on main at 0461e045 (section defaults, media, share copy and editor tests from #673, #677, #678, #691, #693), 2026-11-10 |
| Support state | Maintained test contracts checked against main 9dc18495. This documentation pass does not execute the suites; required checks are mutable repository settings |
| Suite | Location | Runner | Run |
|---|---|---|---|
| Web data | src/lib/{adapters,content,live,directus,schemas,server,utils,state,styles,og,projects}, src/tests |
Vitest project data, Node |
bun run --cwd apps/web test |
| Web DOM | src/lib/components, src/lib/motion, src/routes, src/tests/utils, *.svelte.test.ts |
Vitest project dom, happy-dom |
Same command |
| Site invariants | src/tests/: CSP, security and cache headers, sitemap and OG coverage, legal truth, blog translations, asset guards, payload budget, breakpoints |
Vitest data |
Same command |
| Browser E2E | apps/web/tests/**/*.spec.ts |
Playwright; projects phone, reduced-motion, desktop, tablet, light, dark from @yesid/config/playwright |
bun run --cwd apps/web test:e2e |
| CMS contracts | apps/cms/tests/, apps/cms/scripts/**/*.test.ts |
bun test, offline |
bun run --cwd apps/cms test |
| Shared contracts | packages/shared/src/**/*.test.ts |
bun test |
bun run --cwd packages/shared test |
| Fast local loop | Files changed against origin/main |
Vitest --changed plus the impacted Playwright specs at one phone and one desktop project (#689) |
bun run test:changed |
A test file's directory decides its Vitest project. A component test under a data path fails with document is not defined. In apps/web, a bare bun test runs Bun's native runner. The package test script runs Vitest.
| Mode | Set | Behavior |
|---|---|---|
| Local default | nothing | Exports the fixture, builds, previews on :4173, runs every project |
| Prebuilt | E2E_PREBUILT=1 |
Previews the existing build; CI uses this after replaying the cached build |
| External target | PLAYWRIGHT_BASE_URL |
Runs against a deployment; VERCEL_AUTOMATION_BYPASS_SECRET adds the protection-bypass headers for protected previews |
CI runs fullyParallel with two workers per shard and no retries. Local previews keep file-level parallelism. Waits use locators, URLs and expect.poll; never networkidle (the weather fetch and a clock keep the network busy) and never waitForTimeout. Shared waiters live in tests/_support/helpers.ts. Every Playwright project starts as a returning visitor with sections-choice set to expanded; specs that need fresh storage override it (playwright.config.ts). tests/accessibility-responsive.spec.ts runs 45 axe scans in each of the phone and tablet projects (15 per locale in EN, FR and ES) and is ignored by every other project; the desktop light and dark scans are unchanged.
flowchart LR
PR["PR to main"] --> CL["web.yml classify"]
PR --> PG["web.yml prerelease-guard"]
PR --> B["web.yml build<br/>one prebuilt site"]
PR --> CW["web.yml ci-work"]
CL & B --> E2E["web.yml e2e<br/>4 shards"]
E2E -. "on failure" .-> REP["merged HTML report"]
CL & PG & B & CW & E2E --> CI["web.yml ci"]
PR --> CT["cms.yml test"]
PR --> GL["secret-scan.yml gitleaks"]
PR --> CQ["CodeQL default setup"]
CI & CT & GL & CQ --> Q["merge queue<br/>same workflows on merge_group"]
Q --> MERGE["squash merge to main"]
MERGE --> LD["cms.yml live-diff<br/>read-only"]
| Workflow | Trigger | What runs | Required |
|---|---|---|---|
web.yml classify |
PR to main, merge queue (merge_group), manual |
Selects e2e unless the PR changes only the root README.md, AGENTS.md or GLOSSARY.md; unknown paths, empty diffs and errors select e2e. Runs the .github/scripts policy taken from the PR's base SHA, or the queue's base SHA; an unreadable queue base falls back to main's policy and runs everything |
Through ci |
web.yml prerelease-guard |
PR to main, merge queue (merge_group), manual |
design-prerelease-guard.mjs self-test, then the guard that rejects prerelease design URLs |
Through ci |
web.yml build |
PR to main, merge queue, manual |
One Turbo build of @repo/web with the fixture and PUBLIC_PLAUSIBLE_DOMAIN=yesid.dev, build contracts, payload budgets and CSP check; uploads the prebuilt site as artifact web-build-<sha> for every e2e shard (#687) |
Through ci |
web.yml ci-work |
PR to main, merge queue, manual |
CI-decision tests, Turbo check and test for @repo/web, brand gates, shared-package tests |
Through ci |
web.yml e2e |
After classify and build, alongside ci-work (#684); skipped only when classify finds a docs-only diff |
Four Playwright shards balanced by measured file duration (#685), two workers each, previewing the prebuilt site from build; Chromium cached by Playwright version (#688); blob reports kept 14 days, merged into HTML on failure |
Through ci |
web.yml ci |
Always, after classify, prerelease-guard, build, ci-work and e2e | Fails unless those jobs succeeded, or e2e was skipped by a docs-only classification. Runs the .github/scripts policy taken from the PR's base SHA |
Yes |
cms.yml test |
Every PR, merge queue entry and push to main |
Offline CMS contracts; no path filter so it always reports | Yes |
cms.yml live-diff |
Push to main, or a diff dispatch from main |
Read-only sync:diff against production |
No |
secret-scan.yml |
PR, merge queue, push to main, weekly, manual |
gitleaks over the commit range or full history | Yes |
| CodeQL | GitHub default setup; it does not report on merge_group |
Actions and JavaScript/TypeScript analysis | Yes, as the ruleset's code scanning rule |
dependabot.yml |
Monthly | Grouped GitHub Actions updates, which run every check; npm and Bun packages are updated by hand | n/a |
The main ruleset 15785309 (read 2026-11-10) requires a pull request, the GitHub Actions checks gitleaks, ci and test (strict up-to-date off), and CodeQL code scanning results (errors, and security alerts high or higher). Merges go through a merge queue: squash, ALLGREEN, up to 5 entries built and merged per group, no wait, 45-minute check timeout. Repository admins bypass always (gh pr merge --admin skips the queue). Org ruleset 26919834 also protects the default branch: no deletion, no force push, changes through a pull request. E2E is required through ci.
Measured 2026-11-10 on one PR head (run 42776443667, attempts 1 to 3, runner wait 4 to 17 s): PR decision p50 6m24s, p90 6m28s, against p50 11.0 and p90 13.6 min before. The slowest e2e test step took 254 to 271 s, so the 4-minute shard aim is not met. Under a throttled runner queue, wall time grows with the wait. A queued check is not a result; if jobs never start, check GitHub Actions billing limits.
Shared CI setup is .github/actions/setup: Bun from .bun-version, a Bun cache keyed by bun.lock, then a frozen install. Turbo remote caching is enabled when TURBO_TOKEN and TURBO_TEAM are present and falls back to the local cache otherwise.
| Topic | Rule |
|---|---|
| Test data | Factories in apps/web/src/tests/factories/ (interface-forge and faker), checked against the shared Zod schemas by factories.test.ts |
| Mocks | vi.mock is hoisted to the top of the file; keep mocks at module scope |
| Scope | Test observable behavior, not source text |
| Payload budgets | Root layout entry 30,000 bytes and root payload 180,000 bytes gzip; override with ROOT_LAYOUT_ENTRY_BUDGET_GZIP and ROOT_LAYOUT_PAYLOAD_BUDGET_GZIP |
| Generated content | Tests run against fixture-generated modules; content integrity tests own translation coverage |
Section tests cover first-visit expanded content with a Collapse all control and no stored choice, the bulk choice across pages and reload, and target reveal without changing the stored choice (tests/remembered-sections.spec.ts). These are maintained source contracts, not a fresh test execution in this documentation pass.
| Contract | Maintained verification |
|---|---|
| Hero hold / recovery | HeroBanner.polling.test.ts plus live-kpis.test.ts; automatic and manual failures share a budget, held data keeps ageing, DEMO does not reshuffle on failed polls |
| Reading / Back | detail-reading-position.svelte.test.ts, locale-handoff tests, tests/remembered-sections.spec.ts and tests/back-restore.spec.ts cover target-chain reveal, remembered bulk choice, locale progress and click-time history offset. tests/back-restore.spec.ts runs in the desktop project at one setting per flow, 1440x900 with motion no-preference: Back keeps the settled offset on /projects/yesid-dev and /services/web-development, and normal anchoring resumes after Back when a diagram above the reader grows. Its CPU-throttled departure test runs with reduced motion. Restore policy at other widths and with reduced motion is covered by reading-position-plan.test.ts, reading-session.test.ts, reading-session-history.test.ts, detail-reading.test.ts and locale-handoff-scroll.svelte.test.ts (Lenis and native scroll), not by visual checks at every viewport (#672, 04a2437d). The fast Back case stays in tests/remembered-sections.spec.ts. The outgoing-transition reload-guard case is explicitly fixme |
| Metric evidence contract | Shared schema and fetcher tests validate optional source/date data. Project/Service card tests, tests/impact-proof.spec.ts, tests/service-metric-layout.spec.ts and LLM text tests assert values and labels without source/date output |
| Publish graph | Offline CMS flow tests execute snapshot scripts with adapters and enforce per-column link uniqueness. They do not execute the live Directus handlers |
| Contact | Mocked provider success/error and blank-key tests; no automated real message send |
| Table blocks | packages/shared blocks.test.ts (payload validation, ragged rows, sanitizer strips scripts, handlers and unsafe URLs, semantic HTML), BlockRenderer.test.ts (named focusable region, header and headless tables) and the CMS blog-posts.test.ts export case |
| CMS images, layouts and share copy | CMS build-media.test.ts (orientation, unmirrored block and hero export, optional pairs, fixture placeholders) and build-media-policy.test.ts (Build Bot file grant); web BlockRenderer.test.ts, tests/image-layouts.spec.ts, project-images.test.ts and ProjectDetailPage.image.test.ts (rows, beside, theme pairs, gallery captures); og/project-copy.test.ts and card.test.ts (share copy and fit); CMS article-editor.test.ts (editor data handling). check-article-editor.mjs is a manual local browser check, not CI |
| SSR runtime | The web build script runs scripts/check-ssr-runtime.mjs with plain Node (--no-experimental-require-module) after vite build. It boots the emitted server, requires a missing path to return 404 with the site error page, and fails if any emitted chunk still imports or requires sanitize-html or its parser chain. It runs wherever build runs, including the ci-work job |
| Hosted acceptance | Read every shard on the exact PR head. Conditional skips and fixme cases are verification limits |