Testing

pnpm run test        # unit tests, no stack, a few seconds
pnpm run test:e2e    # 1 end-to-end test, needs the stack
pnpm run check       # audit, format, env, typecheck, unit tests, CI parity, docs site

pretest:e2e runs doctor with --e2e first and names what is missing — the 14 checks of the quickstart’s “Check it”, plus capture-fixtures and the crawl permission for e2e, the name the test uses (fga:grant submitter e2e acme): 16 in all. It checks not just that things are up but that a crawl can run to the end — miss any of these and a crawl fails only at its level report, staying running. It lives outside vitest deliberately: vitest prints “No test files found, exiting with code 1” whenever a global setup throws, and no message written inside can survive that.

Why both layers

The defects this repository actually produced fall into two groups that do not overlap:

defect	unit	e2e
respect_robots arrived null, robots never read	○	○
level-boundary gap collapsed 3000 ms → 521 ms	○	△
content-type on a bodyless POST → 400	○	○
hostParallelism in camelCase arrived null	✗	○
schema defaults not applied on webhook runs	✗	○

The bottom two are invisible to unit tests by definition: arguments are passed by flow.yaml, not by TypeScript. Pick one layer and half of the holes stay open.

The unit tests need nothing

windmill-client is already a dev dependency and plan_level.ts imports only robots-parser, so the scripts import into vitest directly. IO is stubbed: vi.stubGlobal("fetch", …) and vi.mock("windmill-client", …).

captureHost is exported from crawl_host.ts so the pacing loop can be driven with a fake client — call() only does client[method](req, cb), so the fake is a plain object. Its retry argument — how long to wait when every BrowserHive is busy — exists so tests run on real timers at millisecond scale; fake timers were tried against capture-style sleeps elsewhere in this workspace and cost a round trip.

The end-to-end tests start real crawls

Each test starts a crawl through capture-ledger’s API — not Windmill’s run endpoint — because the point is to carry the arguments capture-ledger actually sends: crawl_id, depth, frontier, per_host_delay_ms, host_parallelism, capture_formats, signing, scripts, and artifact_sink where capture-ledger serves the sink. respect_robots is not among them.

The first one asserts four things:

1. capture-fixtures never received /links/hidden, which its robots.txt disallows. The verdict comes from the other end’s request log, because the ledger can only say what it recorded.
2. the crawl succeeded and captured at least one page
3. GET /api/search?q=hub returns a hit — so the indexing step completed inside the flow
4. if capture-ledger serves the sink — an unauthenticated PUT /api/sink/… answers 401 rather than 404 — every archive of the crawl has an objectKey under org/acme/<YYYY-MM>/. A flat key means the sink was never handed out and BrowserHive wrote into its own bucket. That is what happened from capture-ledger #226 until it was fixed, and such a crawl passes the first three.

Breaking ?? true in plan_level.ts and deploying turns both the unit test and the e2e red, and the e2e shows capture-fixtures receiving the forbidden page. That pair was checked once on purpose: if only one layer goes red, the other is watching something it thinks it is watching and is not.

What runs inside the page

The rest look at the scripts a crawl runs in the page, again from capture-fixtures’ request log, which counts every request by its full URL, query string included.

• Registered scripts run, and only when asked. Named, autofetch requests /assets/hero-2x.svg, which a 1280px browser at DPR 1 never asks for itself. Two controls go to the same page: one sends scripts: [], the other no scripts, leaving the choice to the ledger’s defaults, which leave out autofetch (from capture-scripts v0.5.0). Nobody requests it in either.
• A script that does not type-check fails the level at compile, with the number of diagnostics and the first five in the crawl’s error: once through the ledger (its CLI adds and removes a broken script), once as an inline script.
• An inline script runs in the order it was asked for. It fetches /plain-html?inline=<token>&state=<document.readyState>, which leaves a mark in the log: loading when it ran before navigation, complete after load.
• The same inline script asked again as preload runs before navigation. Both crawls send the same source, so the second one hits ts-compile-service’s cache. A cache that remembered whole answers handed back the first crawl’s behavior, and the mark said complete; that is how ts-compile-service v0.2.2 and earlier behave, and this test is red against them.
• However many type errors a script has, the crawl closes as failed and blocks nothing. The inline script raises hundreds of diagnostics. The test checks that error names the count and stays within the ledger’s 2,000 characters, and that the next crawl is accepted. Up to v0.24.2 the reason carried every diagnostic, the ledger refused it, and the crawl stayed running.

The nightly loop refuses to guess

trigger_crawl.ts is the only thing that decides whether the nightly capture succeeded, and the e2e does not touch it — that test posts its own seed to capture-ledger, so the trigger script is never in the picture. It used to have no coverage at all, because POLL_INTERVAL_MS was a 15-second constant and the sleep happens before the first poll — one test, fifteen seconds.

It now takes poll_interval_ms as an argument — the same trick as crawl_host’s busy wait — so the loop runs on real timers at millisecond scale.

The guard worth knowing about is this one:

if (crawl.state !== "succeeded") throw new Error(`知らない状態「…」`);

Without it, a state the script does not recognise falls through both comparisons and returns succeeded with counts of zero. That is exactly what a renamed field looks like — and it happened once, for real: capture-ledger’s runs.status became runs.state, and the pre-rename script was pointed at the post-rename API to watch this fire. It threw, naming the likely cause, instead of reporting a green nightly run that never ran. runs has since been folded into crawls; the guard moved with it, unchanged, and now reads crawl.state.

Nothing else protects the shape of that response: capture-ledger has no serializer schema, and this side does a bare as CrawlState cast.

What is not covered

The e2e seeds its crawl with a URL, so nothing exercises fromTargets — the path that turns capture_targets into seeds, which is the one the nightly job takes. That path through capture-ledger is covered only by unit tests on both sides.