archives

The ledger: one row per capture that produced an archive. It records where each WACZ is, and it is what the signing endpoint looks up.

    .addColumn("id", "uuid", (col) => col.primaryKey().defaultTo(sql`gen_random_uuid()`))
    .addColumn("task_id", "uuid", (col) => col.notNull())
    .addColumn("correlation_id", "text")
    .addColumn("bucket", "text", (col) => col.notNull())
    .addColumn("object_key", "text", (col) => col.notNull())
    .addColumn("source_url", "text", (col) => col.notNull())
    .addColumn("labels", sql`text[]`, (col) => col.notNull().defaultTo(sql`'{}'::text[]`))
    // CaptureResultReport.completeness.complete。`false` は、そのアーカイブに
    // 本文が 1 つ以上欠けているという意味 —— 304 としてしか見なかった URL か、
    // 容量の上限を超えて BrowserHive が落としたもの (後者は v1.11.0 で加わった。
    // それ以前、上限に当たった取り込みは `true` と報告していた)。アーカイブを
    // wacz-validator に渡す前に知っておく価値がある。この field より前の取り込みや、
    // 記録しなかった取り込みでは NULL。
    .addColumn("wacz_complete", "boolean")
    // ledger が受け取った時刻ではなく、BrowserHive 自身が付けた取り込みの時刻。
    // reconciler は何時間も後に行を登録しうるので、履歴を並べ替えてはならない。
    .addColumn("captured_at", "timestamptz", (col) => col.notNull())
    .addColumn("created_at", "timestamptz", (col) => col.notNull().defaultTo(sql`now()`))

The absence of an owner column is deliberate

There is no owner_id and no org_id. Who may read an archive is a relationship, and OpenFGA holds it.

Two places answering the same question will eventually disagree. An org_id here would make it possible to be in a state where the table says one organization and OpenFGA says another.

Instead, registration queues tuples into fga_outbox and the relationship lives in OpenFGA.

Failed captures do not enter

if (report.status !== CaptureStatus.CAPTURE_STATUS_SUCCESS ||
    report.artifacts?.wacz === undefined) {
  log.warn(…, "capture produced no archive; not adding to the ledger");
  return { reason: "no-archive" };
}

A failed capture uploaded nothing. Recording it would have the signing endpoint hand out URLs for objects that do not exist — authorization working perfectly against a 404, which is the hardest kind of breakage to notice.

So the row count here will not match capture_submissions. The gap is correct.

Why double registration cannot happen

Two paths fill the ledger.

the crawl — admits what a level captured as soon as the flow reports it (fast)
the reconciler — reads the manifest keys recorded in capture_submissions for captures the ledger lacks

Both can reach the same capture, and either can be re-run.

.onConflict((oc) => oc.columns(["bucket", "objectKey"]).doNothing())

(bucket, object_key) is UNIQUE, so that is a no-op rather than a duplicate. And when nothing was inserted, no tuples are queued either — if the archive is already in the ledger, its tuples went out the first time.

Column notes

Column	Note
`id`	`gen_random_uuid()`. Becomes `archive:<id>` in OpenFGA
`task_id`	BrowserHive’s task id
`correlation_id`	The thread tying capture-ledger’s log lines to artifact filenames (8 hex digits per submission)
`bucket` / `object_key`	Taken from the server’s own report, not reassembled from a filename
`wacz_complete`	`false` means at least one body is missing — see below
`captured_at`	Capture time; listings are ordered by this, descending

When `wacz_complete` is `false`

It is CaptureResultReport.completeness.complete verbatim. false means the archive is missing at least one body:

a URL only ever seen as a 304
a body BrowserHive dropped for exceeding the size cap

The latter arrived in BrowserHive v1.11.0. Before that, a capture that hit the cap reported true. It is NULL for captures from before the field existed, or ones that were not recorded.

Worth knowing before handing an archive to wacz-validator.

Indexes

archives_pkey                    PRIMARY KEY (id)
archives_bucket_object_key_key   UNIQUE (bucket, object_key)   -- blocks double registration
archives_captured_at_idx         (captured_at DESC)            -- newest first
archives_task_id_idx             (task_id)                     -- lookup from a report