Agent transcripts

Full-fidelity record of every model call an activity attempt makes, for debugging, evals, training data, and audit.

Each activity attempt gets one AgentTranscript row, created lazily on its first record. Its payload is a sequence of gzipped NDJSON parts stored as WorkflowArtifact rows (agent-transcript-part, no runId — a deleted run does not take its transcripts), each with a sha256; the transcript’s own sha256 hashes the ordered part hashes, so a reader can verify integrity end-to-end.

What is recorded

• http_call — every call through the worker’s model fetch: method, host, sanitized path, status, duration, request body, response body. Headers are never stored, and credential query parameters (key, token, api_key, …) are stripped before the path is written.
  • Embedding calls (lib/embeddings.ts, every activity that embeds, bulk re-embedding included) are http_calls with purpose: 'embedding'. The request — the embedded inputs — is kept in full; each vector in the response is replaced by { omitted: 'vector', dimensions } (OpenAI and OpenAI-compatible data[].embedding, Google embedding.values and embeddings[].values) whenever the body parses as JSON, whatever its content-type. They do not count toward the transcript’s modelCallCount.
• harness_turn — written as every Claude Code turn starts, proxied or not: the model, proxied, the system prompt as { preset: 'claude_code', sha256, append } — the full appended text on the adapter’s first turn in a transcript and whenever its sha256 changes, otherwise { preset, sha256, unchanged: true } (with stagedPath when a prompt too long for the command line was staged as a file; append then still holds the full text), the tool names, maxTurns, the MCP server names, loadProjectSettings, and the effort. It carries no endpoint, key, or MCP server address.
• harness_message — Claude Code SDK stream messages, only for non-proxied turns; a proxied turn’s exchanges arrive through the model proxy’s onExchange instead.
• harness_session_file — after every non-proxied Claude Code turn that ended without cancellation or its deadline (a failed turn included), what the turn added to the harness’s session JSONL file (the first regular file matching $HOME/.claude/projects/*/<sessionId>.jsonl in the container): { sessionId, path, offset, bytes, truncated, lines }, each line parsed (a line that does not parse is kept as its string). Turns resume one session, so each records only the bytes after the previous turn’s offset, capped at 4 MiB per turn (truncated: true; the next turn continues from where the cap stopped). bytes is what the read delivered — the delta is copied to a temp file in the container first — so a file that grows during the read loses nothing. A file shorter than the offset (rewritten or compacted) is read from 0 and the record carries reset: true. The offset advances only when the record is stored, so bytes a dropped record held are read again by the next turn; once the transcript is closed or past its cap, no more reads run. The read runs in the background — the turn returns without waiting, and the recorder’s close drains it — each read waits (up to 10 s) for the previous one so offsets stay in order, and is itself bounded to 10 s. A missing file, a failed read, or a session id that is not a UUID records nothing and never fails the turn.
• marker — truncated (the run hit workflow.transcriptMaxBytes), record_dropped (a record arrived after the activity ended).

Secrets are redacted at write time (the same redactJsonValues used by trace capture) with no truncation; the PII scrub is applied only at read/export, opt-in via scrubPii=true. Persistence is best-effort: a failed part write marks the transcript incomplete, logs, counts transcript.persist_failures (alerted on by the bundled Grafana), and writes a transcript.persist_failed trace event — never failing the activity. Transcripts still streaming when the activity returns are drained (bounded to 10 s) before the row closes; the interceptor’s whole close is bounded to 30 s so a hung artifact store cannot stall the activity.

Reading transcripts

• GET /api/v1/workflow-runs/:id/transcripts — a run’s transcripts (visible iff the run is).
• GET /api/v1/transcripts/:id — metadata plus part hashes.
• GET /api/v1/transcripts/:id/content — the records as an NDJSON stream (?format=raw|normalized, ?scrubPii=true).
• POST /api/v1/transcripts/:id/download-link — a short-lived signed link to that stream (see Download links).
• POST /api/v1/transcripts/:id/verify (ADMIN) — re-hashes every part’s gzipped bytes and the transcript hash.
• PUT / DELETE /api/v1/workflow-runs/:id/transcripts/legal-hold (ADMIN, reason required on PUT) — sets or clears a run-wide retention hold in one transaction (the WorkflowRun columns plus every transcript’s runLegalHold* copy).
• PUT / DELETE /api/v1/transcripts/:id/legal-hold (ADMIN) — an independent per-transcript hold (legalHold*), for a transcript whose run was deleted or which must outlive the others.

The two scopes are independent: releasing a run hold leaves per-transcript holds in place, and releasing a transcript’s own hold leaves its run hold in place. legalHold in API metadata is the effective hold (either scope); transcriptHold and runHold report each scope. The sweep deletes only when both are false and the run itself is not held.

• GET /api/v1/platform/transcripts/export (ADMIN) — NDJSON export of { transcript, record } lines over a date range (from/to required, ≤ 31 days; format, scrubPii, agentKey, activityType filters), across all tenants including runless transcripts.
• GET /api/v1/platform/transcripts/access-log (ADMIN) — the access audit, newest first.

Authorization: list and metadata follow the run’s visibility filter; content (and a download link for it) requires run control (owning team or requester) or ADMIN — a shared-repo teammate can see a run but not its raw model traffic. A runless transcript is ADMIN-only; a caller short of the bar gets 404. Service principals get 403 TRANSCRIPTS_NOT_AVAILABLE on the ENGINEER routes and 401 SERVICE_TOKEN_NOT_ALLOWED on the ADMIN ones, which do not admit service tokens at all.

Access audit

Every content read, download-link issue, export, integrity verification and legal-hold change (run-wide and per-transcript) writes a TranscriptAccessLog row: the action (content_read, download_link, export, verify, legal_hold_set, legal_hold_cleared), its scope (transcript, run or export), the actor’s id and role, the transcript and run ids, the request’s parameters (format, scrubPii, the export’s range and filters, a hold’s reason, and via — api or download_link), the client IP and user agent, and an outcome. A content or export row is written before the stream starts, as started, and completed from the response itself — its listeners are attached before the row is inserted, so a client that leaves during the insert is still recorded: ok once the response has finished, integrity_error when a part failed verification (an export that carried on past one still ends integrity_error), stream_error when the read failed, aborted when the response closed before it finished. The transcript sweep marks a row still started an hour after it was written aborted (with completedAt set), which covers a gateway that died mid-stream. A verification ends ok or integrity_error; a hold change or a link issue is written once it has been applied, as ok. A 401 or 404 on the content or download-link route writes a denied row when the caller is identified — the signed-in user, or a download link’s subject once its signature verified (status and via in its parameters, no run id); the response itself is the same whether or not the transcript exists. The rows have no foreign keys, so they outlive the user, run and transcript they name, and no tenant columns: the log is ADMIN-only. The gateway log keeps its own audit lines (transcript_content_read, transcript_export, transcript_legal_hold_set / _cleared) as well.

The failure policy is allow + alert: when the row cannot be written (or completed), the access still goes ahead, and the gateway logs a warning and counts transcript.access_log_failures (by action, alerted on by the bundled Grafana).

GET /api/v1/platform/transcripts/access-log (ADMIN; never callable by the in-app assistant) filters by from/to (ISO instants, to exclusive), actorId, transcriptId, runId and action, pages with limit (at most 200, default 50) and offset, and returns { data, total } newest first, each row with the actor’s email when the user still exists. The dashboard shows it on /admin/audit under Transcript access, with the same URL-kept filters as the change log. Rows are kept for workflow.transcriptAccessLogRetentionDays (default 2555 days, 365 to 3650) and pruned by the transcript sweep.

Download links

A browser cannot attach the API bearer to a plain navigation, so the run page asks for a link instead: POST /api/v1/transcripts/:id/download-link with { format, scrubPii } (same authorization as the content route) returns { url, expiresAt }, where url is the relative content URL with format, scrubPii and a dl token. The browser then navigates to it and the attachment streams straight to disk — nothing is buffered in the page.

The token is base64url(payload).base64url(HMAC-SHA256), keyed by an HKDF-SHA256 subkey of CONFIG_ENCRYPTION_KEY (info auto-swe/transcript-download-link/v1). The payload carries the version of the key it was signed under (kv), and verification uses that version’s key from the keys the process holds — the current one or, during a rotation, CONFIG_ENCRYPTION_KEY_PREVIOUS — so a link issued just before a rotation still works; a version the process holds no key for is invalid. Its payload names the user, the transcript, the format and the PII choice, a random nonce, and an expiry 60 seconds out. Only the content route accepts dl, and only in place of a credential: the signature (compared in constant time), the expiry, the transcript id and the query’s format and scrubPii must all match, the user must still exist, be active and hold at least the ENGINEER role, and the same run-control authorization runs again — a link outlives neither its holder’s access nor its 60 seconds. A refused link answers 401 DOWNLOAD_LINK_EXPIRED or 401 DOWNLOAD_LINK_INVALID. The read is logged with via: download_link. The dl value is redacted from request logs and from the server span’s url.query.

Stream integrity

Every content or export stream ends with a {"kind":"eof","complete":true} sentinel. A part whose sha256 does not verify, whose gunzip output exceeds 64 MiB, or whose artifact cannot be read ends the stream with an integrity_error line (reason: sha_mismatch / too_large / corrupt / read_failed; corrupt means the part’s gzip stream did not decode) and a torn-down connection — a client must treat any stream without the eof line as incomplete. The CLI exits non-zero on that.

Normalized view

format=normalized maps each record through @auto-swe/shared/lib/transcripts/normalize — Anthropic Messages (including thinking blocks and streamed input_json_delta), OpenAI Responses and chat completions (streamed or not), Google generateContent/streamGenerateContent, and Claude Code harness_messages — into one message/tool-call/tool-result shape with usage. Every normalized call carries source — http_call, harness_message, or harness_session_file — so a reconstruction is never mistaken for a real Messages API call. A harness_session_file normalizes to one anthropic call whose messages are the delta’s main-conversation user and assistant lines, with the model those lines name, and whose response.usage sums the assistant lines’ usage (the last report per message id, sidechain calls included); sidechain (subagent) lines, summaries, and unparsed lines stay in the raw record only. Embedding calls (by purpose, or by an /embeddings, :embedContent, or :batchEmbedContents path) and harness_turn records are not conversations and normalize to null, as does a record that matches no known shape: it emits { normalized: null, record }.

Retention

workflow.transcriptRetentionDays (default 365) bounds storage. A daily Temporal schedule (AGENT_TRANSCRIPT_PRUNE_ENABLED, AGENT_TRANSCRIPT_PRUNE_CRON, default 57 4 * * *) runs pruneAgentTranscripts: expired non-held transcripts are deleted (transcript row first — the part’s artifactId is Restrict, so the artifacts cannot go first — then the artifacts; a leftover artifact is reaped by the orphan pass), open transcripts with no update in 24 h become abandoned, and orphaned agent-transcript-part artifacts older than 24 h are deleted. The same sweep deletes transcript access-log rows past workflow.transcriptAccessLogRetentionDays, in batches, and artifacts left pending for more than 24 h (below). workflow.transcriptCaptureEnabled (default true) turns recording off without touching existing rows.

On the S3 backend an artifact is written ahead of its object: the WorkflowArtifact row is created pending with its object key, then the object is uploaded, then the row is marked ready. A failed upload leaves its row pending and the error propagates — the store may have committed the object even though the call failed. getArtifact refuses a row that is not ready, and the readers that treat an artifact row as present evidence (quality-gate evidence, knowledge capture) filter on ready. A failed upload, or a writer that dies between the two, leaves a pending row naming the object; the sweep walks those older than 24 h in batches, deleting the object (a missing one counts as deleted) and then the row. A deletion that fails is logged, counted (pendingArtifactFailures in the sweep’s result) and retried on the next sweep without stopping the passes after it. The Postgres backend writes its row and payload in one insert and has no pending state.

The lesson writer reads a run’s newest implementer transcripts for a code-built digest of failed tools and commands, churned files and final notes (memory.md §2).

The run page’s Transcripts section lists each attempt’s transcript with a download link (raw or normalized) and an ADMIN-only legal-hold toggle that asks for a reason.

Limitations

• http_call and harness_message capture model traffic through the worker fetch and the Claude proxy/harness only — a runtime that shells out to a model API outside modelFetch is not recorded.
• A non-proxied Claude Code turn records no HTTP exchange, so the text of the built-in claude_code preset prompt and the JSON schemas of the harness’s tools are not in the transcript — only the appended prompt and the tool names are (harness_turn). Through the model proxy, every request is recorded whole.
• A non-proxied turn’s session-file delta is capped at 4 MiB per turn and, when cut at the cap, ends at its last newline so no line is split across turns (a single line longer than the cap is delivered cut); a turn that writes more records the rest only if a later turn of the same session runs. A read that fails, exceeds its 10 s bound, or lands after the transcript closed leaves that turn without a harness_session_file (the next turn’s read starts from the same offset), and a turn stopped by cancellation or its deadline is not read at all — a later turn’s read picks its bytes up.
• A session-file delta whose record still exceeds 16 MiB (pathological escaping of unparseable lines) is replaced by a record_truncated marker and its offset does not advance, so later turns re-read it and it is never stored.
• Embedding inputs are recorded in full, so bulk re-embedding (reembedMemory) can fill workflow.transcriptMaxBytes on its own; past the cap, the rest of the attempt goes unrecorded.
• Redaction is key-name- and pattern-based at write time; a secret under an unrecognized key survives into the part.
• The PII scrub is regex- and Luhn-based and only ever applies to string fields — it catches what its patterns catch, nothing more, and only when requested at read/export.
• gzip compresses within a ~32 KB window, so a part does not deduplicate repeated context across calls — long sessions are large; workflow.transcriptMaxBytes bounds them.
• A record that arrives after close’s 10 s drain lands only as a record_dropped marker (or, once the row is final, as a log + counter); an activity killed mid-attempt leaves an abandoned transcript with whatever parts flushed.
• runId survives a deleted run only while the transcript does — retention still applies, and a runless transcript is readable by ADMIN only.
• sha256 detects corruption or tampering of the stored objects; it is not WORM — someone who can rewrite the transcript/part rows holding the hashes can resign edited content.
• The value-shape redactor is pattern-based and misses secret formats it has no pattern for.
• A record over 16 MiB is stored truncated (a record_truncated marker, head kept) — full fidelity is bounded by that per-record cap as well as the transcript cap.
• The access log is allow + alert: when its write fails the read still happens, and only the warning, the transcript.access_log_failures counter and the bundled Grafana alert record it. Like the transcripts, its rows are not WORM — anyone who can write the table can edit or delete them.
• A stream’s outcome is recorded from the gateway’s side: ok means the response finished — every byte handed to the socket — not that the client stored it. A gateway that dies mid-stream leaves its row started until the next sweep marks it aborted, so that outcome also covers a crash.
• A refusal is recorded only when the caller is identified — a signed-in user, or a download link whose signature verified (an expired one included). An unauthenticated request, or one carrying a forged or malformed link, writes no row, so anonymous callers cannot grow the table; such refusals appear only in the gateway’s request log.
• The transcript alerts fire only in Grafana → Alerting with no notification channel out of the box, and exist only in the bundled Grafana unless infra/grafana/provisioning/auto-swe-alerts.yaml is provisioned into another one.
• Download links are not single-use: within its 60 seconds a link can be replayed by whoever holds it (each use is logged). Once a rotation drops CONFIG_ENCRYPTION_KEY_PREVIOUS, links signed under it stop verifying.
• The pending artifact sweep runs inside the transcript prune schedule, so with AGENT_TRANSCRIPT_PRUNE_ENABLED=false an object whose upload outlived its writer is never deleted; an S3 object with no row at all (left by a version that uploaded before inserting its row) is not found by any sweep.