Backfill & import
Import historical telemetry from Claude Code, Langfuse, Helicone, and raw OTLP dumps.
tj backfill imports historical telemetry from external sources into the local DuckDB, where the standard analyzers can read it. Every source maps onto the same internal NormalizedSpan schema, so once imported the data looks exactly like natively-captured telemetry. tj cost, tj optimize, and the Lens web UI treat it no differently.
The posture toward upstream tools is partnership. Keep whatever you use for live tracing, then point tj backfill <source> at the same data to run the local cost-optimization analyzers against it.
Sources
| Source | Command |
|---|---|
| Claude Code (on-disk session JSONL) | tj backfill claude-code |
| Langfuse (live API or JSON dump) | tj backfill langfuse |
| Helicone (live API or JSON dump) | tj backfill helicone |
| Raw OTLP JSON (file or HTTP dump) | tj backfill otlp |
Two input modes
Every adapter except claude-code (which reads a fixed on-disk path) takes one of two modes.
Live API. Pass --source-url and an API key. The adapter calls the source’s REST endpoint and follows pagination.
tj backfill langfuse --source-url https://cloud.langfuse.com --api-key lf_pk_... --since 30d
tj backfill helicone --source-url https://api.helicone.ai --api-key hc_pk_... --since 30dLocal file. Pass --source-file and a JSON or NDJSON dump. This is the right mode for testing, offline analysis, or scripted ingestion from a snapshot.
tj backfill langfuse --source-file ./langfuse-export.json
tj backfill helicone --source-file ./helicone-export.json
tj backfill otlp --source-file ./traces.jsonFor Claude Code, no source flag is needed. The adapter parses ~/.claude/projects/*.jsonl and runs automatically at the end of tj onboard --claude-code.
tj backfill claude-codeTime windows
--since filters ingested records by start time. It accepts the same syntax as tj cost --since: a duration like 30d or 24h, or an ISO-8601 timestamp.
tj backfill otlp --source-url https://example.com/traces.json --since 7dIdempotency
Re-running a backfill is a no-op. Every adapter derives a deterministic span_id from the source’s own identifiers, and the spans table enforces a PRIMARY KEY on span_id, so rows that already exist are skipped rather than duplicated.
| Source | Deterministic span_id derived from |
|---|---|
| Claude Code | (session_id, message_uuid) |
| Langfuse | (traceId, observation_id) |
| Helicone | (request_id) |
| Raw OTLP | the payload’s own (trace_id, span_id) |
Because of this you can schedule backfills nightly via cron without duplicating data, combine overlapping --since windows, and run a backfill alongside the live daemon. The two share one DB with no conflict as long as their source IDs differ, which they will.
Plan tier on backfilled sessions
Backfilled sessions land with plan_tier = 'unknown' because the source data carries no plan-tier identifier. Run tj onboard --reconfigure to set your plan, and tj optimize will render dollar figures correctly for those sessions afterward.
Notes per source
Langfuse reads observations from /api/public/observations. Self-hosted instances work by pointing --source-url at their base URL. Cost is taken from calculatedTotalCost when present, otherwise recomputed from pricing/models.toml.
Helicone POSTs /v1/request/query and maps one request record per span. billing_account comes from the record’s provider field when present, otherwise it is derived from the model name.
Raw OTLP uses the same parser as the live POST /api/v1/spans route, so the receive path and the backfill path always agree on attribute extraction. It handles a {"resourceSpans": [...]} envelope or NDJSON of envelopes, and reports spans_seen, spans_written, spans_skipped, and spans_rejected. Note that this is for JSON dumps only, not live push. For a live collector, send OTLP directly to tj serve instead.