TemporalDBSecretoftheUniverse.com
TemporalDB · Database for research and investing In design

A database that remembers what you knew, and when.

Market data gets revised. GDP is restated, fundamentals are corrected, tickers change hands. TemporalDB is designed to keep every captured response and every revision, so a backtest sees only what was knowable on its date and any result can be replayed later from the same inputs.

Architecture approved · 12 Oct 2026 Implementation planning Not deployed
Data workspace overview with Delta Lake, kdb+ and Security Master explorer cards Data workspace overview with Delta Lake, kdb+ and Security Master explorer cards, light theme
Data Workbench interactive concept (27 Sep 2026, synthetic data): three explorers, one research workflow. This is the front end TemporalDB is designed to feed.
Four kinds of time

“As of” is really four separate questions.

Most tools collapse these into a single as_of date, which is how lookahead bias sneaks into backtests. TemporalDB keeps them apart and makes you say which one you mean.

1

Effective time

When does this fact apply? The quarter a GDP figure measures, the date a price traded.

2

Provider availability

When was this version publicly available, according to evidence such as an ALFRED vintage date?

3

Local knowledge

When did our system actually receive and accept this version?

4

Storage snapshot

Which physical inputs did this particular execution read?

Queries carry explicit context fields: mode, effective_at or a date range, known_at, and knowledge_basis set to local_received or provider_published.

Time machine

Move the knowledge cutoff and watch the answer change.

Pick a dataset, slide known_at, and switch the knowledge basis. The demo applies the design’s rules to a small illustrative history. Values and dates are made up for the demo.

Mode
knowledge_basis
Provider publishedLocally receivedknown_at cutoff

      
Bronze, Silver, Gold

Keep the evidence.
Keep the revisions.
Serve the clean view.

The three layers are about trust, not temperature: Bronze does not mean cold storage and Gold does not mean hot.

Bronze

Immutable evidence

Each captured response is stored as an immutable payload plus a capture record: provider, endpoint, sanitised request, timing, HTTP status, content hash, adapter version and outcome. Credentials are never stored.

  • Every retrieval keeps its own event, even when the bytes are identical
  • Failed and malformed responses are kept for diagnosis, never promoted
Silver

Every revision, with its time

Normalised assertions with stable keys and immutable revision identity: effective interval, provider vintage and its evidence, local receipt, status and a link to whatever it supersedes.

  • Separates new facts, corrections, withdrawals and late additions
  • A row missing from a partial response is not a cancellation
Gold

The current accepted view

The latest cleaned projection over Silver, built from versioned validation, identity and source-precedence rules. History queries use Silver, never today’s Gold pretending to be the past.

  • Published as validated generations with manifests
  • Unresolved records stay inspectable instead of becoming clean values

Captured before it is served

  1. Persist the payload durably and verify its hash.
  2. Commit the capture record and its processing obligation.
  3. Serve: only now may the response be used or normalised.

If capture fails, TemporalDB serves the last captured value with a stale warning, or fails if there is none. A crash between steps must never leave a success record pointing at missing bytes.

Scale path

Start with one of everything.
Add pools when you need them.

The same contracts run on a laptop and on a lab cluster. Switch the view to see what changes.

BDOBB clientsdesktop, browser, iPad
API serverpool of N, no local stateone instance
PostgreSQL controlquery library, jobs, leases and fencing tokens
Acquisition workersshared provider budgets, no duplicate fetches
DuckDB query workerspinned results and execution receipts
DuckLakeDuckDB catalog, local ParquetPostgreSQL catalog, object storage
Why this storage choice. DuckLake keeps its catalog in a SQL database and its data in Parquet files. Its documentation recommends a DuckDB catalog for a single client and PostgreSQL for a multi-user lakehouse, and the DuckLake spec and extension are MIT-licensed. TemporalDB’s design picks DuckLake with PostgreSQL and object storage for shared deployments. Adding API servers does not make the PostgreSQL catalog infinitely scalable; that remains its own capacity question.
Shared query library

A saved query is a recipe.
An execution is a receipt.

Queries live in a shared library with immutable revisions. Widgets bind to a query and revision by default; following the latest revision is an explicit choice. Refresh creates a new execution and never edits the old receipt.

Every execution freezes

  • Query revision
  • Resolved parameters and relative dates
  • TemporalDB context
  • Input manifest
  • Engine and extension versions
  • Result schema and completeness
  • Canonical result hash
  • Ordering contract
Security Master query builder with generated DuckDB SQL, a record inspector and execution context showing the source, context used and reproducibility
Security Master in the Data Workbench concept: structured query builder, generated SQL and an execution context panel. Synthetic data.
Inside BDOBB

Two kinds of widgets:
refreshable or pinned to a receipt.

TemporalDB plugs into BDOBB’s existing table execution path as a new source. Derived tables, pivots, charts and exports carry the parent receipt, so a derived widget cannot quietly query newer data while showing the old receipt.

All screenshots: Data Workbench interactive concept, 27 Sep 2026, synthetic data, no live connections.

Retention and replay

Protected runs can be replayed, not just remembered.

Keeping a snapshot number or an output file is not enough to replay a result. A protected audit or backtest run keeps everything it depended on.

What a protected run keeps

Input snapshots, raw evidence, query revisions, cleaning and projection artefacts, runnable environments and result evidence, indefinitely.

Guarded cleanup

Expiry names explicit snapshot IDs after checking every dependency. Broad age-based expiry is not allowed on protected catalogs. DuckLake supports dry runs and explicit version lists; TemporalDB adds its own dependency checks first.

Recovery as a set

PostgreSQL catalogs, control records and the object data they reference are backed up together, and protected replay is verified after restoring to a separate environment.

Delivery path

Seven stages, each with a gate.

Each stage lands reviewable work. TemporalDB is complete only when all seven pass.

#DeliverableAcceptance gateStatus
1Inventory and canonical time, identity and query contractsActual branches, deployed services, data volumes and coverage recorded; compatibility mappings resolvedPlanned
2PostgreSQL control service, DuckLake adapters, capture and recoveryMultiple workers, capture-failure fallback, crash recovery, no duplicates, catalog and object restorePlanned
3Daily prices, fundamentals, identities and FRED vintage pipelinesBoth knowledge policies, correction examples, lineage; unknown publication history cannot leak into strict queriesPlanned
4Tick-vault migration and acquisition integrationPinned replay, cancellation, resumability, reference joins and NAS load measurementsPlanned
5Shared query library and execution APITwo BDOBB clients, revision conflicts, two API instances, worker loss, paging and permissionsPlanned
6Query data widgets and linked presentationExisting table, pivot and chart execution reused; parent receipt survives derivation, refresh and exportPlanned
7Protected audit runs and migration cutoverConcurrent cleanup cannot break a protected run; restored replay works; legacy limits stay visiblePlanned
Honest status. TemporalDB is an approved architecture, not a running service. Nothing is deployed and no data has been migrated. Moving existing data to a new format makes no API calls, but proving lineage the old data never recorded may require fetching it again, and no new paid bulk fetch starts before its request and storage forecast is reviewed.

Sources