How it works

Metrics Book runs as a set of containers on one VM. Only one of them, the api, faces the internet. Data moves one way: dlt loads it, dbt transforms it, and anfra answers questions from the result.

Metrics Book is not released yet, and the stack is still being built. This page describes the design the build follows. See the changelog for what exists today.

The stack

                      internet
                         │ 443
                 reverse proxy (Traefik under Dokploy, or Caddy)
                         │
             ┌───────────┴────────────┐
             │ api (Hono, Node 24)    │  the only public process
             │ /api/auth  Better Auth │
             │ /api/mcp   connector   │────► cerbos (decisions)
             │ /api/v1    REST        │
             │ /          React SPA   │────► postgres 18 (auth, audit, catalog
             │ /docs      Starlight   │        index, run records, job queue)
             └───────────┬────────────┘
                         │ SemanticEngine: allowlisted ops only, ANFRA_URL
             ┌───────────┴────────────┐
             │ anfra serve --no-apps  │  internal network, no published port
             └───────────┬────────────┘
                         │ reads
         serve.duckdb (views, made at start) ──► MARTS_URI/current/*.parquet
                                                        ▲ atomic switch
   jobs (Python 3.13): runner ── dlt ──► build.duckdb ── dbt ──┘
                         │
     monitoring: victoriametrics, vmalert, alertmanager, node_exporter,
                 cadvisor, postgres_exporter, gatus

Every process is a container in stack/compose.yml. The drawing shows the default duckdb warehouse.

The units

Unit	What it is
api	One Hono app on Node 24. It serves sign-in (/api/auth), the MCP connector (/api/mcp), the REST API (/api/v1), and the built web app and docs as static files.
web	A React single-page app. It holds no data of its own: everything it shows comes from /api/v1, the same API the CLI and service accounts use.
docs	This site, built with Starlight and served at /docs.
jobs	Python. The runner, the dlt pipelines, dbt, the catalog builder and the sample-data generator.
anfra	The semantic engine from Holistics. It turns a metric query into SQL and runs it.
cerbos	Decides who may see which dataset, metric or report.
postgres	Holds sign-in data, the audit log, the catalog index, run records and the job queue.

The MCP tools, the REST routes and the admin routes are thin adapters over one service layer. Every caller reaches data the same way.

What is public

Only api is reachable from outside. anfra, cerbos and postgres sit on internal networks with no route out. anfra’s own MCP endpoint is never exposed. Read Security for what the api does to every request.

The data path

1. Load. dlt writes raw tables into build.duckdb, one schema per source, on the jobs container’s own volume.
2. Transform. dbt builds staging and intermediate models as tables in build.duckdb. It writes marts as Parquet files under MARTS_URI/<run_id>/. dbt tests run in the same build.
3. Switch. Only if dbt build passed, the runner points MARTS_URI/current at the new run’s folder. The pointer is a symlink replaced by an atomic rename, so a reader sees the old marts or the new ones, never a mix. The last three runs are kept, and a rollback points current back at the run before.
4. Serve. anfra reads serve.duckdb, a file that holds only views over MARTS_URI/current/*.parquet. A one-shot container, serve_init, writes it from the dbt manifest before anfra starts.

Build and serve are separate

anfra opens its DuckDB file read-write and can hold it for up to three hours. While it does, nothing else can write to that file. So anfra never reads the file dbt writes. dbt writes build.duckdb and the Parquet marts. anfra reads serve.duckdb, which only points at the marts.

The cost is that a new mart appears on deploy, not mid-run. The set of views changes only when you deploy, which is also the only time the set of marts can change.

The warehouse is a choice

You name the warehouse in stack.yml:

warehouse:
  type: duckdb          # or postgres

One module per warehouse type is the only code that knows which one is in use. dlt, dbt and anfra all support many warehouses, but version 0.1 ships two: duckdb, the default on one VM, and postgres. The end-to-end test runs the e-commerce template on both, every time, so pipelines, models and metrics stay portable. With Postgres there are no Parquet files. Each run builds its marts into its own schema, and the served views are replaced to point at it.

How it grows past one VM

Five rules keep any part movable to its own machine without a rewrite:

• Containers share nothing but Postgres, HTTP APIs and the marts store.
• Every location is a setting: MARTS_URI, ANFRA_URL, CERBOS_URL, DATABASE_URL.
• Jobs coordinate through Postgres: the API inserts a job request, and the runner takes it.
• The api keeps no state, so you can run more than one.
• Only the warehouse module knows which warehouse you use.

When load grows, the first move is small. For queries, add anfra replicas. For builds, use a bigger VM. For the API, add api replicas. For Postgres, point DATABASE_URL at a managed one. Later moves are marts on object storage, jobs on their own machine, or another warehouse.

Next: Core and projects explains where your own files live.