Engineering the fastest FHIR server

Our analytics platform answers questions that need the clinical record and the billing record at the same time: the HbA1c trend for a diabetic cohort next to its claims, or the lab evidence behind a denied inpatient stay. Serving those answers interactively means running the clinical data layer ourselves, so we built our own FHIR server. It is a Rust engine that serves FHIR R4 over REST and GraphQL, mirrors continuously from EHRs like Epic and Cerner, and stores all 146 R4 resource types.

Two goals govern every component. The first is fidelity: every ingested resource must come back byte-for-byte identical to what was stored. The second is speed: the server is designed to beat every FHIR server we can benchmark it against, and it carries a harness that runs the identical workload against HAPI and Medplum to keep that claim testable. Each decision below serves one of the two goals, and usually both.

Two query shapes, two databases

Everything we run falls into one of two query shapes. The first is the cohort traversal: “final inpatient encounters for patients who died in the hospital” crosses Patient and Encounter, filters on status and class, and follows references between resource types. That query wants joins and transactional integrity. The second is the time-series scan: “every hemoglobin A1c for this patient over the last two years” is one patient and one code over a window of an enormous append-mostly table. That query wants sequential reads over data physically sorted the way it will be asked for.

No single database serves both shapes well, so the server uses two. Postgres is the system of record: it owns versioning, idempotency, the reference graph, and the embeddings behind semantic search. ClickHouse is the analytical store for high-volume time-series resource types. Routing is per resource type through one configuration map, and the shipped default sends Observation to ClickHouse and everything else to Postgres.

Tempting as one database would be, we deliberately kept ClickHouse narrow. Its mutations are asynchronous, its point updates and deletes are weak, and it has no transactions, which the FHIR write surface needs constantly for versioned updates and transactional bundles. The relational graph stays in Postgres. Only the proven high-volume shape moves.

Observation is the workload

In every corpus we have loaded, Observation is most of the data. Each vital sign, lab result, and device reading lands as its own Observation resource, so a single inpatient stay produces thousands of them. In the synthetic Synthea populations we benchmark with, Observation is the largest resource type by a wide margin, and live EHR extracts skew harder once device telemetry arrives.

The access pattern is just as lopsided. An Observation is written once and almost never updated, then read back as a series for one patient and one code over a window. That is precisely the shape a ClickHouse MergeTree table is built for, because its sort key makes patient-code-time the physical layout of the data on disk:

-- the observation table; every version is kept (condensed)
CREATE TABLE observation (
    logical_id           String,
    version_id           Int64,
    patient_id           String,
    encounter_id         String,
    status               LowCardinality(String),
    code_systems         Array(String),
    code_codes           Array(String),
    loinc_code           String,
    effective_time       DateTime64(6, 'UTC') CODEC(DoubleDelta, ZSTD(1)),
    value_quantity_value Nullable(Float64) CODEC(Gorilla, ZSTD(1)),
    value_quantity_unit  String,
    doc                  String CODEC(ZSTD(3)),
    canonical_hash       String,
    PROJECTION population_by_code (
        SELECT * ORDER BY (loinc_code, effective_time, patient_id)
    )
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(effective_time)
ORDER BY (patient_id, loinc_code, effective_time)

Three choices in that DDL do most of the work:

• Plain MergeTree, deliberately not ReplacingMergeTree. Replacing collapses version history at merge time, which breaks vread and _history, and it still requires query-time dedup before merges have run, so it buys nothing. Every version is kept, and the current one is deduped at read time with LIMIT 1 BY logical_id, which benchmarks about 35% faster than the argMax alternative (1.68M versus 1.24M rows per second) with far less variance.
• The layout is the query. Partitioning by month prunes whole months out of a window scan, the sort key is the patient-code-time access path, and the projection stores a second sort order for population questions (an HbA1c trend across all patients) that don’t lead with a patient.
• Purpose-built codecs. Timestamps compress under DoubleDelta, float values under Gorilla, and the full document under ZSTD. Token search needs no side tables here: codes travel as parallel (system, code) arrays matched with arrayExists at query time.

On the write side, an entire bundle’s Observations land as one batched insert, and the streaming path uses a buffered inserter that flushes at 100,000 rows or one second, whichever comes first. The ClickHouse path ingests about 41,600 Observations per second on our benchmark hardware against 35,700 for the same rows through Postgres, and the ClickHouse figure includes the Postgres locator row it writes alongside (more on that below).

Store the bytes, index the copies

A FHIR server’s hottest read is GET Patient/123, and most servers answer it by loading rows and reassembling JSON. Ours answers it with a byte copy.

On ingest, each resource is canonicalized once and those exact bytes are stored whole in a doc column (bytea with LZ4 in Postgres, ZSTD-compressed String in ClickHouse). Every field the server queries is additionally flattened into typed columns and side tables, and those copies exist only for indexing. A read is one partial-index lookup plus a byte copy. There is no merge-on-read, no reconstruction step, and no code path that could reassemble the document wrong.

Canonical form is defined once: recursively sorted keys, no insignificant whitespace, and lexically preserved decimals. The last rule is load-bearing. FHIR treats precision as significant, so 1.0 and 1.00 are different values, and a number that round-trips through a float loses that distinction. Decimals therefore never pass through f64 anywhere in the system, and the canonicalizer makes that a compile-time property:

// canonical bytes carry the exact decimal lexeme as received;
// Number::as_str only exists under serde_json's arbitrary_precision
// feature, so any f64 path is a compile error (condensed)
match value {
    Value::Null => buf.push_str("null"),
    Value::Number(n) => buf.push_str(n.as_str()),
    Value::String(s) => write_escaped(s, buf),
    // objects write their keys in sorted order, arrays in place
    ...
}

We evaluated simd-json for the hot parse. It reads a typical Observation about 1.3× faster, and it routes numbers through f64, which destroys decimal lexemes. It stays behind an off-by-default feature flag. A faster wrong answer is still wrong.

Next to the bytes sits canonical_hash = blake3(doc): the idempotency key on ingest and the corruption check on read. A configurable read-verification mode re-hashes served documents (sampled by default), and the mismatch counter has a permanent budget of zero. The fidelity suite round-trips 2,912 official R4 example resources and asserts every one comes back canonically identical.

The Postgres side of the same idea looks like this:

-- every resource type gets this table shape; queried fields are
-- promoted to typed columns, but doc is what reads serve (condensed)
CREATE TABLE encounter (
    pk             uuid PRIMARY KEY,     -- UUIDv7, time-ordered
    logical_id     text NOT NULL,
    version_id     bigint NOT NULL,
    is_current     boolean NOT NULL DEFAULT false,
    deleted        boolean NOT NULL DEFAULT false,
    last_updated   timestamptz NOT NULL,
    patient_id     uuid,                 -- patient-compartment shortcut
    change_seq     bigint NOT NULL,      -- global write order
    doc            bytea NOT NULL,       -- canonical JSON, served verbatim
    canonical_hash bytea NOT NULL,       -- blake3(doc)
    UNIQUE (logical_id, version_id)
) WITH (fillfactor = 90);                -- current-flag flip stays a HOT update
 
CREATE UNIQUE INDEX encounter_current_idx
    ON encounter (logical_id) WHERE is_current;

The partial unique index is the entire hot read path. There is deliberately no GIN index on doc: if a query ever needs to look inside the document, the fix is a promoted column, never a document scan. Index discipline is a standing rule in this codebase, because every index is a tax on the highest-volume write path.

Ingest parses once and writes O(tables)

The write pipeline is four stages: parse, validate, flatten, persist. A resource is parsed into a JSON tree exactly once; validation, canonicalization, and flattening all borrow that one tree. The CPU-bound stages run on a dedicated rayon thread pool, so tokio’s I/O threads never do CPU work. Validation is layered: structural JSON Schema compiled per resource type (about 10 µs for a typical Observation), coded-field sanity checks, then US Core 6.1.0 profile validation with FHIRPath invariants evaluated by our own hand-written FHIRPath engine. Canonicalize plus hash costs 2.47 µs.

Backpressure is structural rather than advisory. Stages connect through bounded channels in 64-entry chunks, and a semaphore caps in-flight chunks at the pool size. A worker that cannot hand off its chunk blocks right there, so a slow database stalls the CPU stage and a stalled CPU stage stalls intake. Memory stays flat regardless of bundle size.

Persistence issues one round trip per table per bundle: multi-row INSERT … SELECT FROM UNNEST(...) statements, switching to COPY BINARY through a staging table above a configured row count. A 10,000-entry bundle costs a number of statements proportional to the tables it touches, never to its size, and a regression test counts the actual SQL statements to keep it that way.

Idempotency runs before any other write logic:

// identical bytes against the live current version are a no-op, and a
// replayed upstream version is skipped the same way (condensed)
if let Some(row) = current.get(&logical_id)
    && !row.deleted
    && row.canonical_hash == flattened.canonical_hash.as_bytes()
{
    return (Plan::Skip, WriteOutcome::UnchangedHash { .. });
}
if let Some(uvid) = upstream_version_id
    && replays.contains_key(&(logical_id, uvid))
{
    return (Plan::Skip, WriteOutcome::UnchangedReplay { .. });
}

This is what makes mirror mode cheap. Re-syncing a window from an upstream EHR re-ingests thousands of unchanged resources, and each one costs a single row read. There is no separate dedup layer to build or to get wrong. The pipeline’s intrinsic throughput is 12,000 to 21,000 resources per second, measured on a laptop.

Flattening is data, and it cannot touch the bytes

Each resource type has a processor that declares which fields become typed columns. The mechanical parts are code-generated from the published FHIR StructureDefinitions, so all 146 types get reference, coding, and identifier extraction for free; the hot columns on the types we query hardest are declared by hand. The declaration is a const table in a small path language, so a column definition reads like the FHIR path it extracts:

// the promoted columns for Observation, declared as data (condensed)
pub static OBSERVATION: &[PromotedPath] = &[
    first("status", &[Field("status")], ValueKind::String),
    first("code_loinc",
        &[Field("code"), Field("coding"),
          Filter { path: &["system"], any_of: LOINC }, Field("code")],
        ValueKind::String),
    first("effective_date_time_start",
        &[Field("effectiveDateTime")], ValueKind::DateTimeStart),
    first("value_quantity_value",
        &[Field("valueQuantity"), Field("value")], ValueKind::Decimal),
    // unit, period bounds, and the string/concept variants elided
];

One definition drives both stores: it produces the promoted columns on the Postgres table and the columns of the ClickHouse row, so the two backends can never disagree about what value-quantity means.

The contract on flatten is that it borrows the parsed tree and clones no subtrees, and in debug builds the pipeline re-canonicalizes the tree after flattening and asserts the hash is unchanged. A processor bug can produce a wrong column, and the harness will catch it, but it structurally cannot corrupt the stored document. Dates get the same never-disagree treatment as columns: the function that turns a FHIR partial date into [start, end) bounds at ingest is the same function the search compiler uses to evaluate ?date=ge2024, so the stored interval and the query interval share one definition.

One query AST for every surface

REST search, GraphQL, semantic search, natural-language query, and the conditional write operations all compile into one typed query AST. Compilation is FHIR-aware and backend-blind; lowering is dialect-aware and FHIR-blind; execution binds parameters and copies bytes. The AST names resource types and promoted columns, never tables or SQL dialects, which is what lets the same tree lower to Postgres SQL or ClickHouse SQL depending on where the type lives.

A chained parameter or a _has is one Semijoin node in that AST, lowered to an indexed EXISTS over the reference edge table. Paging is keyset only; the engine never emits OFFSET, so page N costs the same as page one and pages stay stable under concurrent writes. Unknown search parameters are rejected instead of ignored, because conditional create, update, and delete run their criteria through this same compiler, and a silently dropped filter would let a conditional delete match resources it should never touch.

The lowering is easy to see. A routine Observation search:

-- GET Observation?patient=p1&code=http://loinc.org|4548-4
-- lowers to ClickHouse SQL over a current-version subquery (condensed)
SELECT t.logical_id, t.change_seq, t.doc
FROM (SELECT * FROM observation
      ORDER BY logical_id, version_id DESC LIMIT 1 BY logical_id) t
WHERE t.deleted = 0
  AND t.patient_id = ?
  AND arrayExists((s, c) -> s = ? AND c = ?, t.code_systems, t.code_codes)
ORDER BY t.change_seq ASC
LIMIT 50

On the Postgres side, the side tables carry covering composite indexes in both traversal directions, so the engine’s semijoins execute as index-only scans with zero heap fetches. The measured effect on the worst query shape, a reverse _has probing two side tables at once: 226 ms and 273,921 buffers with ordinary indexes, 11.3 ms and 29,347 buffers with covering ones. Twenty times faster, with the same number of indexes to maintain on ingest.

The natural-language path deserves one safety note. An LLM planner translates the question into a plan object validated against a pinned schema, and that plan is resolved through the identical compiler REST search uses. Model output never reaches SQL as text, and the NL form of a question compiles to the byte-identical AST as its hand-written REST equivalent, which is replay-tested in CI.

Two stores, never a distributed join

Routing Observation to ClickHouse would break FHIR semantics if it were naive, because Observations participate in compartments, $everything, _include, and graph traversal, all of which live in Postgres. The bridge is the locator row: for every routed Observation, Postgres keeps a stub with a placeholder document, the real canonical_hash (so idempotency still fires), and all of its reference edges. The graph machinery works unchanged; only the bytes moved.

Write ordering keeps the stores consistent without a cross-store transaction, since ClickHouse has none to offer. Postgres runs the normal write and remains the sole authority for version numbers and idempotency. ClickHouse rows are written afterward, only for writes that produced a new version, carrying the Postgres-assigned version. A ClickHouse failure after the Postgres commit leaves a locator whose document is briefly absent, and the reconcile pass settles it. We documented that window rather than pretending it away.

Analytics across both stores follow a two-phase cohort pattern. The headline query, “average HbA1c over the last year for patients who died in the hospital,” resolves its cohort in Postgres (Patient?deceased=true&_has:Encounter:patient:class=IMP), then pushes the resulting patient ids into ClickHouse:

// a resolved cohort becomes a predicate; small cohorts inline,
// large ones land in a throwaway Memory table (condensed)
if cohort.len() <= inline_max {  // 10,000
    return Ok(PushedCohort {
        predicate: format!("patient_id IN ({list})"),
        temp_table: None,
    });
}
let table = format!("cohort_{}", Uuid::new_v4().simple());
client
    .query(&format!(
        "CREATE TABLE {table} (patient_id String) ENGINE = Memory"
    ))
    .execute()
    .await?;

The aggregation (count, min, max, average, or a trend line of raw points) then runs entirely inside ClickHouse over the pushed set. Patient-scoped authorization rides the same mechanism: a caller’s scope resolves to patient ids in Postgres and pushes down as the same predicate, and an empty scope compiles to a clause that matches nothing, so the failure mode is a blank page.

The property that makes all of this safe to operate is that routing is invisible to clients, and that is proven rather than promised. The defining test stands up two complete server arms, one with Observation in Postgres and one with it in ClickHouse, seeds both identically, and asserts that reads, version reads, history, every search in the matrix, paging, and $everything return identical results. Moving a resource type between databases is a config edit, and the test suite is the reason we believe that.

The numbers, and how we keep them honest

Every performance claim above comes from a committed baseline with a ratio gate in CI, so a regression fails the build before it ships:

| Path | Measured | | --- | --- | | Canonicalize + hash, typical Observation | 2.47 µs | | Structural validation, typical Observation | 10 µs | | Ingest pipeline, intrinsic | 12,000–21,000 resources/s | | ClickHouse Observation ingest | ≈41,600 rows/s | | Single read, warm cache hit | 307 ns (437 µs uncached, ≈1,420×) | | Reverse _has search at ~84k references | 11.3 ms (226 ms before covering indexes) |

The comparison harness runs the identical experiment against HAPI JPA and Medplum: the same seeded, checksum-pinned Synthea population, the same bulk load, and the same four k6 scenarios (single read, indexed search, the cohort traversal, and a 1,000-row time-series scan). The competitor configurations are tuned per their own documentation and adjusted only for parity, and each adjustment is written down: HAPI’s write-time validation is disabled to match our asynchronous reference reconciliation, and Medplum’s rate limiter is lifted so the workload isn’t throttled before it reaches the server. Numbers from developer laptops gate ratios only; the absolute head-to-head runs on fixed Linux hardware and lands in the repository with hardware and image versions attached.

That last rule matters more to us than a headline. A benchmark you cannot rerun is an anecdote, and this server’s performance claims are designed to be rerun by anyone, against anything, forever.

Where this goes

The server already carries the full surface our platform needs: SMART-on-FHIR and OAuth2 auth with patient-scoped grants enforced inside the query engine, bulk $export, subscriptions, and semantic search whose embeddings are computed in-process so clinical text never leaves the box. A terminology service with UCUM conversion and value-set expansion is next, followed by R5 and R6 code generation on the same version-scoped machinery.

The design has one sentence at its core: store the exact bytes, index disposable copies, and make every query surface compile into one plan. Everything fast about this server follows from it, and everything trustworthy about it is a test someone can run.

Want to see it against your own clinical data? Talk to us.