Performance & Cost

​

Serving Behavior

• A materialized table persists a source’s data as a physical table and routes queries to it.
• A search index embeds a dimension’s values so they are findable by value search, filter suggestions, and the AI agent.

​

How Materialization and Indexing Compose

• Consistent — the index reflects the materialized snapshot, so searchable values match what queries return.
• Stable — a table-backed index refreshes exactly when its source table refreshes. An index on a source you have not materialized refreshes on publish and on demand, and — if you declare a freshness window — on that window.
• Cheap — indexing reuses the table you already built instead of paying to re-scan the warehouse.

​

Deciding What to Persist

​

Configuration

​

Annotations

#@ persist name="orders_fast" refresh="incremental" watermark="order_date" freshness.window="24h"
source: orders is conn.table('sales.orders') extend {
  #(index)
  dimension: status is order_status
}

• name — choose where the materialized table lands (optional; container-qualifiable).
• refresh — "full" (the default) rebuilds the whole copy; "incremental" applies only what changed and requires a watermark (see Incremental Refresh).
• watermark / merge_key — how an incremental table finds and applies new rows (see Incremental Refresh).
• freshness.window — the staleness objective the platform schedules against (see Freshness).

​

Package Manifest

{
  "scope": "package",
  "materialization": { "freshness": { "window": "24h", "fallback": "live" } }
}

• scope: package (the default) — a derived copy is reused across the package’s versions whenever they define the same thing. Maximal reuse, lowest cost.
• scope: version — each version keeps its own copies, with no cross-version reuse. Choose this when you want to own an exact rebuild schedule for a version.

​

Incremental Refresh

#@ persist refresh="incremental" watermark="order_date"
source: daily_revenue is orders -> {
  group_by: order_date
  aggregate: revenue is amount.sum()
}

​

Which shape is yours

#@ persist refresh="incremental" watermark="ingested_at"
source: events is conn.table('raw.events') -> {
  select: ingested_at, event_id, user_id, payload
}

#@ persist refresh="incremental" watermark="updated_at" merge_key="id"
source: accounts is conn.table('raw.accounts')

​

Limits and repairs

• Late data. A row that arrives with a watermark value below the range already covered is never picked up automatically.
• Hard deletes. A row deleted upstream can’t appear in any delta, so a merge_key source retains it. Prefer soft deletes — a tombstone flag arrives as an ordinary update — and keep the flag in the persisted source’s output, filtering it in consuming views instead.

​

Pre-Aggregations

#@ preaggregate grain="order_time.day, category"
measure: total_revenue is amount.sum()

• grain — required: the dimensions the rollup stores. A query is served from the rollup when everything it groups by and filters on is covered by the grain — a coarser truncation of a stored time dimension (order_time.month over a day grain) counts. Anything else falls back to the base source and runs live: “unsupported” and “unaccelerated” are the same, correct outcome.
• #@ -preaggregate pins a measure to the base even when a covering rollup exists — the escape hatch for a consumer that can’t tolerate the rollup’s freshness.

• Additive measures (sum, count, min, max) and avg are re-aggregated from the rollup at any covered grain.
• Non-additive measures (count(distinct), median, percentiles) can’t be correctly re-aggregated to a coarser grain, so a rollup answers them only at exactly its declared grain — other grains run live. You’re told this once, as a publish-time warning on the measure.

​

Freshness

​

Builds and Refreshes

• On publish — Credible builds every persisted source and search index for the new version automatically.
• On demand — trigger a Rerun from the package page (or the runs API) to force a rebuild. You can rerun a whole version, or a single source or dimension — optionally including its upstream persisted sources.
• On a schedule — the platform refreshes derived copies to meet their freshness objectives.

​

Storage Reclamation

​

Next Steps