> ## Documentation Index
> Fetch the complete documentation index at: https://docs.credibledata.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Persistence
> Materialize sources and index dimensions for fast, searchable models
Credible maintains managed, derived copies of your data, and you opt parts of your model into them. Add a single annotation and the platform builds a derived copy of the relevant data, keeps it fresh, reuses it where it can, and wires it into serving—so your model gets faster, cheaper, and more searchable without you managing any of the machinery behind it.
There are two kinds of derived copy, each opted into at its natural grain:
| Derived copy | Goal | What you opt in | Annotation |
| ---------------------- | ------------------------------------- | --------------- | --------------------------- |
| **Materialized table** | Make a source fast and cheap to query | A source | `#@ persist` on the source |
| **Search index** | Make a dimension's values searchable | A dimension | `#(index)` on the dimension |
Both are derived from your published model, kept fresh by Credible, and reused automatically. You choose *what* to persist and *what* to make searchable; Credible owns everything else.
## Serving Behavior
A query on a persisted source **serves from the derived copy when one exists, and otherwise runs live** against your warehouse. The result is always correct—if a derived copy isn't available yet, the query is simply slower, not wrong.
* A **materialized table** persists a source's data as a physical table and routes queries to it.
* A **search index** embeds a dimension's values so they are findable by value search, filter suggestions, and the AI agent.
## How Materialization and Indexing Compose
The two features work together automatically, with no extra configuration. If you index a dimension on a source you have also materialized, Credible builds the search index **from the materialized table** instead of re-scanning the warehouse—and reverts to the warehouse if you un-materialize the source.
You can rely on three properties:
* **Consistent** — the index reflects the materialized snapshot, so searchable values match what queries return.
* **Stable** — a table-backed index refreshes exactly when its source table refreshes. An index on a source you have *not* materialized refreshes on publish and on demand, and—if you declare a freshness window—on that window.
* **Cheap** — indexing reuses the table you already built instead of paying to re-scan the warehouse.
The recommended pattern for an expensive, frequently-searched source is **"materialize the source, then index its dimensions."** Credible sequences the work for you so the index is always built after the table it derives from.
## Deciding What to Persist
Queried often or expensive to compute? Add `#@ persist`.
Values that users or the agent search or filter by? Add `#(index)`.
Credible links them (index-from-table) and orders them automatically.
Neither annotation: queried straight from the warehouse, not searchable.
The annotation alone is the intended usage. Defaults are chosen so the platform can optimize on your behalf—deduplicating copies across model versions and scheduling refreshes to meet a freshness objective. At most, add a freshness window for data with a real staleness requirement.
Two guardrails are enforced when you publish an indexed dimension: it may be partitioned by **at most one required filter**, and it may **not** sit on a source that requires parameters. Both surface as publish-time errors rather than silent wrong answers.
## Configuration
### Annotations
Add the annotation to the source or dimension you want to persist. Options are optional `key="value"` pairs; omit them to accept the platform default.
```malloy theme={null}
#@ persist name="orders_fast" refresh="incremental" freshness.window="24h"
source: orders is table('sales.orders') extend {
#(index refresh="incremental" freshness.window="24h")
dimension: status is order_status
}
```
* `name` — choose where the materialized table lands (optional; container-qualifiable).
* `refresh` — `"full"` rebuilds the whole copy; `"incremental"` updates only what changed.
* `freshness.window` — the staleness objective the platform schedules against (see [Freshness](#freshness)).
### Package Manifest
Reuse scope and the refresh cadence are declared once for the whole package in `malloy-publisher.json`:
```json theme={null}
{
"scope": "package",
"materialization": { "freshness": { "window": "24h", "fallback": "live" } }
}
```
* **`scope: package`** (the default) — a derived copy is reused across the package's versions whenever they define the same thing. Maximal reuse, lowest cost.
* **`scope: version`** — each version keeps its own copies, with no cross-version reuse. Choose this when you want to own an exact rebuild schedule for a version.
Declare **either** a `freshness` objective **or** an explicit `materialization.schedule`, never both. A fixed schedule is the power-tier option and is only valid under `scope: version`.
## Freshness
`freshness.window` is an **objective**, not a fixed refresh time: it tells Credible how stale the derived copy is allowed to get, and the platform schedules refreshes to meet it. This lets Credible batch work, run off-peak, and skip a refresh any recent publish or on-demand run already covered.
The `fallback` setting controls what a query does when a materialized table is older than its window—`live` runs the query against the warehouse instead of serving stale data.
Search indexes surface their staleness on the version page and in search and retrieval responses, so the agent can tell when suggestions come from an older snapshot.
## Builds and Refreshes
* **On publish** — Credible builds every persisted source and search index for the new version automatically.
* **On demand** — trigger a **Rerun** from the package page (or the runs API) to force a rebuild. You can rerun a whole version, or a single source or dimension—optionally including its upstream persisted sources.
* **On a schedule** — the platform refreshes derived copies to meet their freshness objectives.
The version page shows **Materialized sources** and **Indexed dimensions** side by side, each with a simple status and its build and refresh history, so you can see at a glance whether a version is fully built.
## Next Steps
How `#(index)` and `#(doc)` power AI discovery
Publish to build your materialized tables and indexes