Skip to main content
The Intelligent Context Engine is the retrieval system at the core of the Credible platform. It enables AI agents to understand and query your data through natural language by indexing the metadata in your semantic models and finding semantic matches to user questions.

​
How It Works

The system operates in three stages: indexing your semantic model metadata, retrieving relevant entities when queries come in, and generating suggested Malloy queries to address the question.

​
Indexing

When you publish a semantic model to Credible, the platform indexes the metadata encoded in your model to make it searchable. Documentation tags (#(doc)) capture human-readable definitions for each field. These descriptions are indexed and searchable with natural language, allowing AI agents to find data entities based on meaning rather than exact field names. 
source: programs is table('content.programs') {
  dimension:
    #(doc) """The title of the TV program or movie"""
    title is program_name

    #(doc) """The content genre (e.g., Drama, Comedy, Sports, News)"""
    genre is content_category
}
Value indexing (#(index_values)) tells the platform to extract and index distinct values from specific columns. This enables discovery when users search for data values rather than field names. 
source: programs is table('content.programs') {
  dimension:
    #(index_values) 10000
    #(doc) """The title of the TV program or movie"""
    title is program_name
}
Why value indexing matters: Consider a media database where the column is named program_name but contains values like “World Cup Finals” or “MLS Championship”. When a user asks about “soccer games”, the system needs to search the actual values—not just the column name—to find relevant content. Value indexing makes this possible.

​
Retrieval

When a natural language query comes in, the retrieval API finds semantic matches across your indexed models. This is powered by state-of-the-art embedding models—the same technology that powers LLMs—which map related natural language concepts together. This means an AI agent can semantically search for “soccer games” and correctly retrieve both “World Cup Finals” (a program title) and “Sports” (a genre) without exact string matching or prior knowledge of the schema.

​
Query Generation

The retrieval system doesn’t just find relevant data—it also generates a Malloy query intended to address the scope of the natural language request. This gives agents a starting point for analysis that they can refine based on conversation context.

​
Why This Approach?

This retrieval-first design mirrors how analysts actually approach business questions. When a business user comes to a database, they rarely know the exact query or even what data is available. They might have:
  • A goal (“understand customer churn”)
  • A topic (“Q4 sales performance”)
  • A question (“why did revenue drop?”)
  • A hunch (“I think product returns are up”)
The traditional approach requires them to understand the schema first, then translate their question into the right tables, columns, and joins. This creates friction and often requires back-and-forth with data teams. The Intelligent Context Engine enables a different workflow:
  1. Understand the question - The system interprets what the user is trying to learn
  2. Discover relevant data - Semantic search finds data entities that might help, even with imprecise terminology
  3. Suggest analysis approaches - The agent receives not just data, but potential ways to analyze it
  4. Iterate naturally - The agent can refine queries, ask follow-ups, or explore related dimensions
This approach allows AI agents to have the same exploratory, iterative conversations that analysts have—discovering what’s possible before committing to a specific query.

​
Optimizing Your Models for Retrieval

To get the best results from the Intelligent Context Engine:
  1. Add #(doc) tags to dimensions and measures with clear, business-friendly descriptions
  2. Use #(index_values) on high-cardinality dimensions where users might search by value (product names, customer names, titles, etc.)
  3. Write descriptions in natural language - describe what the field means to a business user, not just its technical definition
The richer your metadata, the better the retrieval system can match user intent to your data.