AI

config.ai ships a full AI product on the shared Hono backend: a streaming chat assistant, automated runs on a cron cadence, and capabilities that let the assistant operate the app for the user. Nothing renders until config.ai.enabled is true.

• One Mastra agent (packages/api/src/mastra/agents/assistant/config.ts) serves chat and cloud automated runs alike. Author it with Mastra's own conventions - see Agents.
• Two independent model switches: built-in AI (builtin, billed from credits) and user API keys (byok, model calls never metered). Turn on either or both.

Where it lives

Chat and Automations lead the sidebar; AI configuration sits under Settings. Only surfaces whose flag is on ever render.

Where models come from

• The pane the user picks is the billing choice, encoded in the selection key: a built-in pick always draws credits, an own-key pick always runs on their key and fails with "add your key" when it is gone. Runs on a paired runner CLI are never metered either - the user's own subscription executes them. One decision point (resolveChatRun in packages/api/src/ai/byok.ts) serves chat and cloud automations.
• Pricing needs no upkeep. Built-in rates come from OpenRouter's live listing (refreshed every 5 minutes, applied at run start) times your pricingConfig.credits markup. An unpriceable model is rejected, never billed at a guess.
• Zero setup for end-users. Set config.ai.defaults (default and fallback model as plain OpenRouter ids, plus efforts) and a fresh user arrives pre-configured. Their own stored choice always wins. Conversation titles are a separate config.ai.titles decision, never a user setting.

Metering is structural, not a setting

There is no metered flag and nothing sniffs a provider name: a call draws credits only if the MODEL OBJECT it runs on was built to.

The middleware settles per model call, so every step of an agent loop pays, and clamps maxOutputTokens to what the balance can buy.

Chat

POST /ai/chat (authenticated, rate-limited) resolves the run mode and calls getMastra().getAgent("assistant").stream(...), passing the resolved model as a per-call override and the capability tools as a named app toolset while Mastra persists each turn to a memory thread. GET /ai/models feeds the picker.

• Two surfaces, one session. A full /chat page and an app-wide dock hand the same conversation back and forth; the empty state offers starter chips (config.ai.chat.suggestions, empty by default).
• Reasoning effort is per chat and offers each model's OWN advertised levels; send reasoningEffort in the request body. Web search is a composer toggle, OFF by default: send webSearch: true to let the model search during that turn (see below). An absent flag never searches.
• Picking a model sets the account default (PUT /ai/settings), so the next conversation opens on it. A chat reopened from history keeps the model it was held on.
• A substituted model is never silent. A recoverable failure retries the user's fallback model and names it in the x-ai-fallback-model header, which chat shows inline.
• Debits land on finish, with the stream drained server-side so a disconnected client still pays.

Attachments

The composer takes any file the selected target can actually receive, and offers exactly that - the accepted types are derived from the model's declared input modalities, never from a list built into the UI.

• Three delivery mechanisms, chosen by what the file is. A photo or PDF rides the turn as an attachment where the model declares that modality. A text file is not attached at all - it is read into the message as a fenced, filename-labelled block, exactly as a CLI reads a file into its context, which is why it works on every model without any capability. Anything else needs a filesystem, so it reaches only a local CLI.
• Audio and video ride the same path as PDFs wherever a model declares those modalities (roughly a thousand catalog entries do).
• A dispatched runner run has no filesystem, and that is the point. It is floored - the CLI's file tools are denied outright - so a staged path could never be opened there. Only what a CLI receives natively crosses that wire, which is why Codex takes photos but not PDFs (its document mechanism is a file path) while Claude Code takes both.
• The desktop local lane is the unfloored one. The user is sitting in front of it, so its CLIs keep their file tools and any file at all can be handed over by path - a zip, a spreadsheet, a binary.
• Unknown means yes for photos, no for everything binary. Image input is the norm; PDF and media input are the exception, so offering them on an undeclared model would fail the turn at the provider with the file already uploaded.
• A runner turn is refused, never quietly stripped, with runner_images_unsupported / runner_documents_unsupported shown as a localized notice.

• The text ceiling is a context guard, not just a cost one. Inlined text is billed as ordinary prompt tokens: 3MB of it prices at roughly 786,000 tokens where the same bytes as a PDF reserve 2,000, and it overflows every shipped context window. 64KB is about 16,000 tokens, and the limit is clamped further on a model whose own window is narrow.
• Too large is refused, never truncated. A silently shortened CSV makes the model answer about partial data with nothing saying so. Split the file instead.
• PDFs are never recompressed - a photo is downscaled and re-encoded, but a document has no lossy variant that stays readable.
• A stored attachment is a chip, not a payload. Coding-CLI and runner turns persist only a name and size, so a reopened conversation shows the chip and a replayed turn re-sends nothing. Text files are already part of the message text, so they persist as the message. On the built-in and BYOK lanes an attached turn is stored by Mastra Memory like any other, so a PDF inside the recall window is replayed and billed again, exactly as an attached photo is.
• Billing is the provider's own. No attachment-specific metering: a binary file reserves the same pre-flight allowance an image does, and inlined text is priced as the text it is.

Asking the user a question

The assistant can stop mid-turn and ask, with the answer continuing the same turn. Mastra's built-in ask_user tool suspends the run, the chat renders a card with the options, and answering resumes it.

• A suspended run holds nothing open. No stream, no concurrency slot, no tokens. Answering posts resume back to POST /ai/chat, which resumes the run and streams the rest of the turn - billed like any other turn.
• An answer can only resume its own conversation. The client-supplied runId is matched against runs suspended on that user's own thread before it is resumed; anything else is a 404.
• An unanswered question just stops. The turn never finishes and its snapshot stays in storage. Nothing retries it and nothing is charged.
• Automations never ask. A headless run has nobody to answer, so the tool is absent there and its prompt clause with it - it answers with a default or says what it would have needed instead.

Chat titles

A new conversation is named by a model at its first message, in parallel with the answer, so the title lands while the assistant is still writing. Which model is your decision, per lane (config.ai.titles) - never a user setting: set the cheapest sensible model each provider offers.

• Keep every entry cheap. Naming fires on every conversation's first message, abandoned one-liners included. Reasoning is off at the call site; the CLI entries take effort: "minimal" where the CLI spends reasoning by default.
• Remove an entry to keep prefix titles on that lane.
• Once per conversation, never twice. The thread is claimed with the prompt-prefix title before the title model is called, so the early ask closes the gate the settled turn would otherwise name through.
• A rename always wins. Only a title still equal to the auto-derived prompt prefix is replaced, including one renamed while the model is still naming it.
• A title is stored whole, never trimmed to fit: the prompt asks for a short one and the surfaces ellipsize in CSS. A reply past 120 characters is prose, not a name, and is discarded.
• Nothing fails on it, and nothing retries. No title model, an empty balance, a provider failure, or a reply that is not a title all leave the prompt-prefix title in place - and a failed or rejected naming run logs at warn with the reply sample, so it is diagnosable.

Synced chat history

Every conversation is a server-side Mastra Memory thread in your own Postgres, so history follows the signed-in user to every device.

• Workspace-scoped. A thread is stamped with the organization and project it was created under, and the list shows only the active workspace's conversations - switching projects switches the visible history (desktop local CLI chats scope the same way on-device). Threads created before scoping appear in every scope until their next message stamps them.
• Ordinary application rows. Guarded by your database access controls, readable by whoever operates the app.
• Reopens as it looked. A thread stores each turn's text, tool calls, attached thumbnails and the model's reasoning, so a restored conversation shows the same collapsed Thoughts boxes it showed live.
• Thumbnails travel, originals do not. Coding-CLI and runner turns store a 256px thumbnail so a re-sent transcript stays inside the append budget; the full-resolution image is archived in the browser's own IndexedDB (64MB, least-recently-used first) purely so the enlarged view is sharp. Opening that chat on another device shows the thumbnail. A PDF stores only a name + size chip - see Attachments.
• Two kinds of stored reasoning. Reasoning the provider streamed is saved by Memory with its thought signature and replayed on later turns, signature intact. Reasoning a client sends up - a compacted transcript, a runner CLI turn - has no signature, so it is stored as a display-only part that renders but never reaches a model.
• Bounded twice. config.ai.chats.cap threads per user (default 20, oldest evicted), and a long conversation auto-condenses older turns into a recap once it passes 70% of the selected model's own context window.
• Compact on demand. The chat menu's Compact runs that recap immediately; it is disabled while every turn still fits the window the compaction keeps (the newest 6).

Web search and extract

Set a vendor key and the assistant gets its own web tools, which it calls only when a question needs them. Set neither and nothing changes.

• A key's presence is the switch. No key, no tool - the model is never offered something that would fail.
• One Firecrawl key is the recommended setup. It serves search and extract both, so a single signup gives the assistant web research it can iterate on, and it wins auto-detection whenever several keys are set.
• Parallel is the cheaper alternative on both, at $5 per 1,000 searches on the advanced mode its adapter pins (10 results each) against Firecrawl's ~$0.010-$0.011, and $1 per 1,000 pages extracted against Firecrawl's $0.0032. Opt in per capability with config.ai.webTools.searchEngine / extractEngine: "parallel", or by setting PARALLEL_API_KEY alone. The two knobs are independent, so you can run the cheap engine on one capability and Firecrawl on the other. A named engine whose key is missing ships no tool on that lane rather than routing to the other vendor.
• TinyFish is free at list price on both, rate-limited instead: 30 searches and 150 pages a minute by default, per API key and therefore shared by every user of your deployment, so one user's research loop can hand everyone else a 429. A call bills the user nothing unless you set searchCostUsd / extractCostUsd - on a busy app, set a price or keep a paid engine. Its extract renders the page live. Opt in per capability with config.ai.webTools.searchEngine / extractEngine: "tinyfish", or by setting TINYFISH_API_KEY alone; it never outranks the other two keys.
• It researches, it does not just look up. The tool descriptions and a system clause (aiPrompts.webResearch, injected only on a turn that actually got web_search) tell the model to refine its query and search again when the first results do not settle a question, and to read a page when snippets conflict. Expect a hard question to cost several calls and an easy one to cost none.
• The toggle means "may search", not "search every message". With a search key set, the composer toggle registers the tools and the model decides per turn; a turn it answers from memory costs nothing extra. Without one, an opted-in turn falls back to the provider's NATIVE search, which OpenRouter bills per request (~$0.0175) whether the model needed it or not.
• Every AI surface gets them, gated differently. Chat registers them per turn from the composer toggle. An automated run and the external agent have no toggle to read, so they register whenever a key is set and config.ai.webSearch is not false. Ledger tags follow the surface: ai.chat, ai.automation, ai.mcp.
• Only one lane runs. Configuring your own search engine suppresses the provider-native one, so a turn is never billed for both.
• Billed like model usage — on every lane. Each call is a credits line item (Web search: 3 results, Web extract: example.com) priced through your pricingConfig.credits markup. A repeated search or page is served from a 15-minute cache and costs nothing, as does a call the vendor refused.
• User-key runs pay for tools, not for tokens. This is the one place BYOK draws credits: the model call spends the user's own provider key, but a search spends YOUR vendor key (Firecrawl, Parallel or TinyFish), so the tool call is metered on every lane at the engine's price - which on a free engine is nothing unless you set one. A project with credits disabled meters nothing.
• Change engine or price in config.ai.webTools. Each engine prices its own calls at its vendor's small-plan list price unless you say otherwise, so the defaults track whichever engine actually ran; set searchCostUsd / extractCostUsd once you know your real plan rate, and your number wins on every engine. To margin web tools differently from AI usage, set the per-call cost above your vendor price - the global markup applies on top - or set credits.markupByCategory, which gives webSearch and webExtract their own markup and leaves the cost honest. Adding a provider is one file implementing WebSearchEngine or WebExtractEngine plus one line in packages/api/src/ai/web-tools/registry.ts.

Automated runs

An automation is a saved prompt that fires on a cadence, running the same integration, capability and web tools as chat but headless - same agent (assistant.generate(...), threadless), same model resolution, same metering, so it carries the agent's persona and the paths never drift. Users manage them on /automations as one list, each row badged with its lane; a card opens a detail dialog (prompt, last result, model, device pinning, edit, run-now, delete, and a history modal of the last 25 runs with their outcome and credit cost). They fire on the shared background jobs cron.

| Lane | cloud (the default, stored as no lane at all) or local-cli - see Lanes. |

| Web research | On whenever a vendor key is set and config.ai.webSearch is not false; there is no per-automation toggle. Calls bill the workspace owner, itemized per call, so a chatty automation's cost is visible on the ledger. A user-key automation still pays for its tools - they spend YOUR vendor key. | | How many | A fixed 100 per workspace, the same on every plan - an anti-abuse bound, not a tier feature, so nothing counts it on screen and no upgrade moves it. One creation choke point, so form, chat capability and MCP tool all refuse with 403 ABUSE_LIMIT_REACHED. Counted per workspace across ALL its projects, never per project. Built-ins never count. | | At the concurrency cap | A due run WAITS: it defers to the next 5-minute sweep with nothing charged, nothing metered, and no failure on the row. Run now and the chat capability refuse instead of re-arming - a button press must never become a paid run minutes later. See plan limits. | | End-user authoring | config.ai.userAutomations: false hides the create form, 403s the create route, and withholds the create_automation capability. Listing, running, editing and deleting stay. |

Lanes

An automation's lane decides which store owns it, and therefore who fires it, what it costs, and whether it runs with the desktop app closed.

The author picks the lane first. The create form leads with it and the model picker then offers only that lane's models, so the two most consequential facts about an automation - what it costs and when it stops firing - are chosen rather than inferred from a model key.

A local-cli row is a definition, not something this backend runs. Both frontends render it as ONE row: on a desktop the row carries that machine's own run state and its switch turns the automation on there, and on the web it states that it runs on the desktops where members turn it on.

Built-in automations

Declare automations once in packages/config/src/ai/automations.ts (@repo/config/ai/automations) and every user gets them. The registry is aiAutomations, an array of AiAutomationSpec; the boilerplate ships one disabled example.

• Materialization. Each spec becomes one row per USER with a deterministic id builtin:<id>:<userId> - written when that user first lists their personal automations and, for an enabled spec, by the sweep, so an always-on automation fires even for a user who never opens the page. Never per organization: a second copy would run and bill the same built-in twice per cadence.
• Registry-owned. Users may toggle a built-in (unless toggleable: false) but never edit it, delete it, or change its model. The store enforces every lock, not just the UI: a hidden prompt serves as null, a forced enabled is clamped on read and re-forced each sweep, and a pinned model is rewritten back on drift.
• hidden is a list decision only. The row still materializes, still fires, and stays reachable by id (run history, Run now); it is dropped where the list is served, so use it for background work the user has no decision to make about - which is why it must be forced on.
• Source of truth after deployment. The sweep re-aligns rows to the current specs each tick, so edits propagate within minutes and removing an entry retires its rows. Changing an id retires and re-creates - treat ids as permanent.
• Validated at startup. A broken cadence, a non-kebab-case or duplicate id, an incoherent hidden entry, or defaults for a surface the entry does not target fails the app at boot, naming it.

Capabilities

Capabilities are the tools the assistant uses to operate the app, so a user can ask for a thing instead of clicking through the UI. The shipped set manages automations and integrations and reads the acting user's own account - their profile, plan and credit balance, and model preferences; every call runs with no approval prompt, scoped to the acting user, and audited. Extend it with defineCapability({ name, description, inputSchema, execute }) in a domain file under packages/api/src/ai/capabilities/, spread into APP_CAPABILITIES (packages/api/src/ai/app-capabilities.ts) - it reaches chat and automated runs at once.

The model-primitive toolkit: @repo/ai

@repo/ai re-exports the AI SDK v6 core (ai@^6) provider-agnostically - the model primitives Mastra itself runs on, and your toolkit for a bespoke endpoint or a custom agent tool.

Writing your own AI surface

Your own endpoint gets metering for free if it resolves its model through the same seam the shipped surfaces use, and nothing if it builds a provider model itself.

Build on a resolved model, and give it the run's own id so each call settles under its own key:

import { streamText, convertToModelMessages } from "@repo/ai";
import { Billing } from "@repo/payments";
import { resolveChatRun } from "../../../ai/byok"; // packages/api/src/ai/byok.ts

const { resolved } = await resolveChatRun(session.user.id, config.ai!, {
  model: userSelection,
  runId: crypto.randomUUID(),
  feature: "ai.my_surface",
  resolveBilling: () => Billing(session)
});

const result = streamText({
  model: resolved.model,
  messages: await convertToModelMessages(uiMessages)
});

return result.toUIMessageStreamResponse();

Reach for a provider SDK directly in exactly one case: work you pay for on your own key with no user-facing meter, such as a nightly digest or a moderation pass.

Frequently asked questions

• Which flag turns AI on? config.ai.enabled in packages/config/src/index.ts - see configuration. With it false, /ai/* returns 400 (aiEnabledGuard) and no AI UI renders.
• Credits or BYOK? builtin to resell AI with a markup, byok to let users pay their provider directly, both to offer either. These are the web account's keys - keys added in the desktop app stay encrypted on that device.
• Where do model calls run? Always on the shared backend, so no key reaches the browser. Streaming and client hooks for your own endpoints are framework-specific - see data fetching.
• How do I upgrade Mastra? Stay on the pinned @mastra/* 1.x line: run Mastra's codemod for a breaking change, then the full test suite.