AI

config.ai ships a full AI product on the shared Hono backend: a streaming chat assistant, automated runs on a cron cadence, and capabilities that let the assistant operate the app for the user. Nothing renders until config.ai.enabled is true.

• One Mastra agent (packages/api/src/mastra/agents/assistant/config.ts) serves chat and cloud automated runs alike. Author it with Mastra's own conventions - see Agents.
• Two independent model switches: built-in AI (builtin, billed from credits) and user API keys (byok, model calls never metered). Turn on either or both.

Where it lives

Chat and Automations lead the sidebar; AI configuration sits under Settings. Only surfaces whose flag is on ever render.

Where models come from

• Bring your own key covers every model call: chat, compaction summaries, titles and cloud automations. The one exception is a desktop coding-CLI run, which stays on the CLI's own subscription.
• A held key always wins. A built-in pick runs on the workspace's OpenRouter key when there is one, and on credits otherwise. An own-key pick always runs on a stored key and fails with "add your key" when it is gone, never moving onto credits. Scheduled runs read the key when they fire. One decision point (resolveChatRun in packages/api/src/ai/byok.ts) serves every lane.
• Inside an organization, the owner's keys serve everyone. Keys belong to the workspace's payer, the same person billing charges: in an organization, every member's chats, automations and titles run on the owner's keys, and a member's own keys apply only in their personal workspace. Only the owner sees (last four), replaces or removes them; members' pickers show which providers are connected, and their Models page says the owner manages the keys. Handing the organization over moves the keys with the funding, and a removed member stops using them on their next request.
• Pricing needs no upkeep. Built-in rates come from OpenRouter's live listing (refreshed every 5 minutes, applied at run start) times your pricingConfig.credits markup. An unpriceable model is rejected, never billed at a guess.
• Zero setup for end-users. Set config.ai.defaults (default and fallback model as plain OpenRouter ids, plus efforts) and a fresh user arrives pre-configured. Their own stored choice always wins. Conversation titles are a separate config.ai.titles decision, never a user setting.

Metering is structural, not a setting

There is no metered flag and nothing sniffs a provider name: a call draws credits only if the MODEL OBJECT it runs on was built to.

The middleware settles per model call, so every step of an agent loop pays. Before each call it prices that call's own input (prompt plus tool schemas) and refuses it when the balance cannot cover the input plus a short answer, then clamps maxOutputTokens to what is left AFTER the input (and never above the room the context window leaves nor the model's own output limit from the OpenRouter listing; there is no fixed per-step cap, so a model may use its full output when the balance pays for it) - so a dust balance cannot buy a long prompt whose input alone outruns it. A non-empty balance that cannot cover the message in front of it is refused the same way, before any model call: the user is told this message costs more than they have left rather than that a run stopped, since none started.

Chat

POST /ai/chat (authenticated, rate-limited) resolves the run mode and calls getMastra().getAgent("assistant").stream(...), passing the resolved model through assistantRunModel(model) (a per-call override plus the request context, so Mastra shapes tool schemas for the provider that actually runs) and the capability tools as a named app toolset while Mastra persists each turn to a memory thread. GET /ai/models feeds the picker.

• Two surfaces, one session. A full /chat page and an app-wide dock hand the same conversation back and forth; the empty state offers starter chips (config.ai.chat.suggestions, empty by default).
• Reasoning effort is per chat and offers each model's OWN advertised levels; send reasoningEffort in the request body. Web search is a composer toggle whose request flag is tri-state (see below): omit webSearch and the server decides (its own tools when an engine is configured, never the provider-native lane), send true to add that lane as the fallback, send false for no searching at all.
• Picking a model sets the account default (PUT /ai/settings), so the next conversation opens on it. A chat reopened from history keeps the model it was held on.
• Switching model or provider keeps the conversation, including a switch to or from a paired CLI. A CLI with no session for the chat reads a bounded summary of earlier turns; a CLI on a device another user owns gets only the newest message.
• A turn typed mid-run is queued, not refused. It is listed above the composer, editable and removable, and sent when the run settles - one per settle, in the order it was typed. A queued turn carries its own attachments, so the composer keeps staging photos, PDFs and text files while the assistant is still answering. Send now on a row stops the running turn (the composer's own Stop, so a metered run settles its partial usage) and sends that one ahead of the rest.
• A failed run parks the queue rather than firing every waiting turn into a dead lane. Submitting again releases the head; Send now releases the row it names.
• A substituted model is never silent. A recoverable failure retries the user's fallback model and names it in the x-ai-fallback-model header, which chat shows inline.
• Debits land on finish, with the stream drained server-side so a disconnected client still pays.

Attachments

The composer takes any file the selected target can actually receive, and offers exactly that - the accepted types are derived from the model's declared input modalities, never from a list built into the UI.

• Three delivery mechanisms, chosen by what the file is. A photo or PDF rides the turn as an attachment where the model declares that modality. A text file is not attached at all - it is read into the message as a fenced, filename-labelled block, exactly as a CLI reads a file into its context, which is why it works on every model without any capability. Anything else needs a filesystem, so it reaches only a local CLI.
• Audio and video ride the same path as PDFs wherever a model declares those modalities (roughly a thousand catalog entries do).
• The desktop local lane is the unfloored one. The user is sitting in front of it, so its CLIs keep their file tools and any file at all can be handed over by path - a zip, a spreadsheet, a binary.
• Unknown means yes for photos, no for everything binary. Image input is the norm; PDF and media input are the exception, so offering them on an undeclared model would fail the turn at the provider with the file already uploaded.

• The text ceiling is a context guard, not just a cost one. Inlined text is billed as ordinary prompt tokens: 3MB of it prices at roughly 786,000 tokens where the same bytes as a PDF reserve 2,000, and it overflows every shipped context window. 64KB is about 16,000 tokens, and the limit is clamped further on a model whose own window is narrow.
• Too large is refused, never truncated. A silently shortened CSV makes the model answer about partial data with nothing saying so. Split the file instead.
• PDFs are never recompressed - a photo is downscaled and re-encoded, but a document has no lossy variant that stays readable.
• A stored attachment is a chip, not a payload. Coding-CLI turns persist only a name and size, so a reopened conversation shows the chip and a replayed turn re-sends nothing. Text files are already part of the message text, so they persist as the message. On the built-in and BYOK lanes an attached turn is stored by Mastra Memory like any other, so a PDF inside the recall window is replayed and billed again, exactly as an attached photo is.
• Billing is the provider's own. No attachment-specific metering: a binary file reserves the same pre-flight allowance an image does, and inlined text is priced as the text it is.

Asking the user a question

The assistant can stop mid-turn and ask, with the answer continuing the same turn. Mastra's ask_user tool suspends the run, the chat renders a card with the options, and answering resumes it - or Skip ends the turn there.

• A suspended run holds nothing open. No stream, no concurrency slot, no tokens. Answering posts resume back to POST /ai/chat, which resumes the run and streams the rest of the turn - billed like any other turn.
• An answer or a skip can only reach its own conversation. The client-supplied runId is matched against runs suspended on that user's own thread first; anything else is a 404.
• Only the person answers. A model that writes resumeData into its own tool call (Mastra reads it as an answer) is ignored: the chat model's tool arguments are stripped of it, and ask_user takes an answer only when the user's own resume request recorded it.
• Skipping closes the run. resume: { runId, skip: true } resumes it with { skipped: true } and stops before the next model step, so the thread records "The user chose not to answer" as the tool's result and the next turn sends a valid history. The card reads as skipped, and so does the question's row in a reopened conversation.
• A new message skips a waiting question. Sending instead of answering closes that question first, then runs the turn normally - so a card left open elsewhere cannot resume a run the conversation has moved past. A send refused for credits closes nothing.
• Automations never ask. A headless run has nobody to answer, so the tool is absent there and its prompt clause with it - it answers with a default or says what it would have needed instead.

Chat titles

A new conversation is named by a model at its first message, in parallel with the answer, so the title lands while the assistant is still writing. Which model is your decision, per lane (config.ai.titles) - never a user setting: set the cheapest sensible model each provider offers.

• Keep every entry cheap. Naming fires on every conversation's first message, abandoned one-liners included. Reasoning is off at the call site; the CLI entries take effort: "minimal" where the CLI spends reasoning by default.
• Remove an entry to keep prefix titles on that lane.
• Once per conversation, never twice. The thread is claimed with the prompt-prefix title before the title model is called, so the early ask closes the gate the settled turn would otherwise name through.
• A rename always wins. Only a title still equal to the auto-derived prompt prefix is replaced, including one renamed while the model is still naming it.
• A title is stored whole, never trimmed to fit: the prompt asks for a short one and the surfaces ellipsize in CSS. A reply past 120 characters is prose, not a name, and is discarded.
• Nothing fails on it, and nothing retries. No title model, an empty balance, a provider failure, or a reply that is not a title all leave the prompt-prefix title in place - and a failed or rejected naming run logs at warn with the reply sample, so it is diagnosable.

Synced chat history

Every conversation is a server-side Mastra Memory thread in your own Postgres, so history follows the signed-in user to every device.

• Workspace-scoped. A thread is stamped with the organization and project it was created under, and the list shows only the active workspace's conversations - switching projects switches the visible history (desktop local CLI chats scope the same way on-device). Threads created before scoping appear in every scope until their next message stamps them.
• Ordinary application rows. Guarded by your database access controls, readable by whoever operates the app.
• Reopens as it looked. A thread stores each turn's text, tool calls, attached thumbnails and the model's reasoning, so a restored conversation shows the same collapsed Thoughts boxes it showed live.
• Thumbnails travel, originals do not. Coding-CLI turns store a 256px thumbnail so a re-sent transcript stays inside the append budget; the full-resolution image is archived in the browser's own IndexedDB (64MB, least-recently-used first) purely so the enlarged view is sharp. Opening that chat on another device shows the thumbnail. A PDF stores only a name + size chip - see Attachments.
• Two kinds of stored reasoning. Reasoning the provider streamed is saved by Memory with its thought signature and replayed on later turns, signature intact. Reasoning a client sends up - a compacted transcript, a coding-CLI turn - has no signature, so it is stored as a display-only part that renders but never reaches a model.
• Bounded twice. config.ai.chats.cap threads per user (default 20, oldest evicted), plus config.ai.chats.apiKeyCap threads started by the user's API keys (default 20, capped apart so a key's chats never evict the user's own), and a long conversation auto-condenses older turns into a recap once it passes 70% of the selected model's own context window.
• The composer's meter reads what the runtime measured, not a character-count guess: the tokens the last model step actually carried, cached prompt included, over the window that step ran against. A lane that reports nothing falls back to a ~4-chars-per-token estimate, which is also what the compaction threshold above compares.
• Compact on demand. The chat menu's Compact runs that recap immediately; it is disabled while every turn still fits the window the compaction keeps (the newest 6).

Web search and extract

Set a vendor key and the assistant gets its own web tools, which it calls only when a question needs them. Set none and nothing changes.

Which engine serves a call, per capability and chosen independently:

• A key's presence is the switch. No key, no tool - the model is never offered something that would fail. Any one key covers both capabilities.
• Naming an engine wins over the order. Set config.ai.webTools.searchEngine / extractEngine to "tinyfish", "firecrawl" or "parallel"; a named engine whose key is missing ships no tool on that lane rather than routing to another vendor.
• What one call costs the user: the vendor price x credits.creditsPerUsd x the webSearch / webExtract markup, which ships at 100 (twice cost). Override the vendor price with searchCostUsd / extractCostUsd once you know your plan rate, and your number wins on every engine.
• TinyFish calls are free end to end. A $0 call debits nothing, writes no ledger row, and is never refused for an empty balance, so an out-of-credits user still gets a searching chat. Set a price on a busy app and it becomes an ordinary priced call.
• Where they run: chat (see below), the desktop app-tools lane, automated runs and the external agent. Ledger tags follow the surface: ai.chat, ai.automation, ai.mcp.
• Chat searches by default. A user who has never touched the composer toggle gets these tools whenever an engine is configured, and the model decides per turn - a turn it answers from memory costs nothing extra. Turning the toggle off is remembered and switches them off for that browser. The clients learn whether an engine exists from webTools on GET /ai/models, so a deployment with no key shows the toggle OFF rather than promising a search it cannot run.
• The provider-native lane still needs an explicit yes, and is the fallback for two cases, both of which start with the user turning the toggle ON: no engine is configured at all, or an engine is configured but this run's tools were withheld (the payer lookup failed - see below). Either way our own tools did not register, so there is nothing to suppress it. OpenRouter bills it per REQUEST (~$0.0175) whether the model needed it or not, so an unanswered toggle never reaches it, and a run that DID register our tools suppresses it outright.
• It researches, it does not just look up. The tool descriptions and a system clause (aiPrompts.webResearch, injected only on a turn that actually got web_search) tell the model to refine its query and search again when the first results do not settle a question, and to read a page when snippets conflict. Expect a hard question to cost several calls and an easy one to cost none.
• Billed like model usage, on every lane. Each call is a credits line item (Web search: 3 results, Web extract: example.com). A repeated search or page is served from a 15-minute cache and costs nothing, as does a call the vendor refused.
• User-key runs pay for tools, not for tokens. This is the one place BYOK draws credits: the model call spends the user's own provider key, but a search spends YOURS. A project with credits disabled meters nothing.
• Margin them apart from AI usage with credits.markupByCategory, which gives webSearch and webExtract their own rate and leaves the cost honest. Adding a provider is one file implementing WebSearchEngine or WebExtractEngine plus one line in packages/api/src/ai/web-tools/registry.ts, placed by price: the order is auto-detect precedence.

Automated runs

An automation is a saved prompt that fires on a cadence, running the same integration, capability and web tools as chat but headless - same agent (assistant.generate(...), threadless), same model resolution, same metering, so it carries the agent's persona and the paths never drift. Users manage them on /automations as one list, each row badged with its lane; a card opens a detail dialog (prompt, last result, model, device pinning, edit, run-now, delete, and a history modal of the last 25 runs with their outcome and credit cost). They fire on the shared background jobs cron.

And, with the desktop app:

Lanes

An automation's lane decides which store owns it, and therefore who fires it, what it costs, and whether it runs with the desktop app closed.

The author picks the lane first. The create form leads with it and the model picker then offers only that lane's models, so the two most consequential facts about an automation - what it costs and when it stops firing - are chosen rather than inferred from a model key.

A local-cli row is a definition, not something this backend runs. Both frontends render it as ONE row: on a desktop the row carries that machine's own run state and its switch turns the automation on there, and on the web it states that it runs on the desktops where members turn it on.

Built-in automations

Declare automations once in packages/config/src/ai/automations.ts (@repo/config/ai/automations) and every user gets them. The registry is aiAutomations, an array of AiAutomationSpec; the boilerplate ships one disabled example.

• Materialization. Each spec becomes one row per USER with a deterministic id builtin:<id>:<userId> - written when that user first lists their personal automations and, for an enabled spec, by the sweep, so an always-on automation fires even for a user who never opens the page. Never per organization: a second copy would run and bill the same built-in twice per cadence.
• Registry-owned. Users may toggle a built-in (unless toggleable: false) but never edit it, delete it, or change its model. The store enforces every lock, not just the UI: a hidden prompt serves as null, a forced enabled is clamped on read and re-forced each sweep, and a pinned model is rewritten back on drift.
• hidden is a list decision only. The row still materializes, still fires, and stays reachable by id (run history, Run now); it is dropped where the list is served, so use it for background work the user has no decision to make about - which is why it must be forced on.
• Source of truth after deployment. The sweep re-aligns rows to the current specs each tick, so edits propagate within minutes and removing an entry retires its rows. Changing an id retires and re-creates - treat ids as permanent.
• Validated at startup. A broken cadence, a non-kebab-case or duplicate id, an incoherent hidden entry, or defaults for a surface the entry does not target fails the app at boot, naming it.

Capabilities

Capabilities are the tools the assistant uses to operate the app, so a user can ask for a thing instead of clicking through the UI. The shipped set manages automations and integrations and reads the acting user's own account - their profile, plan and credit balance, and model preferences; every call is scoped to the acting user and audited, and runs without a prompt unless it needs approval. Extend it with defineCapability({ name, description, inputSchema, execute }) in a domain file under packages/api/src/ai/capabilities/, spread into APP_CAPABILITIES (packages/api/src/ai/app-capabilities.ts) - it reaches chat and automated runs at once.

Approve before it runs

A capability that deletes data or spends money declares needsApproval; the chat then pauses on an Approve / Deny card and runs the call only after the click. The shipped delete_automation and run_automation (spends credits) do in the in-app chat - an outside agent over MCP runs them directly, as an API key may - and disconnect_integration (deletes stored credentials) does everywhere, as does every integration MCP tool declared destructive. Reversible edits (update_automation, the model settings) run unprompted.

defineCapability<{ id: string }>({
	name: "delete_record",
	needsApproval: true, // or (input, ctx) => boolean | Promise<boolean>, per call
	approvalPreview: async ({ id }, ctx) => {
		const row = await findRecord(ctx.userId, id); // resolve on the SERVER
		if (!row) return { items: [], refused: true };
		return { items: [{ label: "Delete record", target: { id: row.id, label: row.name } }] };
	},
	// ...description, inputSchema, execute
});

• A rule may read the run context. needsApproval as a function gets the input and the run context and may be async; it is asked again when the call runs, and a rule that throws asks.
• Write approvalPreview from the server's state. Resolve each target yourself, read before from it, and keep label a fixed string. One target twice, more than 10 changes or an over-long value is offered as Deny only.
• A new message cancels a waiting card. The call does not run, and the model is told it was not approved.
• The card settles only once the server takes the click. A call that changed since it was drawn (409 approval_stale) offers Deny only; a busy run slot (429 / 503) says so and stays answerable.
• Clients opt in. The web and desktop apps send approvals: true; a chat client that draws no card gets an immediate refusal instead of a run parked on an invisible card.

The model-primitive toolkit: @repo/ai

@repo/ai re-exports the AI SDK v6 core (ai@^6) provider-agnostically - the model primitives Mastra itself runs on, and your toolkit for a bespoke endpoint or a custom agent tool.

Writing your own AI surface

Your own endpoint gets BYOK and metering for free by resolving its model through the same seam the shipped surfaces use.

Build on a resolved model, and give it the run's own id so each call settles under its own key:

import { streamText, convertToModelMessages } from "@repo/ai";
import { Billing } from "@repo/payments";
import { resolveChatRun } from "../../../ai/byok"; // packages/api/src/ai/byok.ts

const { resolved } = await resolveChatRun(session.user.id, config.ai!, {
  model: userSelection,
  organizationId: session.session.activeOrganizationId ?? null,
  runId: crypto.randomUUID(),
  feature: "ai.my_surface",
  resolveBilling: () => Billing(session)
});

const result = streamText({
  model: resolved.model,
  messages: await convertToModelMessages(uiMessages)
});

return result.toUIMessageStreamResponse();

Work no user is behind, such as pnpm translate running from a git hook on your own key, lives in its own package outside that rule.

Frequently asked questions

• Which flag turns AI on? config.ai.enabled in packages/config/src/index.ts - see configuration. With it false, /ai/* returns 400 (aiEnabledGuard) and no AI UI renders.
• Credits or BYOK? builtin to resell AI with a markup, byok to let users pay their provider directly, both to offer either. These are the web account's keys - keys added in the desktop app stay encrypted on that device.
• Where do model calls run? Always on the shared backend, so no key reaches the browser. Streaming and client hooks for your own endpoints are framework-specific - see data fetching.
• How do I upgrade Mastra? Stay on the pinned @mastra/* 1.x line: run Mastra's codemod for a breaking change, then the full test suite.