Chat Module

The Chat module exposes two endpoints: a streaming chat endpoint and a model listing endpoint. Source: packages/server/src/modules/chat/.

REST Endpoints

`ChatController` (`chat.controller.ts`)

Method	Path	Guards	Description
POST	`/chat`	ClerkAuthGuard + QuotaGuard	Stream LLM response via SSE

`ModelsController` (`models.controller.ts`)

Method	Path	Guards	Description
GET	`/models`	ClerkAuthGuard	List available LLM models

Chat Request

POST /api/v1/chat accepts:

Field	Type	Required	Description
`message`	string	Yes	User's message
`history`	`HistoryMessage[]`	No	Prior conversation turns
`tools`	`ToolDefinition[]`	No	Client-defined tool schemas
`model`	string	No	Model ID (default: `claude-sonnet-4-20250514`)
`maxTokens`	number	No	Max output tokens
`systemPrompt`	string	No	Custom system prompt
`enableWebSearch`	boolean	No	Enable `web_search` built-in tool

`HistoryMessage`

Each history entry has:

{
  role: 'user' | 'assistant';
  content: string | ContentBlock[];  // text, tool_use, tool_result
}

SSE Response

The response is a text/event-stream stream. Each chunk is a JSON object with a type field:

Type	Fields	Description
`text`	`text: string`	Partial text output
`tool_call`	`toolCall: { id, name, args, serverHandled? }`	LLM invoked a tool
`tool_result`	`toolResult: { callId, result }`	Tool execution completed
`error`	`error: string`	Fatal error — stream ends
`done`	`usage: { inputTokens, outputTokens }`	Stream complete

Chat Service (`chat.service.ts`)

`streamChat(request, userId, orgId)`

Async generator that yields ChatChunkDTO objects.

Processing steps:

Validate model ID against the @cosmohq/core model registry.
Serialise history to Anthropic MessageParam[] format:
- Converts tool_use + tool_result blocks from the client's format to Anthropic format.
- Skips server-handled tool calls (e.g. web_search) to avoid re-processing.
Build tool schemas: client-defined tools + optional web_search (if enableWebSearch is true and the model has the web_search capability).
Call anthropic.messages.stream(...) with the full message history.
For each streaming event:
- Yield text deltas as text chunks.
- Yield tool invocations as tool_call chunks.
After stream end, yield done with usage stats.
Call MeteringService.recordLLMUsage(...) with token counts.
Emit a PostHog $ai_generation event with timing and token metrics.

Error Handling

API errors are converted to human-readable messages:

overloaded_error → "Claude is currently overloaded. Please try again in a few moments."
rate_limit_error → "Rate limit exceeded."
Generic → "An error occurred while processing your request."

Models Listing

GET /api/v1/models returns all models registered in @cosmohq/core:

[
  {
    id: "claude-opus-4-7",
    displayName: "Claude Opus 4.7",
    provider: "anthropic",
    tier: "premium",
    contextWindow: 200000,
    maxOutputTokens: 32000,
    capabilities: ["web_search", "extended_thinking"]
  },
  ...
]

On this page