Prompting Haiku

Cosmo runs its interactive surfaces on Claude Haiku 5.5 (claude-haiku-5-5, the fast tier in the model registry at packages/core/src/llm/model-registry.ts). Haiku is fast and cheap, but it rewards a different prompting style than the larger Opus/Sonnet models: it follows concrete, scoped instructions well and drifts when given open-ended, aspirational prompts. This page is the team reference for writing Haiku prompts. It came out of COS-151.

Where Haiku runs

The chat system prompt is assembled by packages/core/src/llm/prompt-stack.ts (layered, pure functions). Both call sites share the same underlying model, so prompt quality here is the main lever on accuracy, latency, and cost.

Guidelines

1. Be imperative, not aspirational

Haiku executes instructions; it does not reliably infer intent from mission statements. Give it a concrete procedure.

• Avoid: "Your job is not to answer questions. Your job is to move work forward."
• Prefer: a numbered per-turn procedure — decide what's needed → retrieve if org-specific → take the concrete step → end with forward motion.

This is exactly the change made to the identity layer in buildIdentityLayer().

2. Scope the task and fix the output shape

An open-ended "summarize this" lets Haiku pad, preamble, and drift. Tell it which facts to keep and what the output should look like.

• Enumerate what to preserve (decisions + reasons, identifiers, open threads, tool results).
• Forbid meta-commentary explicitly ("no preamble, no closing remarks").
• Ask for a fixed format ("bullet points, not prose").

This is the change made to the compaction prompt in context-compactor.ts.

3. Forbid fabrication in-prompt

Haiku will fill gaps with plausible-sounding invention if you let it. State the rule directly: "If a fact is not present in the conversation, do not add it." This is cheaper and more reliable than post-hoc validation for machine-consumed output (like a compaction summary that becomes the model's only memory of earlier turns).

4. Put the load-bearing instruction first

The system prompt is read top-to-bottom every turn. Keep the identity/procedure at the very top (before the behavior contract and tool listings) so the steering directive lands before the model scans what tools exist. The prompt-stack layer order enforces this; don't reorder it without re-running the identity ordering tests.

5. Keep instructions terse

Every token in the system prompt is re-processed each turn (and re-sent, absent caching). A tight, imperative instruction is both cheaper and better-followed than a long aspirational paragraph. Concision is a quality lever here, not just a cost one.

6. Prefer structure the model can execute

Numbered steps, bulleted "keep this / drop that" lists, and explicit enums beat prose. Haiku maps them directly onto its output; prose it has to interpret.

Before / after

Compaction prompt (context-compactor.ts)

Before — open-ended, invites preamble and drift:

System: You are a conversation summarizer. Compress the conversation into a
        brief summary preserving all key context.
User:   Summarize this conversation history in a compact format. Preserve key
        facts, decisions, and context that would be needed to continue the
        conversation. Be concise but complete.

        <conversation>

After — scoped, fixed output shape, no-fabrication rule:

System: You compact conversation history so an assistant can continue the
        conversation without re-reading the original messages. Preserve every
        load-bearing fact; drop pleasantries and filler. Output only the summary
        itself — no preamble ("Here is…"), no closing remarks, no commentary.
User:   Compact the conversation below into terse notes the assistant will rely
        on as its only memory of these turns.

        Keep, and never invent:
        - Decisions made and their stated reasons.
        - Concrete identifiers: names, IDs, file paths, URLs, numbers, dates.
        - Open threads: unfinished tasks, unanswered questions, next steps.
        - Tools that were used and what they returned, when it affects what
          comes next.

        Drop greetings, acknowledgements, and restated context. Write bullet
        points, not prose. If a fact is not present in the conversation, do not
        add it.

        Conversation:
        <conversation>

Measured win: on the eval set (context-compactor.eval — see below), the scoped prompt drops preamble lines entirely, retains identifiers/decisions more reliably, and produces shorter summaries (fewer output tokens for the same retained facts). Because the summary is fed back as the model's only memory of the compacted turns, higher fidelity here directly improves subsequent-turn accuracy.

Identity layer (prompt-stack.ts)

Before — aspirational framing:

Your job is not to answer questions. Your job is to move work forward.
You are proactive, specific, and bias toward action. ...

After — a concrete per-turn procedure:

Move work forward; don't just answer. Be proactive, specific, and bias toward
action: ...

Each turn, work in this order:
1. Decide what the request actually needs — an answer, an action, or facts you
   must retrieve first.
2. If it depends on org-specific facts, retrieve before answering.
3. Take the concrete step: call the right tool, or give a direct answer.
4. End with forward motion — a proposed next step or a heads-up — unless the
   request was purely informational.

Measured win: the numbered scaffold makes Haiku's retrieve-then-act and end-with-forward-motion behavior more consistent turn-to-turn (it aligns the identity layer with the retrieval-first and advisory contracts that follow), without adding prompt tokens the model must re-interpret.

Checklist for a new Haiku call site

• Imperative instructions, not a mission statement.
• Task scoped: state exactly what to produce and what to keep/drop.
• Output shape fixed (format, sections, "no preamble").
• Explicit no-fabrication rule if the output is machine-consumed.
• Load-bearing instruction first.
• As terse as it can be while staying unambiguous.
• A lightweight eval/regression test locking in the win.