Prompting

What is a Prompt?

A prompt is the text you send to an AI model to tell it what to do. Think of it like giving instructions to a smart assistant.

Simple Prompts vs Agent Prompts

Simple Prompt (ChatGPT style)

simple-prompt.txt

User: What's the weather in New York?
Assistant: I don't have access to real-time weather data...

Agent Prompt (what Daydreams creates)

agent-prompt.txt

You are an AI agent that can:
- Call weather APIs
- Send Discord messages
- Remember conversation history

Current situation:
- User asked: "What's the weather in New York?"
- Available actions: getWeather, sendMessage
- Chat context: user123 in #general channel

Please respond with:
<action_call name="getWeather">{"city": "New York"}</action_call>
<output type="discord:message">It's 72°F and sunny in New York!</output>

The Problem: LLMs Need Structure

Without structure, LLMs can't:

Know what tools they have available
Remember previous conversations
Follow consistent output formats
Handle complex multi-step tasks

Example of what goes wrong:

unstructured-problem.txt

User: "Check weather and send to Discord"
LLM: "I'll check the weather for you!"
// ❌ Doesn't actually call any APIs
// ❌ Doesn't know how to send to Discord
// ❌ Just generates text

The Solution: Structured Prompts

Daydreams automatically creates structured prompts that include:

Available Tools - What the agent can do
Current State - What's happening right now
Response Format - How to respond properly
Context Memory - What happened before

structured-solution.txt

Available Actions:
- getWeather(city: string) - Gets current weather
- sendDiscord(message: string) - Sends Discord message

Current Context:
- User: user123
- Channel: #general
- Previous messages: [...]

New Input:
- "Check weather in Boston and send to Discord"

Respond with XML:
<action_call name="getWeather">{"city": "Boston"}</action_call>
<output type="discord:message">Weather in Boston: 65°F, cloudy</output>

How Daydreams Builds Prompts

Every time your agent thinks, Daydreams automatically builds a prompt like this:

1. Instructions

instructions-section.txt

You are an AI agent. Your job is to:
- Analyze new information
- Decide what actions to take
- Respond appropriately

2. Available Tools

tools-section.xml

<available-actions>
  <action name="getWeather">
    <description>Gets current weather for a city</description>
    <schema>{"type": "object", "properties": {"city": {"type": "string"}}}</schema>
  </action>
</available-actions>
 
<available-outputs>
  <output type="discord:message">
    <description>Sends a message to Discord</description>
    <schema>{"type": "string"}</schema>
  </output>
</available-outputs>

3. Current Context State

context-section.xml

<contexts>
  <context type="chat" key="user123">
    Previous messages:
    user123: Hi there!
    agent: Hello! How can I help?
    user123: What's the weather like?
  </context>
</contexts>

4. What Just Happened

updates-section.xml

<updates>
  <input type="discord:message" timestamp="2024-01-15T10:30:00Z">
    What's the weather in Boston?
  </input>
</updates>

5. Expected Response Format

response-format.xml

Respond with:
<response>
  <reasoning>Your thought process here</reasoning>
  <action_call name="actionName">{"argument": "value"}</action_call>
  <output type="outputType">Your response here</output>
</response>

What the LLM Sees (Complete Example)

Here's what a complete prompt looks like:

complete-prompt.txt

You are an AI agent. Analyze the updates and decide what to do.

<available-actions>
  <action name="getWeather">
    <description>Gets current weather for a city</description>
    <schema>{"type": "object", "properties": {"city": {"type": "string"}}}</schema>
  </action>
</available-actions>

<available-outputs>
  <output type="discord:message">
    <description>Sends a message to Discord</description>
    <schema>{"type": "string"}</schema>
  </output>
</available-outputs>

<contexts>
  <context type="chat" key="user123">
    user123: Hi there!
    agent: Hello! How can I help?
  </context>
</contexts>

<working-memory>
  <!-- Previous actions from this conversation -->
</working-memory>

<updates>
  <input type="discord:message" timestamp="2024-01-15T10:30:00Z">
    What's the weather in Boston?
  </input>
</updates>

Respond with:
<response>
  <reasoning>Your thought process</reasoning>
  <action_call name="actionName">{"arg": "value"}</action_call>
  <output type="outputType">Your message</output>
</response>

LLM Response Example

The LLM responds with structured XML:

llm-response.xml

<response>
  <reasoning>
    The user is asking about weather in Boston. I should:
    1. Call the getWeather action to get current conditions
    2. Send the result to Discord
  </reasoning>
 
  <action_call name="getWeather">{"city": "Boston"}</action_call>
  <output type="discord:message">Checking the weather in Boston for you!</output>
</response>

Daydreams automatically:

Parses the <action_call> and runs the weather API
Parses the <output> and sends the Discord message
Saves the <reasoning> for debugging

Advanced Features

Template References

LLMs can reference previous action results within the same response:

template-example.xml

<response>
  <reasoning>I'll get weather, then send a detailed message</reasoning>
 
  <action_call name="getWeather">{"city": "Boston"}</action_call>
 
  <output type="discord:message">
    Weather in Boston: {{calls[0].temperature}}°F, {{calls[0].condition}}
  </output>
</response>

The {{calls[0].temperature}} gets replaced with the actual weather data.

Multi-Context Prompts

When multiple contexts are active:

multi-context.xml

<contexts>
  <context type="chat" key="user123">
    Chat history with user123...
  </context>
 
  <context type="game" key="session456">
    Current game state: level 5, health 80...
  </context>
</contexts>

Key Benefits

Consistency - All agents use the same reliable prompt structure
Clarity - LLMs always know what tools they have and how to use them
Memory - Context and conversation history included automatically
Debugging - You can see exactly what the LLM was told
Extensibility - Easy to add new actions and outputs

Customizing Prompts

You can customize prompts in your contexts:

custom-instructions.ts

const chatContext = context({
  type: "chat",
  schema: z.object({ userId: z.string() }),
 
  // Custom instructions for this context
  instructions: (state) =>
    `You are helping user ${state.args.userId}. Be friendly and helpful.`,
 
  // Custom context rendering
  render: (state) => `
    Chat with ${state.args.userId}
    Recent messages: ${state.memory.messages.slice(-3).join("\n")}
    User mood: ${state.memory.userMood || "neutral"}
  `,
});

Key Takeaways

Prompts are automatically generated - You don't write them manually
Structure enables capabilities - Tools, memory, and context included automatically
LLMs respond with XML - Parsed automatically into actions and outputs
Templates enable complex flows - Reference previous results within responses
Customizable per context - Add specific instructions and state rendering

The prompting system is what makes your agent intelligent - it provides the LLM with everything needed to understand the situation and respond appropriately.

On this page