file: ./content/docs/contributing.mdx meta: { "title": "Contributing", "description": "Contributing to Daydreams." } ## Contributing Looking to contribute? We'd love your help, dreamer. If you are a developer and would like to contribute with code, please check out our [GitHub repository](https://github.com/daydreamsai/daydreams) and open an issue to discuss before opening a Pull Request. ## Star History Star History Chart file: ./content/docs/index.mdx meta: { "title": "Your first agent", "description": "Build your first Daydreams agent." } import { Tab, Tabs } from "fumadocs-ui/components/tabs"; > ⚠️ **Warning**: This is alpha software under active development. Expect > frequent breaking changes and bugs. The API is not yet stable. ## Overview Daydreams is a framework for building autonomous AI agents. At its core, an agent operates through a continuous cycle: 1. **Analyzes** incoming information (inputs) 2. **Reasons** about it using a Large Language Model (LLM) 3. **Decides** on the next steps - either generating a response (output) or performing a task (action) 4. **Feeds results** back into the agent's awareness, creating a continuous loop orchestrated by the LLM This enables you to build agents that can interact with various systems like blockchains, social media platforms, APIs, and more, all based on predefined goals and contextual understanding. ## Installation Install the core Daydreams packages: pnpm add @daydreamsai/core @daydreamsai/cli npm install @daydreamsai/core @daydreamsai/cli bun add @daydreamsai/core @daydreamsai/cli yarn add @daydreamsai/core @daydreamsai/cli You'll also need an LLM provider SDK. For this guide, we'll use OpenAI: pnpm add @ai-sdk/openai npm install @ai-sdk/openai bun add @ai-sdk/openai yarn add @ai-sdk/openai **Important:** Make sure you have an `OPENAI_API_KEY` environment variable set before proceeding. ## Core Concepts Daydreams is built around several key components that work together: ### Essential Components * **[Agent Lifecycle](/docs/concepts/agent-lifecycle)** - The central orchestrator that runs the main loop * **[Contexts](/docs/concepts/contexts)** - Manages state and memory for specific tasks or interactions (e.g., a chat session) * **[Inputs](/docs/concepts/inputs)** - How agents receive information (e.g., CLI messages, API events) * **[Outputs](/docs/concepts/outputs)** - How agents respond or send information (e.g., CLI responses, tweets) * **[Actions](/docs/concepts/actions)** - Tasks agents can perform (e.g., calling APIs, executing transactions) * **[Memory](/docs/concepts/memory)** - How agents store and recall information (working memory, episodic memory) For detailed information about these concepts, visit the [Core Concepts](/docs/concepts/core) section. ## Your First Agent (CLI Echo Bot) Let's build a simple agent that echoes back whatever you type in the command line. This example demonstrates the basic structure and workflow of a Daydreams agent. ### Step 1: Set up your project ```bash title="create a new project" mkdir my-first-agent && cd my-first-agent ``` pnpm add @daydreamsai/core @daydreamsai/cli @ai-sdk/openai zod npm install @daydreamsai/core @daydreamsai/cli @ai-sdk/openai zod bun add @daydreamsai/core @daydreamsai/cli @ai-sdk/openai zod yarn add @daydreamsai/core @daydreamsai/cli @ai-sdk/openai zod ### Step 2: Create your agent Create a file named `agent.ts`: ```typescript title="agent.ts" import { createDreams, context, input, output } from "@daydreamsai/core"; import { cliExtension } from "@daydreamsai/cli"; import { openai } from "@ai-sdk/openai"; import { z } from "zod"; // 1. Define the main context for our agent const echoContext = context({ type: "echo", // No specific arguments needed for this simple context schema: z.object({}), // Instructions that guide the LLM's behavior instructions: "You are a simple echo bot. Repeat the user's message back to them.", }); // 2. Create the agent instance const agent = createDreams({ // Configure the LLM model to use model: openai("gpt-4o-mini"), // Include the CLI extension for input/output handling extensions: [cliExtension], // Register our custom context contexts: [echoContext], }); // 3. Start the agent and run the context async function main() { // Initialize the agent (sets up services like readline) await agent.start(); console.log("Echo agent started. Type 'exit' to quit."); // Run our echo context // The cliExtension automatically handles console input/output await agent.run({ context: echoContext, args: {}, // Empty object since our schema requires no arguments }); // Agent stops when the input loop breaks (e.g., user types "exit") console.log("Agent stopped."); } // Start the application main(); ``` ### Step 3: Run your agent Ensure your `OPENAI_API_KEY` environment variable is set, then run: ```bash title="run the agent" node agent.ts ``` Your agent will start listening for input. Type any message and watch as the agent echoes it back using the LLM and CLI handlers provided by the `cliExtension`. *** ## Next Steps Ready to explore more? Check out these resources: * **[Core Concepts](/docs/concepts/core)** - Deep dive into Daydreams architecture * **[Getting Started Guide](/docs/guides/getting-started)** - More complex examples and use cases * **[API Reference](/docs/api)** - Complete documentation of available methods and options file: ./content/docs/advanced/extensions-vs-services.mdx meta: { "title": "Extensions vs Services", "description": "Understanding the difference between extensions and services in Daydreams." } ## Overview The Daydreams framework uses both Services and Extensions as core architectural components, but they serve different purposes. Let's explore the differences: ## Service (Provider) Services are primarily focused on **Dependency Injection (DI) and Lifecycle Management** for specific functionality, often managing external clients or shared utilities. * **Definition:** Created using `service({...})` * **Key Methods:** * `register(container)`: Adds instances or factories to the DI container. This runs *before* booting. * `boot(container)`: Performs asynchronous initialization after all services have been registered. This runs during `agent.start()`. * **Role:** Manages *how* a dependency is created, configured, initialized, and made available via the `container`. * **Analogy:** Think of a service provider as the **blueprint and setup instructions** for a specific tool or utility. ## Extension Extensions act as **modular packages or bundles** that group related Daydreams features together for easy integration into an agent. * **Definition:** Created using `extension({...})` * **Contents:** * `name` (required): Unique identifier * `services`: Array of Service Provider definitions this extension requires * `contexts`: Definitions for specific Context types * `actions`: Action definitions * `inputs`: Input definitions * `outputs`: Output definitions * `events`: Zod schemas for events the extension might emit * `install(agent)`: Optional hook that runs once when the extension is added during `agent.start()` * **Role:** Organizes and encapsulates a set of related functionalities and declares the services those functionalities need. * **Analogy:** Think of an extension as a **toolbox** for a specific domain (e.g., a "Discord Toolbox"). ## Key Differences | Feature | Service (Provider) | Extension | | :---------------- | :------------------------------------- | :---------------------------------------------------------------------- | | **Primary Focus** | Dependency Injection & Lifecycle | Bundling & Organizing Features | | **Defined By** | `service({...})` | `extension({...})` | | **Contains** | `register`, `boot` methods | `actions`, `contexts`, `inputs`, `outputs`, `services`, `install`, etc. | | **Purpose** | Manages *how* a dependency is provided | Packages *what* features are provided | | **Relationship** | Often **included within** an Extension | **Uses** Services to manage its dependencies | ## In Simple Terms * You define a `service` to tell the system *how* to create and initialize something like an API client. * You define an `extension` to bundle together all the `actions`, `contexts`, `inputs`, and `outputs` related to a feature, and you list the `services` that those features depend on. When you add an `extension` to your agent via `createDreams`, the framework automatically registers the `services` listed within that extension, making them available via the `container` for the extension's components to use. file: ./content/docs/advanced/extensions.mdx meta: { "title": "Extensions", "description": "Building your own modular Daydreams extensions." } Extensions are the primary mechanism for packaging and distributing reusable Daydreams functionality. They bundle together contexts, actions, inputs, outputs, and the service providers they depend on. ## Purpose * **Encapsulate Features:** Group all related components for a specific capability (e.g., Discord integration, ChromaDB support, a custom game interface). * **Simplify Agent Configuration:** Add complex features to an agent by simply including the extension in the `createDreams` configuration. * **Promote Code Reuse:** Share common functionalities across different agents. ## Defining an Extension Use the `extension` helper function exported from `@daydreamsai/core`: ```typescript import { extension, context, action, input, output, service, type AnyAgent, // Import necessary types } from "@daydreamsai/core"; import { z } from "zod"; // Assume necessary components like myApiService, myFeatureContext, etc. are defined elsewhere declare const myApiService: any; declare const myFeatureContext: any; declare const myFeatureAction: any; export const myExtension = extension({ // Required: A unique name for the extension name: "my-feature", // Optional: Service providers required by this extension's components. // These services will be registered and booted automatically. services: [myApiService], // Optional: Context definitions provided by this extension. // These become available for use with agent.run() / agent.send(). contexts: { myFeature: myFeatureContext, }, // Optional: Action definitions provided by this extension. // These become available for the LLM to call. actions: [myFeatureAction], // Optional: Input definitions provided by this extension. // Their 'subscribe' methods will be called on agent.start(). inputs: { "my-feature:event": input({ /* ... input definition ... */ subscribe: (send, agent) => { /* ... listen and call send() ... */ }, }), }, // Optional: Output definitions provided by this extension. // These become available for the LLM to use in tags. outputs: { "my-feature:notify": output({ /* ... output definition ... */ handler: (data, ctx, agent) => { /* ... send notification ... */ }, }), }, // Optional: Events defined by this extension (primarily for typing ctx.emit). events: { myEvent: z.object({ id: z.string() }), }, // Optional: Logic to run once when the extension is added during agent.start(). // Useful for one-time setup that doesn't fit the service 'boot' lifecycle. async install(agent: AnyAgent) { console.log("Installing My Feature Extension!"); // Example: agent.container.resolve('logger').info(...) }, }); ``` ## Usage and Lifecycle 1. **Configuration:** Pass your extension instances to `createDreams` in the `extensions` array: ```typescript import { createDreams } from "@daydreamsai/core"; import { myExtension } from "./my-extension"; import { discord } from "@daydreamsai/discord"; // Example built-in const agent = createDreams({ model: /* ... */, extensions: [ myExtension, discord, // Add other extensions ], // ... other agent config }); ``` 2. **Merging:** When `createDreams` initializes, it iterates through the `extensions` array. For each extension, it merges the defined `contexts`, `actions`, `inputs`, `outputs`, and `events` into the agent's central registries, making them available for use. 3. **Service Registration:** It automatically registers all `services` listed within each extension with the agent's `ServiceManager`. 4. **Installation & Booting:** When `agent.start()` is called: * The `install` method of each extension is executed (if defined). * The `ServiceManager` boots all registered services (calling their `boot` methods, ensuring dependencies are ready). * Input `subscribe` methods are called to start listening for external events. Extensions provide a powerful and organized way to structure agent capabilities, making it easy to combine built-in features with your own custom logic and integrations. file: ./content/docs/advanced/grpo-training-export.mdx meta: { "title": "Training Data Export for GRPO", "description": "This guide explains how to export episodic memories as training data for Group Relative Policy Optimization (GRPO) using the Daydreams AI core package." } ## What is GRPO Training? GRPO (Group Relative Policy Optimization) is a reinforcement learning algorithm designed to enhance reasoning capabilities in large language models. It optimizes memory usage and is particularly effective for tasks requiring complex problem-solving, such as: * Mathematical reasoning * Decision-making scenarios * Step-by-step problem solving * Game-based learning environments **Key Benefits of GRPO:** * Improves reasoning capabilities beyond standard fine-tuning * Optimizes memory usage compared to traditional PPO * Particularly effective for complex problem-solving tasks ## Workflow Overview Your Daydreams agent can build reasoning traces for GRPO training by following this structured workflow: 1. **Define Prompt Sources** - Use static datasets or interactive environments 2. **Generate Reasoning Traces** - Create completions that include thought processes 3. **Store and Save Data** - Export in JSONL format compatible with training tools ## Enabling Automatic Export You can configure Daydreams to automatically export training data after each episode: ```typescript import { createDreams } from "@daydreamsai/core"; const agent = createDreams({ model: openai("gpt-4-turbo"), exportTrainingData: true, trainingDataPath: "./grpo-training-data.jsonl", // Optional, defaults to "./training-data.jsonl" // ... other configuration options }); ``` **Note:** If you don't specify `trainingDataPath`, Daydreams will save the data to `./training-data.jsonl` in your project root. ## Manual Export You can manually export all episodes as training data: ```typescript // Export using the default path from your agent configuration await agent.exportAllTrainingData(); // Or specify a custom path await agent.exportAllTrainingData("./custom-path/grpo-training-data.jsonl"); ``` ## Understanding the Data Format for GRPO Daydreams exports training data in JSONL (JSON Lines) format, optimized for GRPO training. Each line contains a JSON object with: ```json { "prompt": "You are in a dark room with a door to the north.", "completion": "I need to find a way out. I should check if the door is locked.\n\nI found the door was unlocked and was able to exit the room." } ``` The format includes: * **prompt**: The observation or context provided to the agent * **completion**: The agent's reasoning process and action results For interactive environments, ensure completions include both reasoning and an explicit action statement: ```json { "prompt": "You are in a dark room with a door to the north.", "completion": "I need to find a way out. I should check if the door is locked.\n\nAction: try opening the door" } ``` ## Creating Custom Training Pairs for GRPO For advanced use cases, you can create custom training data pairs specifically designed for GRPO: ## Optimizing Data for GRPO Training To maximize the effectiveness of your GRPO training data: 1. **Include diverse scenarios** - Ensure your agent encounters a variety of situations 2. **Capture step-by-step reasoning** - The completion should show the agent's thought process 3. **Format actions consistently** - Use patterns like "Action: \[action]" for easy parsing 4. **Balance task difficulty** - Include both simple and complex reasoning challenges ## Customizing the Export Format If you need a different format for your specific GRPO training framework: 1. Create your own formatter function based on the Daydreams utilities 2. Process the episodic memories to match your required format 3. Save the data using your preferred file structure **Example use case:** You might need to add additional metadata fields like task difficulty or domain type to help with training organization. file: ./content/docs/advanced/introduction.mdx meta: { "title": "Introduction", "description": "Deeper dives into Daydreams capabilities." } This section serves as an exploration of the more conceptually difficult aspects surrounding Daydreams, as well as a collection of projects demonstrating such niche usage. * **[Custom Extensions](/docs/advanced/extensions):** Learn how to bundle your own contexts, actions, services, and more into reusable extensions. * **[Services](/docs/advanced/services):** Understand the dependency injection container and service provider pattern for managing dependencies and component lifecycles. * **[GRPO Training Data Export](/docs/advanced/grpo-training-export):** Discover how to export agent interaction data for advanced model training techniques like Group Relative Policy Optimization. * **[Extensions versus Services](/docs/advanced/extensions-vs-services):** Where and when to use services, extensions, and why. > 💭 **WIP**: Some sections or mentions here may be nascent explorations of the > framework's potential. For those truly pushing the envelope, solving the hardest problems, we want your work to be seen. If we haven't seen you yet, the best place to find us is Discord and X. Dreaming. file: ./content/docs/advanced/services.mdx meta: { "title": "Services", "description": "Dependency Injection & Lifecycle Management." } Daydreams utilizes a Dependency Injection (DI) container and a Service Provider pattern to manage dependencies and component lifecycles effectively. ## Dependency Injection (`container.ts`) At the heart of the framework's modularity is a simple Dependency Injection container, created using `createContainer()`. The container is responsible for instantiating and providing access to various services and components throughout the agent's lifecycle. **Purpose:** * Decouples components by removing the need for them to know how to create their dependencies. * Manages the lifecycle of services (e.g., ensuring only one instance of a database client exists). * Makes components like loggers, clients, or configuration easily accessible. **Core Methods:** * `container.register(token, factory)`: Registers a factory function. A *new instance* is created every time `resolve` is called for the `token`. * `container.singleton(token, factory)`: Registers a factory function, but the instance is created *only once* on the first `resolve` call. Subsequent calls return the same instance. * `container.instance(token, value)`: Registers a pre-existing object instance directly. * `container.resolve(token)`: Retrieves the instance associated with the `token`. Throws an error if the token is not registered. * `container.alias(aliasToken, originalToken)`: Creates an alternative name (`aliasToken`) to resolve an existing `originalToken`. ```typescript import { createContainer, Logger, LogLevel } from "@daydreamsai/core"; // Assuming Logger/LogLevel are exported // Assume DatabaseClient exists declare class DatabaseClient { constructor(uri: string | undefined); } const container = createContainer(); // Register a singleton database client container.singleton("dbClient", () => new DatabaseClient(process.env.DB_URI)); // Register a pre-created config object const config = { apiKey: "123" }; container.instance("appConfig", config); // Register a transient logger (new instance each time) container.register( "requestLogger", () => new Logger({ level: LogLevel.DEBUG }) ); // Resolve dependencies const db = container.resolve("dbClient"); const cfg = container.resolve("appConfig"); const logger1 = container.resolve("requestLogger"); const logger2 = container.resolve("requestLogger"); // Different instance from logger1 ``` The main `Agent` instance, `Logger`, `TaskRunner`, and other core components are typically registered within the container when `createDreams` is called. ## Service Providers (`serviceProvider.ts`) While you could register everything directly with the container, Daydreams uses a Service Provider pattern to organize the registration and initialization (booting) of related services, especially within extensions. **Purpose:** * Groups related service registrations. * Provides a dedicated `boot` phase for initialization logic that might depend on other services already being registered (e.g., connecting a client after its configuration is registered). **Defining a Service Provider:** Use the `service` helper function: ```typescript import { service, type Container } from "@daydreamsai/core"; // Assume MyApiClient exists and has a connect method declare class MyApiClient { constructor(config: { baseUrl: string | undefined }); connect(): Promise; } const myApiService = service({ // Optional: Register dependencies into the container. // Runs before the boot phase. register(container: Container) { container.singleton("apiConfig", () => ({ baseUrl: process.env.API_URL })); container.singleton( "apiClient", (c) => new MyApiClient(c.resolve("apiConfig")) ); }, // Optional: Perform initialization logic after registration. // Runs during agent.start() after all services are registered. async boot(container: Container) { const apiClient = container.resolve("apiClient"); await apiClient.connect(); // Example: Connect the client console.log("My API Client Connected!"); }, }); ``` **Lifecycle:** 1. **Registration:** When a service provider is added to the agent (usually via an extension), its `register` method is called immediately by the `ServiceManager` (created internally in `createDreams`). 2. **Booting:** When `agent.start()` is called, the `ServiceManager` iterates through all registered service providers and calls their `boot` methods *after* all `register` methods have completed. Services and the DI container form the backbone for managing dependencies and initializing components within Daydreams agents and extensions. file: ./content/docs/concepts/actions.mdx meta: { "title": "Actions", "description": "Define capabilities and interactions for your Daydreams agent." } Actions are the primary way Daydreams agents perform tasks, interact with external systems (like APIs or databases), and modify their own state. They are essentially functions that the agent's underlying Large Language Model (LLM) can choose to call based on its reasoning and the current situation. Think of actions as the tools or capabilities you give your agent. ## Defining an Action You define actions using the `action` helper function from `@daydreamsai/core`. Here's the essential structure: ```typescript import { action, type ActionCallContext, type AnyAgent, } from "@daydreamsai/core"; import { z } from "zod"; // Define the structure for the action's arguments using Zod const searchSchema = z.object({ query: z.string().describe("The specific search term or question"), limit: z .number() .optional() .default(10) .describe("Maximum number of results to return"), }); // Define the action itself export const searchDatabaseAction = action({ // Required: A unique name used by the LLM to call this action. // Use clear, descriptive names (e.g., verbNoun). name: "searchDatabase", // Required: A clear description explaining what the action does. // This is crucial for the LLM to understand *when* to use this action. description: "Searches the company knowledge base for records matching the query.", // Optional (but highly recommended): A Zod schema defining the expected arguments. // The framework automatically validates arguments provided by the LLM against this schema. // Use `.describe()` on fields to provide hints to the LLM. schema: searchSchema, // Required: The function that executes the action's logic. // It receives validated arguments, context information, and the agent instance. async handler(args, ctx, agent) { // 'args': The validated arguments object, matching the 'schema'. // Type is automatically inferred: { query: string; limit: number } const { query, limit } = args; // 'ctx': The ActionCallContext, providing access to memory, signals, etc. // 'agent': The main agent instance, for accessing services, logger, etc. agent.logger.info( "searchDatabase", `Searching for: "${query}" (limit: ${limit})` ); try { // --- Perform the action's logic --- // Example: Use a service resolved from the agent's container // const db = agent.container.resolve('database'); // const results = await db.search(query, { limit }); // Simulate an async database call await new Promise((resolve) => setTimeout(resolve, 100)); const results = [ { id: "doc1", title: `Result for ${query}` }, { id: "doc2", title: `Another result for ${query}` }, ].slice(0, limit); // --- End of action logic --- // Return a structured result. This is logged and becomes available // to the LLM in subsequent reasoning steps. return { success: true, count: results.length, results: results, message: `Found ${results.length} results for "${query}".`, }; } catch (error) { agent.logger.error("searchDatabase", "Search failed", { query, error }); // Return an error structure if the action fails return { success: false, error: error instanceof Error ? error.message : "Unknown database error", message: `Failed to search database for "${query}".`, }; } }, }); // Type definition for the ActionCallContext if needed elsewhere type SearchDbContext = ActionCallContext; ``` ## Working with Action Arguments (`args`) The `handler` function receives the action's arguments in the first parameter (`args`). These arguments have already been: 1. Generated by the LLM based on your action's `schema`. 2. Parsed and **validated** by the framework against your `schema`. You can directly use the properties defined in your schema within the handler, with type safety if using TypeScript. ## Managing State (`ctx.memory`) Actions often need to read or modify the persistent state associated with the **current context instance** (e.g., the specific chat session or project the agent is working on). You access this state via `ctx.memory`. ```typescript import { action, type ActionCallContext, type AnyAgent, } from "@daydreamsai/core"; import { z } from "zod"; // Define the expected structure of the context's memory interface TaskListMemory { tasks?: { id: string; title: string; status: "pending" | "completed" }[]; } // Assume myContext is defined elsewhere and uses TaskListMemory export const addTaskAction = action({ name: "addTask", description: "Adds a new task to the current project's task list.", schema: z.object({ title: z.string().describe("The title of the new task"), }), handler(args, ctx: ActionCallContext, agent) { // Access the persistent memory for THIS context instance. const contextMemory = ctx.memory; // Initialize the tasks array if it doesn't exist if (!Array.isArray(contextMemory.tasks)) { contextMemory.tasks = []; } const newTask = { id: agent.utils.randomUUIDv7(), // Generate a unique ID title: args.title, status: "pending" as const, }; // --- Modify the context's state --- contextMemory.tasks.push(newTask); // --- State modification ends --- // The changes to ctx.memory are automatically saved // by the framework at the end of the run cycle. agent.logger.info("addTask", `Added task: ${newTask.title}`); return { success: true, taskId: newTask.id, message: `Task "${newTask.title}" added successfully.`, }; }, }); ``` **Important Memory Scopes in `ctx`:** * `ctx.memory`: (Most commonly used) Persistent memory for the **current context instance**. Use this for state related to the specific chat, project, game, etc. * `ctx.actionMemory`: (Less common) Persistent memory tied to the **action definition itself**, across all contexts. Configure via the `memory` option in `action()`. Useful for action-specific counters, rate limits, etc. * `ctx.agentMemory`: Persistent memory for the **main agent context** (if one was defined globally). Use for global agent settings or state. Always choose the memory scope that matches where your state needs to live. ## Associating Actions with Contexts While you can define actions globally when calling `createDreams`, it's often more organized and efficient to associate actions directly with the `Context` they primarily relate to. This makes the action available *only* when that specific context is active and ensures the action handler has direct, typed access to that context's memory. There are two ways to associate actions with a context definition: **1. Using the `actions` property:** ```typescript import { context, action, type ActionCallContext } from "@daydreamsai/core"; import { z } from "zod"; interface MyContextMemory { value: number; } const myContext = context({ type: "myContext", schema: z.object({ id: string() }), create: () => ({ value: 0 }), // ... other context properties // Define actions directly within the context actions: [ action({ name: "incrementValue", description: "Increments the context's value.", schema: z.object({ amount: z.number().optional().default(1) }), // `ctx.memory` is automatically typed as MyContextMemory here handler(args, ctx, agent) { ctx.memory.value += args.amount; return { success: true, newValue: ctx.memory.value }; }, }), ], }); ``` **2. Using the `.setActions()` method (often better for typing):** ```typescript import { context, action, type ActionCallContext } from "@daydreamsai/core"; import { z } from "zod"; interface MyContextMemory { value: number; } const myContextDefinition = context({ type: "myContext", schema: z.object({ id: string() }), create: () => ({ value: 0 }), // ... other context properties }); const incrementAction = action({ name: "incrementValue", description: "Increments the context's value.", schema: z.object({ amount: z.number().optional().default(1) }), // Handler defined separately or inline handler( args, ctx: ActionCallContext, agent ) { // Type hint might be needed here sometimes, but .setActions() helps inference ctx.memory.value += args.amount; return { success: true, newValue: ctx.memory.value }; }, }); // Associate actions using the chained method const myContextWithActions = myContextDefinition.setActions([ incrementAction, // ... other actions specific to myContext ]); ``` **Benefits of Context-Specific Actions:** * **Scoped Availability:** The `incrementValue` action will only appear in the LLM's available actions when an instance of `myContext` is active in the agent's run. * **Typed State Access:** Within the handler, `ctx.memory` is correctly typed according to the context's memory interface (`MyContextMemory` in this example). * **Organization:** Keeps related logic bundled together. **Note:** This same pattern applies to defining context-specific `Inputs` and `Outputs` using `.setInputs({...})` and `.setOutputs({...})`. ## Interacting with External Systems Actions are the natural place to interact with external APIs, databases, or other services. Remember to use `async`/`await` for any I/O operations. ```typescript import { action } from "@daydreamsai/core"; import { z } from "zod"; export const sendNotificationAction = action({ name: "sendNotification", description: "Sends a notification to a user via an external service.", schema: z.object({ userId: z.string().describe("The ID of the user to notify"), message: z.string().describe("The notification message"), }), async handler(args, ctx, agent) { try { // Example: Get a service client from the DI container // const notificationService = agent.container.resolve('notify'); // await notificationService.send(args.userId, args.message); // Simulate async API call agent.logger.debug( "sendNotification", `Simulating notification to ${args.userId}` ); await new Promise((resolve) => setTimeout(resolve, 150)); return { success: true, message: `Notification sent to ${args.userId}.` }; } catch (error) { agent.logger.error("sendNotification", "Failed to send notification", { args, error, }); return { success: false, error: "Service unavailable", message: "Could not send notification.", }; } }, }); ``` ## Important Considerations for Handlers When writing action handlers, keep these points in mind: 1. **Asynchronous Execution:** Handlers often perform I/O. Always use `async`/`await` for promises. The framework runs handlers concurrently up to a configured limit, so they might not execute instantly if the agent is busy. 2. **Cancellation (`ctx.abortSignal`):** For actions that might run for a long time (e.g., complex calculations, polling), you **must** check for cancellation requests. The `ctx.abortSignal` signals if the overall agent run has been cancelled. ```typescript async handler(args, ctx, agent) { for (let i = 0; i < 100; i++) { // Check for cancellation before long-running work if (ctx.abortSignal?.aborted) { agent.logger.warn("longAction", "Action cancelled by signal"); throw new Error("Action cancelled"); // Or return an appropriate state } // Or use throwIfAborted() which throws if cancelled // ctx.abortSignal?.throwIfAborted(); await performStep(i); // Represents potentially long work } // ... } ``` 3. **Retries (`retry` option):** You can configure actions to automatically retry on failure using the `retry` option in the action definition (e.g., `retry: 3`). If you use retries, try to make your handler **idempotent** – running it multiple times with the same `args` should produce the same final state without unintended side effects (e.g., don't create duplicate records). 4. **Error Handling:** Use `try...catch` blocks within your handler to catch errors from external calls or internal logic. Return a structured error response so the agent (or logs) can understand what went wrong. You can also use the advanced `onError` hook in the action definition for centralized error handling logic. ## How the LLM Chooses Actions The LLM doesn't directly execute your code. Instead: 1. The framework presents the `name` and `description` of all available actions to the LLM in its prompt. 2. Based on its instructions and the current context, the LLM decides which action to use. 3. It formulates the arguments for the action based on the `schema` you provided (including any `.describe()` hints). 4. The framework intercepts the LLM's request, validates the arguments against the `schema`, and then securely executes your `handler` function with the validated `args`. Therefore, clear `name`s and detailed `description`s are vital for the LLM to use your actions correctly. ## Best Practices * **Clear Naming & Descriptions:** Make them unambiguous for the LLM. * **Precise Schemas:** Use Zod effectively, adding `.describe()` to clarify arguments for the LLM. * **Structured Returns:** Return objects from handlers with clear success/error status and relevant data. * **Use `async`/`await`:** Essential for any I/O. * **Handle Errors:** Use `try...catch` and return meaningful error information. * **Check for Cancellation:** Implement `ctx.abortSignal` checks in long-running handlers. * **Consider Idempotency:** Especially if using the `retry` option. * **Choose the Right Memory Scope:** Use `ctx.memory` for context instance state unless you specifically need action (`ctx.actionMemory`) or global (`ctx.agentMemory`) state. * **Keep Actions Focused:** Aim for single responsibility per action. * **Use `agent.logger`:** Log important steps and errors for debugging. Actions are the core mechanism for adding custom capabilities and stateful interactions to your Daydreams agents. file: ./content/docs/concepts/agent-lifecycle.mdx meta: { "title": "Agent Lifecycle", "description": "How Daydreams agents process information and execute tasks." } The core of the Daydreams framework is the agent's execution lifecycle. This loop manages how an agent receives input, reasons with an LLM, performs actions, and handles results. Understanding this flow is crucial for building and debugging agents. Let's trace the lifecycle of a typical request: ## 1. Input Reception * **Source:** An external system (like Discord, Telegram, CLI, or an API) sends information to the agent. This is usually configured via an `extension`. * **Listener:** An `input` definition within the agent or an extension listens for these events (e.g., a new message arrives). * **Trigger:** When the external event occurs, the input listener is triggered. * **Invocation:** The listener typically calls `agent.send(...)`, providing: * The target `context` definition (which part of the agent should handle this?). * `args` to identify the specific context instance (e.g., which chat session?). * The input `data` itself (e.g., the message content). ## 2. `agent.send` - Starting the Process * **Log Input:** The framework logs the incoming information as an `InputRef` (a record of the input). * **Initiate Run:** It then calls the internal `agent.run` method to start or continue the processing cycle for the specified context instance, passing the new `InputRef` along. ## 3. `agent.run` - Managing the Execution Cycle * **Load/Create Context:** The framework finds the specific `ContextState` for the target instance (e.g., the state for chat session #123). If it's the first time interacting with this instance, it creates the state and its associated persistent memory (`ContextState.memory`). It also retrieves or creates the temporary `WorkingMemory` for this specific run. * **Handle Concurrency:** It checks if this context instance is already processing another request. If so, the new input is usually added to the ongoing run. If not, it sets up a new run. * **Setup Run Environment:** It prepares the environment for the LLM interaction, gathering all available `actions`, `outputs`, and relevant context information. * **Start Step Loop:** It begins the main processing loop, which iterates through one or more reasoning steps until the interaction is complete. ## 4. Inside the Step Loop - Perception, Reasoning, Action Each iteration (step) within the `agent.run` loop represents one turn of the agent's core reasoning cycle: * **Prepare State:** The agent gathers the latest information, including: * The current persistent state of the active `Context`(s) (via their `render` functions). * The history of the current interaction from `WorkingMemory` (processed inputs, outputs, action results from previous steps). - Any *new* unprocessed information (like the initial `InputRef` or results from actions completed in the previous step). - The list of currently available `actions` and `outputs`. * **Generate Prompt:** This information is formatted into a structured prompt (using XML) for the LLM. The prompt clearly tells the LLM its instructions, what tools (actions/outputs) it has, the current state, and what new information needs attention. (See [Prompting](/docs/concepts/prompting)). * **LLM Call:** The agent sends the complete prompt to the configured LLM. * **Process LLM Response Stream:** As the LLM generates its response token by token: * The framework **streams** the response. * It **parses** the stream, looking for specific XML tags defined in the expected response structure (``, ``, ``). * The LLM's thought process is extracted from `` tags and logged. * Instructions to perform actions (``) or send outputs (``) are identified. * **Execute Actions & Outputs:** * For each identified ``, the framework validates the arguments against the action's schema and schedules the action's `handler` function to run via the `TaskRunner`. (See [Actions](/docs/concepts/actions) and [Tasks](/docs/concepts/tasks)). - For each identified ``, the framework validates the content/attributes and runs the output's `handler` function to send the information externally (e.g., post a message). (See [Outputs](/docs/concepts/outputs)). * **Wait for Actions:** The agent waits for any critical asynchronous actions scheduled in this step to complete. Their results (`ActionResult`) are logged to `WorkingMemory`. * **Check Completion:** The agent determines if the interaction is complete or if another reasoning step (another loop iteration) is needed based on defined conditions (`shouldContinue` hooks or remaining unprocessed logs). ## 5. Run Completion * **Exit Loop:** Once the loop condition determines no further steps are needed, the loop exits. * **Final Tasks:** Any final cleanup logic or `onRun` hooks defined in the context are executed. * **Save State:** The final persistent state (`ContextState.memory`) of all involved contexts is saved to the `MemoryStore`. * **Return Results:** The framework resolves the promise originally returned by `agent.send` or `agent.run`, providing the complete log (`chain`) of the interaction. This detailed cycle illustrates how Daydreams agents iteratively perceive (inputs, results), reason (LLM prompt/response), and act (outputs, actions), using streaming and asynchronous task management to handle potentially complex interactions efficiently. file: ./content/docs/concepts/contexts.mdx meta: { "title": "Contexts", "description": "Managing state, memory, and behavior for agent interactions." } In Daydreams, a **Context** defines a specific scope or environment for your agent's interactions or tasks. Think of it as a dedicated workspace that holds the state, memory, available tools (actions, inputs, outputs), and specific instructions relevant to that particular job. You might use different contexts for: * Handling individual user chat sessions (`chatContext`). * Managing a specific game state (`gameContext`). * Executing a complex workflow or process (`workflowContext`). * Interacting with a specific external system (`externalApiContext`). Each running **instance** of a context (e.g., the chat session with *user A*) maintains its own unique state and memory, separate from other instances (like the chat session with *user B*). ## Defining a Context You define context types using the `context` function from `@daydreamsai/core`. ```typescript import { context, action, type AnyAgent, type ContextState, } from "@daydreamsai/core"; import { z } from "zod"; // 1. Define the structure of the persistent memory for this context type interface ChatMemory { messageHistory: { sender: "user" | "agent"; text: string }[]; userPreferences: Record; lastInteractionTime?: number; } // 2. Define the Zod schema for arguments needed to identify a specific instance const chatSchema = z.object({ sessionId: z.string().describe("Unique identifier for the chat session"), userId: z.string().describe("Identifier for the user in the session"), }); // 3. Define the context using the `context` function const chatContext = context< ChatMemory, // Type for the persistent memory typeof chatSchema // Type for the identifying arguments >({ // Required: A unique identifier string for this *type* of context type: "chat", // Required: The Zod schema defining the arguments needed to identify // or create a specific *instance* of this context. schema: chatSchema, // Optional: Function to generate a unique string key for an instance from its arguments. // Use this if the 'type' alone isn't unique (e.g., multiple chat sessions). // The full instance ID becomes ":" (e.g., "chat:session-xyz"). key: ({ sessionId }) => sessionId, // Optional: Defines the initial structure and default values for the // context instance's persistent memory (`ctx.memory`). // This runs only if no saved memory exists for this instance. create: (state, agent) => { agent.logger.info( "chatContext", `Creating new memory for session: ${state.key}` ); return { messageHistory: [], userPreferences: {}, // Can use `state.options` if `setup` is used }; }, // Optional: Provides static or dynamic instructions to the LLM *when this context is active*. instructions: (state) => `You are chatting with user ${state.args.userId}. Be helpful.`, // Optional: A description of this context type's purpose. description: "A chat session with a specific user.", // Optional: Function to format the context's *current memory state* for the LLM prompt. // Helps the LLM understand the current situation within this context instance. render: (state) => { // state.memory is typed as ChatMemory here const recentHistory = state.memory.messageHistory .slice(-5) // Show last 5 messages .map((msg) => `${msg.sender}: ${msg.text}`) .join("\n"); return ` ## Recent Chat History (Session: ${state.key}): ${recentHistory || "No messages yet."} ## User Preferences: ${JSON.stringify(state.memory.userPreferences)} `; }, // --- Optional Lifecycle Hooks & Config --- // onStep: async (ctx, agent) => { /* Logic per step */ }, // onRun: async (ctx, agent) => { /* Logic on run completion */ }, // shouldContinue: (ctx) => ctx.memory.messageHistory.length < 50, // Example condition // onError: async (error, ctx, agent) => { /* Handle errors */ }, // model: mySpecificLLM, // Override agent's default model // maxSteps: 15, // Limit run steps for this context type }); // You can then associate actions, inputs, or outputs specifically for this context // (See "Associating Components" section below) // chatContext.setActions([...]); // chatContext.setInputs({...}); // chatContext.setOutputs({...}); ``` **Key Definition Parameters:** * `type`: Unique string identifying the context *type*. * `schema`: Zod schema for arguments needed to identify a context *instance*. Use `.describe()` on fields for clarity. * `key`: (Optional) Function `(args) => string` to create a unique instance key from arguments. * `create`: (Optional) Function `(state, agent) => TMemory` defining the initial structure of the instance's persistent memory (`ctx.memory`). * `render`: (Optional) Function `(state) => string | XMLElement | ...` formatting the current `state.memory` for the LLM prompt. Keep it concise and relevant. * `instructions` / `description`: (Optional) Provide guidance to the LLM about the context. * Lifecycle Hooks (`onStep`, `onRun`, etc.): (Optional) Add custom logic at different points in the execution cycle. * Configuration (`model`, `maxSteps`, etc.): (Optional) Override agent defaults for this context type. ## Context Memory (`ctx.memory`) The most important concept for managing state within a context is its **persistent memory**. * **Definition:** The structure and initial values are defined by the `create` function in your `context` definition. * **Access:** Within context lifecycle hooks (like `onStep`) and within the `handler` of any action, input, or output associated with this context, you can access the **current instance's** memory via `ctx.memory`. * **Persistence:** Changes made to `ctx.memory` during an agent run are **automatically saved** by the framework to the configured `MemoryStore` at the end of the run cycle. * **Loading:** When an agent run starts for a specific context instance, the framework loads its saved memory from the `MemoryStore` into `ctx.memory`. ```typescript // Inside an action handler associated with chatContext: handler(args, ctx, agent) { // ctx.memory is typed as ChatMemory const history = ctx.memory.messageHistory; const lastMsg = history[history.length - 1]; // Modify the memory ctx.memory.lastInteractionTime = Date.now(); ctx.memory.messageHistory.push({ sender: "agent", text: "Acknowledged." }); // This change will be saved automatically. return { success: true }; } ``` ## Associating Components (Actions/Inputs/Outputs) While you can define components globally, it's often better to associate them directly with the context they belong to using the chained `.setActions()`, `.setInputs()`, and `.setOutputs()` methods. This provides better organization and ensures components have typed access to the correct context memory. ```typescript import { chatContext, ChatMemory } from "./chatContext"; // Assuming context is defined elsewhere import { sendMessageAction } from "./actions/sendMessage"; // Assuming action is defined // Associate the action with the chat context const chatContextWithActions = chatContext.setActions([ sendMessageAction, // other actions relevant only to chat... ]); // Now, within sendMessageAction's handler, ctx.memory will be typed as ChatMemory // and the action will only be available when chatContext is active. // Similarly for inputs and outputs: // chatContext.setInputs({ ... }); // chatContext.setOutputs({ ... }); ``` You can also define them inline using the `actions`, `inputs`, or `outputs` properties directly within the `context({...})` definition, though the chained methods often offer superior type inference. ## Working with Context Instances You typically interact with specific context instances through the `agent` object: * `agent.getContext({ context: chatContext, args: { sessionId: "xyz", userId: "user1" } })`: Retrieves the `ContextState` for the specified instance. If it doesn't exist, it creates it (running `create` if needed) and loads its memory. * `agent.run({ context: chatContext, args: { sessionId: "xyz", userId: "user1" }, ... })`: Starts (or continues) the agent's processing loop specifically for this context instance. * `agent.getContextById("chat:session-xyz")`: (Less common) Retrieves a context instance directly by its full ID if you know it. * `agent.saveContext(contextState)`: Manually saves the state. Usually handled automatically by the agent lifecycle. ## Context Memory vs. Working Memory It's important not to confuse the **persistent Context Memory (`ctx.memory`)** with the temporary **Working Memory**. * **Context Memory (`ctx.memory`):** Specific to a context **instance**. Holds the **persistent state** (like chat history). Defined by `create`. Saved/loaded by the `MemoryStore`. Used by `render`. * **Working Memory:** Exists only for the duration of a single `agent.run`. Holds the **temporary log** of inputs, outputs, thoughts, and action calls/results for *that specific run*. Used to build the prompt at each step. ## Best Practices * **Scope Appropriately:** Design contexts around distinct tasks or interaction boundaries. * **Define Memory Clearly:** Use interfaces (like `ChatMemory`) for your context memory structure. Initialize it properly in `create`. * **Keep `render` Concise:** Only include state in `render` that is essential for the LLM's immediate reasoning. Avoid overly large state dumps. * **Use Zod Schemas:** Clearly define `schema` for context arguments with `.describe()`. * **Associate Components:** Link actions, inputs, and outputs to their relevant contexts for better organization and type safety. Contexts provide the structure for organizing your agent's knowledge, state, and capabilities, enabling complex and stateful interactions. file: ./content/docs/concepts/core.mdx meta: { "title": "Introduction", "description": "Understand the fundamental building blocks of the Daydreams framework." } The Daydreams framework is designed around a set of core concepts that work together to enable autonomous agent behavior. Understanding these concepts is key to effectively building and customizing agents. ## Core Architecture A Daydreams agent consists of several key components: ### Contexts Contexts are the foundation of a Daydreams agent. Similar to React components, contexts manage state and rendering for your agent. Each context: * Has a defined schema for initialization * Maintains its own memory state * Provides a rendering function that formats its state for the LLM ```ts const myContext = context({ // Unique identifier for this context type type: "my-context", // Schema defining the arguments needed to initialize this context schema: z.object({ id: z.string(), }), // Function to generate a unique key for this context instance key({ id }) { return id; }, // Initialize the context's memory state create(state) { return { items: [], currentItem: null, }; }, // Format the context for the LLM render({ memory }) { return ` Current Items: ${memory.items.join(", ")} Active Item: ${memory.currentItem || "None"} `; }, }); ``` ### Actions Actions are functions that your agent can call to interact with its environment or modify its state. They're similar to event handlers in React: ```ts action({ name: "addItem", description: "Add a new item to the list", schema: z.object({ item: z.string().describe("The item to add"), }), handler(call, ctx, agent) { // Access the context memory const contextMemory = ctx.agentMemory; // Update the state contextMemory.items.push(call.data.item); // Return a response return { message: `Added ${call.data.item} to the list`, items: contextMemory.items, }; }, }); ``` ### Extensions Extensions are pre-packaged bundles of inputs, outputs, and actions that add specific capabilities to your agent. For example, the `cli` extension adds terminal input/output capabilities. ## The React-like Mental Model If you're familiar with React, you can think of Daydreams in similar terms: * **Contexts** are like React components, managing state and rendering * **Actions** are like event handlers, responding to inputs and updating state * **Extensions** are like pre-built component libraries * The agent itself is like a React application, orchestrating everything This mental model makes it easy to reason about how your agent works and how to structure complex behaviors. *** This section provides a detailed explanation of each fundamental component: * **[Agent Lifecycle](/docs/concepts/agent-lifecycle):** How an agent processes information, makes decisions, and executes tasks in a continuous loop. * **[Contexts](/docs/concepts/contexts):** The mechanism for managing state, memory, and behavior for specific tasks or interactions. * **[Actions](/docs/concepts/actions):** Definable tasks or capabilities that an agent can perform. * **[Inputs](/docs/concepts/inputs):** How agents receive data and trigger processing cycles. * **[Outputs](/docs/concepts/outputs):** How agents communicate results or send information to external systems. * **[Memory](/docs/concepts/memory):** The different ways agents store, retrieve, and utilize information (Working, Episodic, Vector). * **[Prompting](/docs/concepts/prompting):** How instructions and context are formatted for the LLM to guide its reasoning. * **[Tasks](/docs/concepts/tasks):** The system for managing asynchronous operations and background tasks. * **[Services & Extensions](/docs/advanced):** How to integrate external services and extend the framework's capabilities. Explore these pages to gain a deeper understanding of how Daydreams works under the hood. file: ./content/docs/concepts/inputs.mdx meta: { "title": "Inputs", "description": "How Daydreams agents receive information and trigger processing." } Inputs are the mechanism by which Daydreams agents receive information from the outside world. They act as the triggers that initiate or contribute to an agent's processing cycle (`agent.run`). An input could represent a user message, a blockchain event, an API webhook, a sensor reading, or any other data source relevant to the agent's task. ## Defining an Input Input sources are defined using the `input` helper function exported from `@daydreamsai/core`. Each input definition connects an external data source to the agent's core processing loop. ```typescript import { input, context, type AnyAgent } from "@daydreamsai/core"; import { z } from "zod"; import { EventEmitter } from "events"; // Example: Using a simple Node.js EventEmitter // Assume myContext is defined elsewhere declare const myContext: any; // Example: An input source listening to a simple EventEmitter const myEventEmitter = new EventEmitter(); const eventInput = input({ // Required: A unique identifier for this input source type. // Helps in logging and debugging. e.g., "myApi:webhook", "discord:message" type: "custom:event", // Optional (but recommended): Zod schema for the *data payload* that will be // passed into the agent via the `send` function below. This validates // the data structure *before* it triggers the agent's run cycle. schema: z.object({ eventId: z.string().describe("Unique ID for the incoming event"), payload: z.any().describe("The actual data content of the event"), source: z.string().optional().describe("Origin of the event"), }), // Required (usually): Connects to the external data source and calls `send` // when new data arrives. subscribe: (send, agent: AnyAgent) => { // `send` is the crucial function provided by the framework. You call this // to push validated data into the agent for processing. // `agent` is the agent instance, useful for accessing shared services or config. const listener = (eventData: { id: string; data: any; origin?: string; }) => { console.log(`External event received: ${eventData.id}`); // 1. Determine the target Context and its arguments. // This tells the agent *which* context instance should handle this input. // This might be static or dynamically determined based on eventData. const targetContext = myContext; // Replace with your actual context definition const contextArgs = { someId: eventData.id }; // Args matching targetContext.schema // 2. Prepare the data payload according to this input's `schema`. const inputData = { eventId: eventData.id, payload: eventData.data, source: eventData.origin, }; try { // 3. Validate the payload before sending (optional but good practice) eventInput.schema?.parse(inputData); // Use the defined schema // 4. Call `send` to push the input data into the agent. // This triggers the agent's processing cycle for the target context instance. send(targetContext, contextArgs, inputData); } catch (validationError) { agent.logger.error( "eventInput:subscribe", "Invalid event data received", { eventData, validationError } ); // Decide how to handle invalid data (e.g., log, ignore) } }; // Attach the listener to the external source myEventEmitter.on("newEvent", listener); agent.logger.info("eventInput", "Subscribed to newEvent"); // IMPORTANT: Return a cleanup function. // This is called when the agent stops, ensuring you detach listeners, // close connections, clear intervals, etc. return () => { myEventEmitter.off("newEvent", listener); agent.logger.info("eventInput", "Unsubscribed from newEvent"); }; }, // Optional: Pre-process data *after* `send` is called but *before* the log entry // (InputRef) is created. Useful for adding metadata based on context state. handler: async (data, ctx, agent) => { // `data` is the payload passed to `send`. // `ctx` is the ContextState of the target context instance. // Example: Check if this event was already seen in this context instance const seen = ctx.memory.processedEventIds?.includes(data.eventId) ?? false; return { data: data, // Can optionally transform the data here params: { seen: String(seen) }, // Add parameters to the InputRef log entry }; }, // Optional: Customize how this input is represented in logs or prompts. format: (ref) => { // `ref` is the InputRef object (log entry) return `Event Received: ID=${ref.data.eventId}, Source=${ref.data.source ?? "N/A"}`; }, // Optional: One-time setup logic when the agent starts. install: async (agent) => { // E.g., agent.container.resolve('myApiClient').connect(); agent.logger.info("eventInput", "Install hook executed"); }, }); ``` **Key Concepts:** * **`type`**: A unique string identifying your input source. * **`schema`**: (Recommended) A Zod schema defining the structure of the data payload you intend to pass via `send`. Validation happens automatically if provided. * **`subscribe(send, agent)`**: The core function where you connect to your external data source (API, websocket, event emitter, polling mechanism, etc.). * When data arrives, you call the provided `send` function. * You **must** return a cleanup function to disconnect/unsubscribe when the agent stops. * **`send(context, args, data)`**: This framework-provided function is your gateway into the agent. * `context`: The **context definition** object that should handle this input. * `args`: An object matching the `context.schema`, identifying the specific **instance** of the context. * `data`: The payload containing the actual input information, ideally matching this `input`'s `schema`. * **`handler(data, ctx, agent)`**: (Optional) Pre-processing function executed after `send` but before the input log entry is finalized. Allows data transformation or adding metadata (`params`) based on the target context's state (`ctx`). * **`format(ref)`**: (Optional) Customize the string/XML representation of the input log entry (`InputRef`). ## How Inputs Trigger the Agent 1. **External Event:** Your `subscribe` function detects an event or receives data. 2. **`send` Called:** Your code calls `send(context, args, data)`. 3. **Agent Invoked:** The framework receives the call, creates a basic log entry (`InputRef`) for this input, and starts or queues an `agent.run` cycle for the specified `context` instance identified by `args`. 4. **Pre-processing (`handler`):** If you defined an `input.handler`, it runs, potentially modifying the data or adding parameters to the `InputRef`. 5. **Run Cycle:** The agent proceeds with its run cycle ([Agent Lifecycle](/docs/concepts/agent-lifecycle)), processing this new `InputRef` along with other state information to generate a response or perform actions. ## Inputs within Extensions Inputs are commonly bundled within [Extensions](/docs/advanced/extensions) to package integrations cleanly. The structure is the same, but the `input` definitions live inside the `inputs` property of the `extension` definition. ```typescript import { extension, input /* ... */ } from "@daydreamsai/core"; import { z } from "zod"; import { myService } from "./myService"; import { myContext } from "./myContext"; export const myApiExtension = extension({ name: "myApi", services: [myService], // Optional services used by the input contexts: { myContext }, // Optional contexts targeted by the input inputs: { // Input definitions go here, keyed by their type "myApi:webhook": input({ type: "myApi:webhook", // Redundant but ensures key matches type schema: z.object({ /* ... webhook payload schema ... */ }), subscribe(send, agent) { const apiClient = agent.container.resolve("myApiClient"); // Use a service const webhookHandler = (payload: any) => { const validatedData = myApiExtension.inputs["myApi:webhook"].schema.parse(payload); // Determine context/args based on payload const contextArgs = { entityId: validatedData.entityId }; send(myContext, contextArgs, validatedData); }; apiClient.registerWebhookListener(webhookHandler); return () => apiClient.removeWebhookListener(webhookHandler); }, // ... other input properties ... }), // ... potentially other inputs for this extension ... }, }); ``` ## Examples (Keep the existing CLI, Telegram, and Twitter examples here, perhaps slightly simplifying the code snippets to focus on the `subscribe` and `send` pattern.) ### CLI Input (from `@daydreamsai/cli`) ```typescript // Simplified example input({ type: "cli:input", subscribe(send, { container }) { const rl = container.resolve("readline"); const listen = async () => { while (true) { // Basic loop, real implementation handles abort const line = await rl.question("> "); if (line === "exit") break; // Send to a default CLI context instance send(cliContext, { user: "cli_user" }, line); } }; listen(); // Start listening return () => { /* Abort logic here */ }; }, }); ``` ### Telegram Message Input (from `@daydreamsai/telegram`) ```typescript // Simplified example input({ type: "telegram:message", schema: z.object({ /* ... */ }), subscribe(send, { container }) { const tg = container.resolve("telegraf"); tg.on("message", (ctx) => { if ("text" in ctx.message) { const dataPayload = { /* ... extract user, text ... */ }; // Send data to the specific telegram chat context send(telegramChat, { chatId: ctx.chat.id }, dataPayload); } }); // Telegraf handles cleanup return () => {}; }, }); ``` Inputs are the crucial link between your agent and the information it needs to react to, enabling dynamic and event-driven behavior. file: ./content/docs/concepts/memory.mdx meta: { "title": "Memory", "description": "How Daydreams agents store, recall, and learn from information." } Memory is fundamental to how Daydreams agents operate, allowing them to maintain state, recall past interactions, store learned information, and improve their performance over time. The framework provides a flexible system with different types of memory serving distinct purposes. ## Core Memory Components (`BaseMemory`) When creating an agent using `createDreams`, you configure its primary memory system via the `memory` option. This accepts an object conforming to the `BaseMemory` type, which holds implementations for different storage needs: ```typescript import { createDreams } from "@daydreamsai/core"; import { createMemory, createMemoryStore, createVectorStore } from "@daydreamsai/core"; import { createChromaVectorStore } from "@daydreamsai/chroma"; // Example import { createMongoMemoryStore } from "@daydreamsai/mongo"; // Example const agent = createDreams({ model: /* ... */, memory: createMemory( // 1. MemoryStore: For key-value based storage await createMongoMemoryStore({ uri: "mongodb://localhost:27017" }), // 2. VectorStore: For embedding storage and similarity search createChromaVectorStore("my-agent-episodes") ), // Optional: Enable automatic episodic memory generation // generateMemories: true, // exportTrainingData: true, // Optionally save episodes for fine-tuning // trainingDataPath: './agent-training.jsonl' }); ``` The `createMemory` function bundles together two main storage interfaces: 1. **`MemoryStore`**: A key-value store interface for structured data persistence. 2. **`VectorStore`**: An interface for storing vector embeddings and performing similarity searches, primarily used for episodic memory. The `BaseMemory` object passed to `createDreams` makes these stores available throughout the agent's systems. ## `MemoryStore` (Key-Value Storage) The `MemoryStore` handles the persistence of structured data associated with contexts and actions. It defines a simple key-value interface: * `get(key: string): Promise`: Retrieve data by key. * `set(key: string, value: T): Promise`: Store or update data by key. * `delete(key: string): Promise`: Remove data by key. * `clear(): Promise`: Remove all data (use with caution). **Uses:** * Storing `ContextState` snapshots (key: `"context:"`). * Storing persistent `Context` memory (key: `"memory:"`). * Storing `WorkingMemory` snapshots between runs (key: `"working-memory:"`). * Storing persistent `Action` memory (key: defined in `action.memory.key`). **Implementations:** * `createMemoryStore()`: Default in-memory store using a `Map` (data lost on restart). Found in `@daydreamsai/core`. * `createMongoMemoryStore()`: MongoDB-backed implementation. Found in `@daydreamsai/mongo`. * *(Others like SQLite, Redis might be available or could be implemented)* ## `VectorStore` (Embedding Storage & Search) The `VectorStore` is designed for handling high-dimensional vector embeddings, typically used for semantic search and retrieving relevant past experiences (Episodic Memory). * `upsert(contextId: string, data: any): Promise`: Add or update vector embeddings, often associated with a specific context. * `query(contextId: string, query: string): Promise`: Search for vectors similar to the query text within a specific context. * `createIndex(indexName: string): Promise`: Create an index (if required by the backend). * `deleteIndex(indexName: string): Promise`: Delete an index. **Uses:** * Storing `Episode` embeddings for recall. * Potentially storing document embeddings for RAG (Retrieval-Augmented Generation) patterns (though not explicitly shown in the core loop). **Implementations:** * `createVectorStore()`: Default no-op implementation (does nothing). Found in `@daydreamsai/core`. * `createChromaVectorStore()`: Implementation using ChromaDB. Found in `@daydreamsai/chroma`. Requires `OPENAI_API_KEY` for default embedding or a custom embedder. * *(Others like Pinecone might be available or could be implemented)* ## Clarification: Context vs. Memory It's important to distinguish between the concept of a "Context" and "Memory": * **Context (`context({...})` Definition & `ContextState` Instance):** * **Definition:** Think of this as a **blueprint** or template for a specific type of interaction scope (e.g., a chat session, a game state). It defines the *rules*, *behavior*, expected `schema` arguments, the structure of its persistent memory (`create`), how that state is rendered for the LLM (`render`), and its associated `inputs`, `outputs`, and `actions`. * **Instance (`ContextState`):** This is a **runtime instance** of a Context definition, representing a specific, active scope (e.g., the chat session with *user A*). It holds a reference to the definition, the specific `args` used to identify it, and the actual, current **memory payload** for this instance (`ContextState.memory`). * **Memory (`ContextState.memory`, `WorkingMemory`, etc.):** * **Context Memory (`ContextState.memory`):** This is the actual **persistent data payload** or state associated with a specific `ContextState` instance. Its structure is determined by the `context.create` function. This is what's loaded from/saved to the `MemoryStore` and rendered via `context.render`. * **Other Memory Types:** The framework also uses other memory concepts like `WorkingMemory` (transient log for a single run), `ActionMemory` (persistent state for an action definition), and `AgentMemory` (persistent state for the main agent context). These are distinct from the specific `ContextState.memory` of a running context instance. **In essence:** A **Context** defines the **structure and behavior** of an interaction scope. **Memory** (specifically `ContextState.memory`) is the **persistent data** stored *within* a particular instance of that Context. ## Types of Memory Usage ### Working Memory * **Purpose:** Short-term, temporary storage for a single `agent.run` cycle. * **Content:** Holds the sequence of `Log` objects (`InputRef`, `OutputRef`, `Thought`, `ActionCall`, `ActionResult`, `EventRef`, `StepRef`, `RunRef`) generated during the run. * **Lifecycle:** Created at the start of `agent.run`, populated during stream processing, used to build prompts at each step, and potentially snapshotted to the `MemoryStore` between runs. * **Access:** Available as `ctx.workingMemory` within handlers. ### Context Memory * **Purpose:** Persistent state associated with a specific `Context` instance (e.g., chat history for a specific user session). * **Structure:** Defined by the `create` function in the `context` definition. * **Lifecycle:** Loaded from the `MemoryStore` when `agent.getContext` is called, updated by context logic or actions, saved back to the `MemoryStore` via `agent.saveContext`. * **Access:** Available as `ctx.memory` within context-related functions (`render`, `onStep`, etc.) and action handlers. ### Action Memory * **Purpose:** Persistent state associated specifically with an `Action` definition, allowing it to maintain state across multiple calls within different runs or contexts. * **Structure:** Defined by the `memory` option in the `action` definition, using the `memory()` helper. * **Lifecycle:** Loaded from the `MemoryStore` before the action handler runs, potentially updated by the handler, saved back to the `MemoryStore` after the handler completes. * **Access:** Available as `ctx.actionMemory` within the action's handler. ### Agent Memory * **Purpose:** Persistent state associated with the agent's main/root context (if one is defined in `createDreams`). Used for global agent state. * **Lifecycle:** Loaded/saved from the `MemoryStore` using the main agent context's ID. * **Access:** Available as `ctx.agentMemory` within action handlers when a main agent context exists. ### Episodic Memory * **Purpose:** Enables the agent to learn from past experiences by recalling relevant "episodes" (sequences of observation/thought -> action -> result). * **Structure:** Defined by the `Episode` interface (observation, thoughts, result, timestamp, etc.). * **Generation:** Handled by `generateEpisode` (`memory/utils.ts`), which uses an LLM to summarize the `Thought`, `ActionCall`, and `ActionResult`. Triggered automatically if `agent.memory.generateMemories` is true. * **Storage:** The generated `Episode` (or its embedding) is stored in the `VectorStore` via `agent.memory.vector.upsert()`. * **Retrieval:** When a new `InputRef` is processed (`handleInput`), the `agent.memory.vector.query()` method is called to find relevant past episodes based on the input content. Retrieved episodes are added to `WorkingMemory.episodicMemory`. * **Training Data:** Episodes can be exported as prompt/completion pairs for fine-tuning models using `agent.exportAllTrainingData()` or by setting `exportTrainingData: true` on the agent config. By combining these memory types and storage backends, Daydreams agents can maintain short-term focus (Working Memory), long-term context state (Context/Action/Agent Memory), and learn from past interactions (Episodic Memory via VectorStore). file: ./content/docs/concepts/outputs.mdx meta: { "title": "Outputs", "description": "How Daydreams agents send information and responses." } Outputs are how Daydreams agents communicate results or send information to external systems or users. If Inputs are how agents "listen," Outputs are how they "speak" or "act" based on the LLM's reasoning. Examples of outputs include: * Sending a message to a Discord channel or Telegram chat. * Posting a tweet. * Returning a response in a CLI session. * Calling an external API based on the agent's decision (though Actions are often better for this if a response is needed). ## Defining an Output Outputs are defined using the `output` helper function exported from `@daydreamsai/core`. Each definition specifies how the agent should structure information for a particular output channel and how to execute the sending logic. ```typescript import { output, context, type AnyAgent, type ContextState, // Base context state type type OutputRef, // Type for the log entry } from "@daydreamsai/core"; import { z } from "zod"; // Assume myDiscordClient.sendMessage exists declare const myDiscordClient: { sendMessage: (channelId: string, content: string) => Promise; }; declare const myContext: any; // Placeholder for your context type const discordMessageOutput = output({ // Required: A unique identifier for this output type. Used by the LLM. type: "discord:message", // Optional: Description for the LLM. description: "Sends a message to a specific Discord channel.", // Optional: Instructions for the LLM on usage. instructions: "Use this to reply to the user in the Discord channel.", // Optional: Zod schema for the main content of the output. // The LLM provides this content *inside* the tag. // Defaults to z.string() if omitted. schema: z.string().describe("The message content to send."), // Optional: Zod schema for additional attributes the LLM must provide // *on* the tag itself. attributes: z.object({ channelId: z.string().describe("The ID of the Discord channel to send to."), replyToUserId: z .string() .optional() .describe("User ID to mention in the reply."), }), // Required (usually): The function that performs the actual sending logic. handler: async (data, ctx, agent) => { // 'data' contains the validated content (from schema). // 'ctx' includes the ContextState and the OutputRef for this specific call. // Access attributes parsed from the tag via ctx.outputRef.params. const { channelId, replyToUserId } = ctx.outputRef.params ?? {}; const content = data; // Access validated content from schema let messageContent = content; if (replyToUserId) { messageContent = `<@${replyToUserId}> ${content}`; } console.log(`Sending to Discord channel ${channelId}: ${messageContent}`); // Example: await myDiscordClient.sendMessage(channelId, messageContent); await new Promise((res) => setTimeout(res, 50)); // Simulate async // Optional: Return data to update the OutputRef log. // Can also return an array of OutputRefResponse for multiple logs. return { data: { content: messageContent, channelId }, // Updated data for the log params: ctx.outputRef.params, // Typically keep original params processed: true, // Mark this output as fully handled }; }, // Optional: Custom formatting for the OutputRef log. format: (res) => { // Note: 'res' is the OutputRef after the handler possibly updated it const outputData = Array.isArray(res.data) ? res.data[0] : res.data; // Adjust if handler returns array return `Sent Discord message to ${res.params?.channelId}: "${outputData?.content ?? res.content}"`; }, // Optional: Examples for the LLM. examples: [ `Hello there!`, `Got it!`, ], // Optional: Setup logic run when the agent starts. install: async (agent) => { /* ... */ }, // Optional: Conditionally enable this output based on context. enabled: (ctx: ContextState) => { // Example: Only enable if the current context is a discord channel // return ctx.context.type === 'discord:channel'; return true; }, // Optional: Associate with a specific context type. // context: myContext, }); ``` **Key Parameters:** * `type` (string): Unique identifier used in ``. * `description`/`instructions` (string, optional): Help the LLM understand what the output does and when to use it. * `schema` (Zod Schema, optional): Defines the structure and validates the *content* placed *inside* the `` tag by the LLM. Defaults to `z.string()`. * `attributes` (Zod Schema, optional): Defines and validates *attributes* placed *on* the `` tag itself (e.g., ``). These provide necessary parameters for the `handler`. * `handler` (Function): Executes the logic to send the information externally. It receives: * `data`: The validated content from the `schema`. * `ctx`: The context state (`ContextState`) augmented with the specific `outputRef` for this call (`OutputRef`). Attributes parsed from the tag are found in `ctx.outputRef.params`. * `agent`: The agent instance. * It can optionally return an `OutputRefResponse` (or array thereof) to update the log entry or mark it as processed. * `format` (Function, optional): Customizes the log representation of the `OutputRef`. * `examples` (string\[], optional): Provides concrete examples to the LLM on how to structure the `` tag. * `install` / `enabled` / `context` (Functions/Context, optional): Similar to Actions and Inputs for setup, conditional availability, and context scoping. ## LLM Interaction 1. **Availability:** Enabled outputs are presented to the LLM within the `` tag in the prompt, including their type, description, instructions, content schema (`content_schema`), attribute schema (`attributes_schema`), and examples. 2. **Invocation:** The LLM generates an output by including an `` tag in its response stream, matching one of the available types. It must provide any required attributes defined in the `attributes` schema and the content inside the tag matching the `schema`. ```xml This is the message content generated by the LLM. ``` ## Execution Flow 1. **Parsing:** When the framework parses an `` tag from the LLM stream (`handleStream` in `streaming.ts`), it extracts the `type`, `attributes`, and `content`. 2. **Log Creation:** An initial `OutputRef` log is created (`getOrCreateRef` in `streaming.ts`). 3. **Processing:** Once the tag is fully parsed (`el.done`), the engine calls `handleOutput` (`handlers.ts`). 4. **Validation:** `handleOutput` finds the corresponding output definition by `type`. It validates the extracted `content` against the `output.schema` and the extracted `attributes` against the `output.attributes` schema. 5. **Handler Execution:** If validation passes, `handleOutput` executes the `output.handler` function, passing the validated content (`data`) and the context state augmented with the `outputRef` (`ctx`). Attributes are accessed via `ctx.outputRef.params`. 6. **External Action:** The `handler` performs the necessary external operation (e.g., sending the Discord message). 7. **Logging:** The `handler` can optionally return data to update the `OutputRef` log. The `OutputRef` is added to the `WorkingMemory`. ## Outputs vs. Actions While outputs and actions share similar structures, they serve different purposes: | Feature | Actions | Outputs | | --------------- | ------------------------------------------- | ----------------------------------------------- | | Primary purpose | Two-way interaction (call -> result -> LLM) | One-way communication (LLM -> external) | | Return value | Result is crucial for next LLM step | Result usually not directly needed by LLM | | State mutation | Commonly used to update context state | Can update state but less common | | Usage pattern | LLM requests data or triggers process | LLM communicates final response or notification | | Error handling | Errors often returned to LLM for reaction | Errors handled internally (logged/retried) | ### When to Use Outputs vs. Actions * **Use outputs when**: The primary goal is to communicate outward (send a message, display UI, log data), and you **don't** need the result of that communication for the LLM's immediate next reasoning step. * **Use actions when**: You need the **result** of the operation (e.g., data fetched from an API, status of a transaction) for the LLM to continue its reasoning process or make subsequent decisions. ## Best Practices for Outputs 1. **Keep outputs focused**: Each output definition should have a single, clear responsibility (e.g., `discord:message`, `log:event`). 2. **Handle errors gracefully**: The `handler` should contain `try...catch` blocks for external calls and report failures appropriately (e.g., log an error, perhaps emit an `error` event) without crashing the agent. 3. **Consider asynchronous processing**: For outputs involving potentially slow external systems, ensure the `handler` is `async` and handles the operation without blocking the main agent loop excessively. 4. **Track important outputs in context**: If the fact that an output occurred is important for future agent decisions (e.g., remembering a message was sent), update the relevant context memory within the `handler`. 5. **Use descriptive names and schemas**: Clearly define the `type`, `schema`, and `attributes` so the LLM understands exactly how to use the output. Provide good `examples`. Outputs allow the agent to respond and communicate, completing the interaction loop initiated by Inputs and guided by Actions. file: ./content/docs/concepts/prompting.mdx meta: { "title": "Prompting", "description": "How Daydreams structures prompts to guide LLM reasoning and actions." } The interaction between the Daydreams framework and the Large Language Model (LLM) is mediated through carefully structured prompts. These prompts provide the LLM with the necessary context, instructions, available tools (actions and outputs), and current state, guiding its reasoning process and constraining its output format. ## The Main Prompt Template (`mainPrompt`) The core prompt structure is defined in `packages/core/src/prompts/main.ts` within the `mainPrompt` configuration object. It uses a main template string (`promptTemplate`) composed of several sections identified by placeholders: ```text {{intro}} # General instructions for the agent's task {{instructions}} # Step-by-step guide on how to process updates {{content}} # Dynamically generated section with available tools and state {{response}} # Structure definition for the LLM's expected output {{footer}} # Final reminders and important notes ``` Each placeholder (`{{intro}}`, `{{instructions}}`, etc.) corresponds to static text providing overall guidance to the LLM on how it should behave within the framework. ## Dynamic Prompt Generation At each step of the [Agent Lifecycle](/docs/concepts/agent-lifecycle), the framework dynamically generates the content for the `{{content}}` section of the `promptTemplate`. This ensures the LLM always has the most up-to-date information. 1. **Gathering Data (`formatPromptSections`)**: The `formatPromptSections` function (in `packages/core/src/prompts/main.ts`) collects the current state, including: * Available `actions`. * Available `outputs`. * Active `contexts` and their rendered state. * Recent `WorkingMemory` logs (both processed and unprocessed). 2. **Formatting to XML (`packages/core/src/formatters.ts`)**: Various helper functions (`formatAction`, `formatContextState`, `formatOutputInterface`, `formatContextLog`, `formatValue`, `formatXml`) convert these JavaScript objects and data structures into standardized XML strings. This XML format is designed to be clearly parsable by both the LLM and the framework's stream parser. 3. **Rendering (`render`)**: The `render` function (from `packages/core/src/formatters.ts`) injects these dynamically generated XML strings into the main `promptTemplate`, replacing placeholders like `{{actions}}`, `{{outputs}}`, `{{contexts}}`, `{{workingMemory}}`, and `{{updates}}`. ## Key XML Sections in the Prompt The dynamically generated `{{content}}` section typically includes these crucial XML blocks: * **``**: Lists all actions currently enabled for the agent. Each action includes its `name`, `description`, `instructions`, and argument `schema` (as JSON schema). ```xml Fetches the current weather... { "type": "object", "properties": { ... } } ... ``` * **``**: Lists all outputs the agent can generate. Each output includes its `type`, `description`, `instructions`, content `schema` (`content_schema`), attribute `schema` (`attributes_schema`), and `examples`. ```xml Sends a message... { "type": "object", ... } { "type": "string", ... } ... ... ``` * **``**: Displays the state of currently active contexts, as rendered by their respective `render` functions. ```xml user1: Hi there! agent: Hello! How can I help? ... ``` * **``**: Shows *processed* logs (`InputRef`, `OutputRef`, `ActionCall`, `ActionResult`, `Thought`, `EventRef`) from previous steps within the *current* run. ```xml User message ... ... Agent response ``` * **``**: Shows *new*, *unprocessed* logs (typically new `InputRef`s or `ActionResult`s from the previous step) that the LLM needs to analyze and react to in the *current* step. ```xml A new message requiring a response Result is 42 ``` ## Expected LLM Response Structure The framework instructs the LLM (via the `{{response}}` section of the template) to structure its output using specific XML tags: ```xml [LLM's thought process explaining its analysis and plan] [Optional: Action calls] [Arguments based on action schema and format (JSON/XML)] [Optional: Outputs] [Content matching the output's schema] ``` * **``**: The root tag for the entire response. * **``**: Contains the LLM's step-by-step thinking process. This is logged as a `Thought`. * **``**: Used to invoke an action. The `name` must match an available action. The content (arguments) depends on the action's defined `schema` and `callFormat` (defaulting to JSON if schema is complex, but can be XML). The framework parses this content accordingly. * **``**: Used to generate an output. The `type` must match an available output. Any required `attributes` must be included, and the content must match the output's content `schema`. The framework parses this XML structure from the LLM's response stream to trigger the corresponding handlers for actions and outputs. ## Template Engine (`{{...}}`) The prompt template includes a mention of a simple template engine using `{{...}}` syntax (e.g., `{{calls[0].someValue}}`, `{{shortTermMemory.key}}`). As noted in the prompt comments, its primary intended use is for **intra-turn data referencing**. This means allowing an action call within the *same* LLM response to reference the anticipated result of a *previous* action call in that *same* response. Example: ```xml First, I need to create a file, then write to it. { "directory": "/tmp" } { "fileId": "{{calls[0].fileId}}", "content": "Hello!" } ``` Here, the `writeFile` call references the `fileId` expected to be returned by the `createFile` action called just before it within the same LLM response turn. The framework resolves these templates *before* executing the action handlers (using `resolveTemplates` in `packages/core/src/handlers.ts`). This dynamic and structured prompting approach allows Daydreams to effectively leverage LLMs for complex orchestration tasks, providing them with the necessary information and tools while ensuring their output can be reliably processed. file: ./content/docs/concepts/tasks.mdx meta: { "title": "Tasks", "description": "Managing asynchronous operations and concurrency." } Daydreams agents often need to perform asynchronous operations, primarily when executing [Actions](/docs/concepts/actions) that interact with external APIs, blockchains, or other time-consuming processes. The framework includes a `TaskRunner` to manage these operations efficiently. ## Purpose The `TaskRunner` serves two main purposes: 1. **Concurrency Control:** It limits the number of asynchronous tasks (like action handlers) that run simultaneously. This prevents the agent from overwhelming external services with too many requests at once (rate limiting) or consuming excessive local resources. 2. **Prioritization (Future):** While the core framework primarily uses default priority, the underlying queue supports prioritizing tasks, allowing more critical operations to potentially execute sooner. ## Initialization and Configuration A `TaskRunner` instance is automatically created within `createDreams` unless a custom one is provided in the `Config`. Its concurrency limit can be configured: ```typescript import { createDreams, TaskRunner } from "@daydreamsai/core"; // Default TaskRunner with concurrency based on environment or default (e.g., 3) const agent1 = createDreams({ /* ... */ }); // Explicitly configure concurrency const agent2 = createDreams({ // ... other config taskRunner: new TaskRunner(5), // Allow up to 5 tasks concurrently }); // Access the runner later const runner = agent2.taskRunner; ``` The concurrency limit determines how many tasks from the internal queue can be actively running at any given moment. ## Key Implications for Users While you typically don't interact directly with the `TaskRunner` or define new `task` types, understanding how it works is important for writing effective agent components, especially [Actions](/docs/concepts/actions): 1. **Actions are Queued & Run Concurrently:** When the LLM calls an action via ``, its `handler` is scheduled by the `TaskRunner`, not executed immediately. The runner processes tasks concurrently up to its configured limit (default is 3). If more actions are called than the limit allows, they wait in a queue. * *Effect:* Prevents overwhelming external services; actions might not start instantly if the runner is busy. 2. **`async`/`await` is Essential:** Because action handlers are executed asynchronously via the Task Runner, any I/O operations (API calls, database queries, file system access) within your handler **must** use `async`/`await` correctly. The framework waits for the promise returned by the handler (managed by the Task Runner) to resolve before considering the action complete. 3. **Action `retry` Option & Idempotency:** You can configure automatic retries for an action using the `retry` property in its definition. If enabled, the `TaskRunner` will re-execute the handler if it fails (throws an error). * *Best Practice:* If using retries, design your action handlers to be **idempotent** – meaning executing the handler multiple times with the same arguments yields the same final state without unwanted side effects (e.g., avoid creating duplicate database entries). 4. **Action Cancellation (`ctx.abortSignal`):** Action handlers receive an `AbortSignal` within their `ctx` (`ActionCallContext`). * *Requirement:* For long-running handlers (e.g., complex loops, waiting for external processes), you **should** check `ctx.abortSignal.aborted` periodically or use `ctx.abortSignal.throwIfAborted()` and cease execution if the signal is aborted. This allows agent runs to be cancelled cleanly. 5. **`queueKey` Option (Advanced):** Actions can specify a `queueKey`. This directs the `TaskRunner` to use a specific queue for that action. While the default is `'main'`, this advanced feature could be used with custom `TaskRunner` configurations to manage different concurrency limits for different types of actions (e.g., a separate queue with lower concurrency for a rate-limited API). In summary, focus on writing robust, `async`-aware, and potentially idempotent action handlers, leveraging `ctx.abortSignal` for cancellation. The Task Runner handles the scheduling and concurrency control behind the scenes. ## Internal Usage (`runAction`) The most common use of the `TaskRunner` is internal to the framework. When the agent parses an `` from the LLM, the `handleActionCall` function doesn't execute the action's handler directly. Instead, it uses `taskRunner.enqueueTask` to schedule the execution: ```typescript // Simplified logic within handleActionCall (handlers.ts) // ... prepare context and arguments ... result.data = await taskRunner.enqueueTask( runAction, // The task definition for running actions { // Parameters for runAction action, agent, logger, ctx: callCtx, }, { // Options for the task enqueueing debug: agent.debugger, retry: action.retry, abortSignal, priority: 0, // Default priority } ); // ... process result ... ``` This ensures that action executions respect the concurrency limits set for the agent. ## Defining Tasks (`task` helper) The framework uses a `task` helper function (from `@daydreamsai/core/task`) to define named, reusable asynchronous operations that can be managed by the `TaskRunner`. Key framework tasks like `runAction` (executing action handlers) and `runGenerate` (calling the LLM) are defined using this helper. ```typescript import { task, type TaskContext } from "@daydreamsai/core"; // Example task definition (similar to runAction) const myCustomTask = task( // 1. Unique key/name for the task (used for logging/debugging) "agent:run:myCustomTask", // 2. The async function implementing the task logic async (params: { someData: string }, ctx: TaskContext) => { // 'params' are the arguments passed when enqueuing // 'ctx' provides task-specific context ctx.debug("myCustomTask", ["Executing"], params); // Use debug provided via options console.log(`Task ${ctx.callId} running with data:`, params.someData); // ... perform asynchronous work ... await new Promise((resolve) => setTimeout(resolve, 100)); return { success: true, taskId: ctx.callId }; }, // 3. Optional default TaskOptions { retry: 1, // Default retry count } ); // How it might be enqueued: // agent.taskRunner.enqueueTask(myCustomTask, { someData: "hello" }, { priority: 1 }); ``` The `TaskContext` passed to the task function includes: * `callId`: A unique ID generated for this specific task execution. * `debug`: A `Debugger` function instance (configured via `TaskOptions` or defaulting from agent config). While you typically won't need to define new tasks often (most work happens in action handlers), understanding this pattern helps clarify how core operations like `runAction` are structured and managed. ## Direct Usage While primarily used internally for actions, you *could* access the `TaskRunner` via `agent.taskRunner` within your custom code (e.g., inside an action handler or context hook) if you need to manage additional complex, long-running, or resource-intensive asynchronous operations with concurrency control. However, simple `async/await` within action handlers is usually sufficient. The `TaskRunner` provides robust management for the asynchronous operations essential to the agent's functioning, ensuring stability and controlled resource usage. file: ./content/docs/extra-reading/container.mdx meta: { "title": "container.ts" } This file provides a system called a "Dependency Injection (DI) Container", created using `createContainer()`. Its main job is to manage shared resources or services that different parts of your agent might need, like a connection to an external API, a database client, or the agent's logger. It ensures these resources are created correctly and makes them easily accessible wherever needed. ## How to Use You generally **don't create or directly interact** with the container yourself using `createContainer()`. The Daydreams framework creates one automatically when you call `createDreams`. * **Registering:** Services (defined using the `service` helper) or Extensions use the container's `register`, `singleton`, or `instance` methods internally to tell the container *how* to create or find a specific resource (e.g., "Here's how to make the database client"). `singleton` is common for resources you only want one of (like a database connection). * **Accessing:** When your `action` handler (or other component) needs to use a shared resource managed by the container, you access it through the `agent` object: `agent.container.resolve('resourceName')`. For example, to get the logger, you might use `agent.container.resolve('logger')`. ## Benefit The container decouples your code. Your action handler doesn't need to know *how* to create the database client or logger; it just asks the container for it by name (`'database'`, `'logger'`). This makes your code cleaner, easier to test, and simplifies managing shared resources, especially within extensions. If the way a resource is created changes, you only need to update its registration, not every place it's used. ## Anticipated Questions * *"Do I need to call `createContainer()`?"* No, the agent created by `createDreams` already includes a pre-configured container available at `agent.container`. * *"How do things get into the container?"* Typically through `ServiceProvider` definitions (created with the `service` helper), which are often bundled within `Extension`s. The service's `register` method puts things into the container. Core framework components like the default `Logger` are also registered automatically. * *"What's the difference between `register` and `singleton`?"* When registering, `singleton` ensures only *one instance* of the resource is ever created and shared. `register` creates a *new instance* every time `resolve` is called for that name (less common for shared resources). file: ./content/docs/extra-reading/context.mdx meta: { "title": "context.ts" } This file provides the essential `context` function, which you use to define different "modes" or "workspaces" for your agent. Think of each context definition as a blueprint for a specific task or interaction type, like handling a chat conversation, managing a game, or performing a specific workflow. Each active instance of a context (e.g., a specific chat session) gets its own separate memory and state. ## How to Use You'll typically define your contexts in separate files using the `context({...})` function and then pass these definitions to `createDreams`. Key things you define inside `context({...})`: * `type`: A unique name for this type of context (e.g., `"chat"`, `"projectBoard"`). * `schema`: (Using Zod) Defines the arguments needed to identify a *specific instance* of this context (e.g., `{ sessionId: z.string() }` for a chat). * `create`: A function that returns the initial structure and default values for this context's persistent memory (`ctx.memory`). This runs the first time an instance is accessed. * `render`: (Optional) A function that formats the current state (`ctx.memory`) of an instance into text (or XML) for the AI model to understand the current situation within that specific workspace. * `actions`, `inputs`, `outputs`: (Optional, often added via `.setActions()`, etc.) Link specific tools (Actions), data sources (Inputs), and response methods (Outputs) directly to this context type. ## Benefit Contexts allow your agent to manage multiple tasks or interactions simultaneously without getting confused. Each context instance has its own dedicated memory (`ctx.memory`) where it stores relevant information (like chat history or task lists) persistently. The `render` function ensures the AI model gets only the relevant state for the specific task it's working on at that moment. Associating actions/inputs/outputs keeps your agent's capabilities organized. ## Anticipated Questions * *"What's the difference between context memory (`ctx.memory`) and working memory?"* `ctx.memory` is the *persistent* storage for a specific context instance (like chat history saved to a database). *Working memory* is *temporary* storage used during a single agent run cycle to track the steps (inputs, thoughts, actions) of that specific interaction. * *"How do I identify a specific chat session if I have multiple?"* You use the `schema` you define to pass identifying arguments (like a `sessionId`) when calling `agent.run` or `agent.send`. The optional `key` function in the context definition helps create truly unique IDs if needed (e.g., `chat:session-xyz`). * *"How does the AI know what happened in this specific chat?"* The `render` function you define formats the relevant parts of `ctx.memory` (e.g., recent messages) and includes it in the prompt sent to the AI model for that specific context instance. file: ./content/docs/extra-reading/dreams.mdx meta: { "title": "dreams.ts" } This file provides the `createDreams` function, which is the main entry point for building your Daydreams agent. Think of it as the constructor or blueprint for your AI assistant. It takes all your configurations (like which AI model to use, what tools it has, how it remembers things) and assembles them into a ready-to-run agent. ## How to Use You'll call `createDreams({...})` once in your project setup, passing it a configuration object. This object specifies: * `model`: Which language model (like OpenAI's GPT or Anthropic's Claude) the agent should use for thinking (using providers from the Vercel AI SDK). * `extensions`: Pre-built packages of functionality (like Discord integration or file system access) you want your agent to have. * `contexts`: Custom definitions for different tasks or conversations your agent needs to manage. * `actions`: Custom tools or abilities you define for your agent. * `memory`: How the agent should store and recall information (e.g., using in-memory storage or a database like MongoDB). * *(and other optional configurations)* The function returns an `agent` object. You'll then typically call `await agent.start()` to initialize it and `agent.send(...)` or `agent.run(...)` to give it tasks or information to process. ## Benefit It provides a single, organized way to configure and initialize your entire agent. Instead of manually wiring up all the different parts (model, memory, tools), `createDreams` handles the setup and dependencies, letting you focus on defining your agent's capabilities and behavior. ## Anticipated Questions * *"Do I need to provide all configuration options?"* No, many options have sensible defaults (like basic memory storage). You typically only need to provide the `model` and any `extensions` or custom `actions`/`contexts` you want to use. * *"What's the difference between `agent.send()` and `agent.run()`?"* `agent.send()` is typically used when an external event happens (like a user sending a message), providing the input data. `agent.run()` is the underlying method that processes information, reasons, and takes action; `send` usually calls `run` internally. * *"Where do I define things like actions and contexts?"* You usually define them in separate files and import them into your main setup file where you call `createDreams`. file: ./content/docs/extra-reading/formatters.mdx meta: { "title": "formatters.ts" } This file contains helper functions that translate the agent's internal data (like your action definitions, context state, and logs) into the structured XML format the AI model expects to see in its prompt. It also helps format Zod schemas into JSON schemas for the prompt. ## How it Affects You You don't need to call functions like `formatAction` or `formatContextState` directly. The framework uses them automatically when preparing the prompt for the AI model during each step of an `agent.run`. For example: * When you define an `action` with a description and schema, `formatAction` converts that definition into the `` XML block seen in the prompt. * When you define a `render` function for your `context`, the output of your function is placed inside the `` tag within the `` XML block generated by `formatContextState`. * The `formatSchema` function ensures the Zod schemas you define for actions, outputs, etc., are translated into a format the AI model can understand within the prompt's `` tags. ## Benefit These formatters ensure that all the information the agent needs to give the AI model (available tools, current state, recent history) is presented in a consistent, structured way (XML) that the model is trained to understand. This standardization makes the communication between your agent's code and the AI model reliable. You don't have to worry about manually creating complex XML prompts. ## Anticipated Questions * *"Do I need to write XML?"* No. You define your components using JavaScript/TypeScript objects (via helpers like `action`, `context`, etc.). These formatters handle the conversion to XML automatically before sending the prompt to the AI. * *"Why does Daydreams use XML in prompts?"* XML provides a clear way to structure complex information (like nested states, lists of tools with descriptions and schemas) for the AI model, making it easier for the model to parse and understand the different parts of the prompt. * *"What is the `render` function in this file used for?"* It's primarily used internally by the framework to assemble the main prompt template by inserting the formatted XML blocks (like actions, contexts, logs) into the correct placeholders. file: ./content/docs/extra-reading/handlers.mdx meta: { "title": "handlers.ts" } This file holds the internal "handlers" that the Daydreams agent uses during its execution cycle (`agent.run`). When the agent receives input, or when the AI model decides to call an action or send an output, the functions in this file are responsible for processing those requests correctly. Think of it as the agent's internal dispatcher and validator. ## How it Affects You You don't call functions from this file directly. It works behind the scenes, but it's where several important things happen based on how you defined your actions, inputs, and outputs: * **Validation:** When the AI model provides arguments for your `action` or content/attributes for your `output`, the code here validates that data against the `schema` you defined using Zod. If the validation fails, it prevents your `handler` code from running with bad data. * **Parsing:** It parses the arguments/content provided by the AI model (which might be in JSON or XML format) into a usable JavaScript object/value before passing it to your `handler`. * **Template Resolution:** If you use templates like `{{calls[0].someValue}}` in your action arguments (as described in [Prompting](/docs/concepts/prompting)), the `resolveTemplates` function here handles resolving those values *before* your action's `handler` is called. * **Handler Execution:** It prepares the necessary context (including the correct memory scopes like `ctx.memory` or `ctx.actionMemory`) and then calls the specific `handler` function you wrote in your `action`, `input`, or `output` definition. For actions, it uses the `TaskRunner` to queue the execution. * **Error Handling:** It defines specific errors like `NotFoundError` (if the AI calls a non-existent action/output) and `ParsingError` (if validation fails). ## Benefit These handlers ensure that the interaction between the AI model's requests and your custom code (in action/output/input handlers) is safe, validated, and correctly contextualized. It bridges the gap between the AI's structured text output and the execution of your JavaScript/TypeScript functions, handling potential errors and data transformations along the way. ## Anticipated Questions * *"Is this where my `action`'s `handler` function actually runs?"* Yes, functions in this file (specifically `handleActionCall` which uses `runAction` from `tasks/index.ts`) are responsible for preparing the context and ultimately calling the `handler` you defined for your action (via the `TaskRunner`). * *"What happens if the AI provides arguments that don't match my action's Zod schema?"* The validation logic within `prepareActionCall` in this file will catch the mismatch, throw a `ParsingError`, and prevent your action's `handler` from being called with invalid data. * *"How does the agent know which specific context's memory (`ctx.memory`) to give my action handler?"* The logic here (within functions like `prepareActionCall` and `handleOutput`) identifies the correct `ContextState` based on the current run and makes its `memory` available in the `ctx` object passed to your handler. file: ./content/docs/extra-reading/http.mdx meta: { "title": "http.ts" } This file provides a convenient helper object named `http` for making network requests to external APIs or web services from within your agent's actions. It's essentially a smarter version of the standard web `fetch` command. ## How to Use When you write an `action` handler that needs to fetch data from or send data to an external API, you can import and use this `http` object. * For simple GET requests expecting JSON data: ```typescript import { http } from "@daydreamsai/core"; // Inside an action handler: try { const data = await http.get.json<{ someField: string }>( "https://api.example.com/data?id=123" ); console.log(data.someField); return { success: true, result: data }; } catch (error) { console.error("API call failed:", error); return { success: false, error: "API failed" }; } ``` * For POST requests sending JSON data: ```typescript import { http } from "@daydreamsai/core"; // Inside an action handler: const payload = { name: "Widget", value: 42 }; try { const response = await http.post.json( "https://api.example.com/create", payload ); return { success: true, id: response.id }; } catch (error) { // ... handle error ... } ``` * It also includes helpers for specific protocols like `http.jsonrpc(...)` and `http.graphql(...)`. ## Benefit * **Automatic Retries:** The key benefit is built-in automatic retries. If a network request fails due to a temporary network issue or a specific server error (like 500 or 503), the `http` helper will automatically wait a bit and try the request again a few times before giving up. This makes your actions more resilient to temporary glitches. * **Convenience:** Provides shortcuts for common tasks like setting JSON headers, parsing JSON responses, and adding query parameters (`params` option). ## Anticipated Questions * *"Do I have to use this instead of `fetch`?"* No, you can still use the standard `fetch` API directly in your actions if you prefer. However, using the `http` helper gives you the automatic retry logic for free. * *"How do I set custom headers (like Authorization)?"* You can pass standard `fetch` options (like `headers`) as the last argument to the `http` methods (e.g., `http.get.json(url, params, { headers: { 'Authorization': 'Bearer ...' } })`). file: ./content/docs/extra-reading/introduction.mdx meta: { "title": "introduction" } # The Git-Gud Guide to Daydreams This started out as a collection of notes from a hobbyist developer trying to understand the `@daydreamsai/core` package more deeply. The goal isn't to be an exhaustive API reference, but rather a practical guide – "extra reading" – to help you grasp the purpose and role of the key TypeScript files that make up the core framework. ## What's Here? Each page in this section dives into a specific Typescript file from the core library. You'll find explanations focusing on: * **What it is:** A high-level description of the file's purpose. * **How it Affects You / How to Use:** Practical information on whether you interact with it directly and how it fits into building your agent. * **Benefit:** Why this component exists and what advantages it offers. * **Anticipated Questions:** Answers to common questions a developer might have when encountering this part of the framework. ## How to Approach It Think of these pages as supplementary material to the main concepts and tutorials. If you're wondering *why* something works the way it does in Daydreams, or what's happening under the hood when you make a call or define an action, browsing the relevant file explanations here might provide valuable context. You can start by using the search or dive into specific files as you encounter them in your development. file: ./content/docs/extra-reading/logger.mdx meta: { "title": "logger.ts" } This file provides the `Logger` class used throughout the Daydreams framework for recording informational messages, warnings, and errors that occur during agent execution. It helps you understand what your agent is doing and diagnose problems. ## How to Use You don't typically create a `Logger` instance yourself. The agent object returned by `createDreams` already has a pre-configured logger available at `agent.logger`. You use this instance inside your `action` handlers, `context` lifecycle methods, or `service` definitions to log relevant information: ```typescript import { action, type ActionCallContext, type AnyAgent, } from "@daydreamsai/core"; export const myAction = action({ name: "processData", // ... schema ... async handler(args, ctx: ActionCallContext, agent: AnyAgent) { // Log informational message agent.logger.info("processData:handler", "Starting data processing", { inputArgs: args, }); try { // ... do some work ... const result = { status: "completed" }; // Log successful completion (at debug level) agent.logger.debug("processData:handler", "Processing successful", { result, }); return result; } catch (error) { // Log an error agent.logger.error("processData:handler", "Processing failed!", { error, }); throw error; // Re-throw or handle error } }, }); ``` * Common methods are `agent.logger.info()`, `agent.logger.warn()`, `agent.logger.error()`, `agent.logger.debug()`, and `agent.logger.trace()`. * Each method takes a `context` string (often the function/component name), a `message` string, and optional `data` object. ## Benefit Provides a standard way to record what's happening inside your agent. This is crucial for: * **Debugging:** Seeing the flow of execution, variable values, and errors. * **Monitoring:** Understanding how your agent is performing in production. * **Auditing:** Keeping a record of important events or decisions. The default logger prints messages to the console with timestamps, levels, and context, making it easy to follow along. ## Anticipated Questions * *"How can I change the logging level (e.g., see DEBUG messages)?"* You can set the `logLevel` option when calling `createDreams`. For example: `createDreams({ ..., logLevel: LogLevel.DEBUG })`. The levels are `ERROR`, `WARN`, `INFO`, `DEBUG`, `TRACE` (most verbose). * *"Can I send logs somewhere other than the console?"* Yes, the logger is designed with "transports". While the default is `ConsoleTransport`, you could potentially implement custom transports (though this is an advanced topic not covered here) and provide them via the `logger` or `transports` option in `createDreams`. * *"Why provide a `context` string (like `'processData:handler'`)?"* It helps identify *where* in the code the log message originated, which is very useful for debugging complex agents. file: ./content/docs/extra-reading/memory.mdx meta: { "title": "memory.ts" } These files define the agent's memory system. `base.ts` provides the fundamental building blocks: `MemoryStore` for saving and loading the persistent state of your contexts (like chat history), and `VectorStore` for storing "episodic memories" (learned experiences) using vector embeddings for later recall. `utils.ts` contains helpers, primarily for automatically generating those episodic memories using an LLM. ## How to Use You configure the agent's memory system via the `memory` option when calling `createDreams`. * You typically provide implementations for `MemoryStore` and `VectorStore`. * The core package provides simple defaults: `createMemoryStore()` (stores data in memory, lost on restart) and `createVectorStore()` (does nothing). * For real persistence, you'll import and use implementations from other Daydreams packages, like `@daydreamsai/mongo` for MongoDB (`createMongoMemoryStore`) or `@daydreamsai/chroma` for ChromaDB (`createChromaVectorStore`). * The `createMemory` function (exported from `base.ts`) is used to bundle your chosen store implementations together for the `memory` option. ```typescript import { createDreams, createMemory } from '@daydreamsai/core'; // Import specific store implementations import { createMongoMemoryStore } from '@daydreamsai/mongo'; import { createChromaVectorStore } from '@daydreamsai/chroma'; const agent = createDreams({ model: /* ... */, memory: createMemory( // Use MongoDB for context state await createMongoMemoryStore({ uri: 'mongodb://...' }), // Use ChromaDB for episodic memory/vector search createChromaVectorStore('my-agent-episodes') ), // Optional: Enable automatic episodic memory generation // generateMemories: true, // vectorModel: openai('text-embedding-3-small') // Model for embeddings }); ``` * Episodic memory generation (from `utils.ts`) happens automatically in the background if you set `generateMemories: true` in the agent config and provide a `VectorStore`. ## Benefit Allows your agent to have both persistent state (remembering conversations or task progress across restarts via `MemoryStore`) and the ability to learn from past interactions (recalling relevant experiences via `VectorStore` and episodic memory). You can choose storage backends suitable for your needs (simple in-memory for testing, robust databases for production). ## Anticipated Questions * *"Do I need both MemoryStore and VectorStore?"* `MemoryStore` is essential for saving the state of your `context` instances (like `ctx.memory`). `VectorStore` is only needed if you want the agent to use episodic memory (learning from past interactions using embeddings). You can use the default `createVectorStore()` if you don't need episodic memory. * *"What is episodic memory?"* It's a feature where the agent summarizes sequences of thought -> action -> result into "episodes". These are stored as vector embeddings. When the agent encounters a new situation, it can search its `VectorStore` for similar past episodes to potentially inform its current reasoning. (Requires `generateMemories: true` and a `VectorStore`). * *"Where does `ctx.memory` get saved?"* The agent automatically saves the `memory` property of your `ContextState` instances to the configured `MemoryStore` at the end of each run cycle. file: ./content/docs/extra-reading/package-managers.mdx meta: { "title": "package managers" } It's worth mentioning the role of a good package manager, especially for rapid development and monorepos. Wondering which one to use? The simple answer: **[Bun](https://bun.sh/package-manager)** ## **Why?** For the hobbyist developer, Bun offers several compelling advantages: * **Speed:** Bun reduces wait time for installing dependencies, running scripts, and starting your application/agent. * **Simplicity:** Bun acts as a runtime, package manager, bundler, and test runner rolled into one. This eliminates the need to learn, configure, and manage multiple separate tools. Keep it clean. * **Ease of Use:** No more needing separate compilation steps (`tsc`) before running your code (`node index.js`). Bun runs TypeScript directly. Essentially, Bun lets hobbyists focus more on building cool things and less on wrangling complex development toolchains. file: ./content/docs/extra-reading/prompt.mdx meta: { "title": "prompt.ts" } This file offers general tools for working with prompt templates and parsing structured (XML) responses, separate from the main agent prompt defined in `prompts/main.ts`. It provides `createPrompt` for making reusable prompt templates and `createParser` for defining how to extract data from XML text into a specific JavaScript object structure. ## How to Use While the core agent loop uses its own specific prompt, you might use these helpers in more advanced scenarios, perhaps within an `action` handler: * `createPrompt`: If an action needs to call *another* LLM for a sub-task, you could use `createPrompt` to define a reusable template for that specific sub-task prompt. ```typescript import { createPrompt } from "@daydreamsai/core"; const summarizeTemplate = createPrompt<{ textToSummarize: string }>( "Please summarize the following text concisely:\n{{textToSummarize}}" ); // Later, in an action handler: const subTaskPrompt = summarizeTemplate({ textToSummarize: someLongText }); // const summary = await callAnotherLLM(subTaskPrompt); ``` * `createParser`: If an action receives a complex XML response from an external system (or perhaps even from a specialized LLM call), you could use `createParser` to define precisely how to extract the necessary data from the XML tags into a structured JavaScript object. ## Benefit Provides flexible utilities for developers who need to implement custom prompt generation or response parsing logic within their actions or extensions, beyond the standard agent interaction loop. `createPrompt` helps manage reusable prompt strings, and `createParser` offers a structured way to handle custom XML parsing needs. ## Anticipated Questions * *"Is this the main prompt the agent uses?"* No, the main prompt template and its formatting logic are primarily defined in `packages/core/src/prompts/main.ts`. This file (`prompt.ts`) provides more general, optional tools for custom prompt/parsing scenarios. * *"When would I need `createParser`?"* It's less common, but potentially useful if an action interacts with a system that returns data in a specific XML format, and you want a structured way to extract information based on tag names. file: ./content/docs/extra-reading/providers-api.mdx meta: { "title": "providers/api.ts" } This file provides helper functions for interacting with external APIs within your agent's actions or services. The main exported function is `fetchGraphQL`, designed specifically to simplify making requests to GraphQL APIs. ## How to Use If you need to query a GraphQL endpoint from an `action` handler, you can import `fetchGraphQL` from `@daydreamsai/core`. ```typescript import { action, fetchGraphQL } from "@daydreamsai/core"; import type { AnyAgent, ActionCallContext } from "@daydreamsai/core"; const GRAPHQL_ENDPOINT = "https://api.example.com/graphql"; interface UserData { user: { id: string; name: string }; } export const getUserAction = action({ name: "getUserData", schema: z.object({ userId: z.string() }), async handler(args, ctx: ActionCallContext, agent: AnyAgent) { const query = ` query GetUser($id: ID!) { user(id: $id) { id name } } `; const variables = { id: args.userId }; try { const result = await fetchGraphQL( GRAPHQL_ENDPOINT, query, variables ); if (result instanceof Error) { agent.logger.error("getUserAction", "GraphQL query failed", { error: result.message, }); return { success: false, error: result.message }; } agent.logger.info("getUserAction", "Got user data", { user: result.user, }); return { success: true, userData: result.user }; } catch (error) { agent.logger.error("getUserAction", "Fetch failed", { error }); return { success: false, error: "Network error" }; } }, }); ``` ## Benefit `fetchGraphQL` handles the boilerplate of setting up a GraphQL POST request (setting headers, stringifying the query and variables). It also provides basic error handling, returning an `Error` object if the GraphQL response indicates errors, which you can check for using `instanceof Error`. This makes interacting with GraphQL APIs from your actions cleaner and less error-prone than using `fetch` directly for this specific case. ## Anticipated Questions * *"Is there a helper for REST APIs?"* While `api.ts` contains a `fetchRest` function, it doesn't seem to be exported directly via `@daydreamsai/core`. For general REST calls, you would typically use the `http` helper object (from `http.ts`) which provides automatic retries, or the standard `fetch` API. * *"How does this differ from the `http` helper?"* The `http` object provides general-purpose HTTP request helpers (GET, POST, JSON) with automatic retries. `fetchGraphQL` is specifically tailored for the GraphQL protocol, formatting the request body correctly and performing basic GraphQL-specific error checks on the response. file: ./content/docs/extra-reading/serviceProvider.mdx meta: { "title": "serviceProvider.ts" } This file provides the `service` helper function, which you use to define how shared resources or external clients (like an API client, database connection, or special utility) should be managed by the Daydreams framework. It ensures these services are set up correctly and are ready to use when your agent needs them. ## How to Use You typically define a service in its own file using `service({...})` and then include it in an `Extension`. Inside the `service({...})` call, you can define: * `register(container)`: (Optional) A function where you tell the agent's DI Container (`agent.container`) how to create this service instance. Often, you'll use `container.singleton('serviceName', () => new MyServiceClient(...))` here to ensure only one instance is created. * `boot(container)`: (Optional) An `async` function where you perform any necessary initialization *after* all services have been registered (e.g., connecting to the API using credentials maybe resolved from the container). This runs when `agent.start()` is called. ```typescript import { service, type Container } from "@daydreamsai/core"; // Assume MyApiClient class exists declare class MyApiClient { constructor(config: { url: string }); connect(): Promise; } export const myApiService = service({ register(container: Container) { // Tell the container how to create the client (as a singleton) container.singleton( "myApiClient", () => new MyApiClient({ url: "https://api.example.com" }) ); }, async boot(container: Container) { // Initialize the client after registration const client = container.resolve("myApiClient"); await client.connect(); console.log("My API Client connected!"); }, }); // Typically, you would then include `myApiService` in an extension's `services` array. ``` ## Benefit Defining services this way ensures proper setup and teardown, especially for resources needing asynchronous initialization (`boot`). It integrates smoothly with the DI Container, making services easily accessible via `agent.container.resolve('serviceName')` in your actions or other components, without them needing to know the setup details. Bundling services in Extensions makes them reusable. ## Anticipated Questions * *"When should I use a `service` vs just putting logic in an `action`?"* Use a `service` for shared, reusable components, especially those managing connections to external systems or requiring specific setup/initialization steps (`boot`). Actions are better for defining specific *tasks* the agent can perform, which might *use* one or more services obtained from the container. * *"What's the difference between `register` and `boot`?"* `register` runs first and only tells the container *how* to create the service. `boot` runs later (during `agent.start()`) and performs the actual initialization (like connecting), potentially using other services that were registered earlier. * *"Do I need to call `createServiceManager()`?"* No, this is handled internally by `createDreams`. You just define your services using the `service` helper. file: ./content/docs/extra-reading/tasks.mdx meta: { "title": "task.ts" } These files define the system (`TaskRunner`) that manages how your agent runs asynchronous operations, especially the `handler` functions inside your custom `action` definitions. Think of it as a queue manager that prevents your agent from trying to do too many things at once, particularly when actions involve waiting for external APIs or services. ## How it Affects You You don't directly use the `TaskRunner` or the `task` function yourself. However, its existence impacts how you write your `action` handlers: * **Concurrency:** By default, the agent only runs a few action handlers simultaneously (e.g., 3). If the AI model asks the agent to perform many actions quickly, some will wait in a queue managed by the `TaskRunner` before they start executing. This prevents overwhelming external services. * **Asynchronous Code:** Because actions are run through this system, your `action` handlers **must** use `async` and `await` correctly if they perform any operations that take time (like network requests `fetch`, database calls, or even just `setTimeout`). The `TaskRunner` waits for the `Promise` returned by your `async handler` to finish. * **Retries:** You can add a `retry` option when defining an `action`. If the action's handler fails (throws an error), the `TaskRunner` will automatically try running it again a few times. If you use this, try to make your handler logic *idempotent* (safe to run multiple times with the same input). * **Cancellation:** Long-running actions should check for cancellation signals. Inside your `action` handler, the `ctx` object contains an `abortSignal`. You should check `ctx.abortSignal.aborted` periodically in long loops or before/after long waits and stop execution if it's `true`. This allows the agent's overall run to be cancelled cleanly if needed. ## Benefit The `TaskRunner` automatically handles concurrency limits and retries for your actions, making your agent more stable and preventing it from accidentally overloading external systems you interact with. It ensures asynchronous operations are managed correctly within the agent's lifecycle. ## Anticipated Questions * *"Do I need to create a `TaskRunner`?"* No, `createDreams` creates one for you automatically with default settings. * *"How do I know when my action handler actually runs?"* It runs shortly after the AI model calls the action, but it might be delayed slightly if the `TaskRunner`'s queue is busy with other actions. Use `agent.logger` inside your handler to see when it starts and finishes. * *"What if my action needs to run for a very long time?"* Make sure to implement the cancellation check using `ctx.abortSignal.aborted` so the agent can stop it if necessary. file: ./content/docs/extra-reading/types.mdx meta: { "title": "types.ts" } This file acts as the central dictionary for all the data structures used within the Daydreams framework. It defines the specific "shape" (using TypeScript types and interfaces) that different pieces of data should have, such as what information defines an `Action`, what goes into a `Context`, or what the `Agent` object looks like. ## How to Use You generally **don't need to change** this file. However, you'll interact with the types defined here frequently when writing your agent code: * **Type Hints:** When defining the `handler` for your `action`, `input`, or `output`, you'll often use types imported from `@daydreamsai/core` (which ultimately come from this file) to get auto-completion and type safety for the arguments passed to your function (like the `args`, `ctx`, and `agent` parameters). ```typescript import { action, type ActionCallContext, type AnyAgent, } from "@daydreamsai/core"; import { z } from "zod"; // Define the memory structure for a specific context interface MyChatMemory { history: { role: "user" | "agent"; text: string }[]; } // Use ActionCallContext with your memory type for the 'ctx' parameter export const myAction = action({ name: "reply", schema: z.object({ message: z.string() }), handler: async ( args, ctx: ActionCallContext, agent: AnyAgent ) => { // Now, ctx.memory is correctly typed as MyChatMemory ctx.memory.history.push({ role: "agent", text: args.message }); // agent parameter is typed as AnyAgent }, }); ``` * **Defining Memory:** When you define a `context`, you'll often create an `interface` for its specific memory structure (like `MyChatMemory` above). This interface defines the shape of the data stored in `ctx.memory` for that context. * **Understanding Logs:** If you work with the detailed execution logs (`agent.run` results), the types like `InputRef`, `OutputRef`, `ActionCall`, `ActionResult`, `Thought` define the structure of each log entry. ## Benefit Using the types defined here makes your code safer and easier to write. Your code editor can provide helpful auto-completion and immediately warn you if you're using a component incorrectly (e.g., trying to access a property that doesn't exist on the `ctx` object or passing the wrong type of argument to an action). It acts as a form of documentation, clarifying what data is available where. ## Anticipated Questions * *"Do I need to import types directly from `types.ts`?"* No, you should import types directly from the main package entry point: `import type { Action, Context, Agent } from '@daydreamsai/core';`. * *"There are so many types! Which ones are most important?"* The most common ones you'll likely encounter when building your agent are `Agent`, `Context`, `Action`, `Input`, `Output`, `ActionCallContext`, `ContextState`, `WorkingMemory`, `MemoryStore`, `VectorStore`, and the various `Ref` types (`InputRef`, `OutputRef`, etc.) if you inspect execution logs. Many others are for internal framework use. * *"I see types like `AnyAction`. Is it easier to use those instead of specific ones like `Action`?"* While using `AnyAction` might seem simpler because you don't need to specify detailed types, it's generally **not recommended**, especially when starting out. Using specific types gives you significant advantages: 1. **Type Safety:** TypeScript can check your code for errors *before* you run it (e.g., did you misspell a property name in `ctx.memory`? Are you using the action's `args` correctly?). `AnyAction` turns these checks off, leading to potential runtime bugs that are harder to find. 2. **Auto-completion:** Your code editor can provide helpful suggestions for properties and methods when you use specific types, making coding faster and reducing typos. This doesn't work well with `AnyAction`. 3. **Clarity:** Specific types make your code easier to understand for yourself and others. It clearly shows what data an action expects and uses. It's better practice to define Zod schemas for action arguments and interfaces for context memory, then use those in your definitions (e.g., `Action`). file: ./content/docs/extra-reading/utils.mdx meta: { "title": "utils.ts" } This file provides essential "factory" functions that you use to define the building blocks of your Daydreams agent, such as its tools (Actions), how it receives information (Inputs), how it responds (Outputs), how it remembers things specifically for an action (Memory), and how you bundle features together (Extensions). ## How to Use You'll import these functions directly from `@daydreamsai/core` when defining your agent's components, typically in separate files. * `action({...})`: Use this to define a specific capability or tool for your agent. You give it a `name`, `description`, expected arguments (`schema` using Zod), and the `handler` code that runs when the AI decides to use this tool. (See [Actions](/docs/concepts/actions) for details). ```typescript import { action } from "@daydreamsai/core"; import { z } from "zod"; export const myAction = action({ name: "myTool", description: "Does something cool.", schema: z.object({ param: z.string() }), handler: async (args, ctx, agent) => { /* ... */ }, }); ``` * `input({...})`: Use this to define how your agent receives information from the outside world (like a chat message or an API event). You specify how to `subscribe` to the source and how to `send` incoming data into the agent for processing. (See [Inputs](/docs/concepts/inputs)). * `output({...})`: Use this to define how your agent sends information out (like replying to a chat). You give it a `type`, expected content structure (`schema`), and the `handler` code that performs the sending. (See [Outputs](/docs/concepts/outputs)). * `extension({...})`: Use this to package related actions, inputs, outputs, contexts, and services together into a reusable module. You provide a `name` and arrays/objects containing the components this extension provides. (See [Services & Extensions](/docs/advanced)). * `memory({...})`: A specialized helper used within an `action` definition if that specific action needs its own persistent memory across different calls (less common than context memory). You provide a `key` and a `create` function for its initial state. ## Benefit These functions provide a standardized way to define the different parts of your agent. They ensure all the necessary configuration details are provided and integrate smoothly with the agent's lifecycle and the AI model. They abstract away the internal wiring, letting you focus on the logic of your agent's capabilities. ## Anticipated Questions * *"Do I use these functions inside `createDreams`?"* No, you typically use these functions in separate files to define your actions, inputs, etc., and then you import those definitions and pass them *to* `createDreams` in its configuration object (e.g., in the `actions: [...]` or `extensions: [...]` arrays). * *"What's the difference between `action` and `output`?"* Use `action` when the agent needs to perform a task and get a result back to continue thinking (like looking up information). Use `output` when the agent just needs to send information out (like sending a final reply message). file: ./content/docs/guides/deep.mdx meta: { "title": "Deep Research", "description": "This guide will walk you through creating an AI agent that can perform deep research using Daydreams." } You can find a deep-research example in the [examples](https://github.com/daydreamsai/daydreams/tree/main/examples/deep-research) directory. Detailed tutorial coming soon! file: ./content/docs/guides/giga.mdx meta: { "title": "Building a Gigaverse Game Agent", "description": "This guide will walk you through creating an AI agent that can play the Gigaverse dungeon crawler game using Daydreams." } This guide will walk you through creating an AI agent that can play the Gigaverse dungeon crawler game using Daydreams. ## Prerequisites Before starting, make sure you have: 1. A Gigaverse account 2. The following environment variables set up: * `ANTHROPIC_API_KEY`: Your Anthropic API key * `GIGA_TOKEN`: Your Gigaverse authentication token (Bearer token from browser) ## Creating the Agent First, let's create a basic Gigaverse agent: ```ts import { anthropic } from "@ai-sdk/anthropic"; import { createDreams, context, render, action, validateEnv, LogLevel, type Agent, } from "@daydreamsai/core"; import { cliExtension } from "@daydreamsai/cli"; import { string, z } from "zod"; // Validate environment variables const env = validateEnv( z.object({ ANTHROPIC_API_KEY: z.string().min(1, "ANTHROPIC_API_KEY is required"), GIGA_TOKEN: z.string().min(1, "GIGA_TOKEN is required"), }) ); // Define the goal-oriented context const goalContexts = context({ type: "goal", schema: z.object({ id: string(), initialGoal: z.string(), initialTasks: z.array(z.string()), }), key({ id }) { return id; }, create(state) { return { goal: state.args.initialGoal, tasks: state.args.initialTasks ?? [], currentTask: state.args.initialTasks?.[0], }; }, render({ memory }) { return render(template, { goal: memory.goal, tasks: memory.tasks.join("\n"), currentTask: memory.currentTask ?? "NONE", }); }, }); // Create the Gigaverse agent createDreams({ logger: LogLevel.INFO, model: anthropic("claude-3-7-sonnet-latest"), extensions: [cliExtension], context: goalContexts, actions: [ // Actions will be defined below ], }).start({ id: "gigaverse-agent", initialGoal: "Successfully complete a dungeon run in Gigaverse", initialTasks: [ "Start a new dungeon run", "Make strategic combat decisions using rock-paper-scissors mechanics", "Select optimal loot after defeating enemies", "Progress as far as possible in the dungeon", ], }); ``` ## How It Works The Gigaverse agent is built using several key components: 1. **Context Template**: Defines the agent's understanding of the game and its current state: ```ts const template = ` # Gigaverse Dungeon Game You are an AI agent playing a rock-paper-scissors dungeon crawler game. ## Game Rules: - Combat is resolved through rock-paper-scissors mechanics - You can collect loot after defeating enemies - Your goal is to progress as far as possible in the dungeon ## Current Status: Goal: {{goal}} Tasks: {{tasks}} Current Task: {{currentTask}} Make strategic decisions based on enemy patterns and your current state. `; ``` 2. **Game Actions**: The agent can perform several actions in the game: * `attackInDungeon`: Make combat decisions (rock, paper, scissors) or select loot * `getPlayerState`: Retrieve the current game state * `startNewRun`: Begin a new dungeon run ## Understanding Gigaverse Game Mechanics Gigaverse is a dungeon crawler game with the following key mechanics: 1. **Rock-Paper-Scissors Combat**: Battles are resolved using the classic rock-paper-scissors game: * Rock beats Scissors * Scissors beats Paper * Paper beats Rock 2. **Dungeon Progression**: Players advance through the dungeon by defeating enemies. 3. **Loot System**: After defeating enemies, players can select one of three loot options to enhance their character. 4. **Health Management**: Players must manage their health throughout the dungeon run. ## Core Game Actions Let's implement the three main actions needed for the Gigaverse agent: ### 1. Attack in Dungeon This action handles both combat (rock-paper-scissors) and loot selection: ```ts action({ name: "attackInDungeon", description: "Attack in the dungeon using rock-paper-scissors game mechanics", schema: z .object({ action: z .enum([ "rock", "paper", "scissor", "loot_one", "loot_two", "loot_three", ]) .describe("The attack move to make"), dungeonId: z .number() .default(0) .describe("The ID of the dungeon"), }) .describe( "You use this to make an action in a dungeon. If the lootPhase == true then you can select the Loot option, which will then take you to the next phase. If the lootPhase == false then you can select the Rock, Paper, Scissors option." ), async handler(data, ctx, agent) { try { const { action, dungeonId } = data; const payload = { action: action, actionToken: new Date().getTime().toString(), dungeonId: dungeonId, }; const response = await fetch( "https://gigaverse.io/api/game/dungeon/action", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${env.GIGA_TOKEN}`, }, body: JSON.stringify(payload), } ); if (!response.ok) { throw new Error(`Attack action failed with status ${response.status}`); } const result = await response.json(); return { success: true, result, message: `Successfully performed ${action} attack in dungeon ${dungeonId}`, }; } catch (error) { const errorMessage = error instanceof Error ? error.message : String(error); return { success: false, error: errorMessage, message: "Attack action failed", }; } }, }), ``` ### 2. Get Player State This action retrieves the current state of the player in the dungeon: ```ts action({ name: "getPlayerState", description: "Get the current state of the player in the dungeon", schema: z.object({}), async handler(data, ctx, agent) { try { const response = await fetch( "https://gigaverse.io/api/game/dungeon/state", { method: "GET", headers: { "Content-Type": "application/json", Authorization: `Bearer ${env.GIGA_TOKEN}`, }, } ); if (!response.ok) { throw new Error(`Fetch player state failed with status ${response.status}`); } const result = await response.json(); return { success: true, playerState: result, message: "Successfully fetched player's dungeon state", }; } catch (error) { const errorMessage = error instanceof Error ? error.message : String(error); return { success: false, error: errorMessage, message: "Failed to fetch player's dungeon state", }; } }, }), ``` ### 3. Start New Run This action initiates a new dungeon run: ```ts action({ name: "startNewRun", description: "Start a new dungeon run. Use this when the player dies or wants to start a new run from outside the dungeon.", schema: z.object({ dungeonId: z .number() .default(1) .describe("The ID of the dungeon to start. Default is 1."), }), async handler(data, ctx, agent) { try { const { dungeonId } = data; const payload = { action: "start_run", actionToken: new Date().getTime().toString(), dungeonId: dungeonId, }; const response = await fetch( "https://gigaverse.io/api/game/dungeon/action", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${env.GIGA_TOKEN}`, }, body: JSON.stringify(payload), } ); if (!response.ok) { throw new Error(`Start new run failed with status ${response.status}`); } const result = await response.json(); return { success: true, result, message: `Successfully started a new run in dungeon ${dungeonId}`, }; } catch (error) { const errorMessage = error instanceof Error ? error.message : String(error); return { success: false, error: errorMessage, message: "Failed to start a new dungeon run", }; } }, }), ``` ## Agent Decision Making The agent uses the context and player state to make strategic decisions. Here's how the agent approaches the game: 1. **Analyzing Enemy Patterns**: * The agent observes patterns in enemy moves over multiple encounters * It can identify if an enemy favors certain moves (like using "rock" more frequently) * This information helps the agent make counter-moves (like choosing "paper" against a "rock"-favoring enemy) 2. **Loot Selection Strategy**: * After defeating enemies, the agent evaluates the three loot options * It considers the player's current health, inventory, and progression * The agent selects loot that complements the player's build or addresses weaknesses 3. **Adaptive Gameplay**: * As the dungeon difficulty increases, the agent adjusts its strategy * It may prioritize health-restoring items when health is low * The agent can become more conservative or aggressive based on the player's current state 4. **Goal-Oriented Planning**: * The agent maintains awareness of its overall goal and current tasks * It can reprioritize tasks based on the changing game state * This ensures the agent makes decisions that contribute to long-term success ## Integrating with Daydreams This example showcases several key Daydreams features: 1. **Goal-Oriented Context**: The agent maintains a structured goal and task list. 2. **Action Definitions**: Clear, typed actions with Zod schemas for validation. 3. **API Integration**: Seamless interaction with external APIs (Gigaverse in this case). 4. **Error Handling**: Robust error handling to manage API failures gracefully. ## Next Steps * Customize the agent's strategy by modifying the context template * Add more sophisticated decision-making logic * Implement inventory management and character progression * Extend the agent to handle more complex game scenarios For a more thorough demonstration of API implementation for Gigaverse, you can find the complete example in the [examples](https://github.com/daydreamsai/daydreams/tree/main/examples/games/gigaverse) directory. file: ./content/docs/guides/introduction.mdx meta: { "title": "Introduction", "description": "Bootstrap your first Daydreams agent." } ## Bootstrapping a New Agent The easiest way to get started with Daydreams is to use the CLI tool: ### Install the CLI ```bash npm install -g @daydreamsai/create-agent ``` ### Create a new agent ```bash npx @daydreamsai/create-agent dreaming-agent ``` This will scaffold a new agent project with all the necessary files and dependencies. ### Navigate to your project ```bash cd dreaming-agent ``` ### Configure Environment Copy the `.env.example` file to `.env` and fill in your API keys. ```bash cp .env.example .env ``` ### Start your agent ```bash npm start # or use bun # bun run index.ts ``` ## Complete Example Here's a minimal example of a Daydreams agent using the CLI extension: ```ts import { createDreams } from "@daydreamsai/core"; import { cli } from "@daydreamsai/core/extensions"; import { groq } from "@daydreamsai/core/models"; const agent = createDreams({ model: groq("deepseek-r1-distill-llama-70b"), extensions: [cli], }).start(); ``` This will run the agent in the terminal. Talk to it! For more detailed usage, check out the other guides or explore the [Concepts](/docs/concepts/core) section. file: ./content/docs/guides/mcp-guide.mdx meta: { "title": "Model Context Protocol (MCP) Guide for Daydreams" } This guide explains how to use the Model Context Protocol (MCP) integration with Daydreams, allowing your agents to connect to MCP servers and access their resources, tools, and prompts. More information about MPC specification can be found in the [MCP Github](https://github.com/modelcontextprotocol/specification). ## What is MCP? The Model Context Protocol (MCP) is a standardized way for applications to provide context to Large Language Models (LLMs). It separates the concerns of providing context from the actual LLM interaction, allowing for a more modular and flexible architecture. Key benefits of MCP include: * **Standardization**: A common protocol for context exchange between applications and LLMs * **Separation of concerns**: Applications can focus on providing context, while LLMs can focus on processing it * **Modularity**: Connect to multiple context sources simultaneously * **Extensibility**: Add new context sources without changing the core application ## MCP in Daydreams Daydreams provides a built-in MCP extension that allows your agents to: * Connect to multiple MCP servers simultaneously * Access resources from MCP servers * Execute tools provided by MCP servers * Use prompts defined on MCP servers ## Getting Started ### Prerequisites Before you begin, make sure you have: * Daydreams installed * Node.js 18 or later * An MCP server to connect to (or you can use the example server provided) ### Installation 1. Create a new Daydreams project or use an existing one 2. Install the required dependencies: ```bash npm install @daydreamsai/core @modelcontextprotocol/sdk @ai-sdk/anthropic # or yarn add @daydreamsai/core @modelcontextprotocol/sdk @ai-sdk/anthropic # or pnpm add @daydreamsai/core @modelcontextprotocol/sdk @ai-sdk/anthropic ``` ### Basic Setup To connect your Daydreams agent to an MCP server, add the MCP extension to your agent configuration: ```typescript import { createDreams } from "@daydreamsai/core"; import { mcpExtension } from "@daydreamsai/mcp"; import { anthropic } from "@ai-sdk/anthropic"; import path from "path"; // Create an agent with the MCP extension createDreams({ model: anthropic("claude-3-7-sonnet-latest"), // Add the MCP extension with server configuration extensions: [ mcpExtension([ { id: "example-server", name: "Example Resource Server", transport: { type: "stdio", command: "node", args: [path.join(__dirname, "mcp-server-example.mjs")], }, }, ]), ], }).start(); ``` ## Creating a Simple MCP Server Here's how to create a simple MCP server that provides a tool and a resource: ```javascript // mcp-server-example.mjs import { McpServer, ResourceTemplate, } from "@modelcontextprotocol/sdk/server/mcp.js"; import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"; import { ListToolsRequestSchema } from "@modelcontextprotocol/sdk/types.js"; import { z } from "zod"; // Create an MCP server const server = new McpServer({ name: "Demo", version: "1.0.0", }); // Add an addition tool server.tool("add", { a: z.number(), b: z.number() }, async ({ a, b }) => ({ content: [{ type: "text", text: String(a + b) }], })); // Add a dynamic greeting resource server.resource( "greeting", new ResourceTemplate("greeting://{name}", { list: undefined }), async (uri, { name }) => ({ contents: [ { uri: uri.href, text: `Hello, ${name}!`, }, ], }) ); // List available tools server.resource(ListToolsRequestSchema, async () => { return { tools: [ { name: "add", description: "Add two numbers", parameters: { a: { type: "number" }, b: { type: "number" }, }, }, ], }; }); // Start receiving messages on stdin and sending messages on stdout const transport = new StdioServerTransport(); await server.connect(transport); ``` ## Conclusion The MCP integration in Daydreams provides a powerful way to connect your agents to external data sources and tools. By following this guide, you should be able to create agents that can interact with MCP servers and leverage their capabilities to build more sophisticated AI applications. For more information, refer to: * [MCP TypeScript SDK documentation](https://github.com/modelcontextprotocol/typescript-sdk) file: ./content/docs/guides/supabase.mdx meta: { "title": "Supabase", "description": "This guide will walk you through creating an AI agent that utilizes supabase as the memory store." } ## Using Supabase with Daydreams Setup Info: * Vector Model Provider: `gpt-4-turbo` via `@ai-sdk/openai` * Model Provider: `google/gemini-2.0-flash-001` via `@openrouter/ai-sdk-provider` * Memory Store: Supabase via `@daydreamsai/supabase` * Communication method: Command Line via `@daydreamsai/cli` Initialize a project and add our setup packages ```bash bun init bun add @daydreamsai/core @daydreamsai/supabase @daydreamsai/cli @ai-sdk/openai @openrouter/ai-sdk-provider ``` After installing the packages, go to [https://supabase.com/](https://supabase.com/) and create a new project. Once your project is created, you'll need to add the two environment variables necessary for this package to your environment. ```bash # Supabase SUPABASE_URL=YOUR_SUPABASE_URL SUPABASE_SERVICE_KEY=YOUR_SUPABASE_SERVICE_KEY # Other variables used in this example OPENROUTER_API_KEY=YOUR_SUPABASE_SERVICE_KEY OPENAI_API_KEY=YOUR_OPENAI_KEY OPENROUTER_API_KEY=YOUR_OPENROUTER_KEY ``` These variables are provided by Supabase when you create the project and can be found in your project settings:Data API. Next, you need to set up the necessary database structure for the agent's memory. Copy the following SQL code block and paste in the Supabase SQL Editor (found in the sidebar): ```sql -- Enable the pgvector extension if it's not already enabled -- This is crucial for vector similarity search used by SupabaseVectorStore CREATE EXTENSION IF NOT EXISTS vector; -- Function to enable pgvector extension (might be used internally by SupabaseVectorStore) CREATE OR REPLACE FUNCTION enable_pgvector_extension() RETURNS void LANGUAGE plpgsql AS $$ BEGIN CREATE EXTENSION IF NOT EXISTS vector; END; $$; -- Function to execute arbitrary SQL (potentially used by SupabaseVectorStore for initialization) -- SECURITY DEFINER allows the function to run with the privileges of the user who defines it, -- necessary for operations like creating tables or extensions if the calling user doesn't have direct permissions. -- Ensure you understand the security implications before using SECURITY DEFINER. CREATE OR REPLACE FUNCTION execute_sql(query text) RETURNS void LANGUAGE plpgsql SECURITY DEFINER AS $$ BEGIN EXECUTE query; END; $$; ``` Afterards you should see a success message like "Success. No rows returned". With the Supabase setup complete, let's create the agent in our `index.ts`: ```ts // This example shows how to use Supabase with DaydreamsAI. // Vector Model Provider: gpt-4-turbo via @ai-sdk/openai // Model Provider: google/gemini-2.0-flash-001 via @openrouter/ai-sdk-provider // Memory Store: @daydreamsai/supabase // CLI Extension: @daydreamsai/cli import { openai } from "@ai-sdk/openai"; import { createContainer, createDreams, Logger, LogLevel, validateEnv, } from "@daydreamsai/core"; import { createSupabaseBaseMemory } from "@daydreamsai/supabase"; import { z } from "zod"; import { cliExtension } from "@daydreamsai/cli"; import { openrouter } from "@openrouter/ai-sdk-provider"; validateEnv( z.object({ OPENAI_API_KEY: z.string().min(1, "OPENAI_API_KEY is required"), SUPABASE_URL: z.string().min(1, "SUPABASE_URL is required"), SUPABASE_SERVICE_KEY: z.string().min(1, "SUPABASE_SERVICE_KEY is required"), }) ); const agent = createDreams({ container: createContainer(), logger: new Logger({ level: LogLevel.DEBUG }), model: openrouter("google/gemini-2.0-flash-001"), extensions: [cliExtension], memory: createSupabaseBaseMemory({ url: process.env.SUPABASE_URL!, key: process.env.SUPABASE_SERVICE_KEY!, memoryTableName: "agent", vectorTableName: "agentVectors", vectorModel: openai("gpt-4-turbo"), }), }); // Agent starts await agent.start(); ``` Run the agent and chat via the command line interface! ``` bun run index.ts ``` file: ./content/docs/guides/twitter.mdx meta: { "title": "Building a Twitter Agent", "description": "This guide will walk you through creating an AI agent that can interact with Twitter using DreamsAI." } ## Prerequisites Before starting, make sure you have: 1. A Twitter developer account 2. The following environment variables set up: * `GROQ_API_KEY`: Your Groq API key ## Creating the Agent First, let's create a basic Twitter agent: ```ts import { createGroq } from "@ai-sdk/groq"; import { createContainer, createDreams, LogLevel, twitter, } from "@daydreamsai/core"; // Initialize Groq client const groq = createGroq({ apiKey: process.env.GROQ_API_KEY!, }); // Create the agent const agent = createDreams({ logger: LogLevel.DEBUG, container: createContainer(), model: groq("deepseek-r1-distill-llama-70b"), extensions: [twitter], }); // Start the agent await agent.start(); ``` ## How It Works The Twitter agent is created using a few key components: 1. **Groq Integration**: We use Groq's language model for processing and understanding Twitter interactions. 2. **Logger**: Set to DEBUG level for detailed logging during development. 3. **Container**: Manages dependencies and services. 4. **Twitter Extension**: The `twitter` extension provides built-in capabilities for: * Monitoring mentions * Replying to tweets * Posting new tweets * Managing Twitter authentication ## Next Steps * Set up Twitter authentication in your environment variables * Customize the agent's behavior by adding your own extensions * Implement specific use cases like auto-replies or content monitoring For the full twitter example, check out the [Example-Twitter](https://github.com/daydreamsai/daydreams/tree/main/examples/twitter) on Github. file: ./content/docs/providers/ai-sdk.mdx meta: { "title": "AI SDK Integration", "description": "Leveraging the Vercel AI SDK with Daydreams." } Built on top of the [Vercel AI SDK](https://sdk.vercel.ai/docs/introduction), Daydreams seamlessly integrates with different AI providers and models. This means you can easily use any model provider compatible with the Vercel AI SDK ecosystem to power your Daydreams agents. ## Configuring the Model You specify the LLM provider and model when initializing your agent using the `model` property in the `createDreams` configuration object. The value for this property comes directly from the provider functions exported by the respective Vercel AI SDK provider packages. ## Example Usage First, install the necessary provider package. For example, to use OpenAI models: ```bash npm install @ai-sdk/openai # or bun add @ai-sdk/openai ``` Then, import the provider function and pass it to `createDreams`: ```typescript import { createDreams } from "@daydreamsai/core"; import { openai } from "@ai-sdk/openai"; // Import the provider const agent = createDreams({ // Configure the agent to use OpenAI's gpt-4o-mini model model: openai("gpt-4o-mini"), // ... other agent configurations (extensions, contexts, etc.) extensions: [ /* ... */ ], }); // Start the agent await agent.start(); ``` ### Other Providers You can follow the same pattern for other providers: * **Anthropic:** ```bash npm install @ai-sdk/anthropic ``` ```typescript import { anthropic } from "@ai-sdk/anthropic"; // ... model: anthropic("claude-3-7-sonnet-latest"), // ... ``` * **Groq:** ```bash npm install @ai-sdk/groq ``` ```typescript import { createGroq } from "@ai-sdk/groq"; const groq = createGroq(); // Or pass options like apiKey // ... model: groq("llama3-70b-8192"), // ... ``` * **OpenRouter:** ```bash npm install @openrouter/ai-sdk-provider ``` ```typescript import { openrouter } from "@openrouter/ai-sdk-provider"; // ... model: openrouter("google/gemini-2.0-flash-001"), // ... ``` ## API Keys Remember to set the necessary API key environment variables for your chosen provider (e.g., `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GROQ_API_KEY`, `OPENROUTER_API_KEY`). Daydreams relies on the underlying Vercel AI SDK provider to pick up these keys. ## Flexibility This integration allows you to easily switch between different LLMs and providers without changing your core agent logic, simply by modifying the `model` configuration and ensuring the correct provider package is installed and API keys are set. For a list of available providers and models, refer to the [Vercel AI SDK Documentation](https://sdk.vercel.ai/docs/introduction).