LLM Schema and Structured Output Guide

This document explains how to use the schema generator and StructuredOutput patterns with LLMs in this codebase.

Schema Definition Pattern

1. Create a new schema file in src/schemas/:
2. Name: <Purpose>Response.ts
3. Export an interface describing the response structure
4. Add JSDoc comments for all properties
5. Add to SchemaType enum in SchemaTypes.ts

Example schema file:

export interface ExampleResponse {
    /**
     * Description of what this property contains
     */
    propertyName: string;

    /**
     * Array of relevant items
     */
    items: Array<{
        id: string;
        value: number;
    }>;
}

Schema Usage Pattern

1. Get schema with getGeneratedSchema():

const schema = await getGeneratedSchema(SchemaType.ExampleResponse);

1. For routing responses, use the RoutingResponse schema:

const schema = await getGeneratedSchema(SchemaType.RoutingResponse);
const response = await this.modelHelpers.generate<RoutingResponse>({
    message: userInput,
    instructions
});

1. Create StructuredOutputPrompt:

const systemPrompt = `You are an assistant that...`;
const instructions = new StructuredOutputPrompt(schema, systemPrompt);

1. Generate response with modelHelpers:

const response = await this.modelHelpers.generate<ExampleResponse>({
    message: userInput,
    instructions
});

Best Practices

Schema Design

• Use descriptive property names
• Add JSDoc comments for all properties
• Use proper TypeScript types
• Include examples in comments when helpful
• Mark optional properties with ?
• Use arrays for multiple items
• Use nested objects for complex structures

Prompt Design

• Clearly explain the task in system prompt
• Include examples when helpful
• Specify required format in system prompt
• Use schema properties in prompt instructions
• Handle edge cases in prompt

Error Handling

• Validate responses match schema
• Handle malformed responses gracefully
• Log schema validation errors
• Provide fallback behavior
• Track token usage

Example Workflow

1. Define schema:

// src/schemas/ExampleResponse.ts
export interface ExampleResponse {
    /**
     * Array of example items
     */
    items: Array<{
        id: string;
        value: number;
    }>;
}

1. Add to SchemaTypes:

// src/schemas/SchemaTypes.ts
export enum SchemaType {
    // ...
    ExampleResponse = 'ExampleResponse'
}

1. Use in executor:

const schema = await getGeneratedSchema(SchemaType.ExampleResponse);

const systemPrompt = `You are an example assistant...`;
const instructions = new StructuredOutputPrompt(schema, systemPrompt);

const response = await this.modelHelpers.generate<ExampleResponse>({
    message: userInput,
    instructions
});

// Validate and use response
if (response.items && Array.isArray(response.items)) {
    // Process items
}

Common Patterns

Array Responses

Use arrays for multiple items:

items: Array<{
    id: string;
    value: number;
}>;

Nested Objects

Use nested objects for complex data:

metadata: {
    source: string;
    timestamp: number;
    tags: string[];
}

Enums

Use string enums for constrained values:

status: "pending" | "complete" | "failed";

Optional Properties

Mark optional properties with ?:

optionalField?: string;

Token Tracking

Include token usage metadata:

_usage: {
    inputTokens: number;
    outputTokens: number;
}