# Single-Agent Patterns Core architectural patterns for individual agents. Framework-agnostic designs. --- ## Pattern Taxonomy ``` Single Agent ├── ReAct ← Reasoning + Acting interleaved ├── Plan-and-Execute ← Plan first, then execute ├── ReWOO ← Plan without observation ├── Self-Ask ← Decompose into sub-questions ├── Reflexion ← Reflect + memory for learning ├── LATS ← MCTS + reflection ├── Plan-and-Solve ← Explicit plan → solve each step ├── Chain-of-Thought ← Implicit reasoning trace └── Program-of-Thoughts ← Code as reasoning ``` --- ## ReAct (Reasoning + Acting) ### Structure ``` Thought: I need to find the capital of France Action: search_web("capital of France") Observation: Paris is the capital of France Thought: Now I can answer Answer: Paris ``` ### Algorithm ```python def react_loop(goal, max_steps=10): trajectory = [] for step in range(max_steps): # 1. Generate thought thought = llm.generate(prompt=f"{system_prompt}\n{trajectory}\nThought:") # 2. Parse action action, args = parse_action(thought) # 3. Execute observation = execute_tool(action, args) # 4. Record trajectory.append(Thought(thought)) trajectory.append(Action(action, args)) trajectory.append(Observation(observation)) # 5. Check termination if is_final_answer(thought): return extract_answer(thought) raise MaxStepsExceeded() ``` ### Variants | Variant | Modification | Use Case | |---|---|---| | **ReAct** | Original | General purpose | | **ReAct+Reflection** | Add self-critique step | Quality improvement | | **ReAct+Memory** | Retrieve relevant past trajectories | Long-term learning | | **Structured ReAct** | Enforce JSON schema for actions | Reliability, parsing | ### Strengths / Weaknesses | ✅ Strengths | ❌ Weaknesses | |---|---| | Simple, intuitive | Can loop on errors | | Flexible tool use | No explicit planning | | Works with any LLM | Context grows unbounded | | Interpretable traces | No parallel execution | --- ## Plan-and-Execute ### Structure ``` Planner: Generate full plan from goal Executor: Execute each step sequentially Replanner: On failure, revise plan ``` ### Algorithm ```python def plan_and_execute(goal): # Phase 1: Plan plan = planner.generate_plan(goal) # plan = [Step(1, tool, args), Step(2, tool, args), ...] # Phase 2: Execute results = [] for step in plan: try: result = executor.execute(step) results.append(result) except Exception as e: # Phase 3: Replan plan = replanner.replan(goal, plan, results, e) continue return results ``` ### Planner Types | Planner | Description | |---|---| | **LLM Planner** | Single LLM call generates full plan | | **Hierarchical Planner** | High-level → decompose → low-level | | **Constraint-Aware** | Incorporates budgets, dependencies | | **Human-in-Loop** | Plan requires approval before execution | ### When to Use - **Complex, multi-step tasks** with clear dependencies - **High-stakes** where errors costly (need verification) - **Parallelizable** steps (DAG execution) - **Audit requirements** (plan as artifact) --- ## ReWOO (Reasoning Without Observation) ### Key Insight Separate **planning** (reasoning about what to do) from **execution** (doing it). Avoids redundant observations in prompt. ### Three Modules ``` Planner: "I need X, Y, Z" → generates plan with variables Worker: Executes each step, fills variables Solver: Assembles final answer from filled plan ``` ### Example ``` Plan: 1. search(company) → #E1 2. search(ceo of #E1) → #E2 3. search(revenue of #E1) → #E3 Worker executes: #E1 = "Acme Corp" #E2 = "Jane Smith" #E3 = "$10M" Solver: "Acme Corp CEO is Jane Smith with $10M revenue" ``` ### Benefits - **Token efficient** — No intermediate observations in context - **Parallelizable** — Independent steps run concurrently - **Deterministic** — Plan fixed before execution --- ## Self-Ask ### Pattern Decompose complex question into answerable sub-questions. ``` Q: Who won the 2022 World Cup? SubQ: Which teams reached the 2022 World Cup final? A: Argentina and France SubQ: Who won the final between Argentina and France? A: Argentina Final: Argentina won the 2022 World Cup. ``` ### Implementation ```python def self_ask(question): while True: response = llm.generate(f"Question: {question}\nSub-question:") if "Final Answer:" in response: return extract_final(response) sub_q = extract_subquestion(response) sub_a = search_or_knowledge(sub_q) question += f"\nSub-question: {sub_q}\nAnswer: {sub_a}" ``` --- ## Reflexion (Self-Reflection + Memory) ### Loop ``` Episode: Task attempt Reflection: Critique own trajectory → extract lessons Memory: Store lessons in long-term memory Next Episode: Retrieve relevant lessons → apply ``` ### Reflection Prompt ``` You failed the task. Analyze your trajectory: 1. What went wrong? 2. What would you do differently? 3. What general lesson can you extract? Store this lesson for future similar tasks. ``` ### Memory Integration ```python class ReflexionAgent: def __init__(self): self.episodic_memory = EpisodicMemory() self.semantic_memory = SemanticMemory() def run(self, task): # Retrieve relevant lessons lessons = self.semantic_memory.search(task) # Execute with lessons in context result = self.execute_with_context(task, lessons) # Reflect if not result.success: lesson = self.reflect(result.trajectory) self.semantic_memory.store(lesson) return result ``` --- ## LATS (Language Agent Tree Search) ### Algorithm: MCTS + LLM ``` 1. Selection: UCB from root to leaf 2. Expansion: Generate child actions via LLM 3. Simulation: Rollout to terminal (or depth limit) 4. Backpropagation: Update values 5. Reflection: Verbal critique of failed paths ``` ### Key Innovations - **Value function** = LLM self-evaluation of state - **Reflection** = Natural language critique → stored for future - **External feedback** = Environment rewards (test pass, etc.) --- ## Chain-of-Thought / Program-of-Thoughts ### CoT (Implicit) - Single pass, reasoning in natural language - "Let's think step by step" - Zero-shot or few-shot ### PoT (Explicit Code) ``` Question: Calculate compound interest... Program: ```python P = 1000 r = 0.05 n = 12 t = 10 A = P * (1 + r/n)**(n*t) print(A) ``` Execution: 1647.01 Answer: $1647.01 ``` - Generates executable code - Runs in sandbox - Verifiable, precise for computation --- ## Pattern Comparison | Pattern | Planning | Flexibility | Token Efficiency | Parallel | Learning | |---|---|---|---|---|---| | **ReAct** | Implicit | High | Low | No | No | | **Plan-and-Execute** | Explicit | Medium | Medium | DAG | Via replan | | **ReWOO** | Explicit | Medium | High | Yes | No | | **Self-Ask** | Decomposition | Low | High | No | No | | **Reflexion** | Implicit | High | Medium | No | Yes | | **LATS** | Search | High | Low | No | Yes | | **PoT** | Implicit | Low | High | No | No | --- ## Choosing a Pattern | Task Profile | Recommended | |---|---| | Simple QA, lookup | ReAct / Self-Ask | | Complex multi-step, dependencies | Plan-and-Execute | | Many independent lookups | ReWOO | | High-value, needs learning | Reflexion / LATS | | Computation-heavy | PoT | | Creative, open-ended | LATS / ToT | | Production, auditable | Plan-and-Execute | --- ## Hybrid Architectures (Real-World) Most production systems combine patterns: ``` ┌─────────────────────────────────────┐ │ Planner (Plan-and-Execute) │ │ - High-level DAG │ │ - Human approval gate │ └──────────────┬──────────────────────┘ │ ┌──────────┼──────────┐ ▼ ▼ ▼ ┌──────┐ ┌────────┐ ┌────────┐ │ ReAct │ │ ReWOO │ │ PoT │ │ Agent │ │ Agent │ │ Agent │ └──────┘ └────────┘ └────────┘ │ │ │ └──────────┼──────────┘ ▼ ┌─────────────────────┐ │ Reflexion Loop │ │ - Evaluate result │ │ - Extract lessons │ │ - Update memory │ └─────────────────────┘ ``` --- ## Related Notes - [[Foundations/Core-Abstractions.md]] — Building blocks (tools, memory, planning) - [[Architectures/Cognitive-Architectures.md]] — Soar, ACT-R, neural-symbolic - [[Architectures/Control-Flow.md]] — Sequential, parallel, DAG, dynamic - [[Frameworks/LangGraph.md]] — Implements these patterns as graphs - [[Frameworks/AutoGen.md]] — Multi-agent but single-agent patterns apply - [[Evaluation/Agent-Benchmarks.md]] — Benchmarks for these patterns - [[Frontiers/Self-Improving-Agents.md]] — Reflexion → RLAIF evolution --- *Sources: "ReAct" (Yao et al., 2022), "Plan-and-Solve" (Wang et al., 2023), "ReWOO" (Xu et al., 2023), "Self-Ask" (Press et al., 2022), "Reflexion" (Shinn et al., 2023), "LATS" (Zhou et al., 2023), "Tree of Thoughts" (Yao et al., 2023), "Graph of Thoughts" (Besta et al., 2023), "Program of Thoughts" (Chen et al., 2022), LangChain/AutoGen/CrewAI docs.*