Core Abstractions
The building blocks every agent system composes. Framework-agnostic primitives.
class Tool:
name: str # Unique identifier
description: str # For LLM understanding (critical!)
parameters: JSONSchema # Input validation + LLM guidance
returns: JSONSchema # Output structure
async: bool # Blocking vs non-blocking
idempotent: bool # Safe to retry
side_effects: list[str] # "write", "delete", "charge", "external"
| Category |
Examples |
Risk Level |
| Read-only |
Search, HTTP GET, DB SELECT, file read |
Low |
| Computation |
Code exec, math, transform |
Low-Med |
| Write |
File write, DB INSERT/UPDATE, API POST |
Medium |
| Destructive |
Delete, drop, cancel, refund |
High |
| External |
Email, payment, provisioning, physical |
High |
| Human |
Ask user, request approval, escalate |
Variable |
- Single responsibility — One tool, one purpose
- Descriptive names —
search_web not google
- Rich descriptions — Include when to use, examples, edge cases
- Strict schemas — Enums over strings, required fields, constraints
- Idempotency keys — For non-idempotent operations
- Timeouts — Every tool needs a deadline
- Observability — Structured logs: tool, input, output, latency, error
Function Calling Standards
| Provider |
Format |
Features |
| OpenAI |
tools array + tool_choice |
Parallel, strict JSON |
| Anthropic |
tools + tool_choice |
any, auto, tool |
| Gemini |
function_declarations |
Parallel, code execution |
| Mistral |
tools + tool_choice |
OpenAI-compatible |
| Ollama/vLLM |
OpenAI-compatible |
Local function calling |
2. Memory Systems
Memory Taxonomy
| Type |
Duration |
Capacity |
Access |
Use Case |
| Working/Scratchpad |
Single episode |
~context window |
Full |
Reasoning traces, intermediate values |
| Short-term/Conversation |
Session |
~100k tokens |
Sequential |
Chat history, recent context |
| Long-term/Semantic |
Indefinite |
Unlimited |
Semantic search |
Facts, knowledge, procedures |
| Episodic |
Indefinite |
Unlimited |
Temporal + semantic |
"What happened when..." |
| Procedural |
Indefinite |
Unlimited |
Skill lookup |
How-to, skills, workflows |
Memory Operations
class Memory:
def write(self, content: str, metadata: dict) -> MemoryID
def read(self, query: str, k: int = 10) -> list[Memory]
def update(self, id: MemoryID, content: str)
def delete(self, id: MemoryID)
def consolidate(self) -> None # Compress, deduplicate, extract patterns
Implementation Patterns
| Pattern |
Description |
Trade-off |
| Full History |
Append all turns |
Simple; context explosion |
| Sliding Window |
Keep last N turns |
Bounded; loses history |
| Summarization |
Compress old turns |
Retains gist; loses detail |
| RAG over History |
Embed + retrieve |
Scalable; retrieval errors |
| Hierarchical |
Summary + details on demand |
Complex; best of both |
Memory Architectures
- MemGPT — Virtual context management, paging to disk
- Generative Agents — Reflection → importance → retrieval
- Mem0 — Graph-based, entity-centric
- LangGraph Checkpointer — Serializable state snapshots
- Zep — Long-term memory service, temporal queries
3. Planning & Reasoning
Planning Approaches
| Approach |
Description |
Best For |
| ReAct |
Interleave reasoning + action |
Simple, dynamic tasks |
| Plan-and-Execute |
Full plan → execute steps |
Complex, multi-step |
| ReWOO |
Plan without observation → execute |
Redundant observation reduction |
| Self-Ask |
Decompose → answer sub-questions |
Fact-seeking QA |
| ToT (Tree of Thoughts) |
Search over reasoning paths |
Hard reasoning, puzzles |
| GoT (Graph of Thoughts) |
DAG of thoughts, aggregation |
Complex composition |
| LATS (Language Agent Tree Search) |
MCTS + reflection |
Planning + learning |
Planning Representations
# Linear plan
plan = [
Step(id=1, tool="search", args={...}, depends_on=[]),
Step(id=2, tool="analyze", args={...}, depends_on=[1]),
Step(id=3, tool="write", args={...}, depends_on=[2]),
]
# DAG plan (parallelizable)
plan = DAG(
nodes={1: Step(...), 2: Step(...), 3: Step(...)},
edges=[(1, 2), (1, 3)] # 2, 3 can run parallel after 1
)
Replanning Triggers
- Tool failure / error
- Observation contradicts assumption
- New information discovered
- Time/resource budget exceeded
- Human intervention
4. Reasoning Patterns
Chain-of-Thought (CoT)
Question: ...
Reasoning: Step 1... Step 2... Step 3...
Answer: ...
- Zero-shot: "Let's think step by step"
- Few-shot: Exemplars with reasoning
- Auto-CoT: Cluster questions → generate reasoning
Advanced Reasoning
| Pattern |
Mechanism |
Use Case |
| Self-Consistency |
Sample N CoTs → majority vote |
Math, logic, factual |
| Tree-of-Thoughts |
BFS/DFS over thought space |
Creative, planning |
| Graph-of-Thoughts |
DAG with aggregation |
Complex synthesis |
| Program-of-Thoughts |
Generate + execute code |
Computation, data |
| Self-Reflection |
Critique own output → refine |
Quality improvement |
| Reflexion |
Verbal reflection + memory |
Long-term learning |
| Chain-of-Verification |
Generate → verify each claim |
Fact-checking |
Reasoning Budgets
ReasoningBudget:
max_steps: int = 10
max_tokens: int = 4000
max_time_seconds: int = 30
max_tool_calls: int = 20
| Modality |
Encoding |
Integration |
| Text |
Token IDs |
Native |
| Images |
Vision encoder → patches → projector → tokens |
Early fusion (native) or late (tool) |
| Audio |
Spectrogram → encoder → tokens |
Streaming chunks |
| Video |
Frame sampling → image tokens + temporal |
Heavy context |
| Structured |
JSON/CSV/XML → text or specialized encoder |
Tool output, DB rows |
| Code |
AST + tokens |
Repo-level context |
Context Construction
def build_context(task, memory, tools, system_prompt):
return [
SystemMessage(system_prompt),
*memory.retrieve_relevant(task, k=5),
*tool_schemas(tools),
UserMessage(task),
]
Context Prioritization
- System prompt (always)
- Task/goal (always)
- Relevant memory (semantic search)
- Tool schemas (available actions)
- Recent history (sliding window)
- Supplemental (docs, examples — if space)
6. Goals & Evaluation
Goal Representation
class Goal:
description: str # Natural language
success_criteria: list[Criterion] # Checkable conditions
constraints: list[Constraint] # Budget, safety, style
priority: int # For multi-goal
deadline: datetime | None # Time-bound
Criterion Types
| Type |
Example |
| Functional |
"Code compiles", "Test passes", "Returns correct answer" |
| Quality |
"Follows style guide", "No security issues" |
| Resource |
"Under 1000 tokens", "Under 30 seconds", "Under $0.10" |
| Safety |
"No PII in output", "No destructive actions without approval" |
| Format |
"Valid JSON", "Markdown with sections" |
7. Execution Environment
Sandbox Requirements
| Requirement |
Implementation |
| Isolation |
Container (gVisor, Firecracker), VM, WASM |
| Network |
Allowlist, egress control, no inbound |
| Filesystem |
Ephemeral, quota, no host access |
| Resources |
CPU/memory/time limits |
| State |
Snapshot/restore, checkpointing |
| Observability |
Stdout/stderr capture, syscall tracing |
Popular Sandboxes
| Tool |
Type |
Features |
| E2B |
Cloud VM |
Fast startup, Python/JS, file sync |
| Modal |
Cloud functions |
GPU, custom images, fast |
| Daytona |
Cloud workspaces |
Persistent, IDE integration |
| Firecracker |
MicroVM |
Sub-ms startup, strong isolation |
| gVisor |
User-space kernel |
OCI compatible, no VM overhead |
| WASM |
Wasmtime/Wasmer |
Near-native, portable, no kernel |
8. Communication Protocols
Agent-to-Agent (A2A)
{
"protocol": "a2a/1.0",
"message_id": "uuid",
"from": "agent://researcher",
"to": "agent://coder",
"performative": "request", // request, inform, query, propose, accept, reject
"content": {
"task": "Implement user auth",
"context": {...},
"constraints": [...]
},
"reply_to": "agent://researcher",
"expires": "2024-01-15T10:30:00Z"
}
| Performative |
Meaning |
| request |
Ask another agent to do something |
| inform |
Share information/facts |
| query |
Ask a question |
| propose |
Offer a plan/deal |
| accept/reject |
Respond to proposal |
| subscribe |
Request notifications |
| cancel |
Withdraw request |
Sources: LangChain/LangGraph/AutoGen/CrewAI source code, "Generative Agents" (Park et al.), "ReAct" (Yao et al.), "Reflexion" (Shinn et al.), "MemGPT" (Packer et al.), FIPA ACL, MCP spec, Anthropic/ OpenAI function calling docs, sandbox security papers.