85 notes · CC BY 4.0
Agentic Engineering
The research behind course 001: harnesses, tool use, context engineering, evaluation, production concerns.
Working notes from the research behind the course: mechanisms, tables, failure modes and sources. They are written for an engineer, not a beginner — the course is the beginner-facing version of the same material.
| Note | Length |
|---|---|
| Overview | |
| AgenticEngineering — The Source of Truth for AI Agent Systems A curated knowledge depot for building, deploying, and advancing autonomous AI agent systems. Covers foundations through frontier research. | 1024 words |
| Architectures | |
| Cognitive Architectures From classic cognitive science to modern neural-symbolic agents. | 1041 words |
| Control Flow Patterns Execution control structures for agent systems: sequential, parallel, DAG, dynamic routing. | 1399 words |
| Human-in-the-Loop Patterns Designing meaningful human control and collaboration in agent systems. | 1085 words |
| Single-Agent Patterns Core architectural patterns for individual agents. Framework-agnostic designs. | 1264 words |
| ContextEngineering | |
| Advanced RAG Patterns Beyond basic retrieval: HyDE, Graph RAG, Agentic RAG, Multi-modal, and production patterns. | 1005 words |
| Context Window Management Optimizing the finite context window for agent performance. | 848 words |
| Knowledge Graphs Structured knowledge representation for agents: construction, querying, and Graph RAG. | 1057 words |
| Memory Systems Long-term, short-term, episodic, semantic, and procedural memory for agents. | 1403 words |
| Multimodal Context Handling vision, audio, video, and structured data in agent context. | 1108 words |
| RAG Fundamentals Retrieval-Augmented Generation: Grounding LLM responses in external knowledge. | 1158 words |
| Domains | |
| Coding Agents Agents specialized for software development: code generation, refactoring, testing, and repository understanding. | 1546 words |
| Creative Agents Agents for creative tasks: writing, design, image generation, video, music, and content creation. | 1898 words |
| Customer-Facing Agents Agents that interact directly with customers: support, sales, onboarding, and personalization. | 1450 words |
| Data Analysis Agents Agents specialized for data cleaning, exploration, visualization, statistical analysis, and insight generation. | 1986 words |
| Infrastructure Agents Agents for managing, provisioning, monitoring, and optimizing infrastructure: cloud, Kubernetes, networking, and security. | 1575 words |
| Research Agents Agents specialized for information gathering, synthesis, hypothesis generation, and experimental design. | 1564 words |
| Scientific Agents Agents for scientific discovery: hypothesis generation, experimental design, simulation, and automated discovery. | 1657 words |
| Evaluation | |
| Agent Benchmarks Standardized benchmarks for evaluating agent capabilities across tasks and domains. | 1302 words |
| Evaluation Frameworks Tools and platforms for evaluating, monitoring, and improving agent performance. | 1422 words |
| Evaluation Metrics Standardized metrics for evaluating agent performance across dimensions. | 1721 words |
| Offline Evaluation Evaluating agents on static datasets without live execution. | 1152 words |
| Online Evaluation Evaluating agents in production with live traffic, A/B testing, and real-time monitoring. | 1647 words |
| Red Teaming Adversarial testing of agents: jailbreaks, prompt injection, extraction, and safety validation. | 1809 words |
| Foundations | |
| Agent Definitions & Taxonomy What is an agent? A precise taxonomy for the agentic era. | 831 words |
| Core Abstractions The building blocks every agent system composes. Framework-agnostic primitives. | 1444 words |
| History & Timeline From Eliza to autonomous agents — the evolution of agentic AI. | 1159 words |
| LLM Fundamentals Core concepts underlying modern Large Language Models. Essential grounding for agent engineering. | 1199 words |
| Frameworks | |
| AgentKit (LangChain Agent Runtime) LangChain's production agent runtime: graph-based, streaming, checkpointing, horizontal scaling. | 1000 words |
| AutoGen Microsoft's multi-agent conversation framework. Designed for agent-to-agent collaboration. | 1132 words |
| CrewAI Role-based, process-driven multi-agent framework. Optimized for structured team workflows. | 1098 words |
| Framework Comparison & Decision Guide Choosing the right agent framework for your use case. | 1077 words |
| Fringe & Experimental Frameworks Cutting-edge, research-oriented, and niche agent frameworks worth tracking. | 1204 words |
| LangGraph Production-grade stateful graphs for agents. Built on LangChain, used by Replit, Uber, Elastic, Cognition. | 1271 words |
| LlamaIndex Agents LlamaIndex's agent framework: query engines, workflow agents, and multi-agent orchestration built on LlamaIndex's retrieval and indexing. | 929 words |
| OpenAI Agents SDK Official OpenAI framework for building agents. Built on Responses API with native handoffs, guardrails, tracing. | 1166 words |
| PydanticAI Type-safe, structured agent framework built on Pydantic v2. Designed for production reliability. | 1053 words |
| Semantic Kernel Microsoft's enterprise-grade SDK for AI orchestration. Planners, skills, and memory for production AI apps. | 946 words |
| Frontiers | |
| Agent-Computer Interface (ACI) Bridging agents with computer interfaces: desktop control, OS integration, and computer use. | 2759 words |
| Agent Societies Multi-agent systems at scale: emergent behaviors, governance, economics, and coordination at societal levels. | 3641 words |
| Formal Verification Mathematical verification of agent properties: model checking, theorem proving, and verification for AI systems. | 2916 words |
| Lifelong Learning Agents that continuously learn and adapt over their lifetime without catastrophic forgetting. | 2594 words |
| Neurosymbolic Integration Combining neural networks with symbolic reasoning for robust, interpretable, and verifiable agents. | 1860 words |
| Research Directions Cutting-edge research directions and open problems in agentic AI systems. | 4703 words |
| Self-Improving Agents Agents that continuously improve their capabilities through experience, feedback, and autonomous learning. | 2450 words |
| Harness | |
| Building a Minimal Harness This guide walks through building the simplest possible agentic harness to understand the core concepts. We'll build a CLI agent that can read… | 1044 words |
| Reference Implementations Studying existing harnesses is one of the best ways to learn how to build one. Here are the key implementations to study, what to learn from each,… | 778 words |
| Permission System The permission system is critical for safety. It controls what the agent can do, when it needs approval, and how to sandbox dangerous operations. | 930 words |
| Tool Registry The tool registry is the catalog of all capabilities available to the agent. It's responsible for defining, discovering, validating, and executing… | 858 words |
| Sub-Agents Sub-agents are specialized agent instances spawned by the main agent to handle specific tasks. They enable parallelization, specialization, and… | 1011 words |
| The Agent Loop The agent loop is the beating heart of any harness. It's the control structure that determines how the agent thinks, acts, and adapts until it… | 962 words |
| Harness Design Principles Building a great agentic harness requires balancing power, safety, usability, and reliability. These principles guide design decisions. | 968 words |
| Harness Architecture A well-designed agentic harness has a layered architecture. Understanding these layers is key to building or extending one. | 993 words |
| Agentic Harnesses An agentic harness is the runtime environment, control system, and infrastructure that wraps an LLM to turn it into a capable, reliable agent.… | 828 words |
| ModelArchitectures | |
| Inference Optimization vLLM, SGLang, TensorRT-LLM, speculative decoding, quantization, and serving at scale. | 1383 words |
| Mixture of Experts (MoE) Architecture, routing, training, and inference optimization for sparse expert models. | 1065 words |
| Model Routing Dynamic selection and cascading between models for cost, quality, and latency optimization. | 1083 words |
| Multimodal Foundations Vision-language, audio, and unified multimodal architectures for agents. | 1007 words |
| Reasoning Models Chain-of-Thought, Tree-of-Thoughts, self-consistency, process reward models, and test-time compute scaling. | 1246 words |
| Small Language Models (SLMs) Models under 10B parameters: distillation, quantization, edge deployment, and specialization. | 1212 words |
| MultiAgent | |
| Multi-Agent Communication Protocols Standardized communication between agents: message formats, protocols, and patterns. | 1091 words |
| Multi-Agent Coordination Patterns Patterns for orchestrating multiple agents: supervision, swarms, hierarchies, and marketplaces. | 1951 words |
| Delegation & Handoff Patterns for transferring work between agents: task decomposition, context transfer, and seamless handoffs. | 1495 words |
| Emergent Behavior When multi-agent systems exhibit behaviors not explicitly programmed: self-organization, specialization, and collective intelligence. | 1239 words |
| Negotiation & Consensus Mechanisms for agents to reach agreements: voting, auctions, bargaining, and consensus algorithms. | 1825 words |
| Safety & Coordination Ensuring multi-agent systems operate safely: containment, monitoring, guardrails, and failure modes. | 1628 words |
| Production | |
| Compliance Regulatory compliance for agent systems: GDPR, SOC2, HIPAA, PCI-DSS, AI regulations, and audit readiness. | 2008 words |
| Cost Optimization Strategies for optimizing agent system costs: compute, tokens, storage, and operational efficiency. | 1979 words |
| Deployment Architecture Patterns and patterns for deploying agent systems: containers, serverless, edge, and hybrid. | 1427 words |
| Observability Comprehensive observability for agent systems: logging, metrics, tracing, alerting, and debugging. | 1533 words |
| Reliability Patterns for building reliable agent systems: circuit breakers, retries, timeouts, and failure handling. | 2400 words |
| Scaling Strategies Patterns for scaling agent systems: horizontal, vertical, queue-based, and adaptive. | 1950 words |
| Security Security hardening for agent systems: authentication, authorization, data protection, and threat mitigation. | 2697 words |
| StateManagement | |
| Checkpointing Serializable agent state: persistence, recovery, time-travel, and migration. | 1302 words |
| Distributed State Management State management across multiple nodes: consensus, replication, partitioning, and consistency. | 1273 words |
| Event Sourcing Event-driven state management: immutable event log, projections, replay, and audit. | 1274 words |
| Session Management Conversation threads, multi-user isolation, persistence, and lifecycle. | 1210 words |
| Versioning & Migration Schema evolution, state migration, backward compatibility, and safe deployments. | 1370 words |
| ToolUse | |
| API Integration Connecting agents to external APIs: patterns, authentication, resilience, and best practices. | 1402 words |
| Browser Automation Headless browser control for agents: navigation, extraction, interaction, and testing. | 1165 words |
| Code Execution Sandboxed code execution for agents: environments, security, patterns, and tools. | 1316 words |
| Custom Tools Building, registering, and managing custom tools for agents. | 1409 words |
| Function Calling Structured tool invocation: schemas, providers, patterns, and best practices. | 1254 words |
| Model Context Protocol (MCP) Standardized protocol for connecting AI models to tools, data sources, and capabilities. | 1351 words |
| Tool Discovery Dynamic tool discovery, registration, and capability matching for agents. | 1373 words |