Study session step 1 of 13
Agents and tool use — patterns, frameworks, evaluation
An **agent** is an LLM that autonomously directs its own control flow using tools (function calls) to gather information and take actions. Agentic systems sit on a spectrum:
• **Workflows** — LLMs orchestrated through pre-defined code paths. Predictable, auditable, cheaper. Patterns: prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer.
• **Agents** — LLM decides the control flow. Useful for open-ended tasks where the steps aren't known in advance. Higher variance, more expensive, harder to debug.
Anthropic's 'Building Effective Agents' (2024) recommends **starting with the simplest workflow that solves your task** and only adding agentic complexity when you can prove it's needed.
**Tool use loop:** 1. LLM sees tool definitions in the system prompt/API. 2. LLM decides to call tool T with arguments JSON. 3. Runtime executes T, returns tool_result. 4. LLM integrates result, may call more tools or respond.
Stop conditions: max turns, terminal tool invocation, or model emits final text response.
**Frameworks:** • **Anthropic SDK** (messages + tools + tool_result) — thin, direct. • **LangChain / LangGraph** — state-machine graphs, durable. • **CrewAI, AutoGen, AgentOps** — multi-agent orchestration, evaluation. • **Claude computer-use** — screenshot + keyboard/mouse tools for GUI automation. • **SWE-agent, OpenDevin** — software-engineering agents.
**Evaluation:** SWE-bench (software engineering), OSWorld (computer use), AgentBench, GAIA (general assistant), WebArena (web navigation). Absolute scores rise fast; **long-horizon reliability** remains the key open problem — a 95% per-step success rate over 100 steps is <1% task success.
**Production agent considerations:** retry with different tool orderings, checkpointing, human-in-the-loop on high-stakes actions (payments, data deletion), rate limits on self-invocation, budget caps on total tool calls.