Handling Tool Errors Gracefully — PokéAPI¶

This notebook demonstrates how the LLMAgent handles tool errors gracefully.

We define a get_pokemon tool that fetches data from the PokéAPI. When a Pokémon name is not found the tool raises a ValueError. We then give the agent a task with a deliberate typo and observe how it recognises the error and self-corrects.

Setting up the LLM¶

I recommend running these notebooks on Ollama Cloud. It serves much larger models than most laptops can run. The default cloud model is picked by benchmarking candidates on these notebooks, and an automated run checks every notebook against it each week. The notebook status page shows the current model and results.

1. Create an API key at https://ollama.com/settings/keys.
2. Export it before launching Jupyter: export OLLAMA_API_KEY=<your key>.
3. That's all. When a notebook builds its LLM, it detects the key and uses Ollama Cloud.

Prefer to run locally? Install Ollama from https://ollama.com/download and pull the model with ollama pull qwen3:14b. If no key is set, the notebook starts the Ollama service for you, so there is no need to run ollama serve. Local models are smaller, so your outputs will differ from the recorded ones.

Prefer OpenAI or Anthropic?

1. Install the extra: uv sync --extra openai or uv sync --extra anthropic.
2. Export OPENAI_API_KEY or ANTHROPIC_API_KEY before launching Jupyter. If you forget, you will be prompted for it, which keeps the key out of the saved notebook.
3. Set LLM_PROVIDER=openai (or anthropic), or pass provider="openai" to make_llm(). A key on its own never switches providers, so one exported for unrelated work cannot reroute you.

The switch applies wherever a notebook builds its LLM with make_llm(). A notebook that constructs OllamaLLM directly stays on Ollama. Output on gpt-5 or Claude will differ from the recorded outputs, and prompt-sensitive examples (the ch09 evaluator pattern, ch08 supervised trajectories) may behave noticeably differently.

In [1]:

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI
# !pip install llm-agents-from-scratch

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI # !pip install llm-agents-from-scratch

In [2]:

from llm_agents_from_scratch.notebook_utils import make_llm

from llm_agents_from_scratch.notebook_utils import make_llm

Defining the Tool¶

In [3]:

import json
import urllib.error
import urllib.request

from llm_agents_from_scratch.tools.simple_function import SimpleFunctionTool


def get_pokemon(name: str) -> str:
    """Look up a Pokémon by name and return its types and base stats."""
    url = f"https://pokeapi.co/api/v2/pokemon/{name.lower().strip()}"
    req = urllib.request.Request(
        url,
        headers={"User-Agent": "llm-agents-from-scratch/1.0"},
    )
    try:
        with urllib.request.urlopen(req) as resp:
            data = json.loads(resp.read())
    except urllib.error.HTTPError as e:
        if e.code == 404:  # noqa: PLR2004
            raise ValueError(
                f"Pokémon '{name}' not found. "
                "Check the spelling and try again.",
            ) from e
        raise
    types = [t["type"]["name"] for t in data["types"]]
    stats = {s["stat"]["name"]: s["base_stat"] for s in data["stats"]}
    return json.dumps({"name": data["name"], "types": types, "stats": stats})


get_pokemon_tool = SimpleFunctionTool(func=get_pokemon)

import json import urllib.error import urllib.request from llm_agents_from_scratch.tools.simple_function import SimpleFunctionTool def get_pokemon(name: str) -> str: """Look up a Pokémon by name and return its types and base stats.""" url = f"https://pokeapi.co/api/v2/pokemon/{name.lower().strip()}" req = urllib.request.Request( url, headers={"User-Agent": "llm-agents-from-scratch/1.0"}, ) try: with urllib.request.urlopen(req) as resp: data = json.loads(resp.read()) except urllib.error.HTTPError as e: if e.code == 404: # noqa: PLR2004 raise ValueError( f"Pokémon '{name}' not found. " "Check the spelling and try again.", ) from e raise types = [t["type"]["name"] for t in data["types"]] stats = {s["stat"]["name"]: s["base_stat"] for s in data["stats"]} return json.dumps({"name": data["name"], "types": types, "stats": stats}) get_pokemon_tool = SimpleFunctionTool(func=get_pokemon)

Example 1 — Successful Lookup¶

A straightforward lookup with the correct name.

In [4]:

import logging

from llm_agents_from_scratch import LLMAgent
from llm_agents_from_scratch.data_structures import Task
from llm_agents_from_scratch.logger import enable_console_logging

enable_console_logging(logging.INFO)

llm = make_llm()
agent = LLMAgent(llm=llm, tools=[get_pokemon_tool])

task = Task(instruction="What type is Pikachu and what are its base stats?")
result = await agent.run(task)

import logging from llm_agents_from_scratch import LLMAgent from llm_agents_from_scratch.data_structures import Task from llm_agents_from_scratch.logger import enable_console_logging enable_console_logging(logging.INFO) llm = make_llm() agent = LLMAgent(llm=llm, tools=[get_pokemon_tool]) task = Task(instruction="What type is Pikachu and what are its base stats?") result = await agent.run(task)

✓ Using Ollama Cloud (kimi-k2.7-code:cloud)
INFO (llm_agents_fs.LLMAgent) :      🚀 Starting task: What type is Pikachu and what are its base stats?
INFO (llm_agents_fs.TaskHandler) :      ⚙️ Processing Step: What type is Pikachu and what are its base stats?
INFO (llm_agents_fs.TaskHandler) :      🛠️ Executing Tool Call: get_pokemon
INFO (llm_agents_fs.TaskHandler) :      ✅ Successful Tool Call: {"name": "pikachu", "types": ["electric"], "stats": {"hp": 35, "attack": 55, "defense": 40, "special-attack": 50, "special-def...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      ✅ Step Result: Pikachu is an **Electric** type Pokémon.

Its base stats are:

| Stat | Value |
|---|---|
| HP | 35 |
| Attack | 55 |
| Defense | 40 |
...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      No new step required.
INFO (llm_agents_fs.LLMAgent) :      🏁 Task completed: Pikachu is an **Electric** type Pokémon.

Its base stats are:

| Stat | Value |
|---|---|
| HP | 35 |
| Attack | 55 |
| Defense | 40...[TRUNCATED]

In [5]:

print(result)

print(result)

Pikachu is an **Electric** type Pokémon.

Its base stats are:

| Stat | Value |
|---|---|
| HP | 35 |
| Attack | 55 |
| Defense | 40 |
| Special Attack | 50 |
| Special Defense | 50 |
| Speed | 90 |

Pikachu is notably fast with a base Speed of 90, but has low HP and defenses.

Example 2 — Graceful Error Recovery¶

We give the agent a deliberate typo ('Pikachuu'). The tool returns a ValueError and the agent observes the error message, self-corrects the spelling, and retries.

In [6]:

llm = make_llm()
agent = LLMAgent(llm=llm, tools=[get_pokemon_tool])

task = Task(instruction="What type is Pikachuu?")
result = await agent.run(task)

llm = make_llm() agent = LLMAgent(llm=llm, tools=[get_pokemon_tool]) task = Task(instruction="What type is Pikachuu?") result = await agent.run(task)

✓ Using Ollama Cloud (kimi-k2.7-code:cloud)
INFO (llm_agents_fs.LLMAgent) :      🚀 Starting task: What type is Pikachuu?
INFO (llm_agents_fs.TaskHandler) :      ⚙️ Processing Step: What type is Pikachuu?
INFO (llm_agents_fs.TaskHandler) :      🛠️ Executing Tool Call: get_pokemon
INFO (llm_agents_fs.TaskHandler) :      ❌ Tool Call Failure: {"error_type": "ValueError", "message": "Internal error while executing tool: Pok\u00e9mon 'Pikachuu' not found. Check the spelli...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      ✅ Step Result: I need to make the following tool-calls:
{
    "id_": "f918e216-0603-46c8-a259-61e7c4a538ca",
    "tool_name": "get_pokemon",
    "argu...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      🧠 New Step: Call get_pokemon with name "Pikachu" to retrieve its type information after correcting the spelling error.
INFO (llm_agents_fs.TaskHandler) :      ⚙️ Processing Step: Call get_pokemon with name "Pikachu" to retrieve its type information after correcting the spelling error.
INFO (llm_agents_fs.TaskHandler) :      🛠️ Executing Tool Call: get_pokemon
INFO (llm_agents_fs.TaskHandler) :      ✅ Successful Tool Call: {"name": "pikachu", "types": ["electric"], "stats": {"hp": 35, "attack": 55, "defense": 40, "special-attack": 50, "special-def...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      ✅ Step Result: Pikachu is an Electric-type Pokémon. Its base stats are:

- HP: 35
- Attack: 55
- Defense: 40
- Special Attack: 50
- Special Defense: 5...[TRUNCATED]
INFO (llm_agents_fs.TaskHandler) :      No new step required.
INFO (llm_agents_fs.LLMAgent) :      🏁 Task completed: Pikachu is an Electric-type Pokémon. Its base stats are:

- HP: 35
- Attack: 55
- Defense: 40
- Special Attack: 50
- Special Defense...[TRUNCATED]

In [7]:

print(result)

print(result)

Pikachu is an Electric-type Pokémon. Its base stats are:

- HP: 35
- Attack: 55
- Defense: 40
- Special Attack: 50
- Special Defense: 50
- Speed: 90