Chapter 3 — Working with LLMs¶

Setup Instructions¶

To ensure you have the required dependencies to run this notebook, you'll need to have our llm-agents-from-scratch framework installed on the running Jupyter kernel. To do this, you can launch this notebook with the following command while within the project's root directory:

uv run --with jupyter jupyter lab

Alternatively, if you just want to use the published version of llm-agents-from-scratch without local development, you can install it from PyPi by uncommenting the cell below.

Setting up the LLM¶

I recommend running these notebooks on Ollama Cloud. It serves much larger models than most laptops can run. The default cloud model is picked by benchmarking candidates on these notebooks, and an automated run checks every notebook against it each week. The notebook status page shows the current model and results.

1. Create an API key at https://ollama.com/settings/keys.
2. Export it before launching Jupyter: export OLLAMA_API_KEY=<your key>.
3. That's all. When a notebook builds its LLM, it detects the key and uses Ollama Cloud.

Prefer to run locally? Install Ollama from https://ollama.com/download and pull the model with ollama pull qwen3:14b. If no key is set, the notebook starts the Ollama service for you, so there is no need to run ollama serve. Local models are smaller, so your outputs will differ from the recorded ones.

Prefer OpenAI or Anthropic?

1. Install the extra: uv sync --extra openai or uv sync --extra anthropic.
2. Export OPENAI_API_KEY or ANTHROPIC_API_KEY before launching Jupyter. If you forget, you will be prompted for it, which keeps the key out of the saved notebook.
3. Set LLM_PROVIDER=openai (or anthropic), or pass provider="openai" to make_llm(). A key on its own never switches providers, so one exported for unrelated work cannot reroute you.

The switch applies wherever a notebook builds its LLM with make_llm(). A notebook that constructs OllamaLLM directly stays on Ollama. Output on gpt-5 or Claude will differ from the recorded outputs, and prompt-sensitive examples (the ch09 evaluator pattern, ch08 supervised trajectories) may behave noticeably differently.

In [1]:

# Uncomment the line below to install `llm-agents-from-scratch` from PyPi
# !pip install llm-agents-from-scratch

# Uncomment the line below to install `llm-agents-from-scratch` from PyPi # !pip install llm-agents-from-scratch

In [2]:

from llm_agents_from_scratch.notebook_utils import ollama_settings

settings = ollama_settings()

from llm_agents_from_scratch.notebook_utils import ollama_settings settings = ollama_settings()

✓ Using Ollama Cloud (deepseek-v4.1-flash)

Examples¶

The code in the book runs its coroutines with asyncio.run(). A Jupyter kernel already has an event loop running, so this notebook awaits the same coroutines directly at the top level of a cell instead.

Example 1: Joke model for structured_output()¶

In [3]:

from typing import Literal
from pydantic import BaseModel

class Joke(BaseModel):
    """A structured output model for Jokes."""

    subject: Literal["math", "physics", "biology"]
    joke: str

from typing import Literal from pydantic import BaseModel class Joke(BaseModel): """A structured output model for Jokes.""" subject: Literal["math", "physics", "biology"] joke: str

Example 2: Instantiating an OllamaLLM¶

Note: Example 2 constructs OllamaLLM directly, because the constructor is the point of these examples, and Examples 3 through 5 reuse that same llm. settings comes from the setup cell above and resolves the model and host the same way make_llm() does in every other notebook: Ollama Cloud when OLLAMA_API_KEY is set, otherwise a local Ollama service running qwen3:14b. LLM_PROVIDER has no effect here, since these examples are about OllamaLLM. The last section shows how to swap in a different LLM.

In [4]:

swap-llm

from llm_agents_from_scratch.llms.ollama import OllamaLLM

llm = OllamaLLM(**settings)

from llm_agents_from_scratch.llms.ollama import OllamaLLM llm = OllamaLLM(**settings)

Example 3: Using complete() to have an OllamaLLM tell a joke¶

In [5]:

response = await llm.complete("Tell me a joke.")
print(response)

response = await llm.complete("Tell me a joke.") print(response)

response='Why don’t scientists trust atoms?  \nBecause they make up everything!' prompt='Tell me a joke.'

Example 4: Using structured_output() to have an OllamaLLM tell a (structured) joke¶

In [6]:

prompt = "Tell me a joke."
joke = await llm.structured_output(prompt=prompt, mdl=Joke)
print(joke.__class__.__name__)
print(joke)

prompt = "Tell me a joke." joke = await llm.structured_output(prompt=prompt, mdl=Joke) print(joke.__class__.__name__) print(joke)

Joke
subject='math' joke="Why was the equal sign so humble? Because it knew it wasn't less than or greater than anyone else."

Example 5: Hailstone tool call with OllamaLLM¶

In [7]:

def hailstone_step_func(x: int) -> int:
    """Performs a single step of the Hailstone sequence."""
    if x % 2 == 0:
        return x // 2
    return 3 * x + 1

def hailstone_step_func(x: int) -> int: """Performs a single step of the Hailstone sequence.""" if x % 2 == 0: return x // 2 return 3 * x + 1

Eliciting a tool call request¶

In [8]:

from llm_agents_from_scratch.data_structures.llm import ChatMessage
from llm_agents_from_scratch.tools import SimpleFunctionTool

hailstone_tool = SimpleFunctionTool(hailstone_step_func)

user_input = (
    "What is the result of taking the next step of the "
    "Hailstone sequence on the number 3? (use the tool.)\n\n"
    "Be very succinct in your response."
)
user_msg, response_msg = await llm.chat(
    user_input,
    tools=[hailstone_tool],
)
print(response_msg.tool_calls)

from llm_agents_from_scratch.data_structures.llm import ChatMessage from llm_agents_from_scratch.tools import SimpleFunctionTool hailstone_tool = SimpleFunctionTool(hailstone_step_func) user_input = ( "What is the result of taking the next step of the " "Hailstone sequence on the number 3? (use the tool.)\n\n" "Be very succinct in your response." ) user_msg, response_msg = await llm.chat( user_input, tools=[hailstone_tool], ) print(response_msg.tool_calls)

[ToolCall(id_='315267c0-d20f-4f7f-b0bf-b3f41194f85b', tool_name='hailstone_step_func', arguments={'x': 3})]

Executing the tool¶

In [9]:

tool_call = response_msg.tool_calls[0]
tool_call_result = hailstone_tool(tool_call) # a ToolCallResult
print(tool_call_result)

tool_call = response_msg.tool_calls[0] tool_call_result = hailstone_tool(tool_call) # a ToolCallResult print(tool_call_result)

tool_call_id='315267c0-d20f-4f7f-b0bf-b3f41194f85b' content='10' error=False

Sending tool results back to LLM¶

In [10]:

tools_msg, final_response = await llm.continue_chat_with_tool_results(
    tool_call_results=[tool_call_result],
    chat_history=[user_msg, response_msg],
)
print(final_response)

tools_msg, final_response = await llm.continue_chat_with_tool_results( tool_call_results=[tool_call_result], chat_history=[user_msg, response_msg], ) print(final_response)

role=<ChatRole.ASSISTANT: 'assistant'> content='10' tool_calls=None

Swapping the LLM¶

OllamaLLM is the LLM the book builds on, but it is not the only one. Examples 3 through 5 only use the llm from Example 2, so to run them on a different LLM, change that one line and re-run the examples.

make_llm() builds whichever LLM the LLM_PROVIDER setting asks for (see the note at the top of this notebook), which is how every notebook from Chapter 4 onward gets its LLM:

from llm_agents_from_scratch.notebook_utils import make_llm

llm = make_llm()

Or construct one of the two optional providers directly. They implement the same BaseLLM interface as OllamaLLM, so complete(), structured_output(), chat() and continue_chat_with_tool_results() work the same way on each:

Class	Extra	Environment variable
OpenAILLM	uv sync --extra openai	OPENAI_API_KEY
AnthropicLLM	uv sync --extra anthropic	ANTHROPIC_API_KEY

from llm_agents_from_scratch.llms.openai import OpenAILLM
from llm_agents_from_scratch.llms.anthropic import AnthropicLLM

llm = OpenAILLM(model="gpt-5")
llm = AnthropicLLM(model="claude-sonnet-5")

Anything that accepts an LLM in later chapters, starting with LLMAgent, accepts any of them.