Chapter 3 — Working with LLMs¶

Setup Instructions¶

To ensure you have the required dependencies to run this notebook, you'll need to have our llm-agents-from-scratch framework installed on the running Jupyter kernel. To do this, you can launch this notebook with the following command while within the project's root directory:

uv run --with jupyter jupyter lab

Alternatively, if you just want to use the published version of llm-agents-from-scratch without local development, you can install it from PyPi by uncommenting the cell below.

In [1]:

# Uncomment the line below to install `llm-agents-from-scratch` from PyPi
# !pip install llm-agents-from-scratch

# Uncomment the line below to install `llm-agents-from-scratch` from PyPi # !pip install llm-agents-from-scratch

Running an Ollama service¶

To execute the code provided in this section, you’ll need to have Ollama installed on your local machine and have its LLM hosting service running. To download Ollama, follow the instructions found on this page: https://ollama.com/download. After downloading and installing Ollama, you can start a service by opening a terminal and running the command ollama serve.

Examples¶

The code in the book uses asyncio.run() to execute coroutines. To align the book code with this Jupyter notebook, we'll use the nest_asyncio library, which allows for nested async event loops.

In [2]:

# This will allow us to execute `asyncio.run()` calls
import nest_asyncio
nest_asyncio.apply()

# This will allow us to execute `asyncio.run()` calls import nest_asyncio nest_asyncio.apply()

Example 1: Joke model for structured_output()¶

In [3]:

from typing import Literal
from pydantic import BaseModel

class Joke(BaseModel):
    """A structured output model for Jokes."""

    subject: Literal["math", "physics", "biology"]
    joke: str

from typing import Literal from pydantic import BaseModel class Joke(BaseModel): """A structured output model for Jokes.""" subject: Literal["math", "physics", "biology"] joke: str

Example 2: Instantiating an OllamaLLM¶

Note: the below command requies the qwen2.5:3b model to have been pulled. To do so, in terminal run: ollama pull qwen2.5:3b.

In [4]:

from llm_agents_from_scratch.llms.ollama import OllamaLLM

llm = OllamaLLM(model="qwen2.5:3b")

from llm_agents_from_scratch.llms.ollama import OllamaLLM llm = OllamaLLM(model="qwen2.5:3b")

Example 3: Using complete() to have an OllamaLLM tell a joke¶

In [5]:

import asyncio
from llm_agents_from_scratch.llms.ollama import OllamaLLM

async def main():
    llm = OllamaLLM(model="qwen2.5:3b")
    response = await llm.complete("Tell me a joke.")
    print(response)

asyncio.run(main())

import asyncio from llm_agents_from_scratch.llms.ollama import OllamaLLM async def main(): llm = OllamaLLM(model="qwen2.5:3b") response = await llm.complete("Tell me a joke.") print(response) asyncio.run(main())

response='Sure! Here\'s one for you:\n\nWhy don\'t scientists trust atoms?\n\nBecause they make up everything!\n\nThis is a play on the phrase "they make up matter," and in this case, it makes no sense because we can\'t blame or rely on atoms to do things as humans would.' prompt='Tell me a joke.'

Example 4: Using structured_output() to have an OllamaLLM tell a (structured) joke¶

In [6]:

async def main():
    llm = OllamaLLM(model="qwen2.5:3b")
    prompt = ("Tell me a joke.")
    joke = await llm.structured_output(prompt=prompt, mdl=Joke)
    print(joke.__class__.__name__)
    print(joke)

asyncio.run(main())

async def main(): llm = OllamaLLM(model="qwen2.5:3b") prompt = ("Tell me a joke.") joke = await llm.structured_output(prompt=prompt, mdl=Joke) print(joke.__class__.__name__) print(joke) asyncio.run(main())

Joke
subject='math' joke='Why was the math book sad? Because it had too many problems.'

Example 5: Hailstone tool call with OllamaLLM¶

In [7]:

def hailstone_step_func(x: int) -> int:
    """Performs a single step of the Hailstone sequence."""
    if x % 2 == 0:
        return x // 2
    return 3 * x + 1

def hailstone_step_func(x: int) -> int: """Performs a single step of the Hailstone sequence.""" if x % 2 == 0: return x // 2 return 3 * x + 1

Eliciting a tool call request¶

In [8]:

import asyncio
from llm_agents_from_scratch.llms.ollama import OllamaLLM
from llm_agents_from_scratch.data_structures.llm import ChatMessage
from llm_agents_from_scratch.tools import SimpleFunctionTool

hailstone_tool = SimpleFunctionTool(hailstone_step_func)
llm = OllamaLLM(model="qwen2.5:3b")

async def main():
    user_input = (
        "What is the result of taking the next step of the "
        "Hailstone sequence on the number 3?\n\n"
        "Be very succinct in your response."
    )
    return await llm.chat(
        user_input,
        tools=[hailstone_tool],
    )

user_msg, response_msg = asyncio.run(main())
print(response_msg.tool_calls)

import asyncio from llm_agents_from_scratch.llms.ollama import OllamaLLM from llm_agents_from_scratch.data_structures.llm import ChatMessage from llm_agents_from_scratch.tools import SimpleFunctionTool hailstone_tool = SimpleFunctionTool(hailstone_step_func) llm = OllamaLLM(model="qwen2.5:3b") async def main(): user_input = ( "What is the result of taking the next step of the " "Hailstone sequence on the number 3?\n\n" "Be very succinct in your response." ) return await llm.chat( user_input, tools=[hailstone_tool], ) user_msg, response_msg = asyncio.run(main()) print(response_msg.tool_calls)

[ToolCall(id_='96efe041-50ff-45f0-867b-8e493901a28f', tool_name='hailstone_step_func', arguments={'x': 3})]

Executing the tool¶

In [9]:

tool_call = response_msg.tool_calls[0]
tool_call_result = hailstone_tool(tool_call) # a ToolCallResult
print(tool_call_result)

tool_call = response_msg.tool_calls[0] tool_call_result = hailstone_tool(tool_call) # a ToolCallResult print(tool_call_result)

tool_call_id='96efe041-50ff-45f0-867b-8e493901a28f' content='10' error=False

Sending tool results back to LLM¶

In [10]:

async def main():
    return await llm.continue_chat_with_tool_results(
        tool_call_results=[tool_call_result],
        chat_history=[user_msg, response_msg],
    )
    
tools_msg, final_response = asyncio.run(main())
print(final_response)

async def main(): return await llm.continue_chat_with_tool_results( tool_call_results=[tool_call_result], chat_history=[user_msg, response_msg], ) tools_msg, final_response = asyncio.run(main()) print(final_response)

role=<ChatRole.ASSISTANT: 'assistant'> content='The result of taking the next step in the Hailstone sequence on the number 3 is 10.' tool_calls=None