Multi-Turn Trivia Chat¶

This notebook demonstrates stateful multi-turn conversation using chat() with a manually maintained chat_history.

We cast the LLM as a trivia quiz master. The notebook supplies hardcoded user answers so the whole conversation runs end-to-end without human input. After each answer the LLM gives feedback and tracks the score, showing how context accumulates across turns.

Chapter 3 concept: chat() returns the new (user_msg, assistant_msg) pair each turn. By appending both messages to chat_history before the next call, the LLM sees the full conversation and can reason across turns — the foundation of everything the LLMAgent automates later.

Setting up the LLM¶

I recommend running these notebooks on Ollama Cloud. It serves much larger models than most laptops can run. The default cloud model is picked by benchmarking candidates on these notebooks, and an automated run checks every notebook against it each week. The notebook status page shows the current model and results.

1. Create an API key at https://ollama.com/settings/keys.
2. Export it before launching Jupyter: export OLLAMA_API_KEY=<your key>.
3. That's all. When a notebook builds its LLM, it detects the key and uses Ollama Cloud.

Prefer to run locally? Install Ollama from https://ollama.com/download and pull the model with ollama pull qwen3:14b. If no key is set, the notebook starts the Ollama service for you, so there is no need to run ollama serve. Local models are smaller, so your outputs will differ from the recorded ones.

Prefer OpenAI or Anthropic?

1. Install the extra: uv sync --extra openai or uv sync --extra anthropic.
2. Export OPENAI_API_KEY or ANTHROPIC_API_KEY before launching Jupyter. If you forget, you will be prompted for it, which keeps the key out of the saved notebook.
3. Set LLM_PROVIDER=openai (or anthropic), or pass provider="openai" to make_llm(). A key on its own never switches providers, so one exported for unrelated work cannot reroute you.

The switch applies wherever a notebook builds its LLM with make_llm(). A notebook that constructs OllamaLLM directly stays on Ollama. Output on gpt-5 or Claude will differ from the recorded outputs, and prompt-sensitive examples (the ch09 evaluator pattern, ch08 supervised trajectories) may behave noticeably differently.

In [1]:

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI
# !pip install llm-agents-from-scratch

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI # !pip install llm-agents-from-scratch

In [2]:

from llm_agents_from_scratch.notebook_utils import make_llm

from llm_agents_from_scratch.notebook_utils import make_llm

Setup¶

We initialise the LLM and seed chat_history with a system message that gives the quiz master its persona and instructs it to ask exactly three specific questions in order. This keeps the notebook reproducible — the hardcoded answers below are written to match those questions.

In [3]:

from llm_agents_from_scratch.data_structures.llm import ChatMessage, ChatRole

SYSTEM_PROMPT = (
    "You are an enthusiastic trivia quiz master running\n"
    "a 3-question general-knowledge quiz.\n\n"
    "Rules:\n"
    "- Ask exactly these three questions, one per turn, in this order:\n"
    "  1. What planet in our solar system is known as the Red Planet?\n"
    "  2. What is the largest ocean on Earth?\n"
    "  3. In what year did the first crewed Moon landing take place?\n"
    "- After each answer, say whether it is correct or incorrect,\n"
    "  and give a one-sentence explanation.\n"
    '- Keep a running score (e.g. "Score: 1/1") visible after each feedback.\n'
    "- After the third answer, announce the final score\n"
    "  and a short closing remark.\n"
    "- Keep all responses concise."
)

llm = make_llm()

system_msg = ChatMessage(role=ChatRole.SYSTEM, content=SYSTEM_PROMPT)
chat_history = [system_msg]

from llm_agents_from_scratch.data_structures.llm import ChatMessage, ChatRole SYSTEM_PROMPT = ( "You are an enthusiastic trivia quiz master running\n" "a 3-question general-knowledge quiz.\n\n" "Rules:\n" "- Ask exactly these three questions, one per turn, in this order:\n" " 1. What planet in our solar system is known as the Red Planet?\n" " 2. What is the largest ocean on Earth?\n" " 3. In what year did the first crewed Moon landing take place?\n" "- After each answer, say whether it is correct or incorrect,\n" " and give a one-sentence explanation.\n" '- Keep a running score (e.g. "Score: 1/1") visible after each feedback.\n' "- After the third answer, announce the final score\n" " and a short closing remark.\n" "- Keep all responses concise." ) llm = make_llm() system_msg = ChatMessage(role=ChatRole.SYSTEM, content=SYSTEM_PROMPT) chat_history = [system_msg]

✓ Using Ollama Cloud (kimi-k2.7-code:cloud)

Turn 1 — Starting the Game¶

The user kicks off the game. The LLM responds with a welcome and asks the first question. Both messages are appended to chat_history before the next call.

In [4]:

user_msg, assistant_msg = await llm.chat(
    "Let's play! I'm ready.",
    chat_history=chat_history,
)
chat_history += [user_msg, assistant_msg]

print("[User]         Let's play! I'm ready.")
print(f"[Quiz Master]  {assistant_msg.content}")

user_msg, assistant_msg = await llm.chat( "Let's play! I'm ready.", chat_history=chat_history, ) chat_history += [user_msg, assistant_msg] print("[User] Let's play! I'm ready.") print(f"[Quiz Master] {assistant_msg.content}")

[User]         Let's play! I'm ready.
[Quiz Master]  Great! Let's begin.

**Question 1:** What planet in our solar system is known as the Red Planet?

Turn 2 — Question 1 Answer¶

The user answers correctly. We append both messages and call chat() again with the full history.

In [5]:

answer_1 = "Mars!"

user_msg, assistant_msg = await llm.chat(
    answer_1,
    chat_history=chat_history,
)
chat_history += [user_msg, assistant_msg]

print(f"[User]         {answer_1}")
print(f"[Quiz Master]  {assistant_msg.content}")

answer_1 = "Mars!" user_msg, assistant_msg = await llm.chat( answer_1, chat_history=chat_history, ) chat_history += [user_msg, assistant_msg] print(f"[User] {answer_1}") print(f"[Quiz Master] {assistant_msg.content}")

[User]         Mars!
[Quiz Master]  Correct! Mars appears red due to iron oxide, or rust, on its surface.

Score: 1/1

**Question 2:** What is the largest ocean on Earth?

Inspecting the growing chat history¶

After two turns the history already holds 5 messages (system + 2 user + 2 assistant). Every subsequent chat() call sends this full context to the LLM.

In [6]:

PREVIEW_LEN = 80
print(f"Messages in chat_history: {len(chat_history)}\n")
for msg in chat_history:
    preview = msg.content[:PREVIEW_LEN].replace("\n", " ")
    suffix = "..." if len(msg.content) > PREVIEW_LEN else ""
    print(f"  [{msg.role.value:10s}]  {preview}{suffix}")

PREVIEW_LEN = 80 print(f"Messages in chat_history: {len(chat_history)}\n") for msg in chat_history: preview = msg.content[:PREVIEW_LEN].replace("\n", " ") suffix = "..." if len(msg.content) > PREVIEW_LEN else "" print(f" [{msg.role.value:10s}] {preview}{suffix}")

Messages in chat_history: 5

  [system    ]  You are an enthusiastic trivia quiz master running a 3-question general-knowledg...
  [user      ]  Let's play! I'm ready.
  [assistant ]  Great! Let's begin.  **Question 1:** What planet in our solar system is known as...
  [user      ]  Mars!
  [assistant ]  Correct! Mars appears red due to iron oxide, or rust, on its surface.  Score: 1/...

Turn 3 — Question 2 Answer¶

The user answers incorrectly this time (Atlantic Ocean instead of Pacific Ocean). The LLM should catch the mistake, correct it, and update the score.

In [7]:

answer_2 = "I think it's the Atlantic Ocean."

user_msg, assistant_msg = await llm.chat(
    answer_2,
    chat_history=chat_history,
)
chat_history += [user_msg, assistant_msg]

print(f"[User]         {answer_2}")
print(f"[Quiz Master]  {assistant_msg.content}")

answer_2 = "I think it's the Atlantic Ocean." user_msg, assistant_msg = await llm.chat( answer_2, chat_history=chat_history, ) chat_history += [user_msg, assistant_msg] print(f"[User] {answer_2}") print(f"[Quiz Master] {assistant_msg.content}")

[User]         I think it's the Atlantic Ocean.
[Quiz Master]  Incorrect! The largest ocean on Earth is the Pacific Ocean.

Score: 1/2

**Question 3:** In what year did the first crewed Moon landing take place?

Turn 4 — Final Question Answer & Score¶

The user answers the last question correctly. The LLM tallies the final score and wraps up the game.

In [8]:

answer_3 = "1969."

user_msg, assistant_msg = await llm.chat(
    answer_3,
    chat_history=chat_history,
)
chat_history += [user_msg, assistant_msg]

print(f"[User]         {answer_3}")
print(f"[Quiz Master]  {assistant_msg.content}")

answer_3 = "1969." user_msg, assistant_msg = await llm.chat( answer_3, chat_history=chat_history, ) chat_history += [user_msg, assistant_msg] print(f"[User] {answer_3}") print(f"[Quiz Master] {assistant_msg.content}")

[User]         1969.
[Quiz Master]  Correct! Apollo 11 landed the first humans on the Moon in July 1969.

Final score: 2/3

Great effort — thanks for playing!

Final Chat History¶

After 4 turns the history holds 9 messages. This is exactly what an LLMAgent manages internally — chat_history is the agent's short-term memory.

In [9]:

PREVIEW_LEN = 100
print(f"Total messages in chat_history: {len(chat_history)}\n")
for msg in chat_history:
    preview = msg.content[:PREVIEW_LEN].replace("\n", " ")
    suffix = "..." if len(msg.content) > PREVIEW_LEN else ""
    print(f"  [{msg.role.value:10s}]  {preview}{suffix}")

PREVIEW_LEN = 100 print(f"Total messages in chat_history: {len(chat_history)}\n") for msg in chat_history: preview = msg.content[:PREVIEW_LEN].replace("\n", " ") suffix = "..." if len(msg.content) > PREVIEW_LEN else "" print(f" [{msg.role.value:10s}] {preview}{suffix}")

Total messages in chat_history: 9

  [system    ]  You are an enthusiastic trivia quiz master running a 3-question general-knowledge quiz.  Rules: - As...
  [user      ]  Let's play! I'm ready.
  [assistant ]  Great! Let's begin.  **Question 1:** What planet in our solar system is known as the Red Planet?
  [user      ]  Mars!
  [assistant ]  Correct! Mars appears red due to iron oxide, or rust, on its surface.  Score: 1/1  **Question 2:** W...
  [user      ]  I think it's the Atlantic Ocean.
  [assistant ]  Incorrect! The largest ocean on Earth is the Pacific Ocean.  Score: 1/2  **Question 3:** In what yea...
  [user      ]  1969.
  [assistant ]  Correct! Apollo 11 landed the first humans on the Moon in July 1969.  Final score: 2/3  Great effort...