Skill Validation Errors¶

This notebook demonstrates what happens when a skill bundle is malformed during discovery. It covers all fatal errors (the skill is skipped) and cosmetic warnings (the skill is still loaded with a warning).

Type	Class	Consequence
Fatal	MissingSkillMdError	No SKILL.md found in the skill directory
Fatal	InvalidFrontmatterError	Frontmatter delimiters missing or YAML/validation error
Fatal	EmptySkillBodyError	Body after frontmatter is empty or whitespace-only
Cosmetic	NameMismatchWarning	Frontmatter name doesn't match directory name
Cosmetic	NameTooLongWarning	Frontmatter name exceeds 64 characters

Setup Instructions¶

To ensure you have the required dependencies to run this notebook, you'll need to have our llm-agents-from-scratch framework installed on the running Jupyter kernel. To do this, you can launch this notebook with the following command while within the project's root directory:

uv run --with jupyter jupyter lab

Alternatively, if you just want to use the published version of llm-agents-from-scratch without local development, you can install it from PyPI by uncommenting the cell below.

In [1]:

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI
# !pip install llm-agents-from-scratch

# Uncomment the line below to install `llm-agents-from-scratch` from PyPI # !pip install llm-agents-from-scratch

Setup¶

All examples create skill directories under .agents/skills/, the book's adopted project-scoped skills location. Each example cleans up after itself using shutil.rmtree().

Note: Run this notebook from within more-examples/ch06/ so that discover_skills() resolves .agents/skills/ correctly.

In [10]:

import shutil
import warnings
from pathlib import Path

from llm_agents_from_scratch.skills.discovery import validate_skill_dir

SKILLS_DIR = Path(".agents/skills")
SKILLS_DIR.mkdir(parents=True, exist_ok=True)

import shutil import warnings from pathlib import Path from llm_agents_from_scratch.skills.discovery import validate_skill_dir SKILLS_DIR = Path(".agents/skills") SKILLS_DIR.mkdir(parents=True, exist_ok=True)

Fatal Errors¶

Fatal errors prevent a skill from being loaded. During discover_skills(), they are caught internally during the execution of validate_skill_dir(), then re-emitted as SkillSkippedWarning so discovery can continue. Let's see the raw errors raised by calling validate_skill_dir() directly, next.

MissingSkillMdError¶

Raised when the skill directory contains no SKILL.md file.

In [11]:

from llm_agents_from_scratch.errors import MissingSkillMdError

skill_dir = SKILLS_DIR / "missing-skill-md"
skill_dir.mkdir(exist_ok=True)
# no SKILL.md created
try:
    validate_skill_dir(skill_dir)
except MissingSkillMdError as e:
    print(type(e).__name__, ":", e)
finally:
    shutil.rmtree(skill_dir)

from llm_agents_from_scratch.errors import MissingSkillMdError skill_dir = SKILLS_DIR / "missing-skill-md" skill_dir.mkdir(exist_ok=True) # no SKILL.md created try: validate_skill_dir(skill_dir) except MissingSkillMdError as e: print(type(e).__name__, ":", e) finally: shutil.rmtree(skill_dir)

MissingSkillMdError : Missing SKILL.md file in skill directory: .agents/skills/missing-skill-md

Fix: Add a SKILL.md file to the skill directory with valid frontmatter and a non-empty body.

InvalidFrontmatterError¶

Raised when SKILL.md is missing the --- frontmatter delimiters, contains invalid YAML, or fails Pydantic validation (e.g. missing required fields).

In [12]:

from llm_agents_from_scratch.errors import InvalidFrontmatterError

# Case 1: missing frontmatter delimiters
skill_dir = SKILLS_DIR / "invalid-frontmatter"
skill_dir.mkdir(exist_ok=True)
(skill_dir / "SKILL.md").write_text("Just a body with no frontmatter.")
try:
    validate_skill_dir(skill_dir)
except InvalidFrontmatterError as e:
    print("Missing delimiters —", type(e).__name__, ":", e)
finally:
    shutil.rmtree(skill_dir)

from llm_agents_from_scratch.errors import InvalidFrontmatterError # Case 1: missing frontmatter delimiters skill_dir = SKILLS_DIR / "invalid-frontmatter" skill_dir.mkdir(exist_ok=True) (skill_dir / "SKILL.md").write_text("Just a body with no frontmatter.") try: validate_skill_dir(skill_dir) except InvalidFrontmatterError as e: print("Missing delimiters —", type(e).__name__, ":", e) finally: shutil.rmtree(skill_dir)

Missing delimiters — InvalidFrontmatterError : not enough values to unpack (expected 3, got 1)

In [13]:

# Case 2: missing required fields in frontmatter
skill_dir = SKILLS_DIR / "missing-description"
skill_dir.mkdir(exist_ok=True)
(skill_dir / "SKILL.md").write_text(
    "---\n"
    "name: missing-description\n"
    "# description is missing\n"
    "---\n"
    "Some body content.",
)
try:
    validate_skill_dir(skill_dir)
except InvalidFrontmatterError as e:
    print("Missing required field —", type(e).__name__, ":", e)
finally:
    shutil.rmtree(skill_dir)

# Case 2: missing required fields in frontmatter skill_dir = SKILLS_DIR / "missing-description" skill_dir.mkdir(exist_ok=True) (skill_dir / "SKILL.md").write_text( "---\n" "name: missing-description\n" "# description is missing\n" "---\n" "Some body content.", ) try: validate_skill_dir(skill_dir) except InvalidFrontmatterError as e: print("Missing required field —", type(e).__name__, ":", e) finally: shutil.rmtree(skill_dir)

Missing required field — InvalidFrontmatterError : 1 validation error for SkillFrontmatter
description
  Field required [type=missing, input_value={'name': 'missing-description'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing

Fix: Ensure SKILL.md starts with ---, contains at least name and description fields, and ends with a closing --- before the body.

EmptySkillBodyError¶

Raised when the frontmatter is valid but the body after it is empty or whitespace-only.

In [14]:

from llm_agents_from_scratch.errors import EmptySkillBodyError

skill_dir = SKILLS_DIR / "empty-body"
skill_dir.mkdir(exist_ok=True)
(skill_dir / "SKILL.md").write_text(
    "---\n"
    "name: empty-body\n"
    "description: A skill with no body.\n"
    "---\n"
    "   \n",  # whitespace only
)
try:
    validate_skill_dir(skill_dir)
except EmptySkillBodyError as e:
    print(type(e).__name__, "raised (body was empty)")
finally:
    shutil.rmtree(skill_dir)

from llm_agents_from_scratch.errors import EmptySkillBodyError skill_dir = SKILLS_DIR / "empty-body" skill_dir.mkdir(exist_ok=True) (skill_dir / "SKILL.md").write_text( "---\n" "name: empty-body\n" "description: A skill with no body.\n" "---\n" " \n", # whitespace only ) try: validate_skill_dir(skill_dir) except EmptySkillBodyError as e: print(type(e).__name__, "raised (body was empty)") finally: shutil.rmtree(skill_dir)

EmptySkillBodyError raised (body was empty)

Fix: Add instructions to the body section of SKILL.md — the content below the closing ---.

Cosmetic Warnings¶

Cosmetic warnings do not prevent a skill from loading. As you learned in the book, validate_skill_dir() returns them in a list alongside the parsed SkillFrontmatter.

NameMismatchWarning¶

Emitted when the name field in the frontmatter does not match the skill's directory name.

In [15]:

skill_dir = SKILLS_DIR / "my-skill"
skill_dir.mkdir(exist_ok=True)
(skill_dir / "SKILL.md").write_text(
    "---\n"
    "name: different-name\n"  # doesn't match directory 'my-skill'
    "description: A misnamed skill.\n"
    "---\n"
    "Do something useful.",
)
try:
    frontmatter, skill_warnings = validate_skill_dir(skill_dir)
    for w in skill_warnings:
        print(type(w).__name__, ":", w)
finally:
    shutil.rmtree(skill_dir)

skill_dir = SKILLS_DIR / "my-skill" skill_dir.mkdir(exist_ok=True) (skill_dir / "SKILL.md").write_text( "---\n" "name: different-name\n" # doesn't match directory 'my-skill' "description: A misnamed skill.\n" "---\n" "Do something useful.", ) try: frontmatter, skill_warnings = validate_skill_dir(skill_dir) for w in skill_warnings: print(type(w).__name__, ":", w) finally: shutil.rmtree(skill_dir)

NameMismatchWarning : Skill name 'different-name' does not match directory name 'my-skill'.

Fix: Rename either the directory or the name field so they match. Convention is for both to use the same kebab-case name.

NameTooLongWarning¶

Emitted when the skill name exceeds 64 characters.

In [16]:

long_name = "a-very-long-skill-name-that-exceeds-the-sixty-four-character-limit"
print(f"Name length: {len(long_name)} characters")

skill_dir = SKILLS_DIR / long_name
skill_dir.mkdir(exist_ok=True)
(skill_dir / "SKILL.md").write_text(
    f"---\n"
    f"name: {long_name}\n"
    f"description: A skill with a very long name.\n"
    f"---\n"
    f"Do something useful.",
)
try:
    frontmatter, skill_warnings = validate_skill_dir(skill_dir)
    for w in skill_warnings:
        print(type(w).__name__, ":", w)
finally:
    shutil.rmtree(skill_dir)

long_name = "a-very-long-skill-name-that-exceeds-the-sixty-four-character-limit" print(f"Name length: {len(long_name)} characters") skill_dir = SKILLS_DIR / long_name skill_dir.mkdir(exist_ok=True) (skill_dir / "SKILL.md").write_text( f"---\n" f"name: {long_name}\n" f"description: A skill with a very long name.\n" f"---\n" f"Do something useful.", ) try: frontmatter, skill_warnings = validate_skill_dir(skill_dir) for w in skill_warnings: print(type(w).__name__, ":", w) finally: shutil.rmtree(skill_dir)

Name length: 66 characters
NameTooLongWarning : Skill name 'a-very-long-skill-name-that-exceeds-the-sixty-four-character-limit' is 66  characters long, which exceeds the maximum allowed length of 64.

Fix: Shorten the skill name to 64 characters or fewer.

Fatal errors during discover_skills()¶

To re-emphasize the point made earlier, when using discover_skills(), fatal validation errors don't propagate up. Instead, they are caught and re-emitted as SkillSkippedWarning so the rest of the skills directory is still scanned. You can see here with one valid and another invalid skill located in the skills dir, that the broken gets skipped with a warning message.

In [17]:

from llm_agents_from_scratch.skills import SkillScope
from llm_agents_from_scratch.skills.discovery import discover_skills

# valid skill
valid = SKILLS_DIR / "valid-skill"
valid.mkdir(exist_ok=True)
(valid / "SKILL.md").write_text(
    "---\nname: valid-skill\ndescription: A good skill.\n---\nDo something.\n",
)

# broken skill — no SKILL.md
broken = SKILLS_DIR / "broken-skill"
broken.mkdir(exist_ok=True)

try:
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always")
        skills = discover_skills([SkillScope.PROJECT])

    print("Discovered skills:", list(skills.keys()))
    for w in caught:
        print(w.category.__name__, ":", w.message)
finally:
    shutil.rmtree(valid)
    shutil.rmtree(broken)

from llm_agents_from_scratch.skills import SkillScope from llm_agents_from_scratch.skills.discovery import discover_skills # valid skill valid = SKILLS_DIR / "valid-skill" valid.mkdir(exist_ok=True) (valid / "SKILL.md").write_text( "---\nname: valid-skill\ndescription: A good skill.\n---\nDo something.\n", ) # broken skill — no SKILL.md broken = SKILLS_DIR / "broken-skill" broken.mkdir(exist_ok=True) try: with warnings.catch_warnings(record=True) as caught: warnings.simplefilter("always") skills = discover_skills([SkillScope.PROJECT]) print("Discovered skills:", list(skills.keys())) for w in caught: print(w.category.__name__, ":", w.message) finally: shutil.rmtree(valid) shutil.rmtree(broken)

Discovered skills: ['valid-skill']
SkillSkippedWarning : Missing SKILL.md file in skill directory: /home/nerdai/Projects/llm-agents-from-scratch/more-examples/ch06/.agents/skills/broken-skill