Building with the Claude API, part 2: Tool use and RAG

A walkthrough of the Claude Academy course Building with the Claude API (Build with the API track, 67 lessons, 8 quizzes), split into four parts; this is part 2. Each lesson is summarised with my own tables and examples; take the course for the full lessons. All the courses are listed in the Claude Academy course index.

Note: This summary comes mainly from the public lesson text. A few details were added from docs.claude.com or the sibling Bedrock/Vertex courses, such as reranking, contextual retrieval and adaptive thinking. Verify those details against the lessons and current docs.

4. Tool Use with Claude

Flow

1. You send a request that includes the user message and tool schemas.
2. Claude replies with a tool_use block, and stop_reason == "tool_use".
3. Your server runs the function.
4. You send a tool_result back in a user message, along with the full history.
5. Claude gives the final answer.

• The course’s reminder project uses three tools: get_current_datetime, add_duration_to_datetime and set_reminder.

Tool functions

• Give tools well-named, descriptive names and arguments.
• Validate inputs and raise clear errors. Claude reads the error message and can retry with corrected arguments.

from datetime import datetime
def get_current_datetime(date_format="%Y-%m-%d %H:%M:%S"):
    if not date_format:
        raise ValueError("date_format cannot be empty")
    return datetime.now().strftime(date_format)

JSON schemas

• A schema has three fields:
  • name
  • description: 3–4 sentences covering what the tool does, when to use it and what it returns.
  • input_schema: JSON Schema for the arguments.
• You can ask Claude to write a schema from your function and the tool-use docs.
• Course naming convention: <fn>_schema. Type it with ToolParam for editor help.

from anthropic.types import ToolParam
get_current_datetime_schema = ToolParam({
    "name": "get_current_datetime",
    "description": "Returns the current date and time formatted per date_format. Use when the user needs the current time or a reference point for date math.",
    "input_schema": {
        "type": "object",
        "properties": {"date_format": {"type": "string",
            "description": "strftime format string", "default": "%Y-%m-%d %H:%M:%S"}},
        "required": [],
    },
})

Handling responses and sending results

• The assistant reply can contain several blocks, for example a TextBlock followed by one or more ToolUseBlocks. Each tool-use block has id, name, input and type="tool_use".
• Append the whole response.content to history: {"role": "assistant", "content": response.content}.
• Each tool_result must reference the matching tool_use_id. You still pass tools= on the follow-up request.

def run_tool(name, tool_input):
    if name == "get_current_datetime":   return get_current_datetime(**tool_input)
    if name == "add_duration_to_datetime": return add_duration_to_datetime(**tool_input)
    if name == "set_reminder":           return set_reminder(**tool_input)
    raise Exception(f"Unknown tool: {name}")

def run_tools(message):
    results = []
    for block in message.content:
        if block.type != "tool_use":
            continue
        try:
            out = run_tool(block.name, block.input)
            results.append({"type": "tool_result", "tool_use_id": block.id,
                            "content": json.dumps(out), "is_error": False})
        except Exception as e:
            results.append({"type": "tool_result", "tool_use_id": block.id,
                            "content": f"Error: {e}", "is_error": True})
    return results

def run_conversation(messages, tools):
    while True:
        response = client.messages.create(model=model, max_tokens=1000,
                                          messages=messages, tools=tools)
        messages.append({"role": "assistant", "content": response.content})
        if response.stop_reason != "tool_use":
            return response
        messages.append({"role": "user", "content": run_tools(response)})

Fine-grained tool calling (streaming)

• When you stream a tool call, you receive input_json events. Each carries partial_json, the new chunk, and snapshot, the JSON accumulated so far.
• By default the API buffers and validates each complete top-level key before sending it. This makes chunks bursty.
• Fine-grained streaming turns off that validation, so you get faster and smoother chunks. You must then handle invalid or partial JSON yourself.
• The course enables it through its helper. In the API this has been a beta feature (fine-grained-tool-streaming-2025-05-14); check the current docs.

Built-in (Anthropic-defined) tools

Tool	Schema stub	Who executes
Text editor	{"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"} (Claude 4+). Older models use text_editor_20250124 / str_replace_editor.	You. Claude already knows the schema, but you implement view, str_replace, create and insert on your file system.
Web search	{"type": "web_search_20250305", "name": "web_search", "max_uses": 5, "allowed_domains": ["nih.gov"]}	Anthropic. Must be enabled in the Console settings.

• A web search response contains these blocks:
  • TextBlock
  • ServerToolUseBlock, which shows the query Claude used.
  • WebSearchToolResultBlock, which contains WebSearchResultBlock entries with titles and URLs.
  • Text with citations (CitationsWebSearchResultLocation).
• To render these, show the sources as a list and the citations inline.
• With the text editor, Claude can, for example, read main.py, add a function to calculate pi and create test.py. This is a mini coding assistant.

Extra tool patterns (covered in the Bedrock/Vertex versions)

• Batch tool: a batch_tool takes invocations: [{name, arguments}] so Claude calls several tools in one turn.
• Structured data via tools: force a schema tool with tool_choice={"type": "tool", "name": "article_summary"} and read response.content[0].input. This is more reliable than prefill, but needs more setup.
• tool_choice options:
  • {"type": "auto"} is the default; Claude decides.
  • {"type": "any"} means Claude must use some tool.
  • {"type": "tool", "name": …} means Claude must use that specific tool.

---

5. Retrieval Augmented Generation (RAG)

Why RAG

• Putting a whole document into the prompt has three problems:
  • It hits length limits.
  • Quality drops on long prompts.
  • It costs more money and time.
• RAG splits the document into chunks and puts only the relevant chunks into the prompt. This scales better, but you have to choose a chunking strategy and a retrieval method.

Chunking strategies

Strategy	How	Notes
Size-based	Fixed character count + overlap, e.g. chunk_by_char(text, 150, 20)	Easy and works on any text. Can cut words or sentences, which overlap mitigates. The common fallback.
Structure-based	Split on headers, e.g. re.split(r"\n## ", text)	Best for well-formatted Markdown or HTML.
Sentence-based	re.split(r"(?<=[.!?])\s+", text), then group N sentences with overlap	Middle ground.
Semantic	Group related sentences using NLP or embeddings	Most advanced and most costly.

def chunk_by_char(text, chunk_size=150, chunk_overlap=20):
    chunks, start = [], 0
    while start < len(text):
        end = min(start + chunk_size, len(text))
        chunks.append(text[start:end])
        start = end - chunk_overlap if end < len(text) else len(text)
    return chunks

Text embeddings and semantic search

• An embedding is a list of numbers that represents the meaning of a text. You cannot tell what each individual dimension means.
• Anthropic has no embedding model. The course recommends VoyageAI and stores the key in VOYAGE_API_KEY.

import voyageai
vo = voyageai.Client()
def generate_embedding(text, model="voyage-3-large", input_type="query"):
    return vo.embed([text], model=model, input_type=input_type).embeddings[0]

• Full pipeline:
  1. Chunk the source.
  2. Embed each chunk.
  3. Normalize the vectors.
  4. Store them in a vector DB, together with the original text.
  5. Embed the user query.
  6. Find the nearest chunks by cosine similarity, which ranges from −1 to 1. Cosine distance is 1 − similarity.
  7. Put the top chunks into the prompt.
• The course uses its own VectorIndex, with add_vector(embedding, {"content": chunk}) and search(query_embedding, k).

BM25 lexical search

• Semantic search can miss exact terms such as IDs, for example INC-2023-Q4-011.
• BM25 works in four steps:
  1. Tokenize the query.
  2. Count term frequency in each chunk.
  3. Weight rare terms higher; common words such as “a” count for little.
  4. Rank chunks by best match.
• The course’s BM25Index has add_document({"content": …}) and search(query, k).

Multi-index (hybrid) retrieval

• A Retriever(*indexes) class fans each query out to a VectorIndex and a BM25Index. These share a common protocol of add_document and search.

• It merges the result lists with Reciprocal Rank Fusion:

  RRF(doc) = Σ_indexes 1 / (k + rank). k is usually 60; the course uses 1 to keep the worked example readable, giving merged scores of 0.833, 0.75 and 0.583.

Reranking and contextual retrieval (Bedrock/Vertex lessons; also in this course’s outline)

• LLM reranking: after the hybrid search, send the question and the candidate docs to Claude. Give each doc a short ID and ask for only the IDs, e.g. ["1p5g","51n3"], in order of relevance. Returning IDs is much faster than copying text.
  • Trade-off: better ordering, more latency.
• Contextual retrieval: before indexing, ask Claude to write a short context snippet that situates each chunk within the whole document. Store context + "\n" + chunk in both indexes.
  • For very large documents, use the first few chunks plus the chunks just before the target as the context.

---