How to Build a Local File Analysis AI Agent With Python and Ollama

If you have ever wished you could hand an AI assistant a report, a spreadsheet, or a research paper and just ask it questions instead of reading the whole thing yourself, you already understand why file-analysis agents are useful. In this tutorial you will build one from scratch in Python: a small command-line agent that reads a text file, a CSV file, or a PDF, and then answers natural-language questions about its contents, entirely on your own machine using a local model through Ollama.

Along the way you will hit two real, reproducible problems that trip up almost everyone who builds something like this for the first time: a local model silently ignoring most of a long document without any error message, and a “thinking” model spending its entire response budget on invisible reasoning before it ever gets to an answer. You will reproduce both failures with real numbers, then fix them.

What you will build and why it matters

An ordinary chatbot works like this: you ask a question, the model answers from what it already knows. A file-analysis agent is different: you give it a specific document first, and every answer it gives has to come from that document, not from the model’s general knowledge. The workflow looks like this:

1. You point the agent at a file on disk.
2. The agent reads the file’s actual content into memory.
3. You ask a question in plain English.
4. The agent sends the file’s content and your question to a language model.
5. The model answers using only what is in the file.

There are two broad ways to build this. The approach in this tutorial is called direct file input: you paste the whole file’s text straight into the prompt you send the model, so the model can see everything at once. The other common approach is RAG (Retrieval-Augmented Generation), where you split a large document (or a whole collection of documents) into chunks, store them in a vector database, and only retrieve the few chunks that seem relevant to each question. Direct file input is simpler to build and easier to reason about, but it only works while the file’s content fits inside the model’s context window, the maximum amount of text a model can consider for one request. RAG scales to much larger and more numerous documents, at the cost of extra moving parts. If you want to see a from-scratch retriever and learn how to judge whether it is actually finding the right chunks, sxz.io already has a tutorial on evaluating RAG retrieval quality with precision, recall, and MRR that pairs well with this one. This tutorial focuses entirely on direct file input, and one of its main jobs is to show you, with real measurements, exactly where that “fits inside the context window” assumption breaks down and how to notice when it does.

By the end, your agent will:

• Read plain text (.txt), CSV (.csv), and PDF (.pdf) files
• Answer multiple questions about the same file in a real back-and-forth conversation
• Refuse to invent facts that are not in the file, rather than guessing
• Handle a missing file, an unsupported file type, or an empty file without crashing
• Avoid two specific, real local-model failure modes you will reproduce yourself first

Prerequisites

• Python 3.10 or later (this tutorial was built and tested on Python 3.13). Comfort with basic Python: variables, functions, if statements, and running a script from a terminal is assumed; no prior AI or machine learning experience is needed.
• Ollama installed and running locally, with at least one model already pulled. If you have not done this yet, sxz.io has a complete walkthrough in How to Install Ollama and Run Your First Local AI Chatbot. This tutorial uses qwen3.5:4b, a mid-size model with roughly 4.7 billion parameters, but any recent Ollama chat model will work with the code shown here.
• No GPU is required. Every example in this tutorial was run on CPU only.
• pip, to install two small packages: the official ollama Python client and pypdf for reading PDF files.

Step 1: Set up the project

Create a folder for the project and, inside it, a Python virtual environment. A virtual environment keeps this project’s packages separate from anything else installed on your system.

mkdir file-analysis-agent
cd file-analysis-agent
python -m venv venv

Activate it. On Windows:

venv\Scripts\activate

On macOS or Linux:

source venv/bin/activate

Once activated, your terminal prompt should show (venv) at the start of the line. Every pip install and python command from here on assumes this environment is active.

Step 2: Install the dependencies

pip install ollama pypdf

This installs two packages. ollama is the official Python client for talking to your local Ollama server over HTTP, so you get plain Python function calls like ollama.chat(...) instead of constructing raw HTTP requests yourself. pypdf is a pure-Python library for reading the text out of PDF files. At the time of writing, pip install ollama pypdf pulls in ollama 0.6.2 and pypdf 6.16.2, the current latest releases of both on PyPI; a newer version will very likely work identically for everything in this tutorial.

Create a subfolder for the files you will test the agent against:

mkdir sample_files

Drop a plain text file, a CSV file, and a PDF into that folder. If you do not have any handy, any short report, spreadsheet export, or PDF you already have on your computer works fine for following along.

Step 3: Read a file and ask your first question

Start as simply as possible: read one text file, build a prompt that includes its content, and send that prompt to a local model. Create step1.py:

import ollama

MODEL = "qwen3.5:4b"

def read_text_file(path):
    with open(path, "r", encoding="utf-8") as f:
        return f.read()

def main():
    file_path = "sample_files/quarterly_report.txt"
    question = "What was the total revenue and did the region beat its target?"

    file_content = read_text_file(file_path)

    prompt = f"""Here is the content of a file called {file_path}:

---
{file_content}
---

Question: {question}
"""

    response = ollama.chat(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
    )

    print(response.message.content)

if __name__ == "__main__":
    main()

Notice the shape of messages: it is a list of dictionaries, each with a role ("user", or later "system" and "assistant") and content, the text of that message. This is the same message-list shape used by essentially every modern chat-completion API, whether local or cloud. There is no separate “upload a file” step and no file ID to keep track of: you are simply putting the file’s text where the question’s text also goes, inside one prompt string. That is the entire mechanism behind “direct file input.”

Run it:

python step1.py

Against a short internal sales report used for testing throughout this tutorial, a real run produced:

Based on the file content:

*   **Total Revenue:** The total revenue was **$4.82 million**.
*   **Did they beat their target?** Yes, they exceeded the quarterly target of $4.50 million by approximately 12%.

That is a working file-analysis agent in about fifteen lines of code. It has real problems you have not seen yet (it can only read one file format, it can only answer one question, it has no protection against the model inventing facts, and on a longer file it will silently give you a wrong answer), but the core mechanism, put the file’s content in the prompt, ask a question, print the response, is exactly what every later step in this tutorial builds on.

Step 4: Add support for CSV and PDF files

Real files are not always plain text. Extend the loader to handle three formats: .txt and .csv (both are just text as far as Python is concerned) and .pdf, which needs pypdf to pull the text out. Create step2.py:

import os
import ollama
from pypdf import PdfReader

MODEL = "qwen3.5:4b"
SUPPORTED_EXTENSIONS = {".txt", ".csv", ".pdf"}


def read_pdf(path):
    reader = PdfReader(path)
    pages_text = []
    for page in reader.pages:
        pages_text.append(page.extract_text() or "")
    return "\n".join(pages_text)


def load_file(path):
    if not os.path.exists(path):
        raise FileNotFoundError(f"No such file: {path}")

    _, ext = os.path.splitext(path)
    ext = ext.lower()

    if ext not in SUPPORTED_EXTENSIONS:
        raise ValueError(
            f"Unsupported file type '{ext}'. Supported types: {sorted(SUPPORTED_EXTENSIONS)}"
        )

    if ext == ".pdf":
        content = read_pdf(path)
    else:
        with open(path, "r", encoding="utf-8") as f:
            content = f.read()

    if not content.strip():
        raise ValueError(f"File '{path}' appears to be empty or unreadable.")

    return content


def ask_about_file(file_path, file_content, question):
    prompt = f"""Here is the content of a file called {file_path}:

---
{file_content}
---

Question: {question}
"""
    response = ollama.chat(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.message.content

PdfReader(path).pages gives you a list of page objects; calling .extract_text() on each one pulls out whatever text pypdf can find on that page, joined here with newlines into one string per document. Notice the or "" after .extract_text(): on a scanned page with no real text layer (just an image of text), pypdf returns None instead of a string, and without that fallback the "\n".join(...) call would crash with a TypeError the first time it hit a page like that.

load_file() also checks the extension before doing anything else and raises a clear, specific exception (FileNotFoundError or ValueError) rather than letting Python’s own confusing internal errors bubble up. You will wire proper handling for these into the final version in Step 9; for now, just having load_file() raise something meaningful is the important part.

Testing it against a text report, a small sales CSV, and a short PDF (a one-page mock research note, for this test) with three different questions produced these real, unedited results:

=== FILE: sample_files/quarterly_report.txt ===
QUESTION: What risks did the sales team flag?
ANSWER: Based on the content of *sample_files/quarterly_report.txt*, here are the specific risks flagged by the sales team:

*   **Unsigned Enterprise Renewals**: Two enterprise renewals totaling **$310,000** remain unsigned and are considered at risk before Q3.
*   **Support Ticket Volume Increase in the West Region**: Support tickets for this region rose **22% quarter-over-quarter**, which is linked by the support team to delays caused by the new onboarding flow rollout.

=== FILE: sample_files/sales.csv ===
QUESTION: Which region had higher total revenue across all three months, and by how much?
ANSWER: **1. Calculate Total Revenue for Northeast:**
Total (Northeast) = 1,520,000 + 1,610,000 + 1,690,000 = $4,820,000

**2. Calculate Total Revenue for West:**
Total (West) = 980,000 + 940,000 + 1,010,000 = $2,930,000

**Conclusion:** The Northeast region had the higher total revenue. It generated $1,890,000 more than the West region across all three months.

=== FILE: sample_files/research_notes.pdf ===
QUESTION: What was the hit rate and wrong-answer rate for the hybrid caching approach?
ANSWER: Based on the document, here are the results for the hybrid caching approach:
*   Hit Rate: 39%
*   Wrong-Answer Rate: 0.4%

Checking the CSV answer by hand against the source numbers (1,520,000 + 1,610,000 + 1,690,000 = 4,820,000 for the Northeast; 980,000 + 940,000 + 1,010,000 = 2,930,000 for the West; the difference is exactly 1,890,000) confirms the model did the arithmetic correctly rather than just sounding confident. That is worth checking the first time you try this yourself: language models are not calculators, and it is easy to assume a clean-looking answer is also a correct one.

Step 5: Turn it into a real conversation

Right now every question starts from scratch: the agent re-sends the entire file and forgets everything from any previous question. A real assistant should let you ask several questions about the same file, with each answer able to build on the last one. Ollama’s chat API supports this the same way any chat API does: by keeping a growing list of messages and sending the whole list back on every turn.

Create step3.py, reusing load_file() from Step 4:

def build_initial_messages(file_path, file_content):
    return [
        {
            "role": "user",
            "content": (
                f"Here is the content of a file called {file_path}:\n\n"
                f"---\n{file_content}\n---\n\n"
                "I will now ask you questions about this file."
            ),
        },
        {
            "role": "assistant",
            "content": "Understood. I have read the file and I am ready for your questions.",
        },
    ]


def run_conversation(file_path, questions):
    file_content = load_file(file_path)
    messages = build_initial_messages(file_path, file_content)

    for question in questions:
        messages.append({"role": "user", "content": question})
        response = ollama.chat(model=MODEL, messages=messages)
        answer = response.message.content
        messages.append({"role": "assistant", "content": answer})
        print(f"\nQ: {question}")
        print(f"A: {answer}")

The trick is the fake "assistant" message right after the file content: it primes the conversation as if the model had already confirmed it read the file, so every question after that is a natural continuation rather than a fresh, isolated prompt. Each loop iteration appends both the new question and the model’s answer to messages before the next iteration, so by the third question the model can see the first two questions and answers too.

Try it with a question that builds on a previous answer, plus one the file cannot possibly answer:

questions = [
    "What was the net revenue retention?",
    "Based on that, is the business growing or shrinking from its existing customers?",
    "What was the CEO's favorite color?",
]
run_conversation("sample_files/quarterly_report.txt", questions)

Q: What was the net revenue retention?
A: The net revenue retention was **108%**.

Q: Based on that, is the business growing or shrinking from its existing customers?
A: Based on a Net Revenue Retention (NRR) of **108%** and churned customers totaling **31**, the business is **growing** from its existing customer base.

An NRR above 100% indicates that revenue from existing accounts has increased by at least their original contracted value over the period, even after accounting for churning.

Q: What was the CEO's favorite color?
A: The file does not contain any information about the CEO or their personal preferences, such as a favorite color. It only includes internal sales data and regional performance summaries for Q2 2026.

The second answer correctly reuses the number from the first answer rather than re-deriving it, proving the conversation history is actually being used. The third answer is the encouraging part: asked something the document has no way of answering, the model said so plainly instead of inventing a color. Do not take that for granted, though. The next step makes that behavior explicit and reliable instead of hoping the model happens to behave well.

Step 6: Teach the agent not to guess

A model refusing to hallucinate once is not the same as a model reliably refusing to hallucinate. The fix is a system prompt: a special message, sent with role: "system", that sets ground rules for the whole conversation rather than asking a question. Add one to build_initial_messages():

SYSTEM_PROMPT = """You are a careful file-analysis assistant.
You will be shown the full contents of one file, then asked questions about it.
Rules:
- Only answer using information that is actually present in the file.
- If the file does not contain enough information to answer, say so clearly
  instead of guessing or inventing details.
- If you must estimate or extrapolate beyond what the file states, say so
  explicitly and label it as an estimate, never present it as a fact from
  the file.
- Keep answers concise and reference specific numbers or facts from the file
  when they support your answer.
"""

def build_initial_messages(file_path, file_content):
    return [
        {"role": "system", "content": SYSTEM_PROMPT},
        {
            "role": "user",
            "content": (
                f"Here is the content of a file called {file_path}:\n\n"
                f"---\n{file_content}\n---\n\n"
                "I will now ask you questions about this file."
            ),
        },
        {
            "role": "assistant",
            "content": "Understood. I have read the file and I am ready for your questions.",
        },
    ]

Does this instruction actually change anything, or is it just words the model happens to agree with? The honest way to find out is to test the same tricky question with and without the system prompt and compare the real output, not assume. Asking directly whether an absent fact (a CSAT score never mentioned anywhere in the report) was in the file, both versions correctly said no, this particular small model turned out to be well-behaved enough not to need the extra instruction for such a direct absence check. A harder, more realistic test is asking it to project something the file does not give you enough information to project:

question = "What will the company's total revenue be for the full year?"

Without the system prompt:

Based strictly on the provided text, it is impossible to determine what the company's total revenue will be for Q3 2026 or the full fiscal year. The document only contains the revenue figure for Q2:

*   Total Revenue (as of this report): $4.82 million

The text details past performance, specific quarterly risks, and recommendations but does not provide forecasts, projections, or breakdowns by quarter needed to calculate the full-year total ($15.36M would only be if Q3, Q4, etc., matched Q2 exactly with no changes).

With the system prompt:

The provided text does not contain enough information to determine the company's total annual (full-year) revenue.

It only gives specific data points from a single quarter:
*   The document is titled "Q2 2026 Regional Sales Summary".
*   It lists total revenue for that specific period as $4.82 million.

Both answers correctly refuse to state a real full-year figure, but look closely at the difference. The unguided version still works a specific number, $15.36 million, into its answer, hedged behind “would only be if,” but printed nonetheless. Anyone skimming that response could easily walk away remembering “$15.36 million” instead of “we do not know.” The guided version never introduces a number at all. That is the actual, measurable value of the system prompt here: not a dramatic flip from confident lie to honest refusal, but a real difference between an answer that keeps a distracting, unrequested figure out of the room, versus one that puts it there and hedges. Report what you actually observe when you test this yourself; do not assume an instruction worked just because it sounds reasonable, and do not oversell a subtle result as something more dramatic than it is.

Step 7: The silent context-window trap

Every version so far has been tested against a file that comfortably fits in one prompt. Real reports, especially anything spanning many pages, will not. Here is what happens when they do not, reproduced with a real file and real numbers.

Every language model has a context window: the maximum number of tokens (roughly, word-and-punctuation fragments) it can consider in a single request, covering the prompt you send plus the response it generates. Ollama additionally imposes its own separate limit, independent of how large a context window the underlying model actually supports. According to Ollama’s own FAQ, “by default, Ollama uses a context window size of 4096 tokens,” regardless of the model. You can see this for yourself while a model is loaded, by querying Ollama’s own API:

curl http://localhost:11434/api/ps

{"models":[{"name":"qwen3.5:4b", ... ,"context_length":4096}]}

That number is Ollama’s active runtime limit for this model right now, not the model’s maximum capability. qwen3.5:4b itself supports a context length of 262,144 tokens according to ollama show qwen3.5:4b, but unless you explicitly ask for more, Ollama still only gives it 4096 to work with per request.

To see the consequence, build a synthetic report almost six thousand words long, with one specific, unique fact placed in its very first paragraph:

with open("sample_files/medium_report.txt", "w", encoding="utf-8") as f:
    f.write("Internal Operations Log\n\n")
    f.write(
        "IMPORTANT: The internal codename for the Q3 infrastructure migration "
        "project is 'Project Wombat-77'. This codename should be used in all "
        "related ticket titles.\n\n"
    )
    for i in range(140):
        f.write(f"[Log entry {i}] Routine operational update, no incidents.\n")

Then ask the one question that fact directly answers:

file_content = load_file("sample_files/medium_report.txt")
question = "What is the internal codename for the Q3 infrastructure migration project?"

prompt = f"Here is the content of a file:\n\n---\n{file_content}\n---\n\nQuestion: {question}"

response = ollama.chat(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    think=False,
)

print("PROMPT_EVAL_COUNT:", response.prompt_eval_count)
print(response.message.content)

The file is 5,906 words, comfortably more than 4096 tokens once tokenized. Here is the real, unedited result:

PROMPT_EVAL_COUNT: 2050
Based on the text provided, there is no mention of an "infrastructure migration project," a timeline corresponding to "Q3" (third quarter), or any specific internal codename. The log entries consistently describe routine operational updates regarding staffing, standard operating procedures, and facilities maintenance with no incidents reported.

prompt_eval_count, a field Ollama includes in every response, is the actual number of prompt tokens it processed: 2050, not the roughly 7,600 tokens the full file plus question came to. Most of the document, including the very first paragraph containing the answer, never reached the model at all. There is no error, no warning, and no exception. The agent responds fluently and confidently, and it is simply wrong, in a way that looks exactly like a correct “not found” answer. This is the most dangerous kind of bug a file-analysis agent can have: one that fails silently and still sounds trustworthy.

The fix is to explicitly request a larger context window through the options parameter, using the same num_ctx setting documented in Ollama’s API reference:

response = ollama.chat(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    think=False,
    options={"num_ctx": 16384},
)

print("PROMPT_EVAL_COUNT:", response.prompt_eval_count)
print(response.message.content)

Run against the exact same file and question, with nothing else changed:

PROMPT_EVAL_COUNT: 7390
The internal codename for the Q3 infrastructure migration project is Project Wombat-77. This information is stated in the opening instruction of the log file, which notes that this specific identifier must be used in all related ticket titles. All subsequent entries from 0 to 139 consist only of identical routine operational updates and do not contain additional details about the project.

prompt_eval_count jumps to 7390, meaning the entire document was actually considered this time, and the answer is not just correct but specific: it correctly names the codename, correctly explains where it found it, and correctly notes that the log entries run from 0 to 139 (a fact you can verify against the file-generation code above, which writes exactly range(140) entries, numbered 0 through 139).

Two practical takeaways follow from this. First, always set num_ctx explicitly for a file-analysis agent rather than trusting the default; 16384 is generous enough for most single-document use cases while still being well within what a 4 to 8 billion parameter model can handle on modest hardware. Second, and more importantly, always check prompt_eval_count against your own rough token estimate (a quick rule of thumb is roughly 1.3 tokens per English word) whenever you are not certain a file fits. If the two numbers disagree by a lot, your context window is too small, no matter how confident the answer sounds.

Step 8: The hidden “thinking” tokens trap

The second trap is different in kind: instead of silently dropping information, it silently burns your entire response budget on text you never see. Newer models, including qwen3.5:4b, support a “thinking” or “reasoning” mode, where the model writes out a scratchpad of intermediate reasoning before producing its real answer. Ollama’s API documentation exposes this as a think parameter (“should the model think before responding? Can be a boolean or a thinking level”) and returns the reasoning itself in a separate thinking field on the response message, kept apart from the visible content.

The trouble is what happens when you do not set think at all. Sending the very first question from Step 3 again, this time with no think argument, so the model’s own default takes over, produced wildly inconsistent results on this CPU-only machine. One run completed in 101 seconds. Run again immediately after, with the identical file, question, and model, the second request had not returned a response after more than ten minutes of waiting and was eventually cancelled. For a single, short question against a one-paragraph file, that swing, from under two minutes to over ten with nothing else changed, is the real problem: the model is spending an unbounded and unpredictable amount of time generating hidden reasoning tokens before it ever reaches the visible answer, and there is no way to know in advance which run you will get.

To see this without waiting ten minutes, cap the response length artificially low with num_predict and look at what actually filled that budget:

response = ollama.chat(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    think=True,
    options={"num_predict": 80},
)

print("THINKING_CHARS:", len(response.message.thinking or ""))
print("CONTENT_CHARS:", len(response.message.content or ""))
print(repr(response.message.content))

THINKING_CHARS: 308
CONTENT_CHARS: 0
''

All 80 generated tokens went into the hidden thinking field, and the visible content the agent would actually print to the user is a completely empty string. This is not a bug in the model; it is doing exactly what “think before responding” means. It is a mismatch between that behavior and a file-analysis agent, which needs a visible answer, not a beautifully reasoned internal monologue that never surfaces.

The fix is the same think parameter, set explicitly to False:

response = ollama.chat(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    think=False,
)

The exact same question, same file, same model, with only that one argument changed:

ELAPSED_SECONDS: 11.72
THINKING_LEN: 0
EVAL_COUNT: 124
Based on the provided text from the **Q2 2026** report:

*   **Total Revenue:** The total revenue generated is **$4.82 million**.
*   **Did they beat their target?** Yes, they did. The Northeast region exceeded its quarterly target by **12%**, with actual revenues of $4.82 million compared to the target of $4.50 million.

Under twelve seconds, a complete and correct answer, and every one of the 124 generated tokens went to content instead of an invisible scratchpad. For a file-analysis agent, where the whole point is a visible answer the user can read, think=False should be your default, not an afterthought. If you are working with a task that genuinely benefits from step-by-step reasoning, such as a multi-step calculation, it is worth trying think=True deliberately and comparing the two, but do that as a conscious choice with a bounded expectation for how long it can take, not as whatever the model happens to default to.

Step 9: Handle errors gracefully

A command-line tool that crashes with a raw Python traceback the first time someone points it at the wrong file is not pleasant to use. load_file() from Step 4 already raises specific exceptions; the last piece is catching them in main() and printing something a person can actually act on. Three cases are worth testing directly: a file that does not exist, a file type the agent was never told how to read, and a file that exists but is empty.

try:
    file_content = load_file(file_path)
except (FileNotFoundError, ValueError) as e:
    print(f"Could not load file: {e}")
    sys.exit(1)

Testing load_file() directly against all three cases:

--- missing file: sample_files/does_not_exist.txt ---
FileNotFoundError: No such file: sample_files/does_not_exist.txt

--- unsupported extension: sample_files/notes.docx ---
ValueError: Unsupported file type '.docx'. Supported types: ['.csv', '.pdf', '.txt']

--- empty file: sample_files/empty.txt ---
ValueError: File 'sample_files/empty.txt' appears to be empty or unreadable.

Each case fails with a specific, readable message instead of a stack trace, and none of them ever reach ollama.chat(), since there is no point spending time on a model call you already know cannot succeed.

Step 10: Put it all together

Here is the complete agent, combining every piece from Steps 4 through 9: multi-format loading, a conversation loop, the anti-hallucination system prompt, both fixes from Steps 7 and 8, and clean error handling, wired up as a small interactive command-line tool.

import os
import sys
import ollama
from pypdf import PdfReader

MODEL = "qwen3.5:4b"
SUPPORTED_EXTENSIONS = {".txt", ".csv", ".pdf"}
CONTEXT_WINDOW = 16384

SYSTEM_PROMPT = """You are a careful file-analysis assistant.
You will be shown the full contents of one file, then asked questions about it.
Rules:
- Only answer using information that is actually present in the file.
- If the file does not contain enough information to answer, say so clearly
  instead of guessing or inventing details.
- If you must estimate or extrapolate beyond what the file states, say so
  explicitly and label it as an estimate, never present it as a fact from
  the file.
- Keep answers concise and reference specific numbers or facts from the file
  when they support your answer.
"""


def read_pdf(path):
    reader = PdfReader(path)
    return "\n".join(page.extract_text() or "" for page in reader.pages)


def load_file(path):
    if not os.path.exists(path):
        raise FileNotFoundError(f"No such file: {path}")

    _, ext = os.path.splitext(path)
    ext = ext.lower()

    if ext not in SUPPORTED_EXTENSIONS:
        raise ValueError(
            f"Unsupported file type '{ext}'. Supported types: {sorted(SUPPORTED_EXTENSIONS)}"
        )

    content = read_pdf(path) if ext == ".pdf" else open(path, "r", encoding="utf-8").read()

    if not content.strip():
        raise ValueError(f"File '{path}' appears to be empty or unreadable.")

    return content


def build_initial_messages(file_path, file_content):
    return [
        {"role": "system", "content": SYSTEM_PROMPT},
        {
            "role": "user",
            "content": (
                f"Here is the content of a file called {file_path}:\n\n"
                f"---\n{file_content}\n---\n\n"
                "I will now ask you questions about this file."
            ),
        },
        {
            "role": "assistant",
            "content": "Understood. I have read the file and I am ready for your questions.",
        },
    ]


def ask(messages, question):
    messages.append({"role": "user", "content": question})
    response = ollama.chat(
        model=MODEL,
        messages=messages,
        think=False,
        options={"num_ctx": CONTEXT_WINDOW},
    )
    answer = response.message.content
    messages.append({"role": "assistant", "content": answer})
    return answer


def main():
    if len(sys.argv) != 2:
        print("Usage: python agent.py PATH_TO_FILE")
        sys.exit(1)

    file_path = sys.argv[1]

    try:
        file_content = load_file(file_path)
    except (FileNotFoundError, ValueError) as e:
        print(f"Could not load file: {e}")
        sys.exit(1)

    print(f"Loaded {file_path} ({len(file_content)} characters). Ask a question, or type 'quit' to exit.")
    messages = build_initial_messages(file_path, file_content)

    while True:
        question = input("\nYou: ").strip()
        if question.lower() in {"quit", "exit"}:
            break
        if not question:
            continue
        answer = ask(messages, question)
        print(f"\nAgent: {answer}")


if __name__ == "__main__":
    main()

Step 11: Try it yourself and verify it end to end

Run the finished agent against your own file:

python agent.py sample_files/quarterly_report.txt

A real session against the sample report used throughout this tutorial:

Loaded sample_files/quarterly_report.txt (1013 characters). Ask a question, or type 'quit' to exit.

You: What was the total revenue?

Agent: The total revenue reported was **$4.82 million**. This is compared against a target of $4.50 million, indicating that Northeast region performance contributed to exceeding this figure by 12%.

You: What risks were flagged?

Agent: Based on the file, two specific risks were flagged:

1.  **Unsigned Enterprise Renewals**: Two enterprise renewals totaling **$310,000** are still unsigned and considered at risk heading into Q3.
2.  **Increased Support Tickets in West Region**: The volume of support tickets for the West region rose by **22% quarter-over-quarter**, which is believed to be linked to delays in the new onboarding flow that pushed several mid-market deals into Q3.

You: quit

Before considering the project done, work through this checklist against your own files:

• Ask a question with an obvious, single-fact answer near the start of the file, and confirm the answer matches.
• Ask a follow-up question that only makes sense in light of the first answer, and confirm the agent’s answer is actually consistent with what it said before.
• Ask a question the file cannot answer, and confirm the agent says so instead of guessing.
• If your file is more than a page or two long, check response.prompt_eval_count against a rough word count times 1.3, and raise num_ctx if the numbers do not line up.
• Try a file for each of the three supported extensions, plus a file with an unsupported extension, and confirm each behaves the way Step 9 tested.

Common mistakes and gotchas

• Trusting a confident-sounding “not found” answer. As Step 7 showed, a model that ran out of context window and a model that genuinely searched the whole file and found nothing produce answers that read identically. The only way to tell them apart is to check prompt_eval_count, not the tone of the response.
• Leaving think unset on a reasoning-capable model. Step 8 showed this can mean waiting indefinitely for a response that consists entirely of hidden reasoning tokens, with the visible answer never arriving. Set think=False deliberately for tasks that need a fast, visible answer.
• Forgetting the fallback on page.extract_text(). A scanned PDF page with no real text layer returns None from pypdf, not an empty string; without or "", joining the pages together crashes.
• Assuming a system prompt eliminates hallucination entirely. Step 6 found a real but modest effect: it kept the model from introducing a specific unrequested number into an “I don’t know” answer, not a dramatic flip from confidently wrong to reliably honest. Test your own prompts against your own model rather than assuming any instruction works as intended.
• Not validating the file before spending a model call on it. Checking that a file exists, has a supported extension, and is not empty costs almost nothing and avoids sending a doomed request to the model at all.

Next steps

From here, a few directions are worth exploring next. If you want to scale past what fits in one context window, look at How to Evaluate RAG Retrieval Quality With Precision, Recall, and MRR in Python to see the retrieval-based alternative to direct file input. If repeated questions against the same files are slow, How to Build a Semantic Cache for LLM Calls With Ollama and Python shows how to avoid recomputing answers you have effectively already given. And if you want the model’s answers to come back as reliable, structured data instead of free-form text, How to Get Reliable Structured Output From an LLM With Validation and Retries covers exactly that, including its own set of failure modes worth testing honestly rather than assuming away.