AI Sales Assistant for Photography Store

Overview

AI Sales Assistant for Photography Store is an agentic RAG assistant for an online camera store. The manuals are the real thing, 65 of them (and growing); the storefront around them is a catalog I assembled so the assistant has to route between product data and documents rather than only answering from documents. It handles two kinds of questions in the same conversation: what the store sells (price, stock, specs, comparisons between two bodies) and how the gear actually works (menu settings, features, shooting technique), with the second kind grounded in the official manuals of the cameras in the catalog.

The catalog currently holds 65 camera bodies. The knowledge base holds the official manual for each of them, plus 10 photography technique guides, around 41,000 indexed chunks in total. The assistant picks its own tools per question, and it is built to say "I don't have that" instead of guessing a spec.

The point was never a chatbot that sounds convincing. It was an assistant that only says what the catalog and the manuals actually support, and a way to prove that with numbers instead of vibes. Everything else follows from that: the tool design, the citations, the golden set, and the habit of measuring retrieval on its own before looking at the final answer.

Status: in progress. Ingestion, retrieval, the agent and the evaluation harness are working. Tracing, deployment and a chat UI are the next pieces.

What it does

• Catalog answers. Price, stock, sensor format, specs, and side by side comparisons, straight from the store's product data.
• Manual answers. How to set white balance on a specific body, whether it has in body stabilization, how to shoot a time lapse. Manual answers cite the manual they came from.
• Technique answers. General photography questions like exposure, metering or bokeh, answered from the store's technique guides instead of a camera manual.
• Honest gaps. If the catalog does not list a spec and the manual does not cover it, the assistant says so rather than filling the hole with world knowledge.
• Clarifying questions. "How do I change white balance?" without naming a camera gets a question back, not a guess.
• Scope control. Anything unrelated to the store or to photography gets politely declined.

Architecture

The system is four layers, and each one can be run and tested without the one above it.

• API. A FastAPI service. POST /ask runs the agent, POST /search returns retrieved chunks directly. Retrieval is also callable in process, without the agent, which is what the first layer of the eval harness exercises.
• Agent. A ReAct agent built with LangGraph, using Claude Sonnet (currently, but easily changable). It gets four tools and a system prompt, and it owns the control flow from there.
• Retrieval. Hybrid search over Postgres with pgvector, followed by a local cross-encoder reranker.
• Storage. One chunks table for the corpus and one products table for the catalog, both in Supabase Postgres. Every chunk carries its document type, and manual chunks also carry the product slug, which is what makes filtered search possible.

There is no hand wired routing graph. An earlier version had one, with retrieve, grade, rewrite and answer steps wired together by hand, and it was replaced once the tools were good enough for the model to route on its own. That moved all the routing logic into two places I can edit as text: the tool descriptions and the system prompt.

A typical request

Take "does the Z8 have in-body stabilization?", which is a real run from the golden set. The agent guesses the slug z8, misses, and the tool hands back the closest matches, so the second call with nikon-z8 lands. The specs come back without a stabilization field. The tool description tells the model that a missing spec field means "not listed" rather than "no", so instead of answering from that gap it searches the Z8 manual, twice, once in the user's words and once in the manual's, and writes the answer from the passages with the manual cited.

Three details make that loop behave:

• The tool docstrings are the routing rules. They are the only view the model has of these functions, so they say when to use each tool, that a slug has to be resolved first, and that an ambiguous question deserves a clarifying question instead of a guess.
• The system prompt owns scope and honesty. Prices are EUR, comparisons fetch each product separately, and anything the tools do not cover is answered with "I don't know" rather than world knowledge.
• The API response is separate from the trace. Callers get {answer, sources}. Evaluation calls a second entry point that returns the same run plus every tool call, every tool output, and just the outputs that count as evidence.

The four tools

• search_products, the browse and lookup tool. Optional filters for name, brand, type, sensor format and stock, combined with AND. Returns compact rows: slug, name, price, stock, and whether a manual is on file. Every camera in the catalog has one right now, so that last column reads yes throughout, but it is there so partial coverage degrades gracefully instead of silently.
• get_product_info, the detail tool. Takes an exact slug and returns price, stock, description and the specs JSON. On a miss it does not just fail, it suggests the closest slugs by matching the query tokens against slug and name, so the model can correct itself in one step instead of guessing again.
• search_manual, the manual tool. Takes a query plus a product slug, and searches only that camera's manual. If no manual is on file it says so and lists the cameras that do have one.
• explain_technique, the guides tool. Searches the photography technique guides for questions that are not about one camera's menus.

The two search tools return two things at once: the formatted text the model reads, and the chunk list itself as structured data. Sources and evaluation read the structured version, so nothing downstream has to parse the formatted string back apart.

The ingestion pipeline

Ingestion is a plain script: load, chunk, embed, store. The details that matter are in the chunking and the metadata.

• Loading. PyMuPDF for manuals, BeautifulSoup for HTML, plain reads for the text guides. Extracted PDF text goes through a cleanup pass first, which normalizes ligature glyphs and rejoins words broken across a line break.
• Section aware chunking. If a manual has a usable table of contents, each TOC entry defines a section that runs from its page to the next entry's page. Every chunk gets its heading path prefixed, so a chunk starts with something like [nikon-z8 manual — The Photo Shooting Menu > Vibration Reduction]. That puts the section's topic into both the embedding and the keyword index, even when the body text uses different words than the question.
• Fallback chunking. Manuals without a TOC, and the text guides, get token aware fixed chunks: 400 tokens with 50 tokens of overlap, counted with tiktoken.
• Embedding. A local model, bge-small-en-v1.5, 384 dimensions, L2 normalized. Passages are embedded raw and queries get the short instruction prefix the model expects. Local means re-indexing the whole corpus costs time and nothing else, which is what makes chunking experiments affordable.
• Metadata. Every chunk stores doc_type (manual or technique), brand and product. A check constraint guards the document type, and a partial index covers product where it is not null.
• Re-runnable by source. Inserting a file deletes that file's existing rows first, so re-ingesting one manual after a chunking change is a one line command and never leaves duplicates behind.

How hybrid retrieval works

product = nikon-z8
doc_type = manual

[nikon-z8 manual — Specifications]
Vibration reduction (VR): camera on-board,
5-axis image sensor shift...

Vector search is good at meaning and bad at exact strings. Keyword search is the opposite. Running both and merging them costs one extra query and covers both failure modes.

Both arms run inside a single SQL statement, filtered to the same metadata slice, and are merged with Reciprocal Rank Fusion:

with q as (select to_tsquery('english', %s) as tsq),
vec as (                               -- arm 1: nearest by meaning
  select id, content, source,
         row_number() over (order by embedding <=> %s::vector) as rank
  from chunks where true and product = %s and doc_type = %s
  order by embedding <=> %s::vector limit 20
),
kw as (                                -- arm 2: best keyword matches
  select id, content, source,
         row_number() over (order by ts_rank_cd(content_tsv, (select tsq from q)) desc) as rank
  from chunks where content_tsv @@ (select tsq from q) and product = %s and doc_type = %s
  limit 20
)
select coalesce(vec.content, kw.content) as content,
       coalesce(1.0 / (60 + vec.rank), 0)
     + coalesce(1.0 / (60 + kw.rank), 0) as score
from vec full outer join kw on vec.id = kw.id
order by score desc limit 20

A few decisions are worth calling out:

• Fusion by rank, not by score. Cosine similarity and ts_rank_cd are on scales that have nothing to do with each other, so normalizing them against each other would be guesswork. RRF only uses position: each arm contributes 1 / (60 + rank), and a chunk found by both arms adds up. The constant 60 is the standard default from the paper.
• A full outer join, not an inner one. A passage that only one arm found still competes. That is the entire point of running two arms.
• The filter sits inside both arms. Wrong camera leakage is impossible by construction rather than by prompt instruction.
• The keyword arm uses an OR tsquery. "white balance x100v" becomes white | balance | x100v, and ts_rank_cd sorts out which matches were dense.
• ef_search = 200 plus iterative scan. The consequence of filtering inside the vector arm, and the subject of the first challenge below.

Then the reranker. The bi-encoder that produced the embeddings is fast because it never sees the query and the passage together. A cross-encoder does exactly that, one pair at a time, which is far sharper and far too slow to run over 41,000 chunks. So it runs over 20 candidates and keeps 5. Cheap and rough narrows the field, slow and sharp decides.

How it is evaluated

Retrieval and generation fail in different ways, so they are scored separately, in three layers over a golden set of 27 questions grouped by failure mode: concept, entity, cross-section, sparse-data, routing, ambiguous and refusal.

• Layer 1, retrieval alone. Ragas context precision and recall on the rows where the right passage is known. Nothing is generated, only judged, so the loop is cheap enough to run on every chunking or reranking change, which is where most of that work gets done.
• Layer 2, trajectory. Assertions on what the agent actually did: which tools it called, whether it asked for clarification, whether it refused when it should have.
• Layer 3, answer quality. Faithfulness and answer relevancy, judged by Gemini rather than by the model that wrote the answers, so the generator is not grading its own homework.

Not everything passes. In the last full run the trajectory layer was 15 of 17, with one row failing to ask a clarifying question and one flagged for refusal wording, both still open.

Eval runs go through a batched path with bounded concurrency and a per question timeout, because unbounded parallel calls trip rate limits and the backoff ends up slower than running them one at a time.

Challenges and how I solved them

Approximate search was quietly dropping the right passage

A question about the Nikon Z8's stabilization scored zero on retrieval even though the correct passage was clearly in the database. The cause was the approximate vector index. With a product filter over tens of thousands of chunks, the index collects its candidates globally first, the filter then throws almost all of them away, and the real matches inside the slice never make it into the pool. Widening the graph search and turning on iterative scanning fixed it, so the search keeps going until the result limit is met after filtering. It was almost certainly also behind a drop in quality I had seen earlier when the corpus grew from 6 manuals to 65, which I had wrongly blamed on dilution.

Chunk boundaries decided the answers

With plain fixed size chunking, the line that answers a question often sat in the middle of a chunk about something else, which diluted the embedding and made the passage hard to retrieve. I rebuilt chunking to follow the manual's own structure, so a chunk never spans two sections and carries its heading path. Precision went up on two manual rows and slipped slightly on a third. While digging into a row that did not improve, I found the reason: 34 of the 65 manuals have no embedded table of contents, so they silently fall back to fixed size chunking. That is the next lever, and it was only visible because retrieval is measured separately from the answer.

The vocabulary gap between question and manual

Users ask about "stabilization" while the manual says "vibration reduction" and "sensor shift". The correct passage was in the candidate pool, but the reranker would not put it in the top 5. I tested a stronger cross encoder offline on the full retrieval set before touching the pipeline, confirmed it fixed one failing question outright and improved several more, checked that the two small regressions were tail end reordering rather than lost signal, and only then swapped it in.

What is next

• Tracing with Langfuse, so a bad answer can be replayed step by step.
• Heading detection for the manuals that have no table of contents, which would extend section aware chunking to half the corpus.
• Query rewriting for the vocabulary gap, so "stabilization" also searches for the words the manual uses.
• Deployment plus a small chat interface on top of the API.