AI Sales Assistant for Photography Store

Overview

Obscura is an agentic RAG assistant for an online camera store, and the storefront it sits behind. The code is on GitHub.

It answers two kinds of question in one conversation: what the store sells (price, stock, specs, comparisons) and how the gear actually works, grounded in the official manuals. The catalog holds 65+ camera bodies with a manual for each, plus 10 technique guides, just under 41,000 indexed chunks and growing.

The point was never a chatbot that sounds convincing. It was an assistant that only says what the catalog and the manuals support, and a way to prove that with numbers instead of vibes.

Status: in progress. Everything below runs locally. Deployment is the next piece.

What it does

• Catalog answers. Price, stock, specs and comparisons, from the store's product data.
• Manual answers. How to set white balance on a specific body, whether it has stabilization, how to shoot a time lapse. Cited back to the manual.
• Technique answers. Exposure, metering, bokeh, from the store's guides rather than a camera manual.
• Honest gaps. If neither the catalog nor the manual covers it, the assistant says so instead of filling the hole with world knowledge.
• Clarifying questions. "How do I change white balance?" without naming a camera gets a question back, not a guess.

Architecture

POST /ask

{ answer, sources }

• Storefront. A Next.js store with a chat dock. The browser never talks to the agent directly; a server route proxies it, and any product the answer mentions is rendered as a preview card.
• API. FastAPI. /ask runs the agent and returns JSON, /ask/stream sends the same answer token by token as it is written, and /search returns chunks on their own so retrieval can be tested without an agent in front of it.
• Agent. A ReAct agent on LangGraph, running Claude Haiku 4.5. Four tools, one system prompt, and the model owns the control flow. Turns are checkpointed in Postgres and addressed by a thread id, so the client sends one message instead of replaying a transcript.
• Retrieval. Hybrid search over Postgres with pgvector, then a local cross-encoder reranker.
• Tracing. Every ask opens a Langfuse span stamped with the model and a prompt version, so a metric change can be tied back to the prompt that caused it.

There is no hand wired routing graph. An earlier version had one and it was replaced once the tools were good enough for the model to route on its own. That put the routing logic in two places I can edit as text: the tool descriptions and the system prompt.

A typical request

Take "Does the Nikon Z8 have in-body stabilization?", a real run from the golden set. The agent resolves the name to a slug, pulls the product record, and finds no stabilization field in the specs. The tool description says a missing field means "not listed" rather than "no", so it searches that camera's manual instead of answering from the gap, and cites what it found.

Two details make that loop behave:

• The tool docstrings are the routing rules. They are the only view the model has of these functions. That makes them prompt engineering, and they break like prompts: an example list in one docstring was read as an allowlist, and a whole class of question quietly stopped retrieving.
• The API response is separate from the trace. Callers get {answer, sources}. Evaluation calls a second entry point that also returns every tool call and every tool output.

Ingestion

Load, chunk, embed, store. The parts that matter:

• Section aware chunking. If a manual has a table of contents, each entry defines a section, and every chunk is prefixed with its heading path. The section's topic then lands in both the embedding and the keyword index, so a passage stays findable when its body text never uses the words a customer would. Manuals without a TOC fall back to 400 token chunks with 50 of overlap.
• Local embeddings. bge-small-en-v1.5, 384 dimensions. Re-indexing the corpus costs time and nothing else, which is what makes chunking experiments affordable.
• Metadata on every chunk. Document type, brand, and the product slug for manuals. This is what makes filtered retrieval possible.

Adding a manual still means running that script by hand. An automated path from upload to indexed chunks is not built yet.

Hybrid retrieval

Vector search is good at meaning and bad at exact strings. Keyword search is the opposite. Both arms run in one SQL statement, filtered to the same slice, and are merged with Reciprocal Rank Fusion:

with q as (select to_tsquery('english', %s) as tsq),
vec as (                               -- arm 1: nearest by meaning
  select id, content, source,
         row_number() over (order by embedding <=> %s::vector) as rank
  from chunks where true and product = %s and doc_type = %s
  order by embedding <=> %s::vector limit 20
),
kw as (                                -- arm 2: best keyword matches
  select id, content, source,
         row_number() over (order by ts_rank_cd(content_tsv, (select tsq from q)) desc) as rank
  from chunks where content_tsv @@ (select tsq from q) and product = %s and doc_type = %s
  limit 20
)
select coalesce(vec.content, kw.content) as content,
       coalesce(1.0 / (60 + vec.rank), 0)
     + coalesce(1.0 / (60 + kw.rank), 0) as score
from vec full outer join kw on vec.id = kw.id
order by score desc limit 20

• Fusion by rank, not by score. Cosine similarity and ts_rank_cd are on unrelated scales, so normalizing one against the other would be guesswork. RRF only uses position, and a chunk found by both arms adds up.
• A full outer join. A passage only one arm found still competes. That is the point of running two.
• The filter sits inside both arms. A question about one camera cannot surface another camera's manual, by construction rather than by instruction.

Then the reranker. The bi-encoder is fast because it never sees query and passage together; a cross-encoder does exactly that, which is sharper and far too slow for 41,000 chunks. So it rescores 20 candidates and keeps 5.

Evaluation

Retrieval and generation fail differently, so they are scored separately, in three layers over a golden set of 27 questions grouped by failure mode.

• Retrieval alone. Context precision and recall where the right passage is known. Nothing is generated, so the loop is cheap enough to run on every chunking or reranking change.
• Trajectory. Assertions on what the agent did: which tools it called, whether it asked, whether it refused. 15 of 15 on the latest run.
• Answer quality. Faithfulness and relevancy, judged by Gemini rather than the model that wrote the answers.

Two bugs only the harness could have caught: the docstring allowlist above, where correct sounding answers had no evidence behind them, and an empty answer bug that one model never triggered and another hit in 13 of 30 traces.

Routing is stochastic, and the harness makes that visible rather than fixing it: the same question can retrieve on one run and answer without a tool on the next.

Three problems worth the write-up

Approximate search was quietly dropping the right passage. A question about the Z8's stabilization scored zero even though the passage was in the database. With a filter over tens of thousands of chunks, the vector index collects candidates globally, the filter throws almost all of them away, and the real matches never enter the pool. Widening the graph search and turning on iterative scanning fixed it. It was most likely also behind an earlier quality drop I had blamed on corpus dilution.

Chunk boundaries decided the answers. With fixed size chunks, the answering line often sat mid-chunk inside something unrelated, which diluted the embedding. Section aware chunking fixed it on two rows and cost a little on a third. Digging into a row that did not move, I found that 34 of the 65 manuals have no embedded table of contents and silently fall back. Only visible because retrieval is measured on its own.

The vocabulary gap. Users say "stabilization", the manual says "vibration reduction". The right passage was in the pool but the reranker would not rank it top 5. I tested a stronger cross-encoder offline on the full set first, confirmed the two small regressions were tail end reordering rather than lost signal, then swapped it in.

What is next

• Deployment, so the storefront and the agent are reachable rather than local.
• Heading detection for the manuals with no table of contents, which would extend section aware chunking to half the corpus.
• Query rewriting for the vocabulary gap.
• An automated indexing path for new manuals.