Grounded Answers Need Citations, Not Confidence_

A building inspector does not need a chatbot that sounds authoritative. They need the section number.

That constraint shaped everything about AskJohn, a question-answering system I built over the 2026 Florida Building Code. The product requirement was not “answer questions about code.” It was “answer questions about code, and show me exactly which section you got that from, so I can go read it myself.” Those are very different systems.

Confidence is not grounding

The failure mode with a general model on regulatory text is not that it refuses to answer. It is that it answers fluently and slightly wrong, and a fluent wrong answer about egress width or fastener schedules is worse than no answer at all. The person asking usually cannot tell the difference, which is the whole problem.

So the retrieval layer is not a nice-to-have wrapped around the model. It is the product. The model’s job is narrow: take the passages it was handed, answer only from those, and cite them. If retrieval fails, the correct behavior is to say so rather than improvise.

Hybrid retrieval, because code text is not prose

Building code has two properties that break naive vector search.

It is full of exact tokens that must match literally. Section numbers, table references, defined terms with specific legal meaning. Semantic similarity will happily return a passage that is about the same topic while missing the one that actually governs.

It is also cross-referential to a degree that is hard to overstate. A section routinely means nothing without the definition and the two sections it points at.

The retrieval layer runs on ParadeDB, which gives BM25 full text search and pgvector in the same Postgres instance. Lexical search catches the exact section and term matches. Vector search over HNSW indexes catches the “what is the rule about attaching a deck to a house” phrasing where the user does not know the vocabulary. Ingestion parses the corpus into sections, chunks, references, and definitions as separate concerns, so the cross-references are structure rather than something the model has to infer from flat text.

Keeping both in one database matters more than it sounds. There is no separate vector store to keep in sync, no second consistency story, and a chunk and its metadata are one row.

A narrow boundary around the expensive part

The system is two halves separated by a deliberately thin HTTP contract.

The web tier is Astro on Bun. It serves pages, runs retrieval, and streams responses. There is no Python in it.

The appliance is a GPU box that serves the model behind two endpoints: one to answer, one to embed. That is the entire contract.

The reason to draw the line there is development. A boundary that narrow can be faked. There is a mock appliance that implements the same two endpoints and returns canned responses, which means the entire application runs end to end on a laptop with no GPU. Retrieval, ingestion, streaming, and the UI are all exercised without the expensive dependency present.

This is the part I would repeat on any project with a heavy or costly component. Decide early what the smallest possible interface to that component is, then make sure the rest of the system can run against a fake version of it. If the boundary is too wide to mock, it is too wide.

It also keeps the deployment question open. The appliance is addressed over HTTP, so whether it is a machine on a desk or a hosted endpoint is a configuration detail rather than a rewrite.

What I would tell someone starting this

Decide what “correct” means before choosing a model. For this system, correct meant traceable to a section, and that single decision determined the database, the ingestion format, and the shape of the prompt.

Treat retrieval as the engineering surface. Most of the work, and most of the quality, lives there.

Make the expensive dependency optional in development. Not because you will not eventually need it, but because a system you can only run in one place is a system you will test badly.