Natural-language data interfaces promise to remove the queue between a business question and its answer. But when an LLM-generated database response drives a forecast, staffing decision, inventory action, or executive report, fluency becomes a liability: a wrong number can look exactly like a correct one.

That is the problem Zhelun (Allen) Wu addresses in Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact. Wu argues that accuracy alone is insufficient when users cannot inspect generated SQL—or when the consumer is another agent that will act automatically. The essential requirement is a visible boundary between questions the system can answer reliably and questions it must refuse.

The paper proposes a clear architectural invariant: a component capable of fabrication may influence which question is answered, but never which value is returned. In practical terms, an LLM may interpret language, request clarification, or phrase a response. It should not generate the business-critical query or calculate the resulting number.

Why text-to-SQL accuracy is not enough

A conventional text-to-SQL pipeline sends a user’s request to a language model, generates SQL, executes it, and presents the result. That design can fail silently in several ways:

• The model selects a plausible but incorrect table.
• It maps a business term to the wrong column.
• It aggregates at the wrong grain.
• It overlooks an organization-specific fiscal calendar.
• It silently resolves an ambiguity that has no uniquely correct interpretation.

The database may execute the generated SQL successfully. The result may even be numerically plausible. Neither outcome establishes that the query answered the intended question.

Wu calls systems in which outputs arrive as assertions rather than inspectable workings “fact-consumed systems.” A manager using a conversational analytics interface is one example. An automated agent that reads a returned value and initiates an operational workflow is another—and potentially more consequential—example.

The reliability objective should therefore not be “answer as many questions as possible.” It should be: answer supported questions through a governed path, and make every unsupported question fail visibly.

The trusted-kernel architecture

Wu divides the system into a generative shell and a deterministic kernel.

The shell handles the parts of the interaction where errors are visible and recoverable. It can interpret an underspecified request, extract known entities, ask for missing information, and render the final response. If it asks an irrelevant clarification question, the user loses a turn but does not receive an invented value.

The kernel handles the path from a fully specified question to its result. It decides whether the request belongs to an explicitly supported question shape, compiles that shape to a deterministic query, and delegates computation to the database or another governed execution service.

The two regions meet at a human-readable confirmation. Before execution, the system restates the exact question it intends to answer, including material dimensions such as metric, grain, scope, period, and accounting basis. The user assents or corrects it. Only then does deterministic execution begin.

Wu assigns four roles to this design:

1. Interpreter: Extract what is determinate from the request without filling gaps by invention.
2. Elicitor: Ask for missing information or explain a conflict.
3. Grounder: Return a supported, fully specified question or an explicit failure.
4. Compiler: Map the grounded question to the same governed query every time.

The first two roles may use generative models. The last two must not be generative if the architecture is to preserve the paper’s reliability guarantee.

Structural abstention versus confidence thresholds

Many systems attempt “statistical abstention”: generate an answer, estimate confidence, and suppress the answer when confidence falls below a threshold. That can improve a system, but it still depends on the confidence estimate separating safe outputs from unsafe ones.

Wu proposes structural abstention instead. Unsupported questions cannot be represented by the kernel, so no candidate query exists to approve. The system refuses because the request is outside its authored capability—not because a model reports low confidence.

This distinction matters in production. A confidence threshold is another probabilistic component to calibrate and monitor. Structural refusal follows from a deterministic membership test: the request either matches a supported question shape with valid parameters, or it does not.

The trade-off is equally important. Structural abstention limits coverage by design. It converts some potentially answerable requests into visible refusals, preferring a recoverable delay to an undetectable factual error.

A five-decision implementation framework

Wu reduces implementation to five decisions. They provide a useful architecture review for teams building reliable text-to-SQL agents.

1. Define the unit of answerability

Choose an enumerable, reviewable representation of what the system can answer. It might be a set of typed question templates, parameterized reports, governed metric definitions, or a bounded grammar over a semantic layer.

Each unit should fix the dimensions that determine meaning: metric, aggregation, data grain, filters, time semantics, and eligible groupings. Someone who understands the data model must review these definitions.

2. Represent partial requests explicitly

Do not pass only raw conversation text between turns. Use structured state that supports three deterministic operations:

• Is the request complete?
• Which required elements are missing?
• Is the request internally inconsistent or unsupported?

A typed slot frame is often sufficient. For example, a revenue request might require metric, business_unit, period, currency_basis, and aggregation. Missing fields trigger clarification; incompatible combinations trigger correction rather than execution.

3. Choose a trustworthy compilation target

The confirmed request must map deterministically to execution. Possible targets include parameterized SQL, stored procedures, a governed metrics API, or published reports in a semantic layer.

Wu notes that narrower targets reduce the number of ways an authored mapping can be wrong. Where a governed metrics service already exists, compiling to that service is generally safer than emitting unrestricted SQL.

4. Design a meaningful confirmation

Confirmation is not a generic “Are you sure?” prompt. It must expose the assumptions the user can evaluate.

A financial query might confirm the entity, reporting period, metric definition, currency, and accounting basis. An inventory query might confirm product grain, location scope, snapshot time, and whether reserved stock is included. If a material assumption is omitted, confirmation can create false assurance.

5. Define refusal and recovery

Refusal must be a first-class product state. The system should identify the conflict or missing capability and offer an appropriate recovery path: nearest supported questions, selection of one conflicting category, or escalation to an analyst.

The interface should also distinguish any generative fallback from the deterministic path. Otherwise, users may assume that answers from two paths carry the same reliability properties.

Where the architecture works—and where it does not

Wu identifies three conditions under which the coverage trade-off is justified.

First, the output is consumed as fact. The user or downstream agent will not inspect the generated query before relying on the result.

Second, demand is structurally repetitive. Questions vary in parameters but recur in recognizable shapes, such as sales by region and period, inventory by location and product, or spend by cost centre and accounting basis.

Third, wrong answers are more costly than refusals. A refusal creates delay or a human exception; a wrong answer can propagate into a consequential decision.

If users are SQL-literate analysts exploring novel questions in familiar data, Wu explicitly argues that a generative copilot may be the better tool. Those users can inspect the query, their questions are not repetitive, and a bounded kernel would obstruct exploration. Structural abstention is not a universal replacement for text-to-SQL generation.

Practical production limits

The paper is unusually direct about what this design does not guarantee.

A deterministic compiler can faithfully execute a bad definition. If a template maps a metric to the wrong table or aggregation, the result will still be wrong. The architecture relocates trust to authoring and review; it does not eliminate the need for domain expertise, testing, lineage, version control, and change management.

Coverage also grows through deliberate authoring. Teams must measure direct answers, clarification flows, refusals, corrections after confirmation, and escalations. Wu reports that the production system described in the paper did not separately instrument its three dialogue branches and identifies that omission as a measurement the team should have addressed from deployment.

Confirmation adds friction, particularly for experienced users who already express complete questions. Personalization has a cold start. A tool-using fallback introduces non-determinism and additional latency. And a trace of tool calls offers partial auditability, not the full reproducibility of a fixed compiler.

The reported deployment is also limited evidence. Wu describes roughly two years of production operation in one enterprise analytics domain, but deliberately excludes quantitative accuracy, latency, adoption, customer, and business-outcome metrics. The paper presents an architectural case and an existence proof, not a controlled evaluation of user-detected error rates.

A concrete go/no-go decision

Before adopting structural abstention, score the proposed workflow against four questions:

1. Verification: Can the consumer competently inspect the query before acting? If yes, prefer a copilot with review tooling.
2. Repetition: Can most valuable requests be expressed as a manageable set of governed shapes? If no, use an exploratory or hybrid architecture.
3. Error cost: Is a plausible wrong result materially worse than a visible refusal? If no, structural constraints may impose unnecessary friction.
4. Governance readiness: Can domain owners author, test, approve, and version the question-to-execution mappings? If no, the deterministic kernel will merely make ungoverned semantics repeatable.

A strong “no” on verification, combined with strong “yes” answers on repetition, error cost, and governance readiness, is the clearest fit. Mixed cases suggest a hybrid: route supported requests through the kernel and label any generative fallback as a separate reliability class.

Reliable text-to-SQL agents are not built by asking a model to sound less confident. They are built by controlling where generation is allowed, making ambiguity visible, and ensuring that the path from an approved question to a consequential value is deterministic. The resulting system may answer fewer questions. Its advantage is that users—and downstream agents—can know when it has not answered at all.

References

• Wu, Zhelun (Allen). “Never the Number: Structural Abstention for AI Systems Whose Answers Are Consumed as Fact.” arXiv:2608.13926v1, 2026. https://arxiv.org/abs/2608.13926v1