Structure for the AI Noise
Every week brings new claims about what AI can do for analysis. This seminar offers noise suppression: a framework for judging any analytic technology by asking what kinds of claims it can make, and what it costs to build it, to run it, and, above all, to trust what it says.
What LLMs Know, and What Decisions Demand
In the first hour, we apply that framework to large language models and locate both their power and their limits. LLMs command an extraordinary breadth of knowledge, but the claims they natively support are associations expressed in language. Decisions demand more: what happens if we act, what would have happened otherwise, and what is it worth to find out before acting.
The Most Expensive Thing in Analysis Is a Free Answer
On costs, the picture is just as asymmetric. An LLM produces a confident, well-written recommendation at almost no cost. But decision-makers in accountable institutions know that a conclusion is only as good as its audit trail, and an LLM's conclusions arrive without one. The reasoning that produced the recommendation is distributed across billions of parameters that no one can read, so the audit must be reconstructed from the outside, by you, at full cost, and again on the next answer, and the next. We call this recurring burden the verification tax.
Skipped Verification Is Not Forgiven, Only Deferred
The tax is easy to skip, because right and wrong answers read equally well. But skipped verification does not disappear. It accumulates quietly until it is collected by whoever benefits from your being wrong: a competitor, a counterparty, an auditor, a review board, a court, or an adversary.
Reasoning You Can Calculate
In the second hour, we introduce the complement. Bayesian networks occupy precisely the territory LLMs do not: explicit variables, quantified causal relationships, and calculation, probabilistic and causal inference you can run, not prose you can only read. Their cost structure is the mirror ima