The Checking Problem
The risk with AI in research was never that it gets things wrong. It is that it gets things right often enough that you stop looking.
Everyone bracing for AI to get things wrong, is bracing for the wrong thing.
Yes, the models get things wrong. Everyone knows that by now. You read the answer, you raise an eyebrow, you check. The wrongness announces itself, and the checking takes care of it. That failure mode is loud, and loud failures get caught.
The quiet one is the problem. A tool that is right nine times out of ten does not train you to be careful. It trains you to relax. The tenth answer looks exactly like the nine before it, so you wave it through, and this time the number is wrong. Nobody eyeballs it, because the tool has spent two weeks earning a trust it was never supposed to have. That is the checking problem. The better the tool looks, the less anyone checks, and the less anyone checks, the more the one bad answer costs.
In this work that cost is not abstract. An unsourced figure slides into an IC paper. It gets repeated in the memo, quoted in the meeting, and built into the model. By the time anyone asks where it came from, it has three downstream copies and a lot of momentum. The model did not fail there. The process did, because at no point was there a cheap way to look.
Picture how it actually happens. The tool has been solid all week, so when it returns a revenue figure that looks a touch high, you file it as probably fine and move on, because pulling the source means finding the right page in a 200-page document and you have four more sections to get through before lunch. Nothing about that decision is lazy. It is the only rational call when checking is slow. Which is exactly why slow checking is dangerous.
The tempting answer is a better model. If it were right ten times out of ten, checking would not matter. But no one can promise that, and I would be wary of anyone who does. More to the point, it misreads the problem. The issue is not the error rate. It is that checking got too expensive to bother with, so a rational, busy analyst skips it. You do not fix that with a smarter black box. You fix it by making the check cheap.
That is the whole design idea behind how we build. Every answer keeps a citation. The source is one click away, on the exact page. Checking a claim goes from a ten-minute hunt to a one-second glance, and once it is that cheap, people actually do it. The trust moves off the tool and onto the document, which is where it belonged the whole time.
There is a mindset in this, not just a feature. A tool for serious work should be built to be checked, not built to be believed. Confidence is easy to manufacture. A clean sentence and a firm number will earn trust from anyone in a hurry. What is hard, and what actually matters, is making that confidence auditable, so trust is something you verify rather than something you extend.
None of this makes the analyst redundant, and it is not meant to. The reading and the judgement stay human. What changes is that the human stays in the loop by default, because looking is no longer the expensive option. The point of good tooling here is not to be trusted more. It is to make itself easy to doubt.
If you have ever watched a number you were not sure about making it all the way to committee, you already know the checking problem. We built Analyst One around solving it.
See it for yourself: felixresearch.com
