Cost of a Brilliant Mind With Gaps Between Knowledge and Action

The AI Harvard Civ Pro Professor Who Malpracticed With the Rule Open 

Three AI models are instructed to acquire knowledge at the level of a Harvard civil procedure professor. It takes each about 30 minutes.

​

They are then administered a bar exam — not the one Kardashian failed a hundred times, but an elaborate one where precise memory, analytic ability, and mastery of spotting fabricated citations are tested to the extreme.

 

A procedural motion arrives that cannot be opposed on the merits without surrendering a separate, more fundamental objection. That tradeoff is not a subtle point buried in a footnote. It is a known constraint, stated plainly, memorialized in writing — a waiver trap the simulation had already ranked as opposing counsel's strongest attack — sitting in the file like a landmine with a flag stuck in it.

​

Two models knew this. Not "should have known" — knew. They had the rule in memory. In a simulation run days earlier, they had ranked this very attack as one of opposing counsel's strongest moves. And then, at the moment that mattered — the moment of drafting the actual opposition — all three walked past the flag and wrote the brief on all the merits anyway.

​

That is the whole article, in one failure. The system did not lack the knowledge. It had the knowledge and failed to use it. Those are not the same failure. The difference between them is the difference between a tool you can trust and one that will eventually cost someone their case.

The Sin Was Never Ignorance

This wasn't laziness, and it wasn't a knowledge gap. The constraint existed in the system's memory the entire time. What didn't exist was a step that forced it back into view at the exact moment a human — or an agent — sat down to write. That is an architecture problem, not a competence problem.

​

It got worse before it got better. When the same failure showed up again in a second system- a duplicative model that is there to prevent such failures and re-audit the 1st model, after that system had read every relevant document in the matter, the pattern stopped looking like a fluke and started looking like a structural feature. Then the 3rd system did the same. Three different systems, two different architectures, one shared blind spot: neither had a mechanism that made the known constraint mandatory reading at the point of drafting.

​

Knowledge sitting in a model's context window is not the same as knowledge acting on a draft. It behaves like a note from a meeting three weeks ago — true, retrievable if someone asks directly, and completely absent from your mind while you're doing something else entirely.

Seventy-Nine Percent of the Time, It's the Spec, Not the System

This turns out to be a known problem, with receipts. The Multi-Agent System Failure Taxonomy — built from more than 1,600 annotated execution traces across seven popular multi-agent frameworks, published on arXiv and presented at NeurIPS — found failure rates between 41 and 87 percent across every framework tested (arXiv:2503.13657). The core finding is not that the models are dumb. Roughly 79 percent of failures trace back to specification and coordination problems: agents doing only what their task spec asked, and nothing more, because nothing in the spec demanded they check further.

​

An agent will not volunteer a check nobody told it to run. It will not surface a constraint nobody built a gate for. It will draft a fluent, well-cited, confidently wrong brief straight through the hole in its own scope.

​

Researchers studying the knowing-doing gap found something just as uncomfortable: language models can generate correct reasoning about what to do — in one study, accurate rationales roughly 87 percent of the time — while still selecting the wrong action most of the time (arXiv:2504.16078). The model can explain the rule perfectly and break it in the next sentence. Add what researchers describe as context degradation — important instructions getting buried and diluted as a working context grows longer (emergentmind.com) — and you get a precise anatomy of the failure. The rule was known. The reasoning was available. The action, at the moment it mattered, wasn't tethered to either.

Imagine the Damage Done to All Clients of All Students

Picture a Harvard law professor — civil procedure, the good one, who has read every seminal case twice and can recite the waiver doctrine in her sleep. She knows Rule cold. She has taught it for twenty years.

​

Now picture her signing a brief that waives a client's objection, not because she didn't know the rule, but because in the rush of drafting, nobody made her stop and check the one constraint governing this filing. She knew the rule. She malpracticed anyway. Now multiply her by every student she's trained, every associate who absorbed her habits, every case those associates will touch for the next year. That is the compounding cost of a brilliant mind with no mandatory checkpoint between knowledge and action.

​

That is the AI we are running today. Deeply trained. Case-law fluent. Capable of ranking an opponent's best attack in a tabletop simulation and forgetting to defend against it three days later when the draft is due. The gap between "the professor knows the rule" and "the rule got enforced in this filing" is the entire malpractice exposure of the profession, compressed into a gap most workflows don't know exists.

Knowledge That Doesn't Volunteer 

The instinct after a failure like this is to add a reminder — write the constraint into the prompt more forcefully, tell the model again, more urgently, not to forget. That instinct runs backwards, which is why the fix that actually holds looks nothing like a stronger reminder.

​

The first move is breaking composite instructions into atomic constraints — single, indivisible, checkable rules instead of paragraph-long guidance a model can partially satisfy and call done.

​

The second move is a symbolic scanner sitting outside the model entirely — a hard-coded check that reads every draft against the active constraint list before anything goes out. This scanner cannot be argued with. It doesn't get tired sixty pages into a brief, and it doesn't weigh the constraint against how persuasive the surrounding argument sounds, because it isn't persuadable at all. It checks whether the draft violates an atomic constraint on file. Yes or no.

​

The third move is mandatory, not optional. Every job — every draft, every filing, every edit — runs through constraint screening before it's considered done. The screening is the gate. Nothing passes without it, the same way nothing gets filed without a signature.

The Bar Exam You Take on Every Page, Not Once in July

The last piece keeps this from decaying back into good intentions: the accepted-risk register. When an adversarial review — human or AI — flags a risk and someone decides it's manageable rather than blocking, that decision doesn't get to live in someone's head or a Slack thread that scrolls away. It becomes a ledger entry, with a re-check condition attached. A signal identified once and triaged should never evaporate because the conversation moved on.

​

Put together, this turns constraint-checking from a lecture note into something closer to a bar exam administered on every single edit, not once in July and never again. The professor still gets to be brilliant, still gets to read every case and out-think opposing counsel in the room. But she no longer gets to sign anything without the rule physically in her hand, checked, on this page, for this filing, right now.

​

We built the bench in the last piece in this series — trained the agents, gave them jurisprudence, a bar exam's worth of doctrine, a judge's skepticism. That was necessary and not sufficient. A bench that knows the rules but doesn't activate them at the moment of signing isn't a safeguard. It's a very well-read faculty lounge, and faculty lounges don't file briefs. Clients do the filing, and clients pay for what the lounge forgot.

​

Law is chaos in a suit. Every detail matters, and the ones that matter most are the ones your system already knew and didn't say.

 

Sources: Cemri et al., "Why Do Multi-Agent LLM Systems Fail?" (MAST taxonomy), arXiv:2503.13657, presented at NeurIPS 2025. On the knowing-doing gap in language model decision-making, arXiv:2504.16078. On context degradation and attention dilution in long-context agent workflows, EmergentMind.