AI did not fail where I expected

After several rounds of testing Version 1 and Version 2 of the Prendoco workflow playbook, the most uncomfortable conclusion was this: the most dangerous errors did not look like errors.

They were not obvious hallucinations. They were not ridiculous answers. They were not outputs that immediately felt false.

They were plausible. Well written. Structured. Confident. And that is exactly why they were difficult to spot.

Across legal drafting, document review, summaries, clause analysis, email rewriting and workflow execution, I started aggregating recurring AI error patterns. After reducing and combining the observations, 12 categories kept appearing.

AI usually did not fail because it was incapable. It failed because it was insufficiently controlled.

1. When AI fills the gaps without saying so

One of the most common patterns was assumption drift. AI received partial information and, instead of flagging uncertainty, quietly filled in the missing pieces.

A due diligence note became interpreted as a full contractual position. Or the phrase “protection not visible” quietly turned into “the protection is missing.”

That may look like a small wording difference. It is not.

In legal work, moving from “I cannot see it” to “it does not exist” can change the practical consequence, the negotiation position and the risk assessment.

2. When partial material is treated as complete

Another recurring issue was source overreach. AI reviewed an extract and reached conclusions that would only make sense if the entire agreement had been reviewed.

The fix was not complicated, but it mattered:

“Based on this extract only.”

That phrase changes the value of the output. It forces the system to recognise the limits of the analysis. It also helps the lawyer decide how much weight to give the answer.

In legal work, a useful answer should not only say what it sees. It should also say what it cannot know.

3. Weak references and conclusions that are hard to verify

AI can say something like: “Liability exposure appears broad.”

That may be right. But if it does not say where the conclusion came from, the output loses value.

Lawyers need to verify quickly. They need:

Source: Clause 8.2. Relevant wording. Reason for the conclusion. Confidence level.

Not: “Trust me.”

Traceability is not a nice extra in legal workflows. It is part of quality control.

4. When better English changes legal meaning

This was one of the most delicate failure patterns. AI rewrote text to make it clearer, smoother or more natural. But sometimes, the linguistic improvement changed the legal effect.

For example, simplifying “may terminate” into “can terminate freely” may sound clearer. But it does not necessarily mean the same thing.

In contracts, a cleaner sentence is not always a better sentence. Sometimes it is just a different sentence.

The same problem appears in professional emails. AI often assumes that “more professional” means “more formal”. A simple:

“Can you confirm by Friday?”

becomes:

“We would appreciate confirmation at your earliest convenience.”

Longer. More polished. Less useful.

5. Bigger lists, but not better judgment

AI likes structure. That can be useful. But it can also create a false sense of order.

I saw outputs where facts and assumptions were mixed together, legal and commercial issues blurred, risks were combined with missing information, and action points sat beside decisions as if they were the same thing.

The output looked organised. But the categories were unstable.

Another related issue was risk inflation. One real drafting concern became five overlapping findings.

More output. More noise. Less prioritisation.

Lawyers do not need bigger lists. They need better ranking.

The value is not in finding everything. The value is in distinguishing what matters from what does not.

6. The missing “so what?”

In many cases, AI correctly identified an issue, but stopped just before explaining why it mattered.

“Change of control issue identified.”

Fine. So what?

Does it trigger consent? Create a termination risk? Affect leverage? Require escalation to the client?

A good legal review does not end with issue spotting. It should explain the consequence, the priority and the next practical step.

7. Party position, timing and workflow execution

Another recurring weakness was the failure to preserve party position. AI explained wording changes, but did not always identify who benefited from the change.

A clause changes. Risk shifts. Leverage shifts. But the output stays at the surface level of language.

Timing and notice mechanisms also caused problems: notice periods, renewals, reporting deadlines, trigger dates. AI sometimes made these mechanisms too clean, quietly simplifying complexity that mattered.

And not every failure was about legal reasoning.

Sometimes the workflow failed before the legal analysis even started: attachment problems, retrieval failures, incorrect file handling or differences in platform behaviour.

This matters more than many people realise.

ChatGPT inside one environment may behave differently from Copilot inside Microsoft 365. An agent with knowledge files behaves differently from a standard chat. An LLM connected to internal documents behaves differently from one without context.

Sometimes you are not testing the model. You are testing the environment around the model.

What changed in Version 2 of the playbook

Version 2 of the Prendoco workflow playbook changed substantially because of this.

The goal was not only to improve what AI should do. It was also to improve what AI should not do.

In practice, a surprising amount of legal workflow design becomes a set of controls:

Do not assume. Do not overreach. Do not rewrite meaning. Do not invent priorities. Do not negotiate for me. Do not compress nuance.

I still do not think these errors disappear completely. Models are improving. Accuracy is improving. Behaviour is improving. But certain tendencies appear persistent, and they often seem heavily influenced by platform design and built-in behaviour.

The strongest conclusion from all the testing was simple:

Legal AI increasingly looks less like a tool problem and more like a workflow control problem.

Legal notice: I am not a lawyer and I do not provide legal advice. All content is for educational purposes only. Responses generated by language models such as ChatGPT should be verified by qualified professionals before use.