The pattern is consistent enough that researchers have started naming it. A team builds an AI tool on a small set of curated documentation. It works beautifully in demos. They roll it out across the real knowledge base. Accuracy drops to a level the team eventually stops trusting. Adoption stalls. The project gets quietly shelved.

Gartner’s 2025 research found that AI assistants and agents “often underperform when scaled across diverse enterprise information due to data source quality.” McKinsey and Gartner together report that over 60% of enterprises cite hallucination and unreliable outputs as the primary barrier to scaling AI into production.

The instinct, on first hearing those numbers, is to look at the model. Different model. Better model. Bigger context window. The data says that almost never fixes it.

The cause of these failures, in nearly every case, is the documentation feeding the system. This post walks through the five most common failure modes, what to look for in your own setup, and how to address each one.

TL;DR

Most enterprise AI projects fail not because the model hallucinates, but because the underlying documentation is outdated, prose-shaped instead of step-shaped, internally inconsistent, missing coverage on real questions, or partially indexed. Each is a real, fixable problem. Switching models rarely helps; cleaning up the documentation does.

Failure Mode 1: Outdated documentation feeding current questions

The most common failure mode and the easiest to overlook. A document that was correct in 2023 but never updated keeps showing up in 2026 answers. The model uses it as a source. Your team gets steps that reference deprecated CLI flags, end-of-life operating systems, or vendor names that no longer exist (Office 365 became Microsoft 365 in 2022, but a question about “Office 365 license assignment” will still match a five-year-old document that uses the old name).

What to look for:

• Documents older than 18 months on fast-moving topics: cloud platforms, vendor APIs, security policies
• Documents referencing end-of-life software (Windows Server 2008, Office 365, Exchange 2010)
• Documents with no last-updated timestamp at all
• Wrong answers concentrated on older source documents

The fix: treat freshness as a continuous discipline, not an annual cleanup. Manually, this means a quarterly review pass on operational documentation older than 12 months. A platform like ResolvCmd’s Knowledge Studio tracks this for you and surfaces the documents that need attention.

Failure Mode 2: Prose where steps are needed

Documentation written for humans is usually paragraphs. A teammate asks the AI “how do I do X.” The AI pulls a document that explains what X is, not how to do it. It tries to extract steps from the prose, often producing something that sounds plausible but is slightly wrong.

This is the documentation shape problem. Operational questions (how do I, what’s the process for, how do I troubleshoot) need step-shaped sources. Conceptual questions (what is, why does, when should) can use prose. Most AI tools don’t distinguish, and they cover for the mismatch badly.

What to look for:

• The AI returns a paragraph-style explanation when the user wanted steps
• Generated steps that contain plausible verbs but lack specifics (correct outcome, vague mechanism)
• Routine procedural questions where the answer feels off
• Wrong answers concentrated on prose-formatted sources

The fix: convert your highest-traffic prose documents into proper runbooks. The single biggest accuracy lift you can get from one focused effort is here. A small number of high-traffic prose documents usually account for the majority of failures, which means a focused rewrite produces outsized gains.

ResolvCmd helps your team identify which documents are doing this damage and walk through the rewrite with you in control of every save.

Failure Mode 3: Conflicting documents covering the same topic

Two documents cover the same procedure with different steps. Both are valid in some context (different client, different time period, different team). The AI pulls both. It has to pick. The user gets one answer this time and a different answer next time. Trust erodes silently because no single response is wrong, but the inconsistency is.

What to look for:

• Inconsistent answers from the same question asked twice in a week
• Team members noting “the answer was different last time”
• Documents with similar titles (“VPN Setup,” “VPN Configuration,” “Setting Up the VPN”) that diverge in critical steps
• Drift in client-specific procedures that should have been consolidated long ago

The fix: identify and resolve duplicate-topic documents. Pick a canonical version. Archive or merge the rest. Manually, this means a periodic scan of your knowledge base for duplicate-looking titles and overlapping content. ResolvCmd surfaces these conflicts as they show up in real questions, so you find them when they’re actually causing problems.

Failure Mode 4: Coverage gaps for real questions

The documentation gap users do not even realize exists. Your team asks “how do we reset MFA for an offboarded user” 50 times a quarter. No document covers it. Every retrieval is a near-miss. The AI either invents a procedure (the worst outcome) or admits it cannot find one (honest, but does not solve the user’s problem).

What to look for:

• High volume of questions on the same topic with no good source document
• “I asked the AI but it didn’t know” feedback patterns
• Industry-standard procedures absent from your knowledge base entirely
• Topics that come up in your team chat but were never written down

The fix: systematically detect coverage gaps and write the missing documents. Manually, you can scan your top 100 questions over the past 30 days and check which have no good source document. The gaps with the highest volume are the highest-leverage to fill first.

ResolvCmd handles this continuously, so the gaps surface as your team uses the platform rather than waiting for an annual review.

Failure Mode 5: Documentation that exists but isn’t actually indexed

The often-overlooked failure mode. Your team has 5,000 pages of internal documentation. Your AI tool covers 500. The other 4,500 sit outside the system. A question that should match a document in the unindexed pool fails silently. The AI returns a no-match or makes something up. Neither outcome reveals to the user that the answer existed but was unreachable.

This also covers documents that failed to ingest cleanly: PDF parsing failures, encoding issues, oversize files, scanned PDFs without OCR, files in formats the connector does not handle, content excluded by configuration.

What to look for:

• Total document count in your knowledge sources is much higher than what’s actually been indexed
• Ingestion errors that exist in logs but never surface to admins
• Connectors that report success but ingest fewer documents than expected
• Knowledge sources that are connected but never produce hits

The fix: keep an inventory of documents the connector saw, including the ones that didn’t fully ingest. Make it visible to admins so they can decide whether to expand coverage, fix the ingestion error, or exclude the content intentionally.

How to diagnose your own setup

A practical six-step audit you can run this week:

1. Sample 50 of your most recent answers from the AI tool. How many felt right? How many felt off? A healthy setup has the team trusting most answers without rechecking them.
2. For the answers that felt off, check the source documents. How often is the source prose when the question was operational? That’s failure mode 2.
3. Pick 10 of your most-cited source documents and check when they were last updated. How many are over 18 months old? How many reference deprecated technology? That’s failure mode 1.
4. Look for repeated questions with different answers. Same question asked by different people in the past month producing different responses? That’s likely failure mode 3.
5. Group your “no match” questions by topic. A repeated topic with no documentation is failure mode 4.
6. Compare your ingested document count to your total document count. If your knowledge sources have 5,000 documents and the AI tool only indexed 1,200, you have failure mode 5.

Each of these is a real, addressable problem. None is fixed by switching models.

What ResolvCmd does about it

ResolvCmd is the Knowledge Intelligence Platform. We connect to your existing documentation (IT Glue, Hudu, Confluence, Google Drive, more), surface the documents that need attention across all five failure modes, and help your team improve them. The Resolution Engine delivers source-cited answers inside your ticketing system; the longer your team uses it, the more your knowledge base improves.

We treat the documentation layer, not the model layer, as where the work belongs. The model layer is commodity. The documentation layer is where most teams have not yet invested, and where the largest accuracy gains live.

If you want to see how your own documentation looks, start a free trial. Connect a knowledge source. ResolvCmd shows you which documents need attention within hours of the first sync.

Frequently asked questions

Will switching to a better LLM fix my AI accuracy problems?

Almost never. The best models on the market all hallucinate at similar rates when given poor source documentation. The improvement from changing the model is small compared to the improvement from cleaning up the documentation. Gartner’s research consistently identifies data quality, not model quality, as the dominant variable.

How long does it take to fix documentation problems like these?

The diagnostic pass is fast: a week or two of running real questions through your system. The fixes are continuous. Most teams see meaningful accuracy improvement within 30 days of starting systematic improvement work, with continued gains over the following months.

Should I switch to a different AI vendor?

Probably not. Most AI vendors in this space use similar underlying technology. The differences between vendors are small. The differences between knowledge bases are large. Improve the documentation first, then evaluate whether you still have a vendor problem.

Is this only an IT operations problem?

No. The same five failure modes appear in customer support, legal ops, DevOps, and any other knowledge-intensive operation. Documentation quality is the universal bottleneck on enterprise AI. The vocabulary changes by vertical; the failure modes do not.

Where does the 60% hallucination barrier number come from?

From combined Gartner and McKinsey 2026 enterprise AI adoption research, particularly Gartner’s analysis of regulated industries (financial services, healthcare, legal, professional services) where unreliable AI outputs most directly translate to organizational and regulatory exposure. See sources below.

Sources

• Gartner Innovation Insight: Use RAG as a Service to Boost Your AI-Ready Data
• Why Your AI Agents Are Underperforming, Gartner Data
• RAG in 2026: Bridging Knowledge and Generative AI
• What Is AI-Ready Documentation? A 2026 Definition
• Documentation Health Score: A Practical Audit for AI-Readiness