Why AI Assistants Fail at Insurance Policy Terms, and How Agentic Retrieval Changes That
The question many AI assistants get wrong
”Is my e-bike covered if it’s stolen from the bike cellar at night?” An everyday customer question. The correct answer sits in the policy terms, spread across three places: the benefit description for bicycle theft, the exclusion for the night-time clause, and the definition of what counts as a locked room. Read only one of these and you answer incorrectly.
That is exactly what happens to classic AI assistants on a regular basis. Not because the language model is too weak, but because the method it uses to retrieve knowledge was never built for legally structured documents such as general insurance terms and conditions (in German: AVB).
What RAG is good at, and where it falls short
Retrieval-Augmented Generation (RAG) is the standard method AI assistants use to draw on company knowledge: documents are split into text chunks. When a customer asks a question, the system retrieves the most similar chunks and hands them to the language model as context.
For many knowledge sources this works very well: product pages, help articles, FAQ collections. Each passage stands on its own, and the answer lives inside a single chunk.
General insurance terms and conditions are built differently:
- Clauses reference one another. The benefit promised in section 5 only applies together with the exclusion in section 12 and the definition in section 1. A single chunk has no knowledge of these cross-references.
- Similarity is not relevance. Semantic search finds the passage that sounds most similar to the question. The decisive exclusion often doesn’t sound similar to the question at all.
- Structure carries meaning. Whether a clause sits under “Benefits” or under “Policyholder Duties” changes its effect. Splitting a document into chunks discards that structure.
The result: the assistant quotes a real clause and still gets it wrong, because it is missing the rest of the policy document. In regulated industries, that’s not a cosmetic flaw, it’s an operational risk.
What Agentic Retrieval does differently
Agentic Retrieval refers to an AI agent’s direct, tool-based access to complete documents. Instead of receiving pre-split text snippets, the agent works with the document itself: it opens it, builds an understanding of its structure, jumps to the relevant sections, and reads clauses in context, including their cross-references, exclusions, and definitions.
The difference is the same as between a search-engine snippet and a claims handler who opens the actual policy document: one finds a similar passage, the other answers the question.
How this works on the LoyJoy Platform
The AI Agent module of the LoyJoy Platform has tools that let the agent work directly with documents, including PDF, DOCX, and PPTX. The agent decides on its own which tool to use and when:
- RAG for breadth: fast answers from large knowledge bases, such as website content and help articles.
- Document tools for depth: direct work with complete policy documents, contracts, and product information sheets whenever the question requires it.
- Guardrails and traceability: the agent answers exclusively based on approved sources. Integrated AI testing continuously checks answer quality, including after the policy document itself is updated.
For insurers, this means concretely: coverage and benefit questions are answered correctly based on the full policy terms, not on isolated text snippets. The business team uploads the policy document itself and makes it available to the agent, with no IT project required.
And because policy documents and customer conversations are sensitive: LoyJoy runs the chat LLM Gemma 4 on its own hardware in our Münster data center. Processing takes place entirely in Germany, with no external AI subprocessors.
The e-bike example, followed through
Back to the question from the start. The night-time clause, which covers bicycle theft between 10 p.m. and 6 a.m. only under certain conditions, is a good example of why general knowledge isn’t enough: it appears in classic policy documents, while many current tariffs have dropped it. Whether it applies to this particular customer is stated exclusively in their specific policy document. A language model answering from training knowledge guesses. An agent that reads the document knows.
An agent using Agentic Retrieval therefore proceeds like this: it recognizes that the question concerns bicycle theft under home contents insurance, opens the policy document, reads the benefit description, checks the associated exclusions and the definition of a locked room, and then answers, with a source reference, what is covered, under which conditions, and what the customer needs to be aware of. If that isn’t enough, it hands over to the claims team with full context.
The customer gets a binding answer instead of a text snippet. The service team only gets the cases that genuinely need human expertise.
Frequently asked questions
Does Agentic Retrieval replace RAG? No, it complements it. RAG remains the fastest way to answer knowledge questions from large sources. Agentic Retrieval comes into play when documents need to be read in context. The agent chooses the right approach on its own.
Which documents benefit from Agentic Retrieval? Any document where statements reference one another: insurance policy terms, contracts, tariff documents, technical regulations, and product information sheets.
How is answer accuracy ensured? The agent answers only based on approved sources and cites them. Integrated AI testing automatically simulates test questions, for example after every update to the policy document.
Does IT need to run a project for this? No. The business team provides documents itself and configures the agent without programming. IT retains governance, roles, and permissions.
Try it yourself
Upload your own policy document and ask the questions your customers ask. A first agent is ready in 15 minutes.
Or let’s spend 30 minutes together analyzing which of your customer requests AI Agents can already resolve today: Book a demo