AI & data
Build a chatbot that answers from your documents
Retrieval-augmented generation supplies a model with relevant material found in a document collection at answer time. A useful chatbot needs governed sources, access controls and evaluation rather than just an upload facility.
Choose a governed document collection
Begin with coherent, approved procedures or reference material. Assign ownership, version and audience. Remove replaced versions rather than accumulating contradictory instructions.
Check extracted text, especially from scanned PDFs, tables and images. A visually clear file may produce disordered text, which needs correcting before evaluating the model.
Enforce permissions before retrieval
Associate sources with access rights and search only material available to the signed-in user. Removing a citation afterwards does not undo disclosure through generated text.
Define how private, shared and deleted files are handled. Role changes and removals need to propagate to the search index under a maintained process.
Inspect retrieval before judging writing
Prepare real questions and expected supporting documents. Inspect retrieved passages for relevance, currency and sufficient context. Rewriting instructions will not fix retrieval of an obsolete procedure.
Try terminology, synonyms and incomplete questions. Titles, splitting and metadata need evaluation, and insufficient results should be recognised rather than forced into an answer.
Offer answers with useful references
Separate found information from what is missing, and link to accessible sources that genuinely support the claims. Staff should be able to verify an instruction without restarting their research.
When the collection cannot answer, ask for clarification or refer to a person. Retrieval supplies context but does not guarantee every generated sentence is accurate.
Maintain sources and repeat reference questions
Retest important questions after changing documents, indexes or models. Distinguish retrieval failures from omissions and wording problems because their corrections differ.
Pilot a limited collection before expanding. Custom integration can reuse application identity, cases and documents rather than creating a disconnected knowledge tool.
Your checklist
- Assign document ownership and versions.
- Check text extraction.
- Retrieve according to access rights.
- Test questions with expected sources.
- Plan updates, removal and insufficient answers.
Frequently asked questions
Does RAG require training a new model?
No. It supplies retrieved information at request time. Document preparation and indexing are separate from model training.
Can the chatbot be guaranteed never to be wrong?
No. Evaluate results, provide relevant references and define when clarification or human assistance is required.
A project to explore
Jibril.me
Jibril.me combines reading, notes and AI assistance, illustrating the value of keeping assistance close to relevant content and context.
Explore the project