Find answers in your company’s approved knowledge.

Choose questions your documentation can actually answer.

A useful first scope is one question family: opening a service request, choosing an equipment checklist or requesting business travel. For each question, verify that an approved, current and understandable source exists. If the procedure lives only in a colleague’s memory, document it before introducing an assistant.

Retrieval-augmented generation, or RAG, retrieves relevant passages before generating an answer. It does not make every generated statement correct. Retrieval may find an obsolete version, the source may be ambiguous, and a citation may fail to support the sentence beside it. Those are different defects and need separate evaluation.

Worked scenario: which returns procedure applies?

Illustrative scenario: a team member asks how to handle a return for a discontinued product. The assistant finds the current procedure, shows its revision date and identifies the step requiring the commercial owner. If only an archived version exists, it explains that limitation instead of treating the old instruction as current.

Check permissions before content reaches the answer.

Document visibility must be enforced during retrieval. Asking the model to keep already-retrieved confidential material secret does not replace access control. The scope should also cover snippets, links, caches and previous conversations when a person’s authorisation changes.

Azure guidance describes filtering search results by user or group identifiers and makes clear that the filter itself does not authenticate a user. Retrieval therefore needs a verified application identity. The appropriate implementation depends on the repository, available APIs and the organisation’s existing access model.

Build a testable path from sources to answers.

1. 01

   Curate the first collection

   Assign an owner, status, review date and audience to each collection. Define exclusions and what happens when a document is replaced, archived or deleted.

2. 02

   Prepare questions and expected answers

   Include common questions, missing information, ambiguous requests and restricted topics. Evaluate retrieval, answer content, citation support and appropriate abstention separately.

3. 03

   Run with a limited user group

   Collect unresolved questions and corrections without retaining unnecessary conversation data. Categorise each failure as a source, retrieval or answer problem so remediation addresses the underlying cause.

Design the answers the assistant must decline.

Retrieved documents can contain instructions that try to redirect a model. OWASP explains that RAG does not eliminate this risk. The initial scope separates information retrieval from actions in business tools, and includes adversarial document examples in the evaluation set.

Traditional search may be enough for an exact code lookup. An assistant cannot maintain procedures that nobody owns. Ask for source maintenance, deletion handling and a clear path for unanswered questions in the proposal. For multilingual teams, evaluate the supported question and document languages explicitly.

Practical questions