An AI feature should have a defined job. “Add AI” is not a specification. “Help a support agent locate relevant approved documentation and draft a response” gives a team something concrete to design and evaluate.
Evaluate the task, not the demo
Collect representative examples, including ambiguous requests and cases where no answer should be produced. Define what a useful answer looks like and what would make it unsafe or misleading. Keep the evaluation set separate from the examples used to tune the experience.
Treat retrieved text as untrusted input
Documents and tool responses can contain instructions that should not override application policy. Keep system instructions, user requests and retrieved material conceptually separate. Apply permissions before retrieval so the model never receives data the current user is not allowed to see.
Put approvals at the action boundary
Drafting a suggestion and performing an action are different responsibilities. Sending a message, changing an account or committing money should have an explicit approval rule. Limit tool permissions and record enough information to investigate a run without unnecessarily logging sensitive content.
Plan the fallback
Models can be unavailable, expensive or wrong. A product needs a response for each situation: a retry where appropriate, a normal search interface, a request for clarification or a handoff to a person. Good boundaries make AI more useful because users can understand what it can and cannot do.
