Original fictional teaching example. No model result is prefilled.
An editable starting point
- 1 · Text to check
- Original question or request: Where can I download my invoice, and how do I change the company name for next month? Proposed reply: Download your invoice from Billing → Invoices. For future invoices, update Company name in Billing → Details before the next billing date.
- 2 · Your question
- Does the reply address all explicit parts of the request?
- When does Yes apply?
- Every explicit question or request is addressed, including saying when an answer is unavailable.
- When does No apply?
- At least one explicit question or request is not addressed.
These are model probabilities, not measured accuracy or a guarantee. Yes and No describe your rules; choices are relative to the options you supplied.
The task
A reply can sound helpful while missing half the request. This recipe compares a fictional request and draft reply using one Noul question: does the reply address the explicit requirements? It does not check all facts, promise that a remedy will work, or send anything.
The template is original teaching material with no expected result. It loads a single custom question, rather than the illustrated scenario’s separate checks for coverage, next steps, and new commitments.
Define what counts as addressed
Keep the request and proposed reply visibly separated in the material. Replace them with non-sensitive text. Make a short checklist of the request’s explicit parts before running. Decide whether an honest statement that information is unavailable counts as addressing a question, and encode that decision in the rules.
Do not silently require an unstated next step. A factual answer may be complete without one. If your workflow requires a particular action, make that requirement explicit.
Use the result as a review signal
A higher yes probability suggests coverage under your definitions. It is not proof that the reply is accurate, polite, authorized, or safe to send. Review the request and answer side by side; the website neither contacts the recipient nor executes the proposed action.
Remove one answer
Create a request with two separate questions, then remove the answer to one of them from the draft. Also try a reply that acknowledges both questions but answers neither. Decide how your rules should distinguish acknowledgment from an answer. If the model misses the difference, revise the criteria and keep the failed example for later checking.
Read probability versus confidence before choosing any automated threshold. This exercise has not been independently benchmarked for reply quality.
Sources & further reading
Official documentation and linked community reports support this guide. Community observations are attributed to their authors.
How we check our sources