11 Know when not to answer

11 / 11

Goal

Make the system refuse what is the weather in paris — without setting the threshold so high (above 0.5) that it would refuse real questions too.

Search always returns something. Ask a help centre about the weather and the nearest chunks are still the "nearest" — just not near. Hand them to a model and it will either say something irrelevant or, worse, try to be helpful anyway.

A similarity threshold checks the best score before anything reaches the model. Below it, the app says "I don't know" — cheaply and honestly. The skill is in the number: too low and nonsense gets through, too high and real questions get refused. Look at the scores real questions get, and the scores junk gets, and cut between them.

Try it

Build the prompt without a threshold first and see what the model would be asked to work with.

Go to the terminal

Hint

Search for it and look at the best score. set threshold 0.4, then prompt.

Show a solution

search "what is the weather in paris"
prompt
set threshold 0.4
prompt

This interactive environment needs JavaScript. Everything you need to read is on this page, and the solution works the same way for real.

RAG

Starting RAG…