How to assess site search results for relevance

Assess site search by giving it a small set of realistic questions and checking whether the results help a visitor act. An exact title search may work while common synonyms, misspellings or no-result queries fail. Record the term, expected answer and actual results rather than relying on a single impression of the search box.

Build a query set from real language

Choose an exact product or page name, a customer synonym, a broad problem and a likely misspelling. Add a query for which the site genuinely has no answer. If you manage the site, use support questions or search logs to select terms. For a public review, explain that your set is illustrative.

Write down what a good result would be for each term before searching. Otherwise you may change the standard after seeing the page.

Inspect the first useful result

For each query, note whether the relevant page appears, whether the title and snippet explain it and whether filters or categories help narrow a broad result. A result can be technically relevant but unhelpful if every item looks the same. Check whether outdated or duplicate pages crowd out current information.

Do not judge quality by position alone. A clear answer at the third result may still be easier to use than an ambiguous first result.

Test the zero-result route

Enter a term the site does not cover. Does the page offer spelling suggestions, related topics or a clear way to ask for help? A blank screen leaves the visitor with no next step. Note whether the search preserves the typed query so someone can correct it.

The search experience guide covers implementation choices after a review identifies gaps.

Report patterns, not one query

Group failures by synonym handling, content gaps, ranking or interface clarity. Provide the exact queries and date so an owner can repeat the check. If the site changes its content, rerun the same set and add new customer language as it appears.

A five-query audit set

Use one exact page name, one common synonym, one problem phrase, one plausible misspelling and one term the site does not cover. For each, write the expected answer or expected recovery before pressing Search. Record the first useful result, misleading results and what a visitor can do next. If “delivery times” works but “when will it arrive” does not, the issue may be language matching. If both fail because there is no current delivery page, the issue is content. The same test set can be rerun after a change.

Frequently asked questions

Is a search box necessary on every website?

No. A small site with clear navigation may not need one. Test whether visitors can find answers by other routes.

How many queries should I test?

Start with a varied small set, then add real user terms for important services. One exact-name query is not enough.

Should a zero-result page recommend unrelated content?

No. It should offer a helpful correction or alternative route rather than pretending an unrelated page answers the question. Offer spelling help, category links or a clear way to ask for assistance.