Compare search results with a repeatable protocol

1. Define the comparison before opening a search box

Write one question the exercise should answer. Examples include whether a provider's documented service area is represented accurately, whether two search products link to different source types, or whether a changed page is cited for the question it is meant to answer. Keep one primary comparison at a time.

Choose the unit of observation: a result page, an answer, a cited source or a specific factual claim. Do not switch units halfway through the report. A product comparison and a before/after page observation need different collection designs and different conclusions.

Record the intended audience, language and market. Decide whether the result will guide an editorial improvement, a technical investigation or a reporting choice. This keeps the exercise focused on a useful next action.

2. Freeze a query set and a collection sheet

Use questions from a defined source, such as support enquiries, product decisions or an existing topic plan. Assign each an ID. Store the exact wording and version. Keep branded questions separate from questions that do not name the organization.

Set collection rules before looking at winners. Decide which surfaces and modes are included, how many runs you intend to collect, how the schedule works and what counts as an incomplete run. Choose a scope that the team can finish consistently rather than an impressive-looking panel it cannot maintain.

3. Keep context visible and collection consistent

Google documents variation associated with time, location, language, device and personalization. Disabling personalized recommendations does not remove the query's geographic and other context. Record the context the interface exposes instead of treating a browser mode as proof of a universal result.

Within the chosen design, keep wording, mode and settings steady. Use a planned collection window and rotate the order of products if order or timing could influence the comparison. Record a product change or interface update as a break in the collection history.

If one condition changes, label that run separately. Do not silently pool an image-assisted query with text-only queries or a follow-up answer with a fresh conversation. The geography guide explains why the area behind a place query needs its own record.

4. Score the result with a useful rubric

Choose observable criteria. For a source comparison, these may include whether a relevant source is linked, whether the source supports the important claim, whether the date fits the question and whether a needed qualification is included. Define the criteria before scoring the panel.

Keep factual correctness separate from preference. A concise answer can be pleasant to read while omitting a necessary condition. A longer answer can include the condition but be difficult to use. Record the dimensions independently, then explain how they affect the decision.

Open cited pages when evaluating support. The source-reading guide gives a practical sequence for matching a claim to the underlying passage. Save the passage location and your reasoning so another reviewer can resolve disagreements.

• Relevant source linked
• Important claim supported by that source
• Correct service, audience or geographic scope
• Suitable date or version
• Necessary qualification included
• Useful next action for the reader

5. Normalize links without erasing the observation

Retain the URL exactly as displayed, then store a separate normalized destination for analysis. This lets you group the same page reached through different tracking parameters while preserving what the search product actually showed. Record redirects when they materially change the destination.

Decide how domain-level and page-level observations differ. A link to an unrelated homepage is not equivalent to a link that directly supports the answer. Keep owned-domain presence, relevant-page presence and source support as separate fields.

For repeated runs, report counts and denominators. If a result is unavailable, record an exclusion rather than scoring an unseen answer as a factual failure. Review a sample with a second reader when the scoring requires judgment.

6. Publish a method that can be checked

A useful report includes the question, query-set version, collection dates, context, completion rules, scoring rubric and the observations needed to inspect the conclusion. Keep private account information and personal query details out of public evidence; retain the research fields needed to understand the method.

Summarize patterns by question group rather than selecting only striking screenshots. Distinguish a descriptive comparison from an explanation of cause. To investigate a cause, design a separate change with a suitable comparison and a clear verification rule.

End with a decision: improve a source page, clarify a factual boundary, inspect a technical condition or extend the collection. Save the original records. A repeatable method makes the next run comparable and the next editorial decision easier.

• Question and unit of observation stated
• Query set and context recorded
• Collection and exclusion rules fixed
• Raw outputs and original URLs retained
• Rubric applied consistently
• Counts, denominators and a clear next action reported