Home coffee machines: the three models' brand picks overlap only 40%
Ask three web-enabled LLMs the same question — how different can the recommendation lists get? We ran one real category through them.
What we asked
We fixed one real category (home coffee machines) and ran the same batch of questions through three web-enabled LLMs. The questions fall into five groups by buying intent: recommendations, spec comparison, hands-on experience, cleaning and maintenance, and price ranges. Wording stayed identical in each group — only the model changed.
Where the three models differ
Lay the three models' recommendation lists side by side, and the overlap is smaller than you'd expect.
"In this price range, the safe picks are Brand A, Brand B and Brand C. If you want less fuss, A-1's automatic milk frother is a good bet; if extraction consistency matters more, B-2 is steadier."
Switch to Model B with the same questions: only Brand B survives on the list, while the other two are replaced by names we had never seen in this category. Model C ranks yet another way. Brands recommended by all three models make up only about 40% of the brands recommended overall.
What this means
- · If you only perform well in one model, users of the other two won't see you.
- · The claim "AI recommends you" means nothing unless it states which model and which question.
- · To lift overall visibility, find the gaps model by model — don't look at a single average.