Evaluation and User Preference Study on Spatial Diversity

less than 1 minute read

by Tang and Sanderson (ECIR 2010)

This article presents a user study which shows that users prefer search results which are not only (i) relevant but also (ii) cover many different places (=spatial diversity). It summarizes different approaches mentioned in the literature for assessing spatial diversity and performs a user study based on the following three metrics: (i) Geometric Mean (GM), SC-1, and SC-2.


  • Recall considering diversity at rank K: S-recall at K = total number of subtopics found in the K documents / total number of subtopics in the given topic.


  • Crowdsourcing: identify cheaters by inserting "gold standard data" into the task
  • The evaluation is based on a three level preference schema (prefer results, neutral, prefer alternative).