User Interaction Sequences for Search Satisfaction Prediction

Detecting and understanding implicit measures of user satisfaction
are essential for meaningful experimentation aimed at enhancing
web search quality. While most existing studies on satisfaction
prediction rely on users’ click activity and query reformulation
behavior, o‰en such signals are not available for all search sessions
and as a result, not useful in predicting satisfaction. On the other
hand, user interaction data (such as mouse cursor movement) is
far richer than just click data and can provide useful signals for
predicting user satisfaction. In this work, we focus on considering
holistic view of user interaction with the search engine result
page (SERP) and construct detailed universal interaction sequences
of their activity. We propose novel ways of leveraging the universal
interaction sequences to automatically extract informative,
interpretable subsequences. In addition to extracting frequent, discriminatory
and interleaved subsequences, we propose a Hawkes
process model to incorporate temporal aspects of user interaction.
Œrough extensive experimentation we show that encoding the
extracted subsequences as features enables us to achieve signi€-
cant improvements in predicting user satisfaction. We additionally
present an analysis of the correlation between various subsequences
and user satisfaction. Finally, we demonstrate the usefulness of the
proposed approach in covering abandonment cases. Our €ndings
provide a valuable tool for €ne-grained analysis of user interaction
behavior for metric development.