Pattern 15D: Use existing public rating or interaction data as feedback for the system

Problem

User feedback is needed to assess the system and help it improve over time.

Solution

Leverage existing public rating or interaction data as feedback for the system.

Use when

The system collects and displays publicly item-specific feedback, such as ratings or reactions (e.g., like, dislike, sad, celebrate).

How

Use data from existing item-specific public feedback such as ratings, likes/dislikes, reactions, etc., or implement a feedback mechanism that can serve the dual purpose of sharing feedback publicly and evaluating/training the AI system.

If using existing public feedback:

Take an inventory of available item-specific feedback.
Decide what these signals mean for algorithm improvement. For example, how can you leverage the number of “likes” for algorithm improvement?
Select what existing feedback to use for algorithm improvement.

If creating a new feedback mechanism, design the public feedback in a way that helps inform the algorithm as well. Ensure the information collected is meaningful to the user submitting the feedback and the users seeing the feedback, and that it can be leveraged for system improvement.

Consider how to convey to the user that the feedback was received, how it will be used, and whether the user experience changes immediately after giving feedback (see Guideline 16).

Collect, process, and use user feedback data in privacy-conscious ways.

User benefits

It can be efficient for the user to improve the AI system through regular use, without having to provide additional feedback to the system.
Supports personalization.
May improve the experience for all system users.
Gives the user a voice.

Common pitfalls

The meaning of public feedback signals is misinterpreted when considered for algorithm improvement (e.g., a “like” on a social media post is not always a signal that the user wants to see more posts that are similar).
The feedback is designed in a way that is useful for the algorithm but not useful or meaningful for users.
It’s unclear to the user that the public feedback is used for improving the AI system.
It’s unclear to the user how their feedback will be used by the AI system.

Notes

Explicit feedback is information the user intentionally provides for system improvement.

Implicit feedback is information the system garners from user actions to improve itself.