People and AI See Things Different Implications of Mismatched Perception on HCI for AI Systems

People and AI are increasingly interacting and collaborating in the context of critical application domains (e.g., healthcare, finance, transportation, and legal systems). There is often, however, a fundamental mismatch between how humans and machines perceive and reason about the world. This offers opportunities for bringing together multiple perspectives to reach better outcomes. On the other hand, this mismatch can hurt coordination and result in serious failures (e.g., semi-autonomous vehicle accidents and misdiagnoses by clinical decision support systems). We believe a key solution is to ground communications between humans and machines in their common perceptions while allowing people to inspect and verify the AI and appropriately intervene when necessary. Achieving this requires the HCI and AI communities to address several challenges and to co-design HCI-AI patterns that enable verifiability, control, and consistency.