Reverse Vampire UI: Reflecting on AR Interaction with Smart Mirrors

Authors. Sebastian Rigling, Ševal Avdić, Muhammed Enes Özver, Michael Sedlmair

Venue. CHI EA (2025) Extended Abstract

Type. Extended Abstract

Materials. DOI ACM

Abstract. Mirror surfaces can be used as information displays in smart homes and even for augmented reality (AR). The big advantage is the seamless integration of the visual output into the user’s natural environment. However, user input poses a challenge. On the one hand, touch input would make the mirror dirty. On the other hand, mid-air gestures have proven to be less accurate, slower and more error-prone. We propose the use of an AR user interface (UI): Interactive UI elements are visible “on the other side of the mirror” and can be pressed by the user’s reflection. We built a functional prototype and investigated whether this is a viable option for interacting with mirrors. In a pilot study, we compared the interaction with UI elements placed on three different planes relative to the mirror surface: Behind the mirror (reflection), on the mirror (touch) and in front of the mirror (hologram).

@inproceedings{10.1145/3706599.3719930,
    title        = {Reverse Vampire UI: Reflecting on AR Interaction with Smart Mirrors},
    author       = {Rigling, Sebastian and Avdi\'{c}, \v{S}eval and \"{O}zver, Muhammed Enes and Sedlmair, Michael},
    year         = {2025},
    booktitle    = {Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems},
    publisher    = {Association for Computing Machinery},
    series       = {CHI EA '25},
    doi          = {10.1145/3706599.3719930},
    url          = {https://doi.org/10.1145/3706599.3719930},
    abstract     = {Mirror surfaces can be used as information displays in smart homes and even for augmented reality (AR). The big advantage is the seamless integration of the visual output into the user's natural environment. However, user input poses a challenge. On the one hand, touch input would make the mirror dirty. On the other hand, mid-air gestures have proven to be less accurate, slower and more error-prone. We propose the use of an AR user interface (UI): Interactive UI elements are visible ``on the other side of the mirror'' and can be pressed by the user's reflection. We built a functional prototype and investigated whether this is a viable option for interacting with mirrors. In a pilot study, we compared the interaction with UI elements placed on three different planes relative to the mirror surface: Behind the mirror (reflection), on the mirror (touch) and in front of the mirror (hologram).},
    articleno    = {463},
    numpages     = {7},
    keywords     = {augmented reality, mirror, interaction}
}