Publication Details

Download

Mohamed Khamis, Anita Baier, Niels Henze, Florian Alt, Andreas Bulling
Understanding Face and Eye Visibility in Front-Facing Cameras of Smartphones used in the Wild
To appear in CHI '18: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Montreal, QC, Canada, April 21 - 26, 2018. ACM, New York, NY, USA. (bib)

Commodity mobile devices are now equipped with high resolution front-facing cameras, allowing applications in biometrics (e.g., FaceID in the iPhone X), facial expression analysis, or gaze interaction. However, it is unknown how often users hold devices in a way that allows capturing their face or eyes, and how this impacts detection accuracy. We collected 25,726 in-the-wild photos, taken from the front-facing camera of smartphones as well as associated application usage logs. We found that the full face is visible about 29% of the time, and that in most cases the face is only partially visible. Furthermore, we identified an influence of users' current activity; for example, when watching videos, the eyes but not the entire face are visible 75% of the time in our dataset. We found that a state-of-the-art face detection algorithm performs poorly against photos taken from front-facing cameras. We discuss how these findings impact mobile applications that leverage face and eye detection, and derive practical implications to address state-of-the artâs limitations.