For as long as we’ve had computers, we’ve controlled them almost exclusively by hand. Aside from a very few highly specialized, custom systems driven by voice or eye movements, commanding a computer to do anything has required actions taken through a keyboard or mouse. Apple took a major step toward breaking that pattern last year with the introduction of 3-D facial recognition, which it calls Face ID, built into the iPhone X. What seems at first blush to be a simple convenience — the ability to unlock a phone by looking at it — is poised to dramatically alter the course of mobile computing.
How we got here
Apple, of course, didn’t invent facial recognition; it’s been with us for years in biometric authentication, often in high-end access control systems for buildings. And Apple isn’t first to bring facial recognition to mobile computing. Samsung introduced it in the Galaxy S8, in a somewhat imperfect form that could be spoofed with a photograph of a face.
What Apple brought to the party is a high degree of reliability on a mobile device. Thanks to the iPhone’s TrueDepth camera system and Face ID, which Apple says has only a one-in-a-million chance of spoofing, mobile facial recognition has now gone mainstream.
Putting facial recognition to work in iPhone X apps
From an application development perspective, incorporating Face ID requires very little effort. Any app that already supports Touch ID will automatically support Face ID — Apple has done an excellent job of making Face ID easy to use for app authentication.
Authentication is really just the start, however. For other use cases like face detection features for photo apps, Apple released its Vision Framework with iOS 11, making it easy to identify faces, detect features, and classify scenes in images and video.
Get creative!
Depending on the purpose of the app at hand, facial detection opens up entirely new possibilities for improving the mobile user experience and introducing entirely new capabilities. I’m personally intrigued by the benefits it can deliver to communication/chat apps. As a starting point, messaging apps could send an emoji in response to a facial expression, such as the classic stick-out-your-tongue. That approach could be extended beyond emoji to manipulating content, such as playing music to match a mood.
Far-fetched? Hardly. With the right app and the right approach to face detection, I expect we’ll see some very creative uses for facial recognition coming our way, including innovation along the lines of eye/blink tracking.
Expect more hands-free mobile control in future devices
Face ID is likely just the start of a larger trend toward hands-free control of mobile devices. While it will probably be difficult for other phone manufacturers to compete near-term with the quality of Apple’s TrueDepth camera system, innovation along these lines needn’t be limited to facial recognition. I fully expect to see more innovation around voice detection, perhaps beginning with unlocking a phone and then branching into additional voice-command capabilities. Again, depending on the app’s purpose, there can be loads of possibilities.
Those possibilities all revolve around getting past the notion that computing must be hand-driven. Whatever breakthroughs make it into mainstream mobile, we’ll probably look back at the era of mobile computing heavily dependent on finger-tip control as being very quaint indeed.
