Not buttons but gestures? A look into touchless interaction’s struggling history

Interacting with technology without touching it is where Google Motion Sense and Microsoft’s Kinect have already failed. Now Apple is trying, introducing double tap on Apple Watch 9. But why, and with what chance of success? 

Along with other new features, the new Apple Watch Series 9 introduced a Double Tap gesture that lets users control various actions by pinching their thumb and index finger together twice. The new gesture, picked up with the utmost precision by the wearable device thanks to its sensor and the processing power of the new S9 chip, lets users control the Watch’s interface without using the other hand at all. Apple touts it as a great solution for managing the device while carrying a coffee or while doing sports.

The Double Tap allows users to answer or end a call, control music playback, or activate the new WatchOS 10 Smart Stack. Furthermore, double taps can then be used to scroll through the widgets. 
The idea stems from an accessibility feature that users with disabilities have been able to activate on the Apple Watch for quite some time, enabling an array of different gestures (such as wrist rotation and fist clenching). Still, Apple engineered a simplified solution that only focuses on picking up and recognizing a double pinch.

Microsoft Kinect

The history of touchless interfaces is peppered with spectacular failures. Microsoft’s Kinect is probably the best-known one. Sold as a peripheral for the X-Box Console, the device could recognize the player’s body as a 3D object in space, reading their movements as interaction inputs. While the novelty was there, it quickly wore off in the market, but games for the Kinect failed to be compelling enough, and reliability was also an issue. Eventually Microsoft discontinued the product. Apple would later acquire PrimeSense, the company developing the Kinect Sensors. They didn’t apply the know-how to touchless interfaces, though; rather, they turned it into the 3D TrueDepth camera that enabled the development of FaceID.

Among the startups that tried their luck with hand-gesture recognition, Leap Motion is also worth mentioning. The company developed a USB controller that users could position in front of their laptops to enable touch-free interaction with most operative systems. As it often happens, the startup was probably a bit too early. It eventually got acquired and turned into Ultraleap by a UK firm, which is now trying to tout its hands-free gesture control solutions to AR/VR visor manufacturers. 

Google Motion Sense on Pixel 4

Google has also been experimenting with touchless interfaces for a long time. That experience has recently consolidated into Project Soli, which the company has already embedded in products like the second-gen Next Hub. The Motion Sense feature of the Pixel 4 smartphone was based on the same tech. Unfortunately, like many similar attempts by Android manufacturers (LG, for example), the feature failed to get traction and would drain the battery quicker. Motion Sense was later dropped from new Pixel phones, and the outlook on Project Soli does not look good. The website stopped getting updates in 2022 and knowing Google’s penchant for axing large projects without many second thoughts, there’s a high chance that Soli could be next in a long list of failed Google experiments. 

This brings us back to our original question. Will Apple’s double tap gain traction as a widely used and recognized touchless gesture, just like the swipe or the pinch-to-zoom on touch screens? 
The chances are high if Apple properly describes the feature to new users and pushes hard on its discoverability. This simple gesture also has higher stakes for the company than we might initially consider. While it’s a nice novelty on a Watch, it’s also one of the main touchless interaction gestures on the new Apple Vision Pro headset. Therefore, introducing the Double Tap with the Apple Watch Series 9 means normalizing a gesture that users could find natural when trying a Vision Pro for the first time. 

At the same time, Apple engineers don’t shy away from trying out and subsequently removing interaction modes from their devices. Force Touch is a great example, as are software features like the Smart Stacks, which briefly moonlighted in the second instance of WatchOS to just be scrapped and then return in a better and more coherent version in the latest WatchOS 10. 
While Double Tap could follow the same script, we believe it’s an easy-enough feature to use or otherwise disable from the settings at a user’s will that there would be no reason for Apple to remove it in the future, even if users won’t pick it up as much as the company would hope.

Apple Vision Pro

Latest News

Latest on Domus

Read more
China Germany India Mexico, Central America and Caribbean Sri Lanka Korea icon-camera close icon-comments icon-down-sm icon-download icon-facebook icon-heart icon-heart icon-next-sm icon-next icon-pinterest icon-play icon-plus icon-prev-sm icon-prev Search icon-twitter icon-views icon-instagram