Sick of your clunky keyboard? Want to be able to type and use apps on your phone without actually having to touch it? Better yet, do you want to be able to control your apps by gesturing wildly in a way that’s not at all embarrassing to do in public?
Well, great news: Stanford researchers have developed a spray-on smart skin that uses AI to interpret hand movements. In a study published Dec. 28 in the journal Nature Electronics, the authors created a biocompatible material that can be sprayed on your hand and arm like sunscreen. The team said that it can potentially be used to do things like type without a keyboard or even identify objects by touch.
“With the help of machine learning, electronic devices—including electronic gloves and electronic skins—can track the movement of human hands and perform tasks such as object and gesture recognition,” the authors wrote. However, they note that, unlike a glove or most other e-skins, their spray-on skin is unobtrusive and much less bulky—allowing for much easier use.
The spray is actually a nanomesh, or a type of stretchable and mesh-like material that can bend and move with your skin. It contains an array of tiny electrical sensors that pick up on these movements that are then sent to an AI that interprets them. The machine learning approach allows for great customization to each user’s unique hand patterns, as well as quicker gesture mapping.
Here’s a look at it in action:
“As the fingers bend and twist, the nanowires in the mesh get squeezed together and stretched apart, changing the electrical conductivity of the mesh. These changes can be measured and analyzed to tell us precisely how a hand or a finger or a joint is moving,” Zhenan Bao, a chemical engineering professor at Stanford and senior author of the study, said in a statement.
Not only does this have a lot of applications to control devices like phones and laptops, but the study’s authors note that it can also one day be used in virtual reality. This will allow VR platforms to do away with unintuitive controllers and clunky gloves.
It could also be used in a variety of other settings ranging from medical to…well, literally everywhere. For example, surgeons might one day be able to perform surgeries using robots remotely with spray-on smart skin. The team also developed a method of using the skin to recognize objects by simply touching them—kind of like a wearable Google Lens.
Of course, you shouldn’t expect to see a spray can of smart skin on the shelves any time soon. A lot more research needs to go into this in order to refine the material until it’s ready for any sort of commercial application. For now, we’ll just have to continue dealing with our boring old keyboards and touching our filthy smartphone screens.
