In a nutshell: It has long been known that electromagnetic fields (EMF) can do some wonky things to electronic devices. Recently, scientists have attempted to determine whether they can manipulate an EMF in such as way as to make a gadget do what they want. They were successful.

Researchers from the University of Florida and the University of New Hampshire presented work on an "invisible-finger" attack at Black Hat USA 2022 in Las Vegas last week. Using some complicated science, a robotic arm, and multiple antenna arrays, the scientists could remotely simulate a finger touching the capacitive touch screens of several devices.

The method involves using one hidden antenna array to pinpoint the location of the targetted device and another to generate an electromagnetic field with precise frequencies to send voltage signals to the sensors in the display. The processor then interprets these signals as certain types of touch.

The team could simulate taps, long presses, and swipes in any direction on multiple devices, including iPad, OnePlus, Google Pixel, Nexus, and Surface. Hackers could theoretically use an invisible finger attack to remotely do any number of things that would require the user to touch the screen.

"It just acts like your finger is doing the work," said University of Florida PhD candidate and lead presenter at the conference Haoqi Shan. "We can even generate an omnidirectional swipe on the iPad and Surface. We could totally use this to open a gesture-based lock."

During tests, they used the technique to install malware on an Android phone. Shan said they also sent money "using press and hold on PayPal." Some tests were foiled by the EMF's inability to trigger small hitboxes. For example, anything requiring a response to an Android Yes/No dialog would not work because the small yes and no buttons were too close together.

Before worrying about invisible fingers manipulating our gadgets, it's important to note that bad actors are likely a long way off from using this attack vector for several reasons.

Although the researchers didn't mention the cost of equipment, the fact that the technique requires several pieces of likely expensive hardware probably prevents it from being cost-effective. The robotic arm used to precisely position the electromagnetic antenna could run into the thousands of dollars alone. It also requires intimate knowledge of how touch screens work and the precise voltages needed to register the desired gestures.

Furthermore, the range is far too short to be practical in almost any conceivable scenario. Shan stated it is only effective within three to four centimeters — a range fine for labwork but tricky to impossible to pull off in a real-world setting. So it's more of a proof-of-concept for now.

However, Shan also noted to conference-goers that this is a brand new attack vector, and others could undoubtedly improve upon it.

"[This design is] a relatively new type of attack, even for professional researchers, [though] once you gain the knowledge here, you should be able to reproduce what we are doing now," Shan explained. "Maybe you'll come up with a more powerful or much cooler attack."

Mitigation is not dire at the moment. However, Shan says capacitive touch display manufacturers should consider implementing force detection to prevent this type of future intrusion. Some may recall that Apple introduced "Force Touch" to iPhones and other devices in 2014. However, it discontinued the feature in 2018 — at least for iPhones.

The most effective consumer-level mitigation for invisible fingers would be using a Faraday cage. Slipping your phone into a Faraday bag or something similar might not be all that convenient, but case manufacturers could design stylish phone enclosures that eliminate electromagnetic interference. Some wallet makers have already done this to protect credit cards from skimming devices that read a card's NFC chip.

Those interested can check out the white paper and presentation slides at Black Hat USA's website if the above demo video was too tame for your brain.