Researchers Can Read [Some of] What People Are Typing During Video Calls
rigrig writes:
Researchers have demonstrated they can read what people are typing during video calls by looking at shoulder movements.
This horrifying Zoom hack will deter you from ever side-chatting again:
"From a high-level perspective, this is a concern, which obviously has been overlooked for a while," says University of Texas assistant professor of computer science Murtuza Jadliwala, who led the research, examining what could happen if your video meeting were hacked. "And actually, to be really frank, we didn't start this work for COVID-19. This took a year. . . . But we started realizing in COVID-19, when everything [is in video chat], the importance of such an attack is amplified."
As Jadliwala explains, the core problem is that our face-to-face video streams are presented in high fidelity, and their pixels convey more information than we realize. Without using any special machine learning or artificial intelligence techniques, Jadliwala's team figured out how to read the subtle pixel shifts around someone's shoulders to make out their basic cardinal movements: north, south, east, and west.
Applied to a keyboard, these four directions actually mean a lot. If you are typing "cat," you start with the C, move west to the A, then back east to the T. Once researchers figured out how to read these directions through shoulder movements, they were able to create software that could cross-reference them with what they call "word profiles" built with an English dictionary, which turned the maze of directions into meaningful words.
[...] In a lab setting, with a certain chair, keyboard, and webcam-while testing a limited pool of words-the average accuracy of the software was 75%. When the team tested subjects working from home in uncontrolled setups (they were asked to visit any websites, write emails, and enter their passwords), accuracy dropped significantly. The team was able to reverse-engineer 66% of the websites visited, but only 21% of random English words, and about 18% of the passwords typed.
Also at
Tech Xplore
Distinct Today
Daily Mail
Fast Company
Read more of this story at SoylentNews.