Repurposing optical mice for acoustic eavesdropping

Abstract

Acoustic eavesdropping presents a longstanding challenge in the realm of personal information security and privacy preservation. In this work, we introduce a novel eavesdropping method called JerryAttack, which repurposes an optical mouse as a covert eavesdropping device. Specifically, we transform the mouse's integrated low-resolution but high-frame-rate image sensor into a high-speed camera for visual vibrometry, capable of capturing acoustic vibrations from nearby loudspeakers. Our contributions are threefold: First, we utilize the 'pixel grabber' register as a backdoor to extract the pixel stream from the image sensor. Second, we establish an acoustic-optical side channel that enables effective acoustic eavesdropping. Third, we thoroughly explore two attack scenarios: voice profiling and speech reconstruction. Our findings reveal that the sound recovered through our side channel achieves a mean SNR of 7.3 dB, comparable to standard microphone recordings in noisy environments like cafes. Additionally, when combined with a classification neural network, JerryAttack identifies individuals with an overall accuracy of 83.27% across six languages. Moreover, when cooperated with joint channel information, JerryAttack consistently achieves good intelligibility, with a median STOI score exceeding 0.7 in reconstructed results.