CMU and Disney researchers create acoustruments

Researchers at Carnegie Mellon University and Disney Research have developed physical additions to smart phones, such as knobs, buttons, sliders, wheels and switches, that can control the touch screens using the phone’s own ultrasound.

Dubbed “acoustruments,” these add-ons were inspired by the different ways that wind instruments, such as whistles and flutes, utilize different sized holes to produce different sounds. With acoustruments, plastic tubes and other structures connect the phone’s speaker, which emits a continuous stream of ultrasound frequencies, to its microphone. The physical shape of the acoustrument provides different ways to stop or alter the frequency, for example by blocking and opening holes or changing the length of the tubing. These interactions effectively transform the sound before it reaches the phone’s microphone. No electrical circuitry is involved, so acoustruments are inexpensive to make and can be produced with 3-D printers, injection molds, or even by hand.

Gierad Laput, a Ph.D student in Carnegie Mellon’s Human-Computer Interaction Institute (HCII), is the lead author of the study, which was presented last Wednesday at the 2015 Conference on Human Factors in Computing Systems, in Seoul, South Korea. The paper has been named a Best Paper by the conference organizers.

“We’re providing people with tangible interactivity at basically no cost,” said Laput in a Carnegie Mellon press release. “Using smartphones as computers to control toys, appliances, and robots already is a growing trend, particularly in the maker community. Acoustruments can make the interactivity of these new ‘pluggable’ applications even richer.”

As an example of the wide range of possibilities of acoustruments, the researchers created an add-on that acts as a physical alarm clock with an on/off button and a snooze button. The add-on utilizes a pressure sensor to disrupt sound and tell the phone what to do. Researchers have also developed an acoustrument that is a part of the phone case and acts as a camera shutter, one that turns the phone into a toy car, and one that transforms the phone into an interactive doll.

Carnegie Mellon and Disney researchers have determined in their experiments that acoustruments can control a smart phone with 99 percent accuracy. In addition to Laput, the research team included two HCII faculty members, Scott Hudson and Chris Harrison, and Eric Brockmeyer of Disney Research Pittsburgh.