When seeing isn't enough for believing

Credit: Thomas Kwak

Composers inspired by the cosmos—the Kronos Quartet, Brian Eno, Mike Oldfield and others—have produced interesting, if sometimes cringe-worthy, music. Now, science is adding a new wrinkle to spacey tones, using sound to represent information that is typically communicated visually.

The field of “sonification”—the translation of non-audible data into sound—has been around for decades. (Think of the Geiger counter.) Even so, as recently as 1997, it was described in an NSF report as being “in a formative stage.” And then it began expanding: Attendance at the annual conference of the International Community for Auditory Display (ICAD) has almost doubled since then.

UC, Berkeley’s Space Science Lab (SSL) is one group that’s at the vanguard of finding new ways to represent data. Having developed an “iconic” sonification system, scientists at SSL are getting ready to analyze data from a pair of spacecraft, called STEREO A & B, that will study coronal mass ejections and solar winds. Each category of data collected by STEREO (like different energies of solar particles, or the rate at which they hit a detector) is assigned a sound quality (like a note on the scale, or degree of volume), with each instance of a particular data point producing its respective sound quality.

Roberto Morales, a PhD student at Berkeley’s Center for New Music and Audio Technology, wrote computer programs that turn STEREO’s data into sound, to be analyzed by the SSL physicists. Morales’ sonification tools will help focus their attention on certain types of events that merit further investiga­tion. Laura Peticolas, a physicist who oversees Morales’ work, says the algorithms will give the scientists “flexibility…really any color graph can be displayed and then listened to, which is rare in sonification.” Janet Luhman, who will be interpreting STEREO’s data, said her team will now have “the opportunity to hear the spatial and temporal dimensions of space weather together… If I listen to the data I may be better able to sort it out.”

Sonification can enhance understanding of information that would otherwise look like a confusing jumble.

Indeed, sonification’s adherents say that hearing data through iconic sonification, rather than just seeing it, can enhance understanding and enable the recognition of patterns in information that, displayed visually, would look like a confusing jumble. “The human auditory system is the best pattern-recognition device that we have,” said Bruce Walker, a computing and psychology professor at Georgia Tech and president-elect of ICAD. “And when you’re trying to figure out patterns in any complex data set, it turns out to be very effective to use sound in order to determine those patterns.”

Cognitive scientists agree. Auditory representation enables recognition of “certain patterns…that you wouldn’t be able to see in the [visual] sense,” said Marty Woldorff, associate director of Duke’s Center for Cognitive Neurosciences and an expert in sensory perception. Vision tends to work best for spatial data, naturally, but it’s been established that we process temporal information better by hearing it. For instance, abnormal patterns in EEGs are better grasped by ear than eye, allowing for a quicker diagnosis of epilepsy and other disorders. And when it comes to perceiving data, Woldorff added, more can equal better: “If auditory and visual stimuli are synchronously presented…you get enhanced processing.”
In a separate project, Morales also alters the sounds to “write” his own music. Last year, he composed an orchestral suite, “Turning Points,” that was based in part on solar winds. He likes to “play around,” turning the data into a piece with “the aesthetics [he’s] looking for.”

Whatever the relative merits of space-based music (Rush’s 2112, anyone?), STEREO may hold promise both for sonification and for space science. Peticolas says she looks forward “to find[ing] out if we discover anything new in the solar wind from listening to the data rather than looking at it.”

Originally published October 3, 2006

Tags

Share this Stumbleupon Reddit Email + More

Now on SEEDMAGAZINE.COM

  • Ideas

    I Tried Almost Everything Else

    John Rinn, snowboarder, skateboarder, and “genomic origamist,” on why we should dumpster-dive in our genomes and the inspiration of a middle-distance runner.

  • Ideas

    Going, Going, Gone

    The second most common element in the universe is increasingly rare on Earth—except, for now, in America.

  • Ideas

    Earth-like Planets Aren’t Rare

    Renowned planetary scientist James Kasting on the odds of finding another Earth-like planet and the power of science fiction.

The Seed Salon

Video: conversations with leading scientists and thinkers on fundamental issues and ideas at the edge of science and culture.

Are We Beyond the Two Cultures?

Video: Seed revisits the questions C.P. Snow raised about science and the humanities 50 years by asking six great thinkers, Where are we now?

Saved by Science

Audio slideshow: Justine Cooper's large-format photographs of the collections behind the walls of the American Museum of Natural History.

The Universe in 2009

In 2009, we are celebrating curiosity and creativity with a dynamic look at the very best ideas that give us reason for optimism.

Revolutionary Minds
The Interpreters

In this installment of Revolutionary Minds, five people who use the new tools of science to educate, illuminate, and engage.

The Seed Design Series

Leading scientists, designers, and architects on ideas like the personal genome, brain visualization, generative architecture, and collective design.

The Seed State of Science

Seed examines the radical changes within science itself by assessing the evolving role of scientists and the shifting dimensions of scientific practice.

A Place for Science

On the trail of the haunts, homes, and posts of knowledge, from the laboratory to the field.

Portfolio

Witness the science. Stunning photographic portfolios from the pages of Seed magazine.

SEEDMAGAZINE.COM by Seed Media Group. ©2005-2015 Seed Media Group LLC. All Rights Reserved.

Sites by Seed Media Group: Seed Media Group | ScienceBlogs | Research Blogging | SEEDMAGAZINE.COM