Spatial Audio: Sound in Space
Last year, New York University's Steinhardt School of Culture, Education, and Human Development put the finishing touches on a $6.5 million, 7,500-square-foot addition?the new James L. Dolan Recording and Teaching complex at the Department of Music and Performing Arts Professions.
The 3D Research Lab of New York University's Steinhardt School.
For several years now, the video component of audio/video technology has gotten all the attention: digital video, high definition, videoconferencing, digital signage, and now 3D. But could all this focus on what people see lead to a greater emphasis on what they hear?
Take a 3D movie. With advances in video technology, moviemakers can create the illusion that an object or character is coming right at the viewer–virtually jumping out of the screen. But if the character is coming at you, shouldn't his voice be getting louder? Or if not louder, then at least nearer?
Last year, New York University's Steinhardt School of Culture, Education, and Human Development put the finishing touches on a $6.5 million, 7,500-square-foot addition–the new James L. Dolan Recording and Teaching complex at the Department of Music and Performing Arts Professions. "The scale and depth of the technological and acoustical capacity of this facility are unparalleled," Robert Rowe, vice-chairman and director of music composition, said at the time. The AV design was led by Walters-Storyk Design Group Systems and includes two separate 10.2 surround systems, a Dangerous Music ST/ST Monitor Controller for the recording studio's critical listening environment, and a Renkus Heinz multiconfiguration speaker system for a large-screen, HDTV projector-equipped conference/screening room. The facility has universal Crestron control to provide touchscreen interfaces between audio and video systems, HD/SDI and Composite video routing systems, a Yamaha DME 64 Controller, two Soundweb London Blu-16 processors, and extensive microphone and speaker wiring among the various areas. Even bathrooms can act as live recording environments.
Among the complex's AV installations is a new lab devoted to the research of 3D, or spatial, audio–the kind that would make the voice of an Avatar character close in on a movie-goer.
"3D is more of a physically accurate way to simulate sound" says Agnieszka Roginska, associate director of music technology, who oversees the 3D Research Lab. "Surround sound immerses you in a spatial environment, but it does not accurately represent the space. You can tell the difference between being seated in a theater with a surround sound system and being in the real world." That's where 3D audio comes in.
Technology companies have been developing 3D audio solutions for years, many of them aimed at gamers. Creative Labs and QSound Labs, for example, already market 3D sound products. Before she joined NYU, Roginska worked for a Silicon Valley company that did 3D audio simulation projects for NASA, the U.S. Air Force, and the Navy.
Much of the work in training and simulation is an effort to augment people's ability to get data. For example, about 40 years ago sonar (ironically) became more of a visual application, with operators in front of screens. Researchers are now working to reintroduce critical sound information to sonar systems. That's because auditory systems remain better suited to picking up small changes–or transients–in an environment. "If you're out walking in the woods and you hear a branch snap, it doesn't have to be loud, but you're going to hear it," Roginska explains. "But if you were looking at a branch snap, you probably wouldn't see it."
Research into 3D spatial audio looks at sound in several different ways. It analyzes it as it's emitted from a source and travels through a medium such as air, dissects how the medium impacts sound, and finally simulates how the sound interacts with a listener. This last part is the most complicated, and researchers look at it in a variety of configurations.
"Say you're listening to something with headphones," Roginska says. "When you turn your head, what you hear doesn't change, but that doesn't happen in the real world." In a room like the NYU lab, researchers place sensors on a subject's head to track location and orientation (yaw, pitch, and roll), and then recalculate the auditory space to create an accurate simulation of, for instance, sounds that should be coming from behind a person in a game. "These audio clues give you a lot of important information."
But that information is computationally intensive. Only recently were computer processing platforms powerful enough to create realistic graphics and leave enough headroom to simulate spatial audio.
NYU's 3D Research Lab has 16 Genelec speakers, two Genelec subs, and multichannel miking, tracking, and playback options. It boasts extremely low 0.2-second reverb time. "There is a way to use two speakers to produce spatial audio, although it's very difficult and sensitive," says Roginska. "Typically we use a big array, which is what we've designed here."
Iosono, based in Germany, similarly uses many speakers–dozens or hundreds, depending on the application–to bring 3D audio to audiences. "Instead of reproducing the sound at a person's ears, they aim to reproduce the entire wavefield," Roginska says.
Iosono's technology is already installed at Disney World and Mann Theaters in Hollywood, among other places, making it one of the leaders in 3D audio.
Will spatial audio replace surround sound? "The technology is there," says Roginska.