HOME CONSULTING PUBLICATIONS PROJECTS CV (PDF)
Space RAT Space RAT
An Interface for Spatializing Streaming Audio Sources in Virtual Environments

Spatialized sound is generated in tele-collaborative virtual environments by giving the position of each sound to a sound server. This process usually involves a complete catalogue of the sounds on each client machine. Streaming VOIP sound sources typically involve external applications such as the Robust Audio Tool from UCL. I developed an interface through our Bergen Sound Server to spatialize individual streaming sources by modulating RAT 3D position and gain in real-time.

Robust Audio Tool
The Robust Audio Tool is a streaming audio application for conferencing multiple voice-over-IP clients on standard networks. The audio gain and 3D positioning can be set individually for each participant. By adjusting these variables in real-time, a source can be spatialized in stereo relative to the user.

Stereo Spatialization System Diagram
In a tele-collaborative virtual environment, each participant is represented by an avatar in the scene. The audio gain and 3D position of a remote user is calculated as a function of their distance and angle relative to the local user. Our existing 4-channel spatializing sound server receives sound and local user position information via the OSC sound protocol. The server was modified to also recieve the position of each remote user in the world. The RAT application is actually three applications that communicate using the Mbus messaging protocol. The Bergen Server applicaton uses this protocol to control the stream parameters associated with each remote user.

RELATED LINKS

The official RAT website maintained by University College London.
Download location for the latest version of Bergen Sound.
Project page describing the Spatial OSC messaging protocol used by Bergen clients.
The Ygdrasil ratSource node that establishes streaming source identity and position.

MORE DETAILS

Mbus Protocol

Mbus Spoofing
The three individual RAT applications use the Mbus messaging protocol for interprocess communication. Although the software is opensource, its preferable to have a solution that does not require a customized version of the Robust Audio Tool. Bergen Server immitates the RAT control application by using its process id to initialize Mbus communication. Appropriate messages are given to both control the media engine and update the status of the user interface.


Ygdrasil Sound Nodes
Ygdrasil Sound Nodes
Sampled sounds and other sound related nodes created in Ygdrasil communicate with the sound server through the Bergen Client software. Each user can employ an amplitude node to recieve the current amplitude of their voice input to the sound server. This amplitude value can adjust parameters of the avatar mouth to enhance its visual appearence during conversation. Each user can also add a ratSource node that reports their position to the Bergen Server for audio stream spatialization. Because audio communication is usually established before joining a tele-collaborative session, the RAT source SSRC identifier of the cleint can be used to intialize the BERGEN_RAT_SOURCE variable in the Ygdrasil configuration file.