Overview
Symphony is a suite of tools that allows for
the rapid design, prototyping and implementation of applications that
interpret and visualize a variety of biological signals. Symphony seamlessly
integrates a real-time signal processing framework, a 3D visualization
package, and the foundation required to rapidly develop applications.
System Architecture overview. (click image
to enlarge)
The MindGames group has used
Symphony to build a number of diverse applications. Some of these
were "stress tests." Others have gone on to be used in clinical,
rehabilitation and even performing arts settings. Components of
Symphony are also being used by several other research groups at the lab
(see left).
Symphony was recently
selected to be the focus of a
Microsoft Case Study.
|
|
|
Sample Applications
Here are some examples of real-time applications that have been
developed with Symphony.
Other applications in development that use Symphony include Peace Composed
and Aura Lingua.
|
 |
Mind Balance
Goal:
- Real-time non-invasive acquisition of electroencephalogram
signals to control a character
Challenges:
- Precise rendering required to produce signals that
must be detected as
brainwave activity
- Research and development of signal processing occurring in
parallel with app development.
Render speed required:
- 60fps (minimum)
Render speed achieved:
- 100fps+ (on conventional hardware)
Additional information:
- Uses Sponge to separate signal processing and rendering tasks.
- Uses native code to interface with legacy signal acquisition
software.
- Currently being extended by researchers with no prior C#
experience.
Developed with:
- MindGames group and
Univ. College Dublin Elec. Eng. Dept.
For more information:
- Click
here.
|
 |
Still Life
Goal:
- "Magic Mirror" application
used in physiotherapy and live performance settings
Challenges:
- Processor-intensive real-time video analysis
Render speed required:
- 20fps (minimum)
Render speed achieved:
- 20fps-40fps (depending on mode, on conventional hardware)
Max. interactive 3D characters for frame rate > 15fps:
- more than 60
Additional information:
- Uses Sponge to record videos of performances
- Showcased in the Féileacán project at the closing
ceremonies of AAATE2003, September 2003, Dublin.
- Provided the
complete interactive technical component of the Anima Obscura
performance at the UN's World Summit on the Information Society, December
2003.
Developed with:
- MindGames group, Dublin's
Central Remedial Clinic, CounterBalance
and SmartLab
For more information:
-
Click
here.
|
|
|
Signal Processing
An important component of Symphony is a
real-time signal processing framework that has been used in our
applications to process everything from live video to brainwaves. The
framework provides a visual interface that allows a designer to drag and
drop atomic signal processors that perform operations ranging from wavelet
transforms to image
differencing. These can be assembled into real-time signal processing
networks that are used by an application.
|
|
|
MiniSymphony
MiniSymphony is a version of Symphony that runs on the .NET Compact
Framework platform. It was created by removing Symphony's dependency on a
couple of native libraries, and whittling away a bit at its Visualizers.
In a matter of days, the
first version of
Relax
To Win PocketPC (without biometric input) was implemented using
MiniSymphony. The biometric input is currently being implemented by
the group into a new portable version of the game.
|
|
|
Sponge
One aspect of the
signal processing framework, nicknamed Sponge, provides the ability
for Symphony to dynamically harvest available resources on a Local Area
Network. Sponge uses WMI technology to query other computers on the LAN
that are running a sentinel program, and determine what processor and
memory resources they have available. Then, using .NET Remoting, Sponge
can connect to chosen machines and deploy signal processing tasks.
The Sponge server sits in
the System Tray awaiting connections.
The original intention of Sponge was to
facilitate system designs where signal acquisition and processing are
performed on potentially unspecified computers, in a dynamic LAN
environment. Sponge has allowed for this, and also offered some tremendous
fringe benefits. For example, Still Life can use Sponge to record a live
video of a performance by streaming screenshots to another computer over
the LAN in real-time. These can be processed off-line to turn them
into a moving video file.
|
|
|
Contact Information
If you have any questions about
Symphony, please don't hesitate to
contact me!
Codezone article
I recently wrote an article describing
Symphony for Microsoft's CodeZone Magazine (in the context of some of the
group's recent work: the Féileacán
Project, Still Life, Mind Balance and the venerable Relax-to-Win). I have two PDF versions of the article online: a
scanned copy of the article, and a
printer-friendly version that lacks CodeZone's
superior formatting. (Note: the engine was named Symphony
a matter of days after that article went to press.)
- Robert Burke, Architect and Lead Developer (Symphony, Mind
Balance and Still Life)
MindGames Group,
MIT Media Lab Europe
|
