Who Cares About Audio?

Excuse me while I rant a little about audio. Why is there so much progress in the world of computer graphics, while audio is pushed to one side as a secondary concern?

Hardware

I bought my first ever laptop computer last month. I've been very pleased with it. It has 3GB RAM, a 2.2GHz dual core processor and a 320GB HDD. The graphics are a very competent NVidia 8600 chipset with 256Mb RAM. Pretty much everything about it is great. Except the audio. The speakers are atrociously tinny and have no frequency response at all below about 200Hz. The soundcard drivers for Vista are appalling. You can't connect the outputs to a PA system while the power supply is plugged in because they pick up too much noise. Why oh why do top of the range (OK middle of the range) laptops have audio components worth less than £5 in total?

Vista

The headline audio feature in Vista was a switch to a 32-bit mixing engine. The shocker was that XP was still mixing in 16 bit. The ability to adjust volumes from different applications is kind of nice, but is missing a crucial feature - the ability to mute the sound coming from other logged in user's applications. Very annoying if your children left themselves logged into the CBeebies website.

The transition to Vista has been painful for a lot of people, but surely no one more than users of pro audio. If you are using your computer with a pro soundcard, whether USB, FireWire or internal, chances were it didn't have drivers until a full year after Vista's official release. And the current state of drivers is still dismal, with most still in beta, latencies much worse than the XP drivers, and no sign whatsoever of 64 bit drivers on the horizon (with the refreshing exception of Edirol).

The much heralded WaveRT driver model has not been adopted by any of the major pro audio card manufacturers and all the promise behind WASAPI and MMCSS has not been realised. Some FireWire interfaces are being sold off on the cheap, presumably because they don't and never will support Vista (e.g. Tascam FireOne, Mackie Onyx Satelite). Vista is turning out to be a bad OS for a DAW.

.NET Framework

When the .NET 1.0 framework came out, I was disappointed that it included no audio APIs. I set about writing my own .NET audio library (NAudio), and experienced a lot of pain getting interop working with the WaveOut windows APIs. I fully expected that Microsoft would rectify this omission in the next version of .NET. After all, Java had some low-level audio classes in the javax.audio namespace even back then. However, .NET 1.1 came out and all we got was a rudimentary play a WAV file control. Nothing that couldn't be done with a simple wrapper around the PlaySound API.

So I waited for .NET 2.0. But nothing new for audio there. Or in 3.0. Or in 3.5. And the open source community hasn't exactly done anything much. NAudio has picked up no traction whatsoever, and over the last seven years, I have only come across a handful of similar projects (a few wrappers of unmanaged audio libraries here and there, and a couple of articles on CodeProject).

I find it astonishing that Microsoft are happy to leave vast swathes of the Windows API without any managed way of accessing it. It was perhaps understandable that they couldn't wrap the entire API when .NET 1.0 came out due to the sheer size of the task, but that most of the new APIs for Vista have no managed wrappers doesn't make any sense to me. Wrapping WASAPI will not be a trivial task, as as such it seems destined not to be used in .NET for the forseeable future. And though the WaveOut APIs may have been superceded, the MIDI and ACM APIs are still Windows only way of accessing MIDI devices and compression codecs.

Also disappointing is the fact that the managed DirectSound wrapper project has effectively been abandoned with no planned support for any .NET 2.0 wrapper assemblies. The March 2008 DirectX SDK still contains .NET 1.1 wrappers that trigger LoaderLock Managed Debugging Assistants when you try to use them in Visual Studio. All that we have on offer now is the XNA framework, which is solely focused on the game market, and therefore lacks the flexibilities needed for a lot of streaming low-level audio scenarios.

As for the de-facto standards of pro-audio, ASIO and VST, there seem now to be one or two small projects emerging to provide .NET wrappers. How cool would it be to create a VST plugin that allowed you to code in any language (VB, C#, IronRuby, IronPython) and compile it on the fly. OK, the garbage collection nature of .NET means that you will not be using it at super low latencies, but nonetheless, some very creative applications could be created. Yet at the moment, all the development time has to be put into getting the wrappers working, rather than on writing the end products.

Silverlight

Finally, what about Silverlight? The good news is that it supports MP3 and WMA. For most people that would be enough. But again, why does .NET offer us a stream-based audio API which we could directly manipulate samples before they were sent to the soundcard (i.e. a PCM WAV stream)? This would open up the way for all kinds of cool applications. Check out this astonishing Flash plugin for an idea of what would be possible. (interestingly, Flash developers are petitioning for more audio functionality, and they are already enjoying far more freedom than Silverlight offers). Microsoft, how about beating Adobe to the punch on this one?

Even the excellent Silverlight Streaming service is video-centric. You can include audio files as part of an application, but unlike video, you can't upload audio separately, effectively ruling out the possibility of just updating one audio file for an application that used many files.

Conclusion

Why is it that audio is so low on people's priorities? Year on year we see great progression in video cards, cool 3D rendering technologies like WPF, and Silverlight can do rotating translucent videos with just a couple of lines of XAML. But if you simply want to add some EQ to an audio stream, be prepared to write mountains of code, and tear your hair out debugging the exceptions when the garbage collector moves something that should have stayed where it was.