Playing MIDI through SoundFonts with NAudio and MeltySynth

This week I was in Copenhagen speaking about NAudio at the Copenhagen Developer Festival.

I wanted to do a short demo to show off the capabilities of NAudio, and one thing I've always wished was possible with NAudio is to be about to load SoundFonts and play MIDI messages using them. Unfortunately that's quite a large undertaking and I've never got round to implementing it.

Introducing MeltySynth

However, I recently stumbled across a project called MeltySynth, which builds on a couple of other C# audio projects to implement a SoundFont synthesizer and a MIDI file sequencer.

MeltySynth includes a simple NAudio example, showing how you can play a MIDI file using NAudio. The example works just great, but I also wanted to be able to live play MIDI notes from a MIDI keyboard, so I made a few enhancements.

Create the demo project

My demo project itself is a .NET 6 Windows console application that references both MeltySynth and NAudio.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0-windows</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="MeltySynth" Version="2.4.1" />
    <PackageReference Include="NAudio" Version="2.2.0" />
  </ItemGroup>

</Project>

Select a playback device

The first change I wanted to make is to enable selection of the playback device to use. The MeltySynth demo used WaveOut but I switched it to WasapiOut. The following method prints out a list of available endpoints and lets you select one:

private static MMDevice GetAudioEndpoint()
{
    var enumerator = new MMDeviceEnumerator();
    var n = 1;
    Console.WriteLine("Select output device:");
    var endpoints = enumerator.EnumerateAudioEndPoints(DataFlow.Render, 
        DeviceState.Active).ToList();
    foreach (var device in endpoints)
    {
        Console.WriteLine($"{n++} {device.FriendlyName}");
    }
    var deviceNumber = Console.ReadKey(true).KeyChar - '1';
    var endpoint = endpoints.First();
    if (deviceNumber >= 0 && deviceNumber < endpoints.Count)
    {
        endpoint = endpoints[deviceNumber];
    }
    Console.WriteLine($"Selected {endpoint}");
    return endpoint;
}

Load a SoundFont and start playback

In the Main method we create a MidiSampleProvider (from the MeltySynth demo application) and point to a SoundFont (my example uses the timgm6mb SoundFont that you should be able to find a download link for here, but any GM SoundFont will do).

We then select the endpoint, and initialize it with the MidiSampleProvider and call Play. This will initially play silence as there are no MIDI notes for the synthesizer to play.

var player = new MidiSampleProvider("D:\\Audio\\Soundfonts\\TimGM6mb.sf2");

var endpoint = GetAudioEndpoint();

using (var waveOut = new WasapiOut(endpoint, AudioClientShareMode.Shared, 
            true, 20))
{
    waveOut.Init(player);
    waveOut.Play();

Listening for MIDI events

I also wanted to be able to play MIDI, and the simplest thing was to open every MIDI device and listen for events.

This is quite straightforward - MidiIn.NumberOfDevices indicates how many there are, and for each device we create a new instance of MidiIn and subscribe to the MessageReceived event. We then need to call Start on the MidiIn instance.

I've chosen to filter out certain MIDI messages that aren't relevant to my demo, and I also wanted to log some messages to the console for demo purposes.

Finally, if I decide I do want to forward the MIDI message to the SoundFon synthesizer, I call a method I've added to the MidiSampleProvider called PlayNote.

var devices = new List<MidiIn>();
for (var n = 0; n < MidiIn.NumberOfDevices; n++)
{
    var dev = MidiIn.DeviceInfo(n);
    Console.WriteLine($"Listening on {dev.ProductName}");

    var midi = new MidiIn(n);

    midi.MessageReceived += (s, arg) =>
    {
        bool forwardMessage = true;
        switch(arg.MidiEvent.CommandCode)
        {
            case MidiCommandCode.TimingClock:
            case MidiCommandCode.AutoSensing:
            case MidiCommandCode.KeyAfterTouch:
            case MidiCommandCode.ChannelAfterTouch:
                forwardMessage = false;
                return;
            case MidiCommandCode.NoteOn:
                var noteOn = (NoteEvent)arg.MidiEvent;
                if (noteOn.Velocity > 0)
                    Console.WriteLine($"Note On: {noteOn.NoteName} Velocity: {noteOn.Velocity}");
                break;
            case MidiCommandCode.NoteOff:
                // don't log
                break;
            default:
                Console.WriteLine(arg.MidiEvent.CommandCode);
                break;
        }
        if (forwardMessage)
        {
            player.PlayNote(arg.RawMessage);
        }
    };
    midi.Start();
    devices.Add(midi);
}

Passing a MIDI event to the synthesizer

There is a slight mismatch between the ways NAudio exposes MIDI message data and the way that the MeltySynth Synthesizer wants it, so I take the "raw message" and break it up into its constituent parts with some bit manipulation. Then I pass it on to the Synthesizer by calling ProcessMidiMessage.

public void PlayNote(int rawMessage)
{
    int b = rawMessage & 0xFF;
    int data1 = (rawMessage >> 8) & 0xFF;
    int data2 = (rawMessage >> 16) & 0xFF;
    MidiCommandCode commandCode;
    int channel = 0; // zero based channel

    if ((b & 0xF0) == 0xF0)
    {
        // both bytes are used for command code in this case
        commandCode = (MidiCommandCode)b;
    }
    else
    {
        commandCode = (MidiCommandCode)(b & 0xF0);
        channel = (b & 0x0F);
    }

    synthesizer.ProcessMidiMessage(channel, (int)commandCode, data1, data2);
}

Triggering a MIDI File

MeltySynth also has the ability to sequence MIDI files - that is, to work out when each MIDI message in the file needs to be passed to the synthesizer. First, we need to load a MIDI file (note that this is the MeltySynth MidiFile class, not the one in NAudio).

var file = "Demo.mid";
//file = @"C:\Windows\Media\flourish.mid";
var midiFile = new MidiFile(file);

Then to play the MIDI file, you simply pass it through to the Play method on the MidiSampleProvider, with a flag indicating if you want to loop. I made it so that if you pressed spacebar, it triggered the MIDI file, and wait for any key to stop playback.

// Wait until any key is pressed.
var key = Console.ReadKey(false);
if (key.Key == ConsoleKey.Spacebar)
{
    Console.WriteLine("Playing MIDI file...");
    // Play the MIDI file.
    player.Play(midiFile, true);

    Console.ReadKey(false);
}

Cleaning up

Finally, as well as disposing waveOut which we already put in a using block, we must Stop and Dispose each MidiIn device we opened.

foreach (var midiIn in devices)
{
    midiIn.Stop();
    midiIn.Dispose();
}

Summary

One of the coolest things about releasing your work as open source is the ways the community can build on it, extend it and integrate with it. Huge thanks to Nobuaki Tanaka (sinshu) for all the hard work that went into MeltySynth and how easy it was to integrate with NAudio.