V-Drum Explorer: MIDI interface
If this is the first blog post about V-Drum Explorer you've read, see the first post in this series for the background. In this post we'll look at the MIDI interface I use in V-Drum Explorer to send and receive configuration information.
MIDI basics(Apologies to anyone who really knows their stuff about MIDI - this section is the result of bits of reading and experience working with my drum kit. Please leave corrections in comments!)
MIDI is a protocol for communications between musical instruments, computers, synthesizers and the like. There are many different ways of connecting devices together, including dedicated MIDI cables, USB, TCP/IP and Bluetooth, but applications shouldn't need to care much about this. They should only need to worry about the messages defined by the protocol.
MIDI is one-directional, in that APIs expose both input ports and output ports which are effectively independent even if they're connected between the same pair of devices. Messages are sent on output ports and received on input ports.
Most electronic instruments (of all kinds, not just drums) support at least basic MIDI functionality. There are several simple messages with standard representations, including:
- Channel Voice messages (such as note on" and note off")
- Channel Mode messages (such as all sounds off")
- System Exclusive messages which are manufacturer-specific
The channel voice and channel mode messages apply to a specific MIDI channel. It's possible for multiple devices to be daisy-chained together so that they can be controlled over a single port, but using different channels. Each of these messages has a fixed size.
Note: one irritating problem with the protocol is that I haven't found a way of detecting which channel a particular device is on. This is why channel-specific operations in V-Drum Explorer (such as recording instrument sounds) need a user interface element. If there's something I've missed here, I'd love to hear about it.
Only a few operations in V-Drum Explorer need to simulate or react to the channel messages. Most of the time we're interested in the system exclusive messages which allow the configuration data to be fetched and set.
System Exclusive MessagesSystem Exclusive Messages - often abbreviated to SysEx - are manufacturer-specific messages for additional functionality. The size of the data isn't fixed, but is indicated by a byte of 0xF0 indicating the start of the data, and another byte of 0xF7 indicating the end of the data. Each byte within the data has to have the top bit cleared.
The first byte of data is conventionally the manufacturer ID as assigned by the MIDI Manufacturers Association, which allows a device to ignore messages it wouldn't properly understand. For example, all Roland SysEx messages have a first byte of 0x41.
Protocol design note: MIDI is a relatively old protocol, dating back to 1983. I'd hope that these days we'd allow more than one byte for a manufacturer ID, and use a length-prefix instead of restricting every data byte to 7 useful bits. Oh well.
SysEx messages are divided into realtime and non-realtime messages. There are also universal SysEx messages which are non-manufacturer-specific, using fake" manufacturer IDs of 0x7E (for non-realtime) and 0x7F (for realtime).
For the V-Drum Explorer, we're interested in four SysEx messages:
- We send Identity Request (universal, non-realtime)
- We receive Identity Reply (universal, non-realtime)
- We send Data Request (Roland, non-realtime)
- We send and receive Data Set (Roland, non-realtime)
The identity request and identity reply messages are used to discover information about the connected devices. V-Drum Explorer uses them to check whether you have any supported devices connected (currently just the TD-17 and TD-50) and present the appropriate schema for the device.
The Data Request and Data Set messages are more interesting from the perspective of V-Drum Explorer: they're the messages used to communicate the configuration data.
You can think of the configuration data as a bank of memory in the module. (We'll look at the layout next time.) The Data Request message indicates Give me X bytes starting at address Y." The Data Set message indicates Here are some bytes starting at address X." Interestingly, Data Set is used both to respond to Data Request and to set new data in the module. When loading data from the device, I send a Data Request and wait for a corresponding Data Set. When copying data to the device, I just send a Data Set message. The device also sends Data Set messages when the configuration data is modified on the module itself (if a particular setting is turned on).
There are a few tricky aspects to this though, all effectively related to the protocol being two independent streams rather than being request/response in the way that HTTP is, for example:
- Can the code rely on all the data from a single Data Request being returned in a single Data Set? (Answer: no in theory, but in practice if you ask for small enough chunks, it's fine.)
- How long should the code wait for after sending an Identity Request until it can assume it's seen all the Identity Reply messages it's going to?
- How long should the code wait for after sending a Data Request until it can assume there's something wrong if it doesn't get a reply?
- How long should the code wait between sending Data Set messages when copying data to the device?
- What should we do with messages we didn't initiate?
In the V-Drum Explorer MIDI code, I've modeled Identity Request / Identity Reply using an event for I've seen this device", and I just leave half a second for all devices to respond. For Data Request / Data Set things are a little trickier. For better or worse, I expose an asynchronous API along the lines of
async Task<byte[]> RequestDataAsync(int address, int size)
I keep a collection of address/size pairs I'm waiting to see data about", and when I receive a Data Set message I see whether I'm expecting it, I complete the corresponding task, with a buffer of unexpected messages for diagnostic purposes.
The current implementation uses the Sanford.Multimedia.Midi library, which at least assembles multiple packets into SysEx messages for me. I'm trying to move to managed-midi though, as that's more cross-platform. (In particular, it should make a Xamarin client possible, as well as running a CLI on a Raspberry Pi.) The managed-midi library is quite low-level, so I'll need to perform the reassembly myself - but that shouldn't be too hard.
One thing I'm really hoping is that I can reimplement the VDrumExplorer.Midi project with almost no changes to VDrumExplorer.Wpf.
Next time...In the next post, I'll go into more detail about the layout of the configuration data, and the annoyance of 7-bit addressing.