Beginner's guide to MIDI coding in VM

[drohnee]

Thanks for all of these helpful suggestions and resources. Apologies if I have shifted the focus of this thread, but hopefully other newcomers will find the comments provided by more experienced devs as helpful as I do.

The inspiration for my original question about creating a queue of MIDI messages is a type of sequencer that I have in mind. This idea is still in early stages so has not been carefully considered. Based on MIDI notes coming in to an input jack and the values of a few UI controls, this will generate an ongoing stream of MIDI messages at the output. My assumption had been I would do this by creating a queue (or some other appropriate data structure) of notes each time a new input comes in via MIDI, and then check timestamps to determine when to send out each of these programmatically defined notes. But now that I am aware of the MidiEvent class, I see there is probably a better way to implement what I had in mind.

I was originally just talking about "persistence" in the sense of maintaining state within a single instance of the module but across different calls to ProcessSample(), and had not considered the related issue of persistence across multiple instances of a module that Colin raised. For now I do not think the couple of ideas I am considering need to grapple with the more complicated shared state across multiple instances of a module, but it is interesting to be aware of that and think about the implications.

[Steve W]

public void ProcessSample()
This function gets called once for every sample that gets generated by Voltage. Typically, this function will read input signals from input jacks, process the audio through a DSP algorithm, and write output signals to output jacks.

All audio generated by Voltage is fixed at 48 kHz, regardless of the sample rate of the host application. As a result, you can build fixed sample-rate DSP algorithms that do not need to adjust for sample rate changes.

It is always a good idea to write intelligent and efficient code in your ProcessSample() algorithm. If a module is not connected to any outputs, for example, it may be possible to skip sections of CPU-intensive DSP processing. You will always want to write a value of 0 to any outputs, even if you’re skipping DSP processing, so that values do not get “stuck” on a cable when a module’s inputs are disconnected.

I believe he was asking about persisting MIDI data between calls to ProcessSample(), not between uses of the module. And the short answer to that is, using any data structure you want, so long as you don't make it an automatic variable local to the call.

I think of "sample" as a term related to audio. I have never thought of midi data (bits, bytes, commands) as samples.

Does VM conflate audio and midi and use ProcessSample() to handle both midi and audio?

[UrbanCyborg]

Don't let the terminology throw you; audio and MIDI are just two input streams to the module. ProcessSample() got called that because it's the lowest common denominator in terms of time-slice available to the module developer, corresponding to the sample rate.

Reid

[ColinP]

Basically what Reid said.

There are two main ways that VM communicates with a module: one is an event dispatcher called Notify() and the other is ProcessSample().

ProcessSample() is called for every sample at an average rate of 48,000 times per second. I say average because it actually gets called faster than that but in bursts that correspond to the number of samples you set as the buffer size in VMs audio settings. So you get a spurt then a rest period where the CPU tries to do everything else it needs to do.

Notify() on the other hand operates generally at a much slower speed. So when a knob is turned or a bit of customized display needs redrawing or a timer event fires for instance. Most modules ask to get a timer event 20 times per second and you can do various none urgent jobs by handing this event in Notify()

But MIDI needs to be handled more responsively than just every 1/20th of a second so rather than use the Notify() timer event we handle MIDI inside ProcessSample() even though most of the time there's nothing to do. Hence the remark about VoltageMIDIJack GetMessages() returning null.

[Steve W]

Thanks for the explanations of what "sample" means in terms of VM. I am used to interrupt driven midi data processing but ProcessSample() seems to be more driven by audio processing than midi. In terms of serial midi communication I have used rates from 31.25 Kbps to 115.2 Kbps. So as I understand it. VM's processing of 64 bits of audio data at 48 KHz should not be causing problems with the handling of midi data (which I have been seeing) except insofar as extensive audio processing might "get in the way of" (so to speak) of processing midi data.

With other languages I developed personal strategies/methods for prioritizing midi input that were interrupt driven.

Not really confused about audio and midi outside the context of VM, just interested in finding ways to develop really tight processing code (efficient time-wise). So, these explanations of what VM does and how it does what it does is helpful! Thanks for taking the time to help me get up to speed.

[Waverley Instruments]

My understanding is that all processing in VM is done one sample at a time, otherwise we'd see jitter with audio and MIDI with different buffer sizes, and we don't. So to my mind, It's not quite the same as a "render block" that has to populate a buffer as is customary in other DSP wrappers / implementations. There will of course be an audio buffer, but I think the general idea is that we don't have to worry about that

This is my understanding, having spoken to the guys at CA about it when our MIDI sequencer was in development:

Any MIDI messages that are read from a MIDI jack during ProcessSample() are for that particular sample, and any MIDI messages that are output to a MIDI jack will be received by the next module one sample later.


-Rob @ WI

[ColinP]

AFAIK ProcessSample() does not run at a regular 1/48,000 time per second rate. I used to imagine it did (coming at this from an interrupt driven graphics background where there's only double or perhaps triple buffering i.e. the video buffer is two or three samples in length) but when you think about it, that would be really inefficient and fragile as you'd need to ALWAYS compute the sample value in that fixed timeframe.

But the cost of computing the sample will vary greatly from sample to sample depending on the exact mechanics of each module in a patch. Some samples will compute very quickly and others rather slowly, By batch processing the samples and running as fast as possible through the buffer all the spare time ends up in one "gasp for air" once the buffer is full and it doesn't matter if a minority of samples take longer than 1/48,000th of a second to compute.

So sample-time isn't real-time. But as INTERNAL MIDI signals are also computed in sample-time rather than real-time then everything lines up and there isn't any jitter.

However, EXTERNAL MIDI signals don't exist in sample-time they exist in something closer to real-time so there's potential for jitter there unless there's some timestamping of incoming and outgoing MIDI messages going on behind the scenes to compensate. I've not tested whether there is or isn't but have encountered some practical problems when MIDI syncing with external kit so I've always just assumed that there isn't invisible MIDI timestamping going on. But the problems I've encountered could be down to something else.

[Centripidity]

AFAIK ProcessSample() does not run at a regular 1/48,000 time per second rate.

I'm pretty sure that's true. When I started VM development I made the mistake of trying to time events using calls to system timing methods like System.nanoTime() and that just doesn't work because the time at which ProcessSample() is called is sometime earlier than when the sample will appear at the module's output.

What I assume happens is that calling an output object's SetValue() does nothing more than put that value into a buffer (as Colin suggests) and that a separate thread (internal to VM and running with very high accuracy @48,000 times per second) takes the next value in the buffer and passes it out of the module and on to the appropriate inputs of other modules.

[ColinP]

What I assume happens is that calling an output object's SetValue() does nothing more than put that value into a buffer (as Colin suggests) and that a separate thread (internal to VM and running with very high accuracy @48,000 times per second) takes the next value in the buffer and passes it out of the module and on to the appropriate inputs of other modules.

I doesn't need to be that complicated, each module output just has a single value buffer. This value might actually be stored inside cable or bus objects but effectively VM remembers the state of every output from the previoue sample.

So the main thread does something like this...

Code: Select all

while audio buffer is not full
    for each module in patch
        for each input in module
            inputValue = 0
            for each cable connected to input
               inputValue += value from cable (the value being the output of the previous sample calculation)
         run ProcessSample() in module
    store values of any cables attached to the audio output sockets in the audio buffer
sleep until audio driver empties the audio buffer by making a copy of it

Credit to Grant for figuring out the essence of this. There's a thread on it from a long while ago.

[utdgrant]

Credit to Grant for figuring out the essence of this. There's a thread on it from a long while ago.

That'll be this thread.