StateInfo Max Size?

[collidermod]

When using GetStateInformation and SetStateInformation, what's the max size those bytes can be? This doesn't seem to be documented anywhere.

[jgillmanjr]

Presumably the max amount of stored data would be one byte multiplied by the maximum number of elements in an array, right?

[collidermod]

@jgillmanjr It's possible to create your own stateInfo using GetStateInformation. I'm getting strange behavior when that exceeds 400-500kb, which is why I'm wondering if there's a known limit to what that can be. Hoping someone from Cherry responds...

[jgillmanjr]

Huh, interesting, now you have my curiosity as well.

[creart]

I have asked that same question to CA some time ago and got this reply from Andy:

'I'm told there shouldn't be a limit, unless it's a constraint like physical RAM '

[collidermod]

I have asked that same question to CA some time ago and got this reply from Andy:

'I'm told there shouldn't be a limit, unless it's a constraint like physical RAM '

Huh, alright, thanks. @creart what did you find was the practical limit? 400kb-ish seems to be, at least for the system I'm on, about the limit of when things will still behave 'normal'

[honki-bobo]

I haven't had any problems with data size so far. I've just tested this with 3 stereo wave files of about 2.5 MB each (about 7.5 MB in total). The get stored and restored correctly and everything behaves as it should as far as I can tell.

@collidermod Can you share some more information what you are trying to store and restore and how you do that?

[collidermod]

@honki-bobo I'm working on a sequencer that has many steps, tracks, and patterns. My first couple attempts at a whole-module preset was sized in the 3.5MB to 5MB range.

Since the original post, I've actually found some ways to get the size down. For example, an optimization is to only store steps which have data in them. Instead of having `ArrayList<Step>` with size 64, I am instead storing only non-empty steps, along with their index, which makes it easy to reconstruct, as I just fill in the ArrayList with an empty step until I come across the next non-empty step index.

I've also been experimenting with bit packing into a double. For example, with up to 64 steps, in cases where it's possible to distill a feature of a step into a boolean, this can be represented as a `boolean[64]`, which can then be bit packed into into a single java `double`. The difference between `ArrayList<Thing>` and `double` actually made about a 180kb difference on preset size in some of the testing I did.

Right now I'm hovering around 275kb - 300kb preset size, which seems reasonable given the quantity of data it's storing (and an accomplishment since I started at 3.5MB). It's a bit of a balance between preset size and processing processing time. I might still fiddle with it, but I'm happy where it's at now.

[jgillmanjr]

Glad to know the term is called bitpacking (I was thinking about using a double as a transfer medium for eight bytes of arbitrary data for something else).

[collidermod]

@jgillmanjr Yes! It totally works. Despite that CA has recommended values -5.0 to 5.0 as representing "voltage" it's actually possible to transmit all 64 bits of a double across VM jacks.

Depending on what you're doing, perhaps we could collaborate. If you're familiar with Select Bus for eurorack, I had a similar thing I had created and was considering open sourcing it. Right now, I have a working version of a module-to-module binary protocol that supports transmitting the following:

• clock

• time signature

• shuffle amount

• transport commands (play, pause, stop, etc.)

• numbered state change (ie. like MIDI program change)

• pending state changes (ie. transmitting a chain of program changes)

• pitch quantization

I had originally developed this as a way to connect my own sequencer to other modules, but realized the utility in making it an open protocol. MIDI does some of this already, but the way that MIDI forces realtime, and the weirdness of cramming in things like pending program changes and pitch quantization information made me look for another option. Email me at cv (at) collidermodular (dot) com if you want to talk about this more.

It works by packing a double to have a header, identifying the message type, then encoding the rest of the data in the remaining bits/bytes of the double.