August 15, 2024

3DOF MIDI Gesture Controller

What the puck!

Guitar Effects Chain with an Expression Pedal

As a guitarist, I always thought more could be done with floor based controllers. In particular, the expression pedal. Expression pedal examples include effects such as the wah-wah or simple parameter controls (e.g., volume or delay).


 

They’re obviously limited to one degree of freedom (DOF). You use your foot to control the pitch of the pedal and the pitch determines the parameter value.

9DOF

What if we could introduce more degrees of freedom? Not just pitch, but also roll and yaw. We could potentially also consider movement (forward, backwards, left, right), but lets start relatively simple and stick with angles.

What would be the benefit of this? Well, the first and most obvious is the ability to control 3 parameters with one foot, which ought to lead to more control while playing live.

What’s more, is that for the experimental musician, this opens up new sound possibilities. What new sounds could be generated by changing all 3 parameters at once, or 2 of them?

In other words, the combination of pitch, yaw and roll forming a “gesture”. So we’re now talking gesture based audio control. Some example combinations are given below.


  • Volume, Distortion, Flanger
  • Phaser, Pitch Shift, Chorus
  • Reverb, Overdrive, Delay
  • Delay 1, Delay 2, Chorus
  • …etc…

From 1DOF to 3DOF

Human Ball Joint

My first thought was to use a ball joint. Ball and socket joints are a type of joint that moves throughout three or more planes of motion in multiple directions. Ball and socket joints are found in the human skeleton, for example (the shoulders and hips and control movement essential for all daily functions). Therefore, a ball joint ought to be able to support pitch, yaw and roll. 3 degrees of freedom as opposed to 1.

From Pedal to Puck

After deciding on a ball joint, it occurred to me that the controller I was envisaging could be controlled not just by foot – but also by hand. It’s not just for guitarists. It could be used desk based in the studio during recording, or live as a DJ’s turntable, or for some completely different purpose not related to music (a camera controller, stage lighting…). An expression pedal “holds position” as opposed to, say, a game controller that always “returns to center” when pressure is removed. So all things considered, the design shifted to a puck shape, that would support hand or foot control.

Why MIDI?

Regardless of the above, I still wanted to stick with music for now. So a decision was made to go with MIDI (Musical Instrument Digital Interface). It’s a way to connect devices that make and control sound – such as synthesizers, samplers, and computers – so that they can communicate with each other.

In this instance, the puck simply needs to connect to a PC as a USB MIDI Controller. Thereafter, everything else comes down to software. Whether you’re using guitar effects software, synthesizers, DJ software or other, you’ll typically find they all support MIDI and allow you to assign controls. This avoids the need to get involved with making actual effects (whether digital or analogue), since what happens to the audio is effectively offloaded. The puck itself does not need to receive or pass on the audio, it just needs to issue MIDI messages.

Lastly, MIDI is a protocol. As such, we know in the future we could swap it out for something else. For example, the puck could present itself as a HID (human interface device) much like a keyboard or mouse. Or, the puck could act as a serial device and simply pass angular data to the PC. MIDI could be swapped out for OSC (Open Sound Control), and so on.

  • Studio: Bass, Mid, Treble
  • DJ: Track 1 Volume, Track 2 Volume, Track 1/2 Mixing
  • Gamer: Movement, Strafing, Heading
  • …etc…

Hardware, Software & Implementation

Puck

The implementation is pretty simple!

An RP2040 microcontroller connected to a 6 Axis AHRS IMU Sensor. This is held in a case, which was 3D printed, holding the ball, and connecting to a PC via USB.

The software simply reads the values from the sensor, transposes them to be within acceptable MIDI ranges, and broadcasts the result as a MIDI CC message.

Lastly, a button was added for calibrating or, more specifically, recentering yaw (If the pucks base were to be rotated, for example, during use). There is a “this is forwards” indicator on the case, but a button is still useful.