Using Processing to control Sonic Pi

Having beeen inspired by the superb Christmas Card from MeHackit (do look at it and download and play with the code) I resolved to take another look at Processing and how it can be used to control Sonic Pi. Previously I had only used it to run a conversion script to convert MusicXML files to Sonic Pi format, but seeing this Christmas Card showed that it is capable of far more. As a newbie to using the program in earnest I looked at some of the numerous examples at and saw how easy it was to get information on mouse coordinates. I choose the example constrain which has a filled ellipse follow the mouse coordinates, but bounded by an enclosing box and decided to modifiy this to send coordinates to Sonic Pi using the sync command osc features added in Sonic PI 2.11. With reference to the MeHackit code, it was easy to add OSC commands to send the mouse x and xy coordinates to sonic pi, where they could be received and scaled to control the note pitch and cutoff values for notes played with the tb303 synth. This is just an example. You could control any parameters you wish, or add detection for mouse down as well in a more complex example. The processing script used is:

import oscP5.*; //libraries required
import netP5.*;

OscP5 oscP5;
NetAddress sonicPi;

float mx;
float my;
float easing = 1; //change to 1 to get immediate following
int radius = 24;
int edge = 100;
int inner = edge + radius;

void setup() {
size(640, 360);
oscP5 = new OscP5(this, 8000);
sonicPi = new NetAddress("",4559);

void sendOscNote(float mx,float my) {
OscMessage toSend = new OscMessage("/notesend");
toSend.add(mx); //add mx and my values as floating numbers
oscP5.send(toSend, sonicPi);
void draw() {

if (abs(mouseX - mx) > 0.1) {
mx = mx + (mouseX - mx) * easing;
if (abs(mouseY - my) > 0.1) {
my = my + (mouseY- my) * easing;

mx = constrain(mx, inner, width - inner);
my = constrain(my, inner, height - inner);
rect(edge, edge, width-edge, height-edge);
ellipse(mx, my, radius, radius);
sendOscNote(mx,my); //send the mx and my values to SP

To use it, install processing 3 from and paste the script into the sketch window which opens when you run it. Save the sketcch with a suitable name and location. YOu need to add the oscP5 library from the sketch=>import libarary…=>add library menu selection. YOu can then run the sketch and the ellispe should follow the mouse around inside its rectangle.

On the Sonic Pi side paste in the code below and run it.

use_synth :tb303
live_loop :os do
  nv=sync "/notesend"
  #puts nv #uncomment and comment next line to see OSC input
  #scale the mx and my values in nv[0] and nv[1] appropriately
  #raw mx 124(left)-516(right) and raw my 124 (top)-236(bottom
  puts (40+(nv[0]-124)/392*60).to_i.to_s+" "+(190-nv[1]/2).to_s
  play (40+(nv[0]-124)/392*60).to_i,cutoff: (190-nv[1]/2),sustain: 0.04,release: 0.01
  sleep 0.05

Run the SP program which will wait for input from teh Procdessing script. Run the Processing script and move the mouse around to alter the position of the filled ellipse (circle). Moveing it left-right will alter the pitch of the note from 40 to 100. Moving it up and down will alter the cutoff from 72 (bottom) to 128(top). Note it is possible to slice the note by moving the mouse bottom right where the pitch 100 is significantly above the cutoff value 72 so you hear nothing.

I hope that this simple example will inspre both you and I to explore further the use of Processing with Sonic Pi.

Here is a link to a video of the files in action


2 thoughts on “Using Processing to control Sonic Pi

  1. Hi,
    This looks great, but I’m having trouble making it work. When I run the Processing script, Sonic Pi throws an error:

    “Runtime Error: [buffer 3, line 7]
    Thread death +–> :live_loop_os
    undefined method `/’ for nil:NilClass”

    Does this mean something to you?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s