A visualiser for Sonic PI (resizable version)

I received a request to see if the visualiser I recently published could a) be used on a split screen with Sonic Pi, each taking up half the screen, and b) be used on a second monitor attached to the computer.I looked at the code, and produced this version which uses a resizable screen, which can also be dragged onto a a second screen monitor if attached to the main computer. The main change is in the initial screen setup. This is now all done in the setup function, not in the settings function which becomes redundant. Also all references to displayHeight and displayWidth are replaced by height and width respectively.

In testing the new version, I found that the star routine could causeit to crash. This was down to the way I was generating the parameters, and I have altered the star function calls so that the second radius parameter no longer uses a / when computing its value. The two calls are shown below:

First version:

star(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, circles[i]*stardata[0], circles[i]/stardata[1], int(stardata[2]+random(stardata[3])));

New resizable version:

star(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, circles[i]*stardata[0]*0.25, circles[i]*stardata[1]*0.25, int(stardata[2]+random(stardata[3])));

The new processing code is shown below:

//Visualiser for use with Sonic Pi 3 written by Robin Newman, September 2017
// based on an original sketch https://github.com/andrele/Starburst-Music-Viz
//THIS VERSION FOR RESIZEABLE SCREEN: CAN BE GRAGGED TO SECOND MONITOR
//Changes: changed to resizable screen, updated for Processing 3, added colour, added rectangle and star shapes
//added OSC input (from Sonic Pi) to alter parameters as it runs, removed slider inputs.
//input OSC:   /viz/float       updates STROKE_MAX, STROKE_MIN and audioThresh
//             /viz/pos         updates XY random offset values (can be zero)
//             /viz/col         updates RGB values
//             /viz/shapes      sets shapes to be used from S, E and R (or combinations thereof)
//             /viz/stardata    sets data for star shapes
//             /viz/rotval      turns rotation of shapes on/off
//             /viz/shift       turns XY shift across screen on/off
//             /viz/stop        initiates sending stop all signal back to Sonic Pi port 4557

import ddf.minim.analysis.FFT;
import ddf.minim.*;
import oscP5.*; //to support OSC server
import netP5.*;

Minim minim;
AudioInput input;
FFT fftLog;

int recvPort = 5000; //can change to whatever is convenient. Match with use_osc comand in Sonic Pi
OscP5 oscP5;
NetAddress myRemoteLocation; //used to send stop command b ack to Sonic PI

// Setup params
color bgColor = color(0, 0, 0);

// Modifiable parameters
float STROKE_MAX = 10;
float STROKE_MIN = 2;
float audioThresh = .9;
float[] circles = new float[29];
float DECAY_RATE = 2;
//variables for OSC input
float [] fvalues = new float[5]; //STROKE_MAX, STROKE_MIN,audioThresh values
int [] cols = new int[3]; //r,g,b colours
int [] positions = new int[2];// random offset scales for X,Y
int [] stardata = new int[4];// data for star shape, number of points, random variation
int shiftflag = 0; //flag to control xy drift across the screen set by OSC message
int two = 0; //variable to force concentric shapes when more than one is displayed
String shapes = "E"; //shapes to be displayed, including multiples from S,E,R
int rotval =0;
int xoffset = 0,yoffset = 0;
int xdirflag = 1,ydirflag = 1;

void setup() { 
 size(400, 400,P2D);
 frame.setResizable(true); 
 //noLoop();
  frameRate(60);
   myRemoteLocation = new NetAddress("127.0.0.1",4557); //address to send commands to Sonic Pi
  minim = new Minim(this);
  input = minim.getLineIn(Minim.MONO, 2048); //nb static field MONO referenced from class not instance hence Minim not minim

  fftLog = new FFT( input.bufferSize(), input.sampleRate()); //setup logarithmic fast fourier transform
  fftLog.logAverages( 22, 3); // see http://code.compartmental.net/minim/fft_method_logaverages.html

  noFill();
  ellipseMode(RADIUS); //first two coords centre,3&4 width/2 and height/2
  fvalues[0]=1.0;
  fvalues[1]=0.0;
  fvalues[2]=0.32;
  cols[0] = 255;
  cols[1]=0;
  cols[2]=150;
  positions[0] = 50;
  positions[1]=40;
  stardata[0]=2;
  stardata[1]=4;
  stardata[2]=3;
  stardata[3]=5;
  /* start oscP5, listening for incoming messages at recvPort */
  oscP5 = new OscP5(this, recvPort);
  background(0);
}

void draw() {
  background(0);
  pushMatrix();
  //calculate changing xy offsets: shiftflag set to 0 to siwtch this off
  xoffset += 10*xdirflag*shiftflag;
  yoffset += 10*ydirflag*shiftflag;
  if(shiftflag==0){xoffset=0;yoffset=0;} //reset offset values to zero if shifting is off
  //reverse directions of shifting when limits reached
  if (xoffset >width/3){xdirflag=-1;}
  if (xoffset < -width/3){xdirflag=1;} if (yoffset > height/3){ydirflag=-1;}
  if (yoffset < -height/3){ydirflag=1;}
  //transform to new shift settings
  translate(width/2+xoffset, height/2+yoffset); //half of screen width and height (ie centre) plus shift values

  //optional rotate set by OSC call
  rotate(float(rotval)*(2*PI)/360);
  
  //get limits for stroke values and audiThreshold from OSC data received
  STROKE_MIN=fvalues[0];
  STROKE_MAX=fvalues[1];
  audioThresh=fvalues[2]; 
  //println("fvalues: ",STROKE_MIN,STROKE_MAX,audioThresh); //for debugging

  // Push new audio samples to the FFT
  fftLog.forward(input.left);

  // Loop through frequencies and compute width for current shape stroke widths, and amplitude for size
  for (int i = 0; i < 29; i++) {

    // What is the average height in relation to the screen height?
    float amplitude = fftLog.getAvg(i);

    // If we hit a threshold, then set the "circle" radius to new value (originally circles, but applies to other shapes used)
    if (amplitude < audioThresh) { circles[i] = amplitude*(height/2); } else { // Otherwise, decay slowly circles[i] = max(0, min(height, circles[i]-DECAY_RATE)); } pushStyle(); // Set colour and opacity for this shape circle. (opacity depneds on amplitude) if (1>random(2)) {
      stroke(cols[0], cols[1], cols[2], amplitude*255);
    } else {
      stroke(cols[1], cols[2], cols[0], amplitude*255);
    }
    strokeWeight(map(amplitude, 0, 1, STROKE_MIN, STROKE_MAX)); //weight stroke according to amplitude value

    if (shapes.length()>1) { //if more than one shape being drawn, set two to 0 to draw them concentrically
      two = 0;
    } else {
      two = 1;
    }
    // draw current shapes
    if (shapes.contains("e")) {
      // Draw an ellipse for this frequency
      ellipse(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, 1.4*circles[i], circles[i]);
    }
    if (shapes.contains("r")) {
      rectMode(RADIUS); 
      rect( random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, 1.4*circles[i], circles[i]);
    }
    if (shapes.contains("s")) {
      strokeWeight(3); //use fixed stroke weight when drawing stars
      //star data Xcentre,Ycentre,radius1,radius2,number of points
      star(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, circles[i]*stardata[0]*0.25, circles[i]*stardata[1]*0.25, int(stardata[2]+random(stardata[3])));
    }
    popStyle();

    //System.out.println( i+" "+circles[i]); //for debugging
  } //end of for loop
  popMatrix();
}

void oscEvent(OscMessage msg) { //function to receive and parse OSC messages
  System.out.println("### got a message " + msg);
  System.out.println( msg);
  System.out.println( msg.typetag().length());

  if (msg.checkAddrPattern("/viz/float")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      fvalues[i] = msg.get(i).floatValue();
      System.out.print("float number " + i + ": " + msg.get(i).floatValue() + "\n");
    }
  }

  if (msg.checkAddrPattern("/viz/pos")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      positions[i] = msg.get(i).intValue();
      System.out.print("pos number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }

  if (msg.checkAddrPattern("/viz/col")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      cols[i] = msg.get(i).intValue();
      System.out.print("col number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }
  if (msg.checkAddrPattern("/viz/shapes")==true) {
    shapes=msg.get(0).stringValue();
    //for(int i =0; i<msg.typetag().length(); i++) {
    // shapes += msg.get(i).stringValue().toLowercase();      
    //}
    System.out.print("shapes code "+ shapes + "\n");
  }
  if (msg.checkAddrPattern("/viz/stardata")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      stardata[i] = msg.get(i).intValue();
      System.out.print("stardata number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }
  if (msg.checkAddrPattern("/viz/rotval")==true) {
    rotval =msg.get(0).intValue();
    System.out.print("rotval code "+ rotval + "\n");
  }
  if (msg.checkAddrPattern("/viz/shift")==true) {
    shiftflag =msg.get(0).intValue();
    System.out.print("shiftflag code "+ shiftflag + "\n");
  }
  if (msg.checkAddrPattern("/viz/stop")==true) {
    kill(); //stop Sonic Pi from running
  }
}

//function to draw a star (and polygons)
void star(float x, float y, float radius1, float radius2, int npoints) {
  float angle = TWO_PI / npoints;
  float halfAngle = angle/2.0;
  beginShape();
  for (float a = 0; a < TWO_PI; a += angle) {
    float sx = x + cos(a) * radius2;
    float sy = y + sin(a) * radius2;
    vertex(sx, sy);
    sx = x + cos(a+halfAngle) * radius1;
    sy = y + sin(a+halfAngle) * radius1;
    vertex(sx, sy);
  }
  endShape(CLOSE);
}

void kill(){ //function to send stop message to Sonic Pi on local machine
  OscMessage myMessage = new OscMessage("/stop-all-jobs");
   myMessage.add("RBN_GUID"); //any value here. Need guid to make Sonic PI accept command
  oscP5.send(myMessage, myRemoteLocation); 
}

A modified Sonic Pi driver program is shown below.

#Program to drive Sonic Pi 3 visualiser written in "processing"
#by Robin Newman, September 2017
#see article at https://rbnrpi.wordpress.com
#This program for resizeable version of visualiser
#set up OSC address of processing sketch
use_osc '127.0.0.1',5000
#select shapes to show
osc "/viz/shapes","se"  #"s" "e" "r" Star,Ellipse, Rectangle or combination
sleep 0.1

live_loop :c do
  #choose starting colour for shapes
  osc "/viz/col",rrand_i(0,64),rrand_i(128,255),rrand_i(0,255)
  sleep 0.1
end

live_loop :f do
  #set Stroke max min widths and audioThreshold
  osc "/viz/float",([8.0,5.0,3.0].choose),[1.0,2.0].choose,(0.4+rand(0.3))
  sleep 2
end

#set range of random positional offset (can be 0,0)
#automatically disabled when showng more than one shape
osc "/viz/pos",10,0

#control "bouncing" shapes around the screen 1 for on 0 for off
osc "/viz/shift",0

live_loop :s do
  #setup star data inner/outer circle radius, number of points
  #and random variation of number of points
  osc "/viz/stardata",[1,2,3].choose,[1,2,4].choose,5,1
  sleep 2
end

rv=0 #variable for current rotation
live_loop :r do
  rv+=5*[1,1].choose # choose rotation increment
  rv=rv%360
  osc "/viz/rotval",rv #change rv to 0 to disable rotation
  sleep 0.1
end

#Now setup the sounds to play which will trigger the visualiser
use_bpm 60
set_volume! 5
use_random_seed 999

with_fx :level do |v|
  control v,amp: 0 #control the volume using fx :level
  sleep 0.1
  
  in_thread do #this loop does the volume control
    control v,amp: 1,amp_slide: 10 #fade in
    sleep 140
    control v,amp: 0,amp_slide: 10 #fade out
    sleep 10
    osc "/viz/stop" #send /viz/stop OSC message to sketch
    #sketch sends back a /stop_all_jobs command to port 4557
  end
  
  #  This drum loop is written by Eli see https://groups.google.com/forum/#!topic/sonic-pi/u71MnHnmkVY
  #  used with his permission. I liked it, and it has good percussive output
  #  to drive a visualiser
  live_loop :drums do
    this_sample = [:loop_compus, :loop_tabla, :loop_safari].ring
    start = [ 0.0 , 0.125 , 0.25 , 0.375 , 0.5 , 0.625 , 0.75 , 0.875 ].ring
    sample this_sample.look , beat_stretch: 4, start: start.look, rate: 0.5
    sleep 1
    tick
  end
end

If you are using the original version, I suggest you alter the line that calls the star drawing routine as for the resizable version, and use the new Sonic Pi driver program with it.

You can download the new versions here

The original article shows how to set the system up.

Advertisements

A visualiser for Sonic Pi 3

Every since Sonic PI had a transparency mode added (not available on the Raspberry Pi versions) I have been interested in adding a Visualiser graphics display as a backdrop. The built in Scope can give attractive displays, but I wanted something with a bit more colour, and which covered the whole screen. I did some experiments with the iTunes visualiser which added quite a nice backdrop, but only with a significant delay between the audio and the fluctuations of the display. The recent arrival of Sonic Pi 3 allowed for further possibilities, because it enabled the use of OSC messaging. I did a trawl of the internet and came across a promising looking processing sketch which produced a pattern which reacted to incoming audio. It was written several years ago and was only monochrome based. I played around with this, upgrading it to work on the latest version of Processing, and added some colour. I experimented with adding further  basic shapes to the display (it originally used an ellipse primitive, set up to give concentric circles, which could also be driven to produce a star burst effect). I added rectangles and star shapes and experimented with off-setting these as the program ran, and also with using more than one basic shape at the same time. I then added some code so that the sketch could receive incoming OSC messages sent from Sonic Pi 3, which could be used to control various parameters for the shapes, such as stroke width, colour, and offsets on the screen. I added further flexibility such as the ability to rotate the shapes, and to shift the whole display vertically and horizontally across the screen. The final setup works well with Sonic Pi 3, which controls the patterns both with the Audio signal it produces, and also with OSC messages which can be sent to the sketch display code. These can be timed appropriately with the musical content.

The code for the sketch is shown below

//Visualiser for use with Sonic Pi 3 written by Robin Newman, September 2017
// based on an original sketch https://github.com/andrele/Starburst-Music-Viz
//Changes: changed to full screen, updated for Processing 3, added colour, added rectangle and star shapes
//added OSC input (from Sonic Pi) to alter parameters as it runs, removed slider inputs.
//input OSC:   /viz/float       updates STROKE_MAX, STROKE_MIN and audioThresh
//             /viz/pos         updates XY random offset values (can be zero)
//             /viz/col         updates RGB values
//             /viz/shapes      sets shapes to be used from S, E and R (or combinations thereof)
//             /viz/stardata    sets data for star shapes
//             /viz/rotval      turns rotation of shapes on/off
//             /viz/shift       turns XY shift across screen on/off
//             /viz/stop        initiates sending stop all signal back to Sonic Pi port 4557

import ddf.minim.analysis.FFT;
import ddf.minim.*;
import oscP5.*; //to support OSC server
import netP5.*;

Minim minim;
AudioInput input;
FFT fftLog;

int recvPort = 5000; //can change to whatever is convenient. Match with use_osc comand in Sonic Pi
OscP5 oscP5;
NetAddress myRemoteLocation; //used to send stop command b ack to Sonic PI

// Setup params
color bgColor = color(0, 0, 0);

// Modifiable parameters
float STROKE_MAX = 10;
float STROKE_MIN = 2;
float audioThresh = .9;
float[] circles = new float[29];
float DECAY_RATE = 2;
//variables for OSC input
float [] fvalues = new float[5]; //STROKE_MAX, STROKE_MIN,audioThresh values
int [] cols = new int[3]; //r,g,b colours
int [] positions = new int[2];// random offset scales for X,Y
int [] stardata = new int[4];// data for star shape, number of points, random variation
int shiftflag = 0; //flag to control xy drift across the screen set by OSC message
int two = 0; //variable to force concentric shapes when more than one is displayed
String shapes = "E"; //shapes to be displayed, including multiples from S,E,R
int rotval =0;
int xoffset = 0,yoffset = 0;
int xdirflag = 1,ydirflag = 1;

void settings() {
  fullScreen(P3D);
}

void setup() {  
  frameRate(60);
   myRemoteLocation = new NetAddress("127.0.0.1",4557); //address to send commands to Sonic Pi
  minim = new Minim(this);
  input = minim.getLineIn(Minim.MONO, 2048); //nb static field MONO referenced from class not instance hence Minim not minim

  fftLog = new FFT( input.bufferSize(), input.sampleRate()); //setup logarithmic fast fourier transform
  fftLog.logAverages( 22, 3); // see http://code.compartmental.net/minim/fft_method_logaverages.html

  noFill();
  ellipseMode(RADIUS); //first two coords centre,3&4 width/2 and height/2
  fvalues[0]=1.0;
  fvalues[1]=0.0;
  fvalues[2]=0.32;
  cols[0] = 255;
  cols[1]=0;
  cols[2]=150;
  positions[0] = 50;
  positions[1]=40;
  stardata[0]=2;
  stardata[1]=4;
  stardata[2]=3;
  stardata[3]=5;
  /* start oscP5, listening for incoming messages at recvPort */
  oscP5 = new OscP5(this, recvPort);
  background(0);
}

void draw() {
  background(0);
  pushMatrix();
  //calculate changing xy offsets: shiftflag set to 0 to siwtch this off
  xoffset += 10*xdirflag*shiftflag;
  yoffset += 10*ydirflag*shiftflag;
  if(shiftflag==0){xoffset=0;yoffset=0;} //reset offset values to zero if shifting is off
  //reverse directions of shifting when limits reached
  if (xoffset >displayWidth/3){xdirflag=-1;}
  if (xoffset < -displayWidth/3){xdirflag=1;} if (yoffset > displayHeight/3){ydirflag=-1;}
  if (yoffset < -displayHeight/3){ydirflag=1;}
  //transform to new shift settings
  translate(displayWidth/2+xoffset, displayHeight/2+yoffset); //half of screen width and height (ie centre) plus shift values

  //optional rotate set by OSC call
  rotate(float(rotval)*(2*PI)/360);
  
  //get limits for stroke values and audiThreshold from OSC data received
  STROKE_MIN=fvalues[0];
  STROKE_MAX=fvalues[1];
  audioThresh=fvalues[2]; 
  //println("fvalues: ",STROKE_MIN,STROKE_MAX,audioThresh); //for debugging

  // Push new audio samples to the FFT
  fftLog.forward(input.left);

  // Loop through frequencies and compute width for current shape stroke widths, and amplitude for size
  for (int i = 0; i < 29; i++) {

    // What is the average height in relation to the screen height?
    float amplitude = fftLog.getAvg(i);

    // If we hit a threshold, then set the "circle" radius to new value (originally circles, but applies to other shapes used)
    if (amplitude < audioThresh) { circles[i] = amplitude*(displayHeight/2); } else { // Otherwise, decay slowly circles[i] = max(0, min(displayHeight, circles[i]-DECAY_RATE)); } pushStyle(); // Set colour and opacity for this shape circle. (opacity depneds on amplitude) if (1>random(2)) {
      stroke(cols[0], cols[1], cols[2], amplitude*255);
    } else {
      stroke(cols[1], cols[2], cols[0], amplitude*255);
    }
    strokeWeight(map(amplitude, 0, 1, STROKE_MIN, STROKE_MAX)); //weight stroke according to amplitude value

    if (shapes.length()>1) { //if more than one shape being drawn, set two to 0 to draw them concentrically
      two = 0;
    } else {
      two = 1;
    }
    // draw current shapes
    if (shapes.contains("e")) {
      // Draw an ellipse for this frequency
      ellipse(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, 1.4*circles[i], circles[i]);
    }
    if (shapes.contains("r")) {
      rectMode(RADIUS); 
      rect( random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, 1.4*circles[i], circles[i]);
    }
    if (shapes.contains("s")) {
      strokeWeight(3); //use fixed stroke weight when drawing stars
      //star data Xcentre,Ycentre,radius1,radius2,number of points
      star(random(-1, 1)*amplitude*positions[0]*two, random(-1, 1)*amplitude*positions[1]*two, circles[i]*stardata[0], circles[i]/stardata[1], int(stardata[2]+random(stardata[3])));
    }
    popStyle();

    //System.out.println( i+" "+circles[i]); //for debugging
  } //end of for loop
  popMatrix();
}

void oscEvent(OscMessage msg) { //function to receive and parse OSC messages
  System.out.println("### got a message " + msg);
  System.out.println( msg);
  System.out.println( msg.typetag().length());

  if (msg.checkAddrPattern("/viz/float")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      fvalues[i] = msg.get(i).floatValue();
      System.out.print("float number " + i + ": " + msg.get(i).floatValue() + "\n");
    }
  }

  if (msg.checkAddrPattern("/viz/pos")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      positions[i] = msg.get(i).intValue();
      System.out.print("pos number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }

  if (msg.checkAddrPattern("/viz/col")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      cols[i] = msg.get(i).intValue();
      System.out.print("col number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }
  if (msg.checkAddrPattern("/viz/shapes")==true) {
    shapes=msg.get(0).stringValue();
    //for(int i =0; i<msg.typetag().length(); i++) {
    // shapes += msg.get(i).stringValue().toLowercase();      
    //}
    System.out.print("shapes code "+ shapes + "\n");
  }
  if (msg.checkAddrPattern("/viz/stardata")==true) {
    for (int i =0; i<msg.typetag().length(); i++) {
      stardata[i] = msg.get(i).intValue();
      System.out.print("stardata number " + i + ": " + msg.get(i).intValue() + "\n");
    }
  }
  if (msg.checkAddrPattern("/viz/rotval")==true) {
    rotval =msg.get(0).intValue();
    System.out.print("rotval code "+ rotval + "\n");
  }
  if (msg.checkAddrPattern("/viz/shift")==true) {
    shiftflag =msg.get(0).intValue();
    System.out.print("shiftflag code "+ shiftflag + "\n");
  }
  if (msg.checkAddrPattern("/viz/stop")==true) {
    kill(); //stop Sonic Pi from running
  }
}

//function to draw a star (and polygons)
void star(float x, float y, float radius1, float radius2, int npoints) {
  float angle = TWO_PI / npoints;
  float halfAngle = angle/2.0;
  beginShape();
  for (float a = 0; a < TWO_PI; a += angle) {
    float sx = x + cos(a) * radius2;
    float sy = y + sin(a) * radius2;
    vertex(sx, sy);
    sx = x + cos(a+halfAngle) * radius1;
    sy = y + sin(a+halfAngle) * radius1;
    vertex(sx, sy);
  }
  endShape(CLOSE);
}

void kill(){ //function to send stop message to Sonic Pi on local machine
  OscMessage myMessage = new OscMessage("/stop-all-jobs");
   myMessage.add("RBN_GUID"); //any value here. Need guid to make Sonic PI accept command
  oscP5.send(myMessage, myRemoteLocation); 
}

The basis of a visualiser depends on doing a fast fourier transform analysis of the incoming audio, and calculating amplitudes related to the different frequency components of the audio, which is continuously monitored input buffer by input buffer. I don’t intend togo anywhere near the complex mathematics involved, but there are a lot of useful articles at different levels which you can read on the subject. I quite liked this one on the Fourier Transform. https://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/. Also it is useful to look at the documentation of the minim library analysis section http://code.compartmental.net/minim/index_analysis.html if you want more detailed information on the calls employed.
You may find it easier to look at the original sketch from which I started https://github.com/andrele/Starburst-Music-Viz before taking on board the additions I have added to make the sketch more flexible and wide ranging.
I have added various transformations. Starting at the beginning of the main draw() function there are x and y offsets which increase each time the loop iterates, until a maximum offset is reached, when they reverse in direction. This causes the shapes to move smoothly left and right and up and down the screen. a shiftflag eanbles this to be siwtched on and off by one of the control OSC messages.
There follows an optional rotate command which can rotate the axis of the shapes being drawn, again controlled by an incoming OSC message.
Next values for setting the limits on the stroke size being used to render the shapes are read from data received by an OSC message, together with an audioThreshold setting.
A buffer from the input audio is now processed, and amplitude values for different frequency ranges are stored in an array of “circle” settings. NB the name circles is used for the variable as this was the only shape used in the original.Perhaps it might be better names shapes() now as there are three different classes of shapes used. A new value for the “radius” is stored if it is less than the audioThreshold setting, otherwise, a “decayed” value of the currently stored value from the previous iteration is stored. (suitable minimum values are set).
rgb colour values are set using values received from an OSC message, and these are then swapped at random, before setting the stroke colour attributes  to be used.
Next a flag two is set according to whether  one or more than one shapes have been selected. In the latter case the shapes are forced to be drawn concentrically, by nullifying the offset values by setting two = 0. The selected shapes are then drawn, before the loop starts a further iteration.

The oscEvent(OscMessage msg) function is triggered when an OSC message is received. It contains sections to parse the various OSC messages which can be sent from Sonic PI to the processing sketch. The parameters for each command are used to update various lists containing information used by the draw function.eg cols[ ] holds rgb values, startdata[ ] holds the parameters for the star shapes, fvalues[ ] holds the floating point values for the STROKE_MAX, STROKE_MIN  and audioThresh settings. These and other similar settings are updated when the relevant OSC messages are received, so Sonic PI can control the operation of the sketch as it runs.
The star functions draws the star shape. It is lifted straight from the star example here https://processing.org/examples/star.html.
The final function kill( ) is used to send a /stop-all-jobs OSC message back to Sonic PI to stop all programs running on that machine. It can be triggered by a /viz/stop OSC message being sent from Sonic PI to the Sketch.

As far as Sonic Pi is concerned, it produces two sorts of input. First the audio that it produces is fed back to the sketch which looks at the default audio input. On my Mac I used the program Loopback ( https://rogueamoeba.com/loopback/ ) to perform this. This is a paid for program, but you can use it for free for 10 minutes or so at a time.It is based on the previous free SoundFlower utility, but this has not been fully updated for recent MacOS and you may find it difficult to get it to work instead. The setup I used is shown below:Note that Sonic Pi is added as a Source and that its output is monitored through the speakers, so that the audio is fed both there and to the processing sketch. This loopbackdevice will appear in the list of devices in the MIDI audio setup program, and should be selected as the default input device,

Secondly, Sonic PI is used to send OSC messages to the processing sketch to control its various parameters. this can be done by sending “one off” messages or by putting the message sender into a live loop, sending messages at regular intervals. An example is shown below, where the OSC messages are combined with the program producing the music, but they can  be run as a separate program in a different buffer, which is an advantage if you are visualising a linear piece and do not want to restart it every time you press run to update the OSC messages sent.

#Program to drive Sonic Pi 3 visualiser written in "processing"
#by Robin Newman, September 2017
#see article at https://rbnrpi.wordpress.com
#set up OSC address of processing sketch
use_osc '127.0.0.1',5000
#select shapes to show
osc "/viz/shapes","e"  #"s" "e" "r" Star,Ellipse, Rectangle or combination
sleep 0.1

live_loop :c do
  #choose starting colour for shapes
  osc "/viz/col",rrand_i(0,64),rrand_i(128,255),rrand_i(0,255)
  sleep 0.1
end

live_loop :f do
  #set Stroke max min widths and audioThreshold
  osc "/viz/float",([8.0,5.0,3.0].choose),[1.0,2.0].choose,(0.4+rand(0.3))
  sleep 2
end

#set range of random positional offset (can be 0,0)
#automatically disabled when showng more than one shape
osc "/viz/pos",20,0

#control "bouncing" shapes around the screen 1 for on 0 for off
osc "/viz/shift",0

live_loop :s do
  #setup star data inner/outer circle radius, number of points
  #and random variation of number of points
  osc "/viz/stardata",[1,2].choose,[1,2,3].choose,5,2
  sleep 4
end

rv=0 #variable for current rotation
live_loop :r do
  rv+=5*[-8,1].choose # choose rotation increment
  rv=rv%360
  osc "/viz/rotval",rv #change rv to 0 to disable rotation
  sleep 0.1
end

#Now setup the sounds to play which will trigger the visualiser
use_bpm 60
set_volume! 5
use_random_seed 999

with_fx :level do |v|
  control v,amp: 0 #control the volume using fx :level
  sleep 0.1
  
  in_thread do #this loop does the volume control
    control v,amp: 1,amp_slide: 10 #fade in
    sleep 140
    control v,amp: 0,amp_slide: 10 #fade out
    sleep 10
    osc "/viz/stop" #send /viz/stop OSC message to sketch
    #sketch sends back a /stop_all_jobs command to port 4557
  end
  
  #This drum loop is written by Eli see https://groups.google.com/forum/#!topic/sonic-pi/u71MnHnmkVY
  #used with his permission. I liked it, and it has good percussive output
  #to drive a visualiser
  live_loop :drums do
    this_sample = [:loop_compus, :loop_tabla, :loop_safari].ring
    start = [ 0.0 , 0.125 , 0.25 , 0.375 , 0.5 , 0.625 , 0.75 , 0.875 ].ring
    sample this_sample.look , beat_stretch: 4, start: start.look, rate: 0.5
    sleep 1
    tick
  end
end

As this program runs, you can alter the parameters eg shape(s) chosen, shift parameter etc. and rerun. Effectively live coding the visualiser. The program uses a great percussion loop written by Eli, which he said I might use. https://groups.google.com/forum/#!topic/sonic-pi/u71MnHnmkVY

There is a video of the program in operation on youtube here

and you can download the code for the sketch and Sonic Pi programs here

You can download and install Processing from https://processing.org/download/

You will also have to install the libraries Minim and oscP5 from the Sketch=>Import Library… menu.
To use, set up the loopback audio and select it as the default input device. Load and play the sketch in processing, then run the Sonic Pi program on the same computer. You can adjust the transparency of the Sonic Pi screen on the Preferences Visuals tab to make it semi transparent, or minimise Sonic Pi once it is running, if you are not going to do live coding of the control OSC messages.

Sonic Pi 3 drives the LEDs on pi-topPULSE module

The new pi-topPULSE module contains a 7×7 LED array, a microphone and an audio amplifier. The latter can be used by Sonic Pi as its output audio path, and Sonic PI version 3 can also be used to control the LEDs on the PULSE module using OSC messages, which can be received by a python program which decodes them, and uses the commands to control the LEDs.

I have written a full article giving a detailed explanation of how this is done, which you can read here.

There is also a link in the article to the code which is used, and to a video of the program in action on youtube.

8 note polyphonic gated synth for Sonic Pi 3

A problem with playing midi input on Sonic Pi, is that it is difficult to use the incoming note duration directly. This is because a midi signal specifies the start of a note, but doesn’t specify its duration, instead sending another signal when the note finishes. Conversely Sonic Pi programs its internal synths to play a given pitch for a set duration, which is specified when the note starts playing. Thus, you have to compromise and set a convenient duration time for a note when it starts, which is hopefully not far off the correct duration for the note. You can mask this to a certain extent by using :release to specify the time, so that the note dies away as it is played, and also adding reverb can also help to mask the note durations. There are two alternatives. one is to process the midi input in advance, and use an algorithm to convert to to Sonic Pi notation. Such an algorithm was produced byHiroshi TACHIBANA and I wrote an article about its use  here  and have used it extensively to convert midi files for Sonic Pi use. However, this is no good for live performances, and so I have recently worked on a second method to solve the problem. That is to use a gated synth. In this case, it works by switching on a note as the midi note_on signal is received, and turns it off again as the corresponding note_off signal is received, so the note can be any duration, as specified by the length of the incoming midi note.

I had recently experimented with a “long note” synth, where I started a note of long duration (say 1000 seconds) and then switched it on and off by altering the volume of the note. You could also alter the pitch of the note and other characteristics such as the cutoff value as the note played. It looked promising so I developed this into a polyphonic system with up to 8 parallel notes playing and attempting to switch them on and off by gating the volume setting. In the event, it became rather complicated, and the processing power needed caused timing errors and program drop outs, so eventually I abandoned this in favour of a similar although simpler system. In the new system, instead of switching the note’s volume on and off, I started a new note at the beginning of each midi note_on event, and then maintained that note until the corresponding note_off event. I decided on having up to 8 possible notes playing at the same time, and devised a rather crude scheduling system which allocated a free “note generator” to each note as it was demanded, and then kept track of that note until it was finished, when the “note-generator” was again marked as available for re-use. I just had to keep track of which note was which, so that I knew which  one to release when a note_off event was received. (In fact the note_off events where just note_on events with the velocity (or volume) parameter set to zero. The timing of the loops concerned was fairly critical, and I made a lot of use of the set and get commands available in Sonic Pi 3 for recording events and values in the timing log from which they could be retrieved. The system is running close to the limit, and has several timing error warnings, but it does seem robust to keep going regardless.However one consequence is that at present it is not viable to run on a Raspberry Pi 3, although I have tested it satisfactorily on a Mac PowerBook, and on Ubuntu 17.04

The program is split into two parts, because it is too long for a single buffer. However the first part, which merely sets up state information, only needs to be run once, and then remains active until Sonic Pi is shut down. Otherwise the program is fairly easy to follow, the heart consisting of 8 identical synth engines, together with a midi input live_loop which contains the logic to allocate the synths to the received notes.
To keep the program short enough to fit the buffers, I am afraid that I have not added my usual copious comments to it.

You can download the program files here together with an xml file to generate the touch OSC file. IT would be fairly easy to substitute another means of supplying the external input (choice of synth etc) if required.

There is also a video of the project in action here

Sonic Pi 3 Synth Driver

Following on from the PS3 project, I used some of the same code, and expanded and modified it to produce a project I entitled Sonic Pi 3 Synth Driver. This incorporates the “Long Note” synth introduced in the PS3 project, which produces a continuously sounding synth note, which can have various characteristics such as synth, pitch, cutoff, volume and reverb altered. The project drives Sonic Pi3 using OSC messages from TouchOSC running on an iPad. Full details are in this article which includes links to a video and to the code used.
TouchOSC is available on the App store, and is also available for Android.

Control Sonic Pi 3 with a PS3 wireless controller

Recently I published a video on YouTube showing Sonic Pi 3 being controlled by a PS3 wireless games controller. This post contains a link to the software required to do this, should you wish to try it out.

Two files are included. The first, ps3.py should be run on a Raspberry Pi, into which the wireless dongle of the PS3 controller is plugged. Two pieces of supporting software need to be installed first.

sudo apt-get update
sudo apt-install joystick
sudo pip3 install python-osc

The second software file is a Sonic Pi 3 script, which should be loaded into a buffer on Sonic Pi 3, which can be running either on the Raspberry Pi, or on an external computer.

NB if you want to run Sonic Pi 3 on the Raspberry Pi you need to wait for the Raspbian Stretch to be released, or upgrade a new copy of Rasbian Jessie to Stretch yourself. The released version of Sonic Pi 3 will NOT run on Raspbian Jessie
This is specified on the command line when running the ps3.py script by adding an argument –sp xxx.xxx.xxx.xxx where xxx.xxx.xxx.xxx is the IP address of the computer running Sonic Pi.
In addition, on the IO preferences in Sonic Pi you must make sure that the option to receive remote OSC messages is ticked

To run the ps3.py file, make sure it is executable by typing:
chmod 755 ps3.py
then type
./ps3.py     or        ./ps3.py –sp xxx.xxx.xxx.xxx        Putting in the appropriate IP address of your remote Sonic Pi 3 computer.
As an alternative you can type python3 ps3.py
or         python3 ps3.py –sp xxx.xxx.xxx.xxx

The software can be downloaded from here