test OSC controller for Sonic Pi

I haven’t written anything for some time on this site. Instead I have ben putting material onto the in-thread.sonic-pi.net site. However, recently I was asked for help on that site from someone wanting to control various aspects of a playing sample using OSC messages, and because this requires familiarity with a number of techniques I thought it would make a good topic for an article. I have written about OSC control before, but the program I will describe incorporates some useful techniques which come together to give an efficient and responsive control setup, which can hopefully be helpful to others developing particular setups.
For an OSC source, I have chosen to use TouchOSC which can run on either IOS or Android based phones or tablets. I can highly recommend it as a very flexible yet powerful controller device, with a very modest price, which is well worth purchasing.

The brief I was given asked for the ability to control the volume, fx, and pan of a playing sample, and also for the ability to change the sample playing, all by means of OSC inputs.
I designed the TouchOSC screen shown below on my iPhone, which you can click for a larger version.
This has three sliders on it labelled Volume, Reverb and Pan, which will be used to control the playing sample. Below that there are four linked toggle switches, set up so that only one can be active at a time. These are used to select one of four looping samples. The selected sample will play continuously (looping around) until another sample is selected, and this sample will take over the next time the previously selected one finishes playing. Underneath the switches are four green LEDs, which indicate the currently playing sample. Thus you can select the next sample to start playing whilst another one is playing, and the green led will then switch as the new sample starts playing. The other slider controls can be altered at anytime and will affect the currently playing (and subsequently selected) sample’s sound, in real time.

One important setup to do on Sonic Pi is to switch off enforce timing guarantees in the Audio Tab of the Preferences Screen. This is so that Sonic Pi can cope with the very large number of OSC messages received while the sliders are being altered. It doesn’t matter if it can’t quite keep up, and removing this guarantee allows it some latitude with no noticeable effect on the overall performance. So much for the description of what the test program will do. Let us look first at the OSC messages that TouchOSC will send when a control is altered.

The three sliders will send the OSC messages
/test/slider/vol with variable data 0->1 as the Volume slider is moved.
/test/slider/reverb with variable data 0->1 as the Reverb slider is moved.
/test/slider/pan with variable data -1->1 as the Pan slider is moved. (0 when centred)

The four switches will send OSC messages
/test/sample/n/1 with data 0 if the switch is off and 1 if it is on, where n is 1,2,3 or 4 depending on which switch is pressed. I fact, because the switches are set in exclusive mode on Touch OSC, two switches will respond each time a new switch is pushed. The old switch which is turning off sends data 0 to signify off, and the newly selected switch sends 1 to show it has been switched on. So if for example the 1st switch is currently active and the 4th switch is pushed the messages will be:
/test/sample/1/1, [0] and /test/sample/4/1,[1]

There is one further point to note. When Sonic Pi receives the OSC messages it prepends /osc BEFORE each message, so that the cue system can differentiate message source between incoming OSC and incoming midi which have /midi prepended.
This means that in Sonic Pi you have to look for the message
/osc/test/slider/pan for example, rather than /test/slider/pan
Also if you are using the very latest 3.2dev Sonic Pi then it is actually a more extensive change that takes place, and instead of /osc being prepended you get something like
/osc:192.168.1.218:9000 In other words you also get details of the IP address and destination port used by the originating TouchOSC. You are unlikely to need this information and you can ignore it by looking for the message
/osc*/test/slider/pan for example. It is a good idea to use this /osc* always instead of /osc
because your code will then work with version 3.1 AND version 3.2dev and later.
While we are on the differences between the two versions, another change which affects the TouchOSC end, is that Sonic Pi version 3.1 looks for incoming OSC messages on port 4559, whereas version 3.2dev switches to port 4560 for incoming OSC messages. This is because port 4559 was in fact internationally allocated for a different use, whereas port 4560 was free to use.

So now we can consider how Sonic Pi is set up to receive and utilise these OSC messages to control the playing sample in the way that we want.
We could have a separate live loop to deal with each slider to extract the information relating to the slider position, but it is more efficient if we can just use one, and decide in that loop which of the three sliders has in fact been moved.
Similarly, it is more efficient if we can just use one live loop to extract the information as to which of the four switches is currently on, and use that to preselect the next sample to be played.  Both of these aspirations are achieved by means if using wild cards in the pattern to be matched by an incoming OSC message, and when that pattern is matched, by using a function I have named parse_osc to determine what was actually matched in the incoming message.
We will also need a live_loop to control the playing of the current sample, and we want to be able to alter the characteristics such as volume and pan position as it plays.
Finally we want to be able to apply an fx around the playing sample (in this case to add reverb, although it could be something else), and we want to be able to adjust the level of this effect (in this case by altering the room: parameter) as the sample is playing and for the changes in the effect to apply.
These features raise two problems. First, how can be change parameters of a playing sample or of an effect that has been applied, after is has started? The answer, which is detailed in section 7 of the built in Sonic Pi Tutorial is to use the control command. This uses a reference to the playing sample or to the fx which is created when it starts playing and when the fx is started. The second problem is how can we utilise this reference when the values we want to change are obtained in a different live loop which doesn’t have direct access to the required references. The answer is to store the reference in the time state system using the set command, when the references are created, and to retrieve them in the separate live_loops which will be used to alter the parameters.

Well, after that lot your head is probably swimming a bit, as there is a lot to take in in what has been a fairly abstract setting. Let’s have a look at the program and relate these task to the various sections.

#testOscController.rb
#test OSC controller to adjust amp/reverb/pan/samplename
#written by Robin Newman, October 2019 to illustrate techniques
#
# NB switch OFF Enfore timing guarantees in the prefs Audio tab for this program to work
# It processes many OSC messages from slider inputs and needs some leeway to process them
#
#uses TouchOSC as input device, but should be modifiable for other OSC sources
#makes use of function parse_osc. This uses an undocumented internal function
#of Sonic Pi _event which returns full information on a given event
#Sonic Pi accepts wild cards when matching triggers from OSC messages.
#parse_osc lets you input the wild card string used, and from that to
#determine the actual OSC message which triggered the match.
#So using this you can use one live_loop to parse and deal with several similar
#events, such as those from sliders (three used here)
#or multiple switches (1 4 way togggle switch used here)
#I include a verbose version of parse_osc which prints out the data it is
#working with so you can follow the process,
#which involves Ruby string handling methods
#simply switch over which parse_osc version is uncommented to change them over
use_debug false
use_osc_logging false
use_cue_logging false
use_osc "192.168.1.218",9000 #set ip and port of TouchOSC to receive messages

#comment out ONE of the two parse_osc definitions, quiet or verbose
define :parse_osc do | path | #quiet version
  v = get_event(path).to_s.split(",")[6]
  if v != nil
    return v[3..-2].split("/")
  else
    return ["Could not decipher osc path..."]
  end
end

##| define :parse_osc do |path| #verbose version
##|   puts "op1: #{get_event(path)}"
##|   puts "op2: #{get_event(path).to_s.split(",")}"
##|   v= get_event(path).to_s.split(",")[6]
##|   if v != nil
##|     puts "op3: #{v}"
##|     puts "op4: #{v[3..-2].split("/")}"
##|     return v[3..-2].split("/")
##|   else
##|     return ["error"]
##|   end
##| end

#function switches leds on TouchOSC on/off
define :ledSwitch do |n|
  in_thread do #run in a thread so as not to impact timings
    #leds named from 1-4, but indices go 0-3 so add 1 to x and n
    4.times do |x| #switch all leds off
      osc "/test/led/"+(x+1).to_s,0
    end
    sleep 0.05 #makes sure touchOSC has time to respond
    osc "/test/led/"+(n+1).to_s,1 #switch selected led on
  end
end

#setup list of samples to use
slist=[:loop_amen,:loop_weirdo,:loop_electric,:loop_mika]
#set starting values
set :scurrent,:loop_amen #initial sample
set :sIndex,0 #index of initial sample in list
set :rv,0 #initial reverb room: size
set :pan,0 #initial pan: setting
set :vol,0.5 #initial amp: setting

#send settings to sliders and switch to initialise

osc "/test/slider/vol",0.5
osc "/test/slider/pan",0
osc "/test/slider/reverb",0
osc "/test/sample/1/1",1 #switch in vertical row 1, position 1 set to on

#start fx reverb, wrapped round all sound generation
with_fx :reverb,room: get(:rv),mix: 0.7 do |r|
  #save pointer to fx reverb in :r
  set :r,r
  
  #loop to select current sample. (changes at end of current sample)
  live_loop :sampleControl do
    use_real_time
    data = sync "/osc*/test/sample/*/*" #respond to any switch
    if data==[1.0] #just get data if pushed ie 1 #filter out all but switch "on"
      sn = parse_osc"/osc*/test/sample/*/*" #get actual switch pushed
      puts sn #show response from osc_parse
      #now choose 4th element and convert to integer. Subtract 1 to index from 0
      sn=sn[3].to_i - 1
      set :sIndex,sn #store sample index
      set :scurrent,slist[sn] #store current sample name
      puts get(:scurrent) #print current sample name selected
    end
  end
  
  #loop to get and set slider data
  live_loop :sliders do
    use_real_time
    data = sync "/osc*/test/slider/*" #respond to all sliders
    slider=parse_osc  "/osc*/test/slider/*"
    puts slider #show response from parse_osc
    slider=slider[3]
    case slider
    when "vol"
      set :vol,data[0]
      puts get(:vol)
      control get(:sp),amp: 2 * get(:vol)
    when "reverb"
      set :rv,data[0]
      puts get(:rv)
      control get(:r),room: get(:rv)
    when "pan"
      set :pan,data[0]
      puts get(:pan)
      control get(:sp),pan: get(:pan)
    end
  end
  
  live_loop :playSample do
    #here I have used beat_stretch: opt to set all samples to 2 beats
    #however will work with any sample length
    sp=sample get(:scurrent),amp: 2 * get(:vol),pan: get(:pan),beat_stretch: 2
    set :sp,sp
    ledSwitch get(:sIndex) #switch led as sample starts playing
    #change following line to sample_duration get(:scurrent)
    #if not using beat_stretch
    sleep 2#sample_duration get(:scurrent)
  end
end #reverb

The program starts with some commented lines which says a little bit about the parse_osc function which is used to retrieve information about the last OSC messages that triggered a sync to match a given osc pattern containing wild card * characters. This function which I have written utilises an undocumented internal function within Sonic Pi called get_event. This essentially produces very full details of a given event (identified by the supplied osc message string). Crucially it contains the missing information that we need which is the actual information that matched the wild cards in our string. What the function does is to apply some Ruby string handling jiggery pokery to extract the information in the form of a list of text strings. From this list we can get the information we require. I have included two versions of the function. The normal one, and a verbose copy which prints out the information at various stages as it is processed, which hopefully will help you to understand how it works.

I then switch off most cue logging, to reduce a huge stream of messages on the log screen., and specify the IP address and import port on which OSC messages can be sent back to TouchOSC to control the green lLEDs, and to initialise the sliders and switches when the program first starts running.

There follows the two versions of the parse_osc function with the verbose one commented out.
A second function :ledSwitch is then defined. This is used to select one of the green LEDs. Essentially only one of them should be on at a time, so it works by switching them all off, using an osc call of the form “/test/led/2″,0 and then switching on the one specified by the input parameter n. Because this will be run from within the sample playing loop, the body of the function is run is a thread, so that it doesn’t alter the timing of the sample loop at all. There is a small delay of 0.05 incorporated in the function which would otherwise cause an incorrect duration of the sample loop.
A list of the four sample loops I used is then setup, and starting values are set up for various values started in the time state using the set function.
Four further osc messages are sent to TouchOSC to initialise the sliders and the sample switch. Because the switches are set to exclusive mode in TouchOSC we don’t have to switch off the ones not currently in use. That is handled automatically by TouchOSC.

Now we start the fx :reverb. Note how we get a reference to it by adding do |r| at the end of the line. It is immediately stored in the time state using set :r,r so we can retrieve it elsewhere to alter the :room parameter.
The remaining three live loops reside inside the fx effect scope.

The first of these is live_loop :sampleControl
This is used to select the next sample to be played continuously until a fresh selection is made. It has use_real_time set to give a rapid response time*/test/sample/*/*” to match incoming OSC messages. When a match is synced, the associated data is stored in the list data. First we check to see if data=[1.0] ie if the message has come from a switch turning on. We are not interested in any switches which have just turned off. The parse_osc function is used to determine which switch was pushed, by extracting the data which matched the wild cards signified by the three * in the string. The first one allows for extra bits added by Sonic Pi 3.2 (more detail in the explanation of the next live_loop below), and the second and third relate to the position of the switch in our switch array. These will hav the form /1/1   /2/1   /3/1 or   /4/1 relating to the four switches in use. The are effectively the x,y coordinates of the switches in our array which only has one horizontal row in this case and there for all vertical coordinates are 1. The parse_osc function will return lists like
[“osc”, “test”, “sample”, “2”, “1”]  )for Sonic Pi 3.1 (here switch 2 responded, or
[“osc:192.168.1.218:9000”, “test”, “sample”, “3”, “1”] for Sonic Pi 3.2dev (here switch 3 responded)
We can extract the switch number from the 4th item in this list and then convert it to an integer. As the sample list we use is indexed from 0 we subtract 1 from the result to get our sample index. sn=sn[3].to_i – 1 which is then stored in the time state using set :sIndex,sn
As a bit of overkill I also stored the sample name in the time state using|
set :scurrent,slist[sn] and also printed it on the log screen.

The second of these is live_loop :sliders  As the name implies, this retrieves information whenever any of the three sliders is altered. To get an immediate response it is run with use_real_time.
It waits until an incming OSC message matches the string “/osc*/test/slider/*” The two * are wild cards which match one any additional characters in the relevant section. The * after osc allows for version 3.2dev info previously discussed, and the * after slider/* will match vol, reverb or pan. To find out which caused the trigger we use the _osc function. This returns a list of the form [“osc”, “test”, “slider”, “vol”] for Sonic Pi 3.1 or
[“osc:192.168.1.218:9000”, “test”, “slider”, “vol”] for Sonic Pi 3.2dev
From the 4th item in this list we can retrieve “vol” to show that in this case it was the vol slider that caused the match. Similarly for “reverb’ or “pan” The data variable will contain the value of teh slider in a 1 element list eg [
0.517403244972229]
A case statement is used to process each of the possibilities, with the relevant value being stored in the time state eg set :vol,data[0] or set :pan,data[0] or set :rv,data[0]
We can immediately retrieve this value and use it to control the relevant parameter for thesample which is playing. We will see shortly that this has a reference stored in :sp in the time state and we simply using a command like control get(:sp),amp: get(:vol) or(:pan)
or for the reverb we use the reference :r we stored above and use
control get(:r), room: get(:rv)
So this slider loop controls all the parameters affected by the three sliders.

The third and final live_loop  :playSample plays our current sample continously._stretch: parameter to set all the sample loop durations to 2 beats, but this was just a mater of taste. There is no reason why you cant use sample of different lengths with this program and let each one take as long as it needs. To do this omit the beat_stretch: 2  and later on in the loop change the sleep time from sleep 2 to sleep sample_duration get(:scurrent) and the loop will sleep for the duration of the currently playing sample.
We start the sample playing, but ensure that we get a reference to it, so that we can change the parameters as it plays by using:
sp=sample get(:scurrent),amp: 2 * get(:vol),pan: get(:pan), beat_stretch: 2
and immediately saving the reference to the time state using set :sp,sp so that it can be used from the sliders loop.
I then call the ledSwitch function with the current get(:sIndex) so that updates as soon as a new sample starts playing. Finally the loop sleeps the appropriate time as discussed above.
The final end finishes the scope of the fx reverb

I hope that this detailed explanation gives an insight into how the program operates, and also shows how powerful OSC messaging can be in controlling Sonic Pi, also shown is the power of the time state system, and its use in allowing different parts of the program to link together.
Similar programs can be written using midi functions to give similar control.

The TouchOSC file

Here are some photos of the TouchScreen editor used to create the template I used. Each has a different element highlighted so that you can see the data associated in the panel to the left of the image. You can double click each image to expand it.

Both the program file and the TouchScreen template can be downloaded from this link

The TouchSCreen template should be downloaded you should then compress (zip) it and rename the resulting file testcontrol.touchosc This file should be opened in your TouchScreen editor and then synced to your phone or tablet in the usual TouchScreen way. You should enable OSC and adjust the host address to match that of the computer running your Sonic Pi copy. Set the outgoing part to 4559 for Sonic Pi 3.1 and 4560 for Sonic Pi 3.2dev or later. Set the incoming port to 9000, and note the ip address of your tablet or phone shown below. You should adjust the use_osc line in the program to match this information. In my case with my iPhone it was ip 192,1681.218 and port 9000.

A video of the program in action is available here