pi-topPULSE controlled by Sonic Pi 3

Recently I purchased a pi-topPULSE module to add to my existing pi-topCEED. You can see the white module plugged into my pi-topCEED in the photo above. I purchased this principally to gain a very convenient built in amplifier and speaker to use with Sonic Pi, without the need for any external cables speakers, amplifiers etc. The volume output is more than adequate for one or two people to listen, although it is only mono and not stereo. If you follow the install procedure at https://github.com/pi-top/pi-topPULSE it switches the audio output automatically to use the built in speaker, when Sonic Pi is launched.
However the pi-topPULSE is much more than just a speaker. IT also contains a microphone, and a grid of 49 LEDs arranged in 7 rows of 7 LEDs. These are RGB LEDs which can be set to any colour and brightness, and which are controlled with the help of a python script.

I had recently manually updated my Raspbian Jessie distribution to the forthcoming Raspbian Stretch, and also installed Sonic Pi 3 onto that, and I thought that it wouod be an ideal opportunity to see If I could control the PULSE LEDs from Sonic Pi, utilising some of the new IO (Input/Output) features which are added with this new release. In particular it can send and receive OSC (Open Sound Control) messages. This is a format which is used to communicate information between widely disparate devices, not necessarily musical, although it can be used with many such. Fortunately python supports OSC messaging if you load an appropriate library, and I decided to use the python-osc library.
This can be installed onto your Rasberry PI using the command:

sudo pip3 install python-osc

With this preparation, you have all that you need to run the two programs needed for the project.

First I had to understand the commands to drive the PULSE LEDs. I tried out the impressive led-demo.py script supplied with the software, which certainly put the LEDs through their paces, but I found it more useful to look at the script ledmatrix.py in the ptpulse library. The last section of this file details the external commands to use in controlling the LEDs. I won;t go through all of these here, especially as I only used a few of them in the demonstration, but their names, syntax and actions are all detailed there.

I used five separate commands from the ptpulse ledmatrix  library
ledmatrix.setpixel(x,y,r,g,b) which sets the display map for the pixel at x,y to have the colour values r,g,b where x is the horizontal position from 0->7 and y is the vertical postion from 0-> 7
ledmatrix.setall(r,g,b) which sets the display buffer entry for all leds to the colour r,g,b
ledmatrix.show() which updates the LEDs with the current information held in the display buffer
ledmatrix.off() which clears the display buffer and immediately updates the LEDs (to off)
ledmatrix.brightness(b) where b is the brightness value (0->1). This applies to subsequent changes to the display buffer and LEDs.

I accessed these commands remotely by using an oscserver, which received and decoded a variety of OSC messages, and then called combinations of these four commands, as appropriate to control the PULSE LEDs. The listing of the server is shown below.

#oscserver to communicate with Sonic Pi
#messages received control the leds on an attached pi-topPULSE
#run with:    python3 oscserver.py
#or to use with a remote Sonic Pi computer use ip address of THIS PI
#instead of "localhost" 127.0.0.1   eg:
#python3 oscserver.py --ip 

import argparse
from time import sleep

from ptpulse import ledmatrix

from pythonosc import dispatcher
from pythonosc import osc_server

delta = 0.04 #timing  correction for RPi3 to sync leds and sound with PULSE

def handle_led(unused_addr,args, x,y,r,g,b):
  sleep(delta)
  print("Set Pixel",x,y,"to",r,g,b)
  ledmatrix.set_pixel(x,y,r,g,b)
  ledmatrix.show()

def handle_clear(unused_addr):
  sleep(delta)
  print("Clearing Leds")
  ledmatrix.off()

def handle_setall(unused_addr,args, r,g,b):
  sleep(delta)
  print("Set All Leds",r,g,b)
  ledmatrix.set_all(r,g,b)
  ledmatrix.show()

def handle_bright(unused_addr,args, b):
  sleep(delta)
  print("Set Brightness %4.2f" % (b)) #adjust to 2 decimal places
  ledmatrix.brightness(b)


if __name__ == "__main__":
  parser = argparse.ArgumentParser()
  parser.add_argument("--ip",
      default="127.0.0.1", help="The ip to listen on")
  parser.add_argument("--port",
      type=int, default=8000, help="The port to listen on")
  args = parser.parse_args()

  dispatcher = dispatcher.Dispatcher()
  dispatcher.map("/pulse/xyrgb",handle_led,"x","y","r","g","b")
  dispatcher.map("/pulse/clear",handle_clear)
  dispatcher.map("/pulse/setall",handle_setall,"r","g","b")
  dispatcher.map("/pulse/bright",handle_bright,"b")
  

  server = osc_server.ThreadingOSCUDPServer(
      (args.ip, args.port), dispatcher)
  print("Serving on {}".format(server.server_address))
  server.serve_forever()

The script is based on a simple_server script bundled as an example with the python-osc library. The server is set up at the end of the script. It waits for incoming OSC messages which arrive at the specified port (8000 by default). The server listens either on 127.0.0.1 (localhost) by default, in which case it only responds to messages which originate on the local Raspberry Pi running the script, or, if you specify the IP address of the local Pi (assuming it is connected to a network) using the –ip argument as in the example at the top of the script, then it will also respond to OSC messages received on port 8000 from ANY network connected source.
OSC messages consist of two parts, An address which is of the form “/myosc/address/is_this” and a following comma separated list of data items. The incoming OSC messages are parsed, and a dispatcher links any messages whose addresses match one of the four specified, “/pulse/sxrgb”, “/pulse/clear”, “/pulse/setall” and “/pusle/bright” to a corresponding handler function handle_led, handle_clearhandle_setall and handle_bright which are detailed in the first part of the script. Notice the way the dispatcher handles listing the parameters inside speech marks. These parameters MUST match exactly those which are sent, and are passed on to the handler functions.

The handler functions retrieve the arguments, which are then used to print a message on the terminal running the server, and then to call the relevant ledmatrix commands previously discussed to activate the LEDs. Again notice the rather unusual way in which the arguments are listed in the fiunction calls, eg def handle_led(unused_addr,args, x,y,r,g,b): the first unused_addr merely consists of the OSC address which matched the call to the handler. As such, we already know what that is and so it is unused and ignored. The remaining arguments need to be preceded by the entry args, I must confess to not being quite sure why (I’m no python guru!) , but it is necessary for it work.
The beginning of the script loads in the various python libraries necessary. The sleep(delta) is a fine tuning to synchronise the LED changes with the sound from Sonic Pi. It allows for latency in the sound channel, as the LEDs can change slightly ahead of hearing the sound.

With the oscserver,py script started (python3 oscserver.py to run everything on the local Pi) you now run the Sonic Pi program from a Sonic Pi buffer. The program can be loaded using the LOAD Button on Sonic Pi, or it can be pasted into an empty buffer. It is listed below:

#Sonic Pi3 PULSE-leds workout by Robin Newman, August 2017
#requires the python script osc-server.py to be running to communicate with PULSE-leds
#This requires the python library python-osc to be loaded
#Which can be done with the command: sudo pip3 install python-osc
#Then run the server with python3 oscserver.py
#Finally start this program running in Sonic Pi3

#Each vertical column of LEDS corresponds to notes with a different synth
#The higher the pitch of the note, the higher up the column is the illuminated LED
#Every so often the LEDS are "cleared" to a different background colour
#The LEDs are controlled by the oscserver program, which receives OSC commands
#sent from Sonic Pi to tell it what to do.
#There are four such commands
#"/pulse/clear" which clear leds to current RGB colour
#"/pulse/xyrgb" which sends x,y and r,g,b values
#"/pulse/setall" which sets all LEDs to the r,g,b values specified
#"/pulse/bright" which sends the brightness value b to be used.
#All of these are sent to "localhost" at port 8000 which is where the python oscserver is listening
use_osc_logging true #show osc messages in the log
use_random_seed 2017  #change for different patterns
use_osc "localhost",8000
#use_real_time

osc "/pulse/clear"
sleep 0.004 #make sure LEDs cleared
load_sample :drum_bass_hard
load_sample :drum_snare_hard
load_sample :drum_cymbal_closed

set :stopflag, false
at [0,20,40,60],[1.0,0.5,0.25,0.125] do |delay|
  set :t,delay
end
at 85 do
  set :stopflag, true
end

live_loop :pulse do
  #use_real_time
  r=rrand_i(40,255)
  g=rrand_i(40,255)
  b=rrand_i(40,255)
  a=(r+g+b)/200 - 1 #nb integer division here
  bright=[0.6,0.8,1][a]
  osc "/pulse/bright",bright
  sleep 0.004 #make sure brightness sent first
  x=rand_i(7);y=rand_i(7)
  use_synth [:saw,:tri,:mod_pulse,:piano,:prophet,:pulse,:pluck][x]
  tick
  play 48+y*2,amp: (a+1).to_f/3,release: get(:t) if look%20!=0 or look == 0
  osc "/pulse/xyrgb",x,y,r,g,b if look%20!=0 or look == 0
  osc "/pulse/setall",r,g,b if look%20==0  and look > 0
  sleep get(:t)-0.004 #compensate for brighness delay
  osc "/pulse/clear" if look%80 == 0 and look > 0
  if get(:stopflag) == true
    osc "/pulse/clear"
    cue :endphase #cue lastnotes and LED phase
    stop #this live_loop
  end
end

set :bass_rhythm, ring(9, 0, 9, 0,  0, 0, 0, 0,  9, 0, 0, 3,  0, 0, 0, 0)
set :snare_rhythm, ring(0, 0, 0, 0,  9, 0, 0, 2,  0, 1, 0, 0,  9, 0, 0, 1)
set :hat_rhythm, ring(5, 0, 5, 0,  5, 0, 5, 0,  5, 0, 5, 0,  5, 0, 5, 0)

live_loop :drums, auto_sync: :pulse do
  #use_real_time
  sf = 0.05
  sample :drum_bass_hard, amp: sf*get(:bass_rhythm).tick
  sample :drum_snare_hard, amp: sf*get(:snare_rhythm).look
  sample :drum_cymbal_closed,amp: sf*get(:hat_rhythm).look
  tval= get(:t)

  #puts tval
  if tval <0.3
    sleep tval
  elsif tval <= 0.5
    sleep tval/2
  else
    sleep tval/4
  end
  stop if get(:stopflag) == true #this liveloop
end


live_loop :lastnotes ,sync: :endphase do
  #use_real_time
  use_synth :pulse
  n=scale(:c4,:minor_pentatonic,num_octaves:3).tick
  if look<32
    play n,release: 0.1
    sleep 0.1
  else
    play [72,84,89,96],sustain: 1.8,release:0.1
    sleep 1.9
  end
  stop if look==32 #this live look
end

#This section fades the LEDs up and down several times before
#leaving them all white for the final chord
#They fade out and are blanked when the music ends

sync :endphase

osc "/pulse/bright",1 #set full brightness
osc"/pulse/clear"
10.times do |x|
  v=(255*x/10).to_i
  osc "/pulse/setall",0,v,0
  sleep 0.05
end
2.times do
  10.times do |x|
    v=(255*(x+1)/10).to_i
      osc "/pulse/setall",v,255-v,v
      sleep 0.05
    end
    10.times do |x|
      v=(255*(9-x)/10).to_i
      osc "/pulse/setall",v,255-v,v
      sleep 0.05
    end
  end
  10.times do |x|
    v=(255*x/10).to_i
     osc "/pulse/setall",0,255-v,0
    sleep 0.05
  end

osc "/pulse/setall",255,255,255 #set all white
sleep 2
100.times do |x|
osc "/pulse/bright",(99-x).to_f/100
sleep 0.01
osc "/pulse/setall",255,255,255
sleep 0.01
end

osc "/pulse/clear" #clear all LEDs
puts "FINISHED!"

If you are unfamiliar with Sonic Pi then this may appear a bit daunting, however if you look at it piece by piece it is not too hard to follow. Initially there are a load of comment lines which give some explanation of the program operation.
use_osc_logging true makes Sonic Pi print all of the OSC messages sent on the log screen so that you can see what is happening. You can turn it off by changing true to false.
There are a number of #use_real_time commands which are all commented out. These are often used when you wish to synchronise for examples external instrument to Sonic PI with the minimum delay. In this case they are not necessary, as the time at which the program produces output is not important, as long as that output is synchronised to the sounds produced. Initially I experimented with using them, but it does tax the Raspberry Pi3 quite a bit, and so I discarded them.
The use_random_seed 2017 command lets you change the sequence of LEDs/sounds activated by the program. They are deterministic in that the same number (here 2017) will always give exactly the same sequence, but different numbers will give different sequences.
use_osc “localhost”,8000 tells Sonic Pi where to send the OSC messages. In this case to the local machine (address 127.0.0.1) at port 8000. If you want to use the program from a remote machine running Sonic PI 3, then localhost should be changed to the actual IP address of the pi-topCEED with the connected PULSE module. Also the oscserver on that machine should be started with the alternative command python3 oscserver,py –ip <ip.address.of.ceedPi> so that it can listen to incoming OSC messages from the intranet.
The first osc messages is sent to clear the LEDs on the PULSE, using osc “/pulse/clear”, followed by a very small sleep value to make sure it has completed before the next osc command arrives. then I load in the three samples used in the drums live_loop. This is probably unnecessary without the use_real_time command, but it obviates any problems with timing if they are preloaded. In fact with the use_real_time commands active I also added a sleep 2 here, now removed.
I reproduce the next section below:

set :stopflag, false
at [0,20,40,60],[1.0,0.5,0.25,0.125] do |delay|
  set :t,delay
end
at 85 do
  set :stopflag, true
end

This may be unfamiliar even to those who have used Sonic Pi before. A new feature in Sonic Pi 3 lets you store values in the event timeline using a set command. Such values can be retrieved later using a get command. This is the preferred technique to use when communicating between live_loops and the main linear part of the program. Here I store a setting :stopflag with the value false. Later in the program it has its value changed to true. The at command is very useful for triggering events which you want to happen at different times in the program. I use it here to alter the value of :t another value stored in the time event system. It is altered by the at command at times 0,20,40 and 60 to have values 1.0,0.5,0.25 and 0.125 It is used to alter the speed at which the sounds and LEDs change as the program progresses. A separate at command changes the value of :stopflag after 85 seconds and initiates the endphase of the program.
The meat of the program consists of three live_loops. These are sections of code which run repeatedly in a loop. Potentially they run concurrently, although the third loop, :lastnotes is delayed from starting until it receives a cue :endphase which is sent by the first live_loop :pulse just before it is stopped following a change in the :stopflag value triggered by the at command at 85 seconds  as discussed previously. The first loop is shown below:

live_loop :pulse do
  #use_real_time
  r=rrand_i(40,255)
  g=rrand_i(40,255)
  b=rrand_i(40,255)
  a=(r+g+b)/200 - 1 #nb integer division here
  bright=[0.6,0.8,1][a]
  osc "/pulse/bright",bright
  sleep 0.004 #make sure brightness sent first
  x=rand_i(7);y=rand_i(7)
  use_synth [:saw,:tri,:mod_pulse,:piano,:prophet,:pulse,:pluck][x]
  tick
  play 48+y*2,amp: (a+1).to_f/3,release: get(:t) if look%20!=0 or look == 0
  osc "/pulse/xyrgb",x,y,r,g,b if look%20!=0 or look == 0
  osc "/pulse/setall",r,g,b if look%20==0  and look > 0
  sleep get(:t)-0.004 #compensate for brighness delay
  osc "/pulse/clear" if look%80 == 0 and look > 0
  if get(:stopflag) == true
    osc "/pulse/clear"
    cue :endphase #cue lastnotes and LED phase
    stop #this live_loop
  end
end

on each pass it selects fresh values for r, g and b chosen at random from an integer range 40->254 (the range it to one LESS than the final figure) These are combined by the line a=(r+g+b)/200 -1 to produce an integer in the range 0 to 2. (note integer division takes place so there are no fractions) This number selects a value for the variable bright which is used to control the brightness of the LEDs. The current bright value is sent to the server using the command osc “/pulse/bright”,b    x and y coordinates are then chosen at random in the range 0->6 inclusive. The x coordinate is used to choose a synth from a list. The tick command is a counter which increments each time we go round the live_loop. The associated look command retrieves the current value of tick without incrementing it. It is used to control the logic of when notes are played, and when the entire LED screen is cleared to a new colour. A note is played using the selected synth with an amplitude related to the brightness setting, and a release time given by the set variable :t which is retrieved for the purpose using get(:t). The note is played  on all passes of the loop apart from those which are a multiple of 20. On those passes which ARE a multiple of 20 the screen is cleared to the current RGB colour, otherwise the LED corresponding to the current x,y coordinates is set to the current rgb values with the current brightness. Every multiple of 80 passes through the loop the screen is completely cleared.
This all continues until the value of stopflag changes to true which is checked in the line
if get(:stopflag) == true This happens when the at command triggers the change after 85 seconds. The consequences are that the LED screen is cleared, then a cue :endphase is sent to start the :lastnotes live_loop and to stop the :drums live_loop. Finally the :pulse live loop is stopped.

The :drums live_loop is the second of the three live_loops. It starts at the same time as the :pulse live_loop and provides a percussion accompaniment.

set :bass_rhythm, ring(9, 0, 9, 0,  0, 0, 0, 0,  9, 0, 0, 3,  0, 0, 0, 0)
set :snare_rhythm, ring(0, 0, 0, 0,  9, 0, 0, 2,  0, 1, 0, 0,  9, 0, 0, 1)
set :hat_rhythm, ring(5, 0, 5, 0,  5, 0, 5, 0,  5, 0, 5, 0,  5, 0, 5, 0)

live_loop :drums, auto_sync: :pulse do
  #use_real_time
  sf = 0.05
  sample :drum_bass_hard, amp: sf*get(:bass_rhythm).tick
  sample :drum_snare_hard, amp: sf*get(:snare_rhythm).look
  sample :drum_cymbal_closed,amp: sf*get(:hat_rhythm).look
  tval= get(:t)

  #puts tval
  if tval <0.3
    sleep tval
  elsif tval <= 0.5
    sleep tval/2
  else
    sleep tval/4
  end
  stop if get(:stopflag) == true #this liveloop
end

It is preceded by three set statements which store the rhythms for the three “instruments” used, which are all generated by built in audio samples in Sonic PI. These rhythms are stored in what are called rings. These are lists where the elements can be accessed by means of a tick counter which points to each element in turn sequentially. However when the value of tick exceeds the number of items in the ring it wraps round and starts from the beginning of the ring, in a circle. The numbers in each ring represent relative amplitudes of volumes. They are scaled by sf (the scale factor) which is set to a value of 0.05. Each sample is played in turn, with a volume set by the value of the current “ticked” element. If the element value is 0 then the sample does not sound: this gives the rhythm. Note that the second and third rings are indexed by look rather than tick. look has the same value as tick, but when it is referenced it does NOT increase its value, whereas tick does. The next section of the loop chooses a delay before the next iteration starts. It is governed by the tval which is retrieved from the get(:t) which was set by the at command as discussed earlier. depending on the value of tval one of three different sleep values are chosen and implemented. In the same way that the live_loop :pulse was stopped, live_loop :drums is also stopped when the value of stopval is changed to true by the penultimate line of the live loop.

live_loop :lastnotes ,sync: :endphase do
  #use_real_time
  use_synth :pulse
  n=scale(:c4,:minor_pentatonic,num_octaves:3).tick
  if look<32
    play n,release: 0.1
    sleep 0.1
  else
    play [72,84,89,96],sustain: 1.8,release:0.1
    sleep 1.9
  end
  stop if look==32 #this live look
end

The third live_loop :lastnotes is activated when it receives the cue :endphase from the :pulse live_loop, which triggers the sync :endphase It uses a synth called :pulse (appropriately) to cycle through a list of notes in a c minor pentatonic scale over three octaves (32 notes in all) choosing the notes using the same .tick mechanism used to get the percussion rhythms. The first 32 times (tick starts at 0) notes from the scale are played in sequence. On the 33rd pass (when tick =32) a chord is played for 1.8 seconds, dying away over 0.1 seconds. Then the live loop is stopped.

#This section fades the LEDs up and down several times before
#leaving them all white for the final chord
#They fade out and are blanked when the music ends

sync :endphase

osc "/pulse/bright",1 #set full brightness
osc"/pulse/clear"
10.times do |x|
  v=(255*x/10).to_i
  osc "/pulse/setall",0,v,0
  sleep 0.05
end
2.times do
  10.times do |x|
    v=(255*(x+1)/10).to_i
      osc "/pulse/setall",v,255-v,v
      sleep 0.05
    end
    10.times do |x|
      v=(255*(9-x)/10).to_i
      osc "/pulse/setall",v,255-v,v
      sleep 0.05
    end
  end
  10.times do |x|
    v=(255*x/10).to_i
     osc "/pulse/setall",0,255-v,0
    sleep 0.05
  end

osc "/pulse/setall",255,255,255 #set all white
sleep 2
100.times do |x|
osc "/pulse/bright",(99-x).to_f/100
sleep 0.01
osc "/pulse/setall",255,255,255
sleep 0.01
end

osc "/pulse/clear" #clear all LEDs
puts "FINISHED!"

The last part of the program is a linear sequence of commands which starts when it receives the :endphase cue picked up by the line sync :endphase from the :pulse live_loop.
it resets the brightness to maximum, clears all the LEDs and then uses a series of four 10.times loops which are used to alter the colours of all of the LEDs together, first by bringing up a green colour, then doing a sequence twice which fading out the green whilst bringing up red and blue to give a magenta colour, then reversing this again. Finally the green is faded out again, before all the LEDs are set to white. After a 2 second pause the LEDs are gradually dimmed over a second using a 100 step loop until they are all out. With an “overkill”  belt and braces command all the LEDs are cleared and the program finishes.

You can download the two scripts from my gist site here

You can see a video of the programs in operation here