New Sonic Pi Theremin using Time of Flight Laser sensor

EDIT an update to the osctheremin.py script now lets the project use the latest version (0.0.2) of the python library, which is much easier to install. See changes marked ** in the article


The Final Project in action

The new VL53L1X sensor breakout board

Previously I have done two Theremin projects for Sonic Pi. In each case they have used Ultrasonic Sensors, which have worked, but not very well, as they suffer from reflections and did not give very precise consistent readings.

Out with the old in with the new!

Recently Pimonroni have started selling the latest version of a Time of Flight infra-red laser sensor, model vl53l1x. Adafruit have sold earlier versions of this sensor previously, but I was attracted to the new one, because the pinouts of the breakout board were arranged so that it could plug directly onto the GPIO pins of a Raspberry Pi without the need for any intermediate breadboard or additional wiring. I purchased one a couple of days ago. It is supplied with a right angled set of pins, which would enable you to plug it into a breadboard, but also with a 5 way socket which can plug directly onto the GPIO pins. I elected to use the latter, but I bent the pins of the 5 way socket straight as I wanted the breakout board to be horizontal for the theremin application. It was fairly easy to solder as the receptor holes were well away from any components on the board so there was little chance of experiencing problems.

As I said, the sensor breakout board will plug directly onto gpio pins. However you need to be extremely careful to make sure you use the right set of pins. It uses the 3v3 supply plus the pins for GP2, GP3 GP4 and gnd which  are adjacent. These pins are located on the inner row of GPIO pins starting from the far end of the board to the USB sockets. Make sure you don’t use the outer row which will put 5v on the board!Sensor board plugged onto GPIO pins. Inside strip nearest 5 pins to the end

Once you have the connector soldered on, make sure your Pi is switched OFF and plug the sensor in, taking care to get the correct location described above. You can then startup the Pi.

The Python software required.
I strongly suggest you start with the latest version of Raspbian Stretch on your computer. I used a new installation of the version released on 2018-06-27. Make sure it is up to date by using sudo apt-get update.

Altogether three extra python libraries need to be installed for the Theremin project to work Pimoroni supply a python library to support the sensor, and  you also need to load the smbus2 python library and finally you need the python-osc libarary which I use for the OSC communications with Sonic Pi. Since the sensor uses the i2c bus you need to ensure that i2c is enabled using the Raspberry Pi Configuration utility. To install the python packages you will need an internet connection, but once installed a network is not needed for the project to run on a Raspberry Pi 3, unless you want to involve two computers, one handling the sensor and the other playing Sonic Pi. I sometimes do this using my Mac to play Sonic Pi and a Pi3 to run the sensor. More about this option later on.

Installing the Python librarires

The project runs on python 3.5 which is required for the sonic-python library to function. First install these two packages by typing:

sudo pip3 install smbus2
sudo pip3 install python-osc

** Edit problems I had with the vl54l1x pythoin library are now solved. Instead of the section following in blue, install the latest library using:

sudo pip3 install vl53l1x

I experienced some problems in installing the vl53l1x python library that Pimoroni supply. When I first started the project the current version was 0.0.1, and although it appeared to install using sudo pip3 install vl53l1x it didn’t seem to work. Eventually I found that I needed to add a soft link file for it to find the installed library. Phil Howard, from Pimoroni who set up the library then moved to version 0.0.2, but although this then installed for python 3.5 it seemed to introduce a jitter effect which spoilt the operation of the Theremin, althouhg it would work. So at the time of writing I suggest the following installation procedure.

sudo pip3 install vl53l1x==0.0.1
cd /usr/local/lib/python3.5/dist-packages
ls

you should see amongst other files one like
vl53lx_python.cpython-35m-arm-linux-gnueabihf.so
This needs to have an alias pointing to it so that it appears as vl53lx_python.so
To do this type

sudo ln -s vl53lx_python

then push the ESC key and it should autocomplete the file name for you giving something like

 sudo ln -s  vl53lx_python.cpython-35m-arm-linux-gnueabihf.so

add on the end a space and

 vl53l1x_python.so

giving the line below, before pressing return

 sudo ln -s  vl53lx_python.cpython-35m-arm-linux-gnueabihf.so  vl53l1x_python.so

Hopefully  a later release may solve this problem.

The script to extract range information from the sensor and send it to Sonic Pi

I have successfully completed a range of projects which utilise a sensor and use a python script to send resulting data to Sonic Pi via OSC messages, and the theremin follows the same method. I started with the graph.py example script included on the Pimoroni github site at https://raw.githubusercontent.com/pimoroni/vl53l1x-python/master/examples/graph.py This illustrated how the sensor readings can be extracted and printed on the screen. I modified this and produced a new script which incorporated support for OSC and which sent the readings via an OSC message with the address “/range”. By default this message is sent to the local host 127.0.0.1 ie the same computer as the one on which the sensor is located. However I also added the possibility to send it to an external IP address, and this will enable Sonic Pi running on a different computer eg my MacBook to receive it and utilise the data. The difference is in the way that the script, called osctheremin.py is started.  Normally I start it using ./osctheremin.py (having set it as an executable file using chmod +x osctheremin.py The first line of the script is #!/usr/bin/env python3 which ensures that it is run using python 3.5. When you want to use it with Sonic Pi on a different computer you start the program using ./oscthermin.py –sp 192.168.1.129 (substituting the relevant IP address of the second computer, here 192.168.1.129). and in this case the OSC messages are directed to that IP address and not to the local host.

** EDIT. The code for the following script has been altered to allow the latest python library to be used. The line

tof.start_ranging(1) # Start ranging, 1 = Short Range, 2 = Medium Range, 3 = Long Range

has been updated to

tof.start_ranging(2) # Start ranging, 1 = Short Range, 2 = Medium Range, 3 = Long Range

The updated code for the script is shown below:

#!/usr/bin/env python3
#script to read vl53l1x sensor and send reading via OSC to Sonic pi
#written by Robin Newman August 2018
#with thanks to Pimoroni's Phil Howard @Gadgetoid for the graph.py example
#on which I based some of this script.
import time
import sys
import signal
from pythonosc import osc_message_builder
from pythonosc import udp_client
import argparse
import VL53L1X

MAX_DISTANCE_MM = 800 # Set upper range

"""
Open and start the VL53L1X ranging sensor
"""
tof = VL53L1X.VL53L1X(i2c_bus=1, i2c_address=0x29)
tof.open() # Initialise the i2c bus and configure the sensor
tof.start_ranging(2) # Start ranging, 1 = Short Range, 2 = Medium Range, 3 = Long Range (LINE UPDATED)

sys.stdout.write("\n")

running = True

def exit_handler(signal, frame):
    global running
    running = False
    tof.stop_ranging() # Stop ranging
    sys.stdout.write("\n")
    sys.exit(0)

signal.signal(signal.SIGINT, exit_handler)

def control(spip):
    sender=udp_client.SimpleUDPClient(spip,4559)
    while running:
        distance_in_mm = tof.get_distance() # Grab the range in mm
        distance_in_mm = min(MAX_DISTANCE_MM, distance_in_mm) # Cap at our MAX_DISTANCE
        sender.send_message('/range',distance_in_mm)
        sys.stdout.write("\r") # Return the cursor to the beginning of the current line
        sys.stdout.write ("range %3d " % (distance_in_mm))
        sys.stdout.flush() # Flush the output buffer, since we're overdrawing the last line
        time.sleep(0.01)
    
if __name__=="__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--sp",
    default = "127.0.0.1", help="The ip of the Sonic Pi computer")
    args = parser.parse_args()
    spip=args.sp
    print("Sonic Pi on ip",spip)
    control(spip)

The Script falls into five parts:

  • First the environment is set for python3, and the various libraries required are imported. A max distance limit is also set. This will give a constant reading when there is no obstacle within range.
  • Secondly the sender is initialised, and scanning started. A global variable running is set to True. A reading is taken using tof.get_distance(). It is clamped to a maximum value of 800 and the resultant number is sent using an OSC address of “/range”
  • Third an exit handler is set to shut down the sensor, and close the program when a ctrl-C signal is received. The running variable is set to False to stop the sampling loop.
  • The main sampling loop takes place in the function control, which has one parameter spip when it starts to set up the IP address to which the OSC messages will be directed. When a reading is taken it is sent via the OSC message to the IP address given in the variable spip.
  • The final section of the script is used read any argument provied when the script is started. It looks for –sp <ip.address.for.spmachine> and sets the spip variable to what is supplied, or, if there is no argument to the default value of 127.0.0.1, the local machine. It prints the IP information before calling the control(spip) loop.

I installed the osctheremin.py script in my pi home folder. To make it executable, you should start a terminal window, and navigate to the folder containing the script. Then type:

chmod +x osctheremin.py

You can then start it running in preparation for the Sonic Pi program (running on the same computer) by typing:

./osctheremin

If you then move your hand around in front of the sensor then the number displayed in the terminal window (800 when there is nothing in front of the sensor) should change, showing that the sensor is working.

The Sonic Pi half of the Theremin

So far we have looked at the sensor, and the python script which gets readings from it and sends them to a specified IP address as OSC messages. We use the program Sonic Pi version 3.0.1 of which is installed as standard on the Raspberry Pi Stretch distribution. You can start the program running from the “Raspberry” menu Programming subsection. It is quite a complex program and takes a little while to start up. When it has started up you can immediately test to see if it is receiving the OSC messages carrying the sensor data. On the bottom right of the screen there is a panel titled Cues. If you have the osctheremin.py script running you should see a series of entries /osc/range 800 appearing in this window. If you put your hand in front of the sensor, then this number should vary.between about 1 and 800.
( In fact it may increase slighty above 1 if you get VERY close to the sensor.) All being well, this shows that the sensor is working, and that it is communicating with Sonic Pi.

I have written three programs to run on Sonic Pi, each producing a different theremin sound. The first, entitled ToF_ThereminSmooth.rb produces a traditional theremin output, consisting of a conitinous note whose pitch is altered according to the distance of an obstacle placed in front of the sensor. To maintain sanity I set a maximum range (smaller than the 800 set by the sensor program) which limits the range of the note, and causes it to cease when there is no obstacle in the way. The listing is shown below and contains comments which indicate what is going on.

The First Sonic Pi program with the terminal window running in the foreground

#Sonic Pi theremin using time of flight sensor
#smooth continuous note pitch version
#by Robin Newman, August 2018
#works on Mac or Pi3
with_fx :reverb,room: 0.8,mix: 0.7 do
  use_synth :dpulse
  #start long note at zero vol. It will be controlled by k later
  k = play 0,sustain: 10000,amp: 0
  set :k,k #store k pointer in time-state as :k
  live_loop :theremin do
    use_real_time
    b = sync "/osc/range"
    v= (b[0].to_f/10) #scale and convert range data to a float
    puts v #put value on screen
    if v<=48.2
      #control the note retrieving :k pointer and adjusting pitch and amp
      control get(:k),note: v+47.9,amp: 0.7,amp_slide: 0.04,note_slide: 0.04
    else
      #if range too high set vol to 0
      control get(:k),amp: 0,amp_slide: 0.04
    end
  end
  
end #fx reverb

The program works by starting a very long note lasting 10000 seconds. Initially this is at zero volume, but it has a reference to it stored in variable k. This in turn is stored in the time-state so that it can be retrieved inside the following live loop :theremin. The live_loop waits for a trigger from a received OSC message. Sonic Pi prepends all received OSC messages with /osc to indicate their nature, just as received midi information is prepended by /midi. So the loop is waiting for a message addressed to /osc/range rather than just /range. When this is detected, the data is retrieved in the first entry of the list b as b[0] this is converted to a float and divided by 10 and printed to the log window. If the number is <=48.2 then the control pointer (retrieved with get(:k)) is used to reset the note to be played, and to set its amplitude to 0.7. Otherwise the note amplitude is set to 0 muting it. The loop runs quite fast and so as the range value varies the note sounding varies too. The note_slide: value smoothes out the transitions to that there is a continuously varying smooth sound.

A warning

All the Theremin Sonic Pi programs employ a long sounding note that lasts 10000 seconds. The note does not sound if there is no reflected sensor signal, but is still using resources. You MUST stop the Sonic Pi program BEFORE you run it a second time. Also you should WAIT until the log window shows the message Pausing SuperCollider Audio Server BEFORE running a second time. Otherwise Sonic Pi will quickly run out of resources, and may even freeze your Raspberry Pi. Secondly, if the note gets stuck sounding a continuous tone, you MAY be able to salvage the situation by switching to an empty buffer and running the single command kill get(:k) in that buffer. Otherwise you may need to quit and restart Sonic Pi. If all else fails, as a last resort, you may need to cut the power to the Raspberry Pi.

The second program called ToF_ThereminDiscrete.rb is very similar. The difference is that the note value in b[0] is this time scaled by dividing by 12.0 and the result is then converted back to an integer, so that the resultant number varies in discrete steps. It is also used to pick out a note from a 5 octave g major scale produced as a list which is indexed by the scaled range value v. So as the range reading alters, different notes from this scale are chosen and played. A different synth :tri is also selected. Otherwise the operation of the program is essentially similar.

#Sonic Pi theremin using time of flight sensor VL531X
#Discrete note version (c: major {changeable})
#by Robin Newman, August 2018
with_fx :reverb,room: 0.8,mix: 0.7 do
  use_synth :tri
  #start long note at zero volume: will be controlled later by k
  k = play octs(0,2),sustain: 10000,amp: 0
  set :k,k #store k in time-state reference :k
  live_loop :theremin do
    use_real_time
    b = sync "/osc/range" #get osc message from sensor script
    v= (b[0]/12.0).to_i #scale reading and convert result to integer
    puts v #print on screen
    # puts scale(:g3,:major,num_octaves: 5).length #for debugging puts total no notes
    if v < 36
      #change the scale if you wish. (may need to change note range too)
      nv=scale(:g3,:major,num_octaves: 5)[v] #get note pitch from c scale
      #control the note and set the pitch and volume
      control get(:k),note: nv,amp: 1,amp_slide: 0.04
    else
      #if v is too high then set volume to zero
      control get(:k),amp: 0,amp_slide: 0.04
    end
  end
  
end #fx reverb

The final example is called ToF_ThereminTb303Frenzy.rb As the name suggests, this uses the :tb303 synth, but it is also more complex than the other two programs, as it incorporates a rhythm section and the notes also have their cutoff values varied.

#Sonic Pi theremin using time of flight sensor
#tb303 frenzy version!
#by Robin Newman, August 2018
#experiment changing octs(v+25.8,2) to octs(v+47.9,1) line22

#may want to increase sample default amp to 3 on a Mac
#and decrease tb303 vol from 1.4 to 0.5 on a Mac

use_sample_defaults amp: 2 #for percussion samples
with_fx :reverb,room: 0.8,mix: 0.7 do #can try gverb as well
  use_synth :tb303
  #start long continous note at zero volume: controlled later on by k
  k = play 0,sustain: 10000,amp: 0
  set :k,k #save k pointer in time state as :k
  live_loop :theremin do #start thermin loop
    use_real_time
    b = sync "/osc/range" #wait for input from python script
    v= (b[0]/8.0)#scale the received data as a float
    puts v #print value on screen
    if v<=48.2 #set limit for high note
        cue :drums #send cue to :dr live loop ty sync percussion
        control get(:k),note: octs(v+25.8,2).tick,amp: 1.4,amp_slide: 0.04,note_slide: 0.04,cutoff: 1.5*v+57,pan: 0.8*(-1)**look 
        else #if outside range then set note volume to zero
          control get(:k),amp: 0,amp_slide: 0.04
        end
        end

        live_loop :dr do
          use_real_time
          sync :drums #only run when theremin playing a note
          tick
          #use three samples with spread to give some funky rhythm
          sample :bd_haus if spread(2,5).look
          sample :elec_twip if !spread(5,8).look
          sample :drum_cymbal_closed if spread(5,8).look
          sleep 0.2 #wait then go back for next cue
        end

        end #fx reverb

A second live loop is incorporated which plays three percussion samples, using the spread function to produce some rhythm. The loop only plays when there is an active note being played, and this is controlled by the sync :drums command, with a corresponding :cue drums added to the theremin live_loop. The note reverts to the continuously variable variety used in the first example, and further interest is added by including a controlled cutoff value related to the notes frequency, and a changing pan setting for each successive note. With suitable hand waving some quite zany sounds can be produced hence the program name. The video also illustrates a small modification that can be made to the program.

Resources

The Sonic Pi programs and the osctheremin.py script can be downloaded from my gist site here

For reference the Pimoroni library site is here, although you can install it just by using pip3, but it does have an examples folder of scripts that can be used with the sensor. The sensor itself can be purchase from here

There is a video of the project in action on a Raspberry Pi here

Advertisements