EDIT The programs have been updated for library version 0.0.5 and also for Sonic Pi 3.2.2 with changes to the OSC syntax and a new port 4560 instead of 4559.
The Final Project in action
The new VL53L1X sensor breakout board
Previously I have done two Theremin projects for Sonic Pi. In each case they have used Ultrasonic Sensors, which have worked, but not very well, as they suffer from reflections and did not give very precise consistent readings.
Out with the old in with the new!
Recently Pimonroni have started selling the latest version of a Time of Flight infra-red laser sensor, model vl53l1x. Adafruit have sold earlier versions of this sensor previously, but I was attracted to the new one, because the pinouts of the breakout board were arranged so that it could plug directly onto the GPIO pins of a Raspberry Pi without the need for any intermediate breadboard or additional wiring. I purchased one a couple of days ago. It is supplied with a right angled set of pins, which would enable you to plug it into a breadboard, but also with a 5 way socket which can plug directly onto the GPIO pins. I elected to use the latter, but I bent the pins of the 5 way socket straight as I wanted the breakout board to be horizontal for the theremin application. It was fairly easy to solder as the receptor holes were well away from any components on the board so there was little chance of experiencing problems.
As I said, the sensor breakout board will plug directly onto gpio pins. However you need to be extremely careful to make sure you use the right set of pins. It uses the 3v3 supply plus the pins for GP2, GP3 GP4 and gnd which are adjacent. These pins are located on the inner row of GPIO pins starting from the far end of the board to the USB sockets. Make sure you don’t use the outer row which will put 5v on the board!
Sensor board plugged onto GPIO pins. Inside strip nearest 5 pins to the end
Once you have the connector soldered on, make sure your Pi is switched OFF and plug the sensor in, taking care to get the correct location described above. You can then startup the Pi.
The Python software required.
I strongly suggest you start with the latest version of Raspbian Stretch on your computer. I used a new installation of the version released on 2018-06-27. Make sure it is up to date by using sudo apt-get update.
Altogether three extra python libraries need to be installed for the Theremin project to work Pimoroni supply a python library to support the sensor, and you also need to load the smbus2 python library and finally you need the python-osc libarary which I use for the OSC communications with Sonic Pi. Since the sensor uses the i2c bus you need to ensure that i2c is enabled using the Raspberry Pi Configuration utility. To install the python packages you will need an internet connection, but once installed a network is not needed for the project to run on a Raspberry Pi 3, unless you want to involve two computers, one handling the sensor and the other playing Sonic Pi. I sometimes do this using my Mac to play Sonic Pi and a Pi3 to run the sensor. More about this option later on.
Installing the Python librarires
The project runs on python 3.5 which is required for the sonic-python library to function. First install these two packages by typing:
sudo pip3 install smbus2 sudo pip3 install python-osc sudp pip3 install vl53l1x
The script to extract range information from the sensor and send it to Sonic Pi
I have successfully completed a range of projects which utilise a sensor and use a python script to send resulting data to Sonic Pi via OSC messages, and the theremin follows the same method. I started with the graph.py example script included on the Pimoroni github site at https://raw.githubusercontent.com/pimoroni/vl53l1x-python/master/examples/graph.py This illustrated how the sensor readings can be extracted and printed on the screen. I modified this and produced a new script which incorporated support for OSC and which sent the readings via an OSC message with the address “/range”. By default this message is sent to the local host 127.0.0.1 ie the same computer as the one on which the sensor is located. However I also added the possibility to send it to an external IP address, and this will enable Sonic Pi running on a different computer eg my MacBook to receive it and utilise the data. The difference is in the way that the script, called osctheremin.py is started. Normally I start it using ./osctheremin.py (having set it as an executable file using chmod +x osctheremin.py The first line of the script is #!/usr/bin/env python3 which ensures that it is run using python 3.5. When you want to use it with Sonic Pi on a different computer you start the program using ./oscthermin.py –sp 192.168.1.129 (substituting the relevant IP address of the second computer, here 192.168.1.129). and in this case the OSC messages are directed to that IP address and not to the local host.
#!/usr/bin/env python3
#script to read vl53l1x sensor and send reading via OSC to Sonic pi
#written by Robin Newman August 2018 updated August 2020 for version 0.0.5 of library
#with thanks to Pimoroni's Phil Howard @Gadgetoid for the graph.py example
#on which I based some of this script.
import time
import sys
import signal
from pythonosc import osc_message_builder
from pythonosc import udp_client
import argparse
import VL53L1X
MAX_DISTANCE_MM = 800 # Set upper range
"""
Open and start the VL53L1X ranging sensor
"""
tof = VL53L1X.VL53L1X(i2c_bus=1, i2c_address=0x29)
tof.open() # Initialise the i2c bus and configure the sensor
tof.set_timing(66000,70) #This line is added from the original version to set refresh timings
tof.start_ranging(2) # Start ranging, 1 = Short Range, 2 = Medium Range, 3 = Long Range (LINE UPDATED)
sys.stdout.write("\n")
running = True
def exit_handler(signal, frame):
global running
running = False
tof.stop_ranging() # Stop ranging
sys.stdout.write("\n")
sys.exit(0)
signal.signal(signal.SIGINT, exit_handler)
def control(spip):
sender=udp_client.SimpleUDPClient(spip,4560) #port updated to 4560
while running:
distance_in_mm = tof.get_distance() # Grab the range in mm
distance_in_mm = min(MAX_DISTANCE_MM, distance_in_mm) # Cap at our MAX_DISTANCE
sender.send_message('/range',distance_in_mm)
sys.stdout.write("\r") # Return the cursor to the beginning of the current line
sys.stdout.write ("range %3d " % (distance_in_mm))
sys.stdout.flush() # Flush the output buffer, since we're overdrawing the last line
#time.sleep(0.01)
if __name__=="__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--sp",
default = "127.0.0.1", help="The ip of the Sonic Pi computer")
args = parser.parse_args()
spip=args.sp
print("Sonic Pi on ip",spip)
control(spip)
The Script falls into five parts:
- First the environment is set for python3, and the various libraries required are imported. A max distance limit is also set. This will give a constant reading when there is no obstacle within range.
- Secondly the sender is initialised, and scanning started. A global variable running is set to True. A reading is taken using tof.get_distance(). It is clamped to a maximum value of 800 and the resultant number is sent using an OSC address of “/range”
- Third an exit handler is set to shut down the sensor, and close the program when a ctrl-C signal is received. The running variable is set to False to stop the sampling loop.
- The main sampling loop takes place in the function control, which has one parameter spip when it starts to set up the IP address to which the OSC messages will be directed. When a reading is taken it is sent via the OSC message to the IP address given in the variable spip.
- The final section of the script is used read any argument provied when the script is started. It looks for –sp <ip.address.for.spmachine> and sets the spip variable to what is supplied, or, if there is no argument to the default value of 127.0.0.1, the local machine. It prints the IP information before calling the control(spip) loop.
I installed the osctheremin.py script in my pi home folder. To make it executable, you should start a terminal window, and navigate to the folder containing the script. Then type:
chmod +x osctheremin.py
You can then start it running in preparation for the Sonic Pi program (running on the same computer) by typing:
./osctheremin
If you then move your hand around in front of the sensor then the number displayed in the terminal window (800 when there is nothing in front of the sensor) should change, showing that the sensor is working.
The Sonic Pi half of the Theremin
So far we have looked at the sensor, and the python script which gets readings from it and sends them to a specified IP address as OSC messages. We use the program Sonic Pi version 3.0.1 of which is installed as standard on the Raspberry Pi Stretch distribution. You can start the program running from the “Raspberry” menu Programming subsection. It is quite a complex program and takes a little while to start up. When it has started up you can immediately test to see if it is receiving the OSC messages carrying the sensor data. On the bottom right of the screen there is a panel titled Cues. If you have the osctheremin.py script running you should see a series of entries /osc/range 800 appearing in this window. If you put your hand in front of the sensor, then this number should vary.between about 1 and 800.
( In fact it may increase slighty above 1 if you get VERY close to the sensor.) All being well, this shows that the sensor is working, and that it is communicating with Sonic Pi.
I have written three programs to run on Sonic Pi, each producing a different theremin sound. The first, entitled ToF_ThereminSmooth.rb produces a traditional theremin output, consisting of a conitinous note whose pitch is altered according to the distance of an obstacle placed in front of the sensor. To maintain sanity I set a maximum range (smaller than the 800 set by the sensor program) which limits the range of the note, and causes it to cease when there is no obstacle in the way. The listing is shown below and contains comments which indicate what is going on.
The First Sonic Pi program with the terminal window running in the foreground
#Sonic Pi theremin using time of flight sensor
#smooth continuous note pitch version
#by Robin Newman, August 2018 updated August 2020
#works on Mac or Pi3
with_fx :reverb,room: 0.8,mix: 0.7 do
use_synth :dpulse
#start long note at zero vol. It will be controlled by k later
k = play 0,sustain: 10000,amp: 0
set :k,k #store k pointer in time-state as :k
live_loop :theremin do
use_real_time
b = sync "/osc*/range" #syntax updated
v= (b[0].to_f/10) #scale and convert range data to a float
puts v #put value on screen
if v<=48.2
#control the note retrieving :k pointer and adjusting pitch and amp
control get(:k),note: v+47.9,amp: 0.7,amp_slide: 0.04,note_slide: 0.04
else
#if range too high set vol to 0
control get(:k),amp: 0,amp_slide: 0.04
end
end
end #fx reverb
The program works by starting a very long note lasting 10000 seconds. Initially this is at zero volume, but it has a reference to it stored in variable k. This in turn is stored in the time-state so that it can be retrieved inside the following live loop :theremin. The live_loop waits for a trigger from a received OSC message. Sonic Pi prepends all received OSC messages with /osc to indicate their nature, just as received midi information is prepended by /midi. So the loop is waiting for a message addressed to /osc/range rather than just /range. When this is detected, the data is retrieved in the first entry of the list b as b[0] this is converted to a float and divided by 10 and printed to the log window. If the number is <=48.2 then the control pointer (retrieved with get(:k)) is used to reset the note to be played, and to set its amplitude to 0.7. Otherwise the note amplitude is set to 0 muting it. The loop runs quite fast and so as the range value varies the note sounding varies too. The note_slide: value smoothes out the transitions to that there is a continuously varying smooth sound.
A warning
All the Theremin Sonic Pi programs employ a long sounding note that lasts 10000 seconds. The note does not sound if there is no reflected sensor signal, but is still using resources. You MUST stop the Sonic Pi program BEFORE you run it a second time. Also you should WAIT until the log window shows the message Pausing SuperCollider Audio Server BEFORE running a second time. Otherwise Sonic Pi will quickly run out of resources, and may even freeze your Raspberry Pi. Secondly, if the note gets stuck sounding a continuous tone, you MAY be able to salvage the situation by switching to an empty buffer and running the single command kill get(:k) in that buffer. Otherwise you may need to quit and restart Sonic Pi. If all else fails, as a last resort, you may need to cut the power to the Raspberry Pi.
The second program called ToF_ThereminDiscrete.rb is very similar. The difference is that the note value in b[0] is this time scaled by dividing by 12.0 and the result is then converted back to an integer, so that the resultant number varies in discrete steps. It is also used to pick out a note from a 5 octave g major scale produced as a list which is indexed by the scaled range value v. So as the range reading alters, different notes from this scale are chosen and played. A different synth :tri is also selected. Otherwise the operation of the program is essentially similar.
#Sonic Pi theremin using time of flight sensor VL531X
#Discrete note version (c: major {changeable})
#by Robin Newman, August 2018 updated August 2020
with_fx :reverb,room: 0.8,mix: 0.7 do
use_synth :tri
#start long note at zero volume: will be controlled later by k
k = play octs(0,2),sustain: 10000,amp: 0
set :k,k #store k in time-state reference :k
live_loop :theremin do
use_real_time
b = sync "/osc*/range" #get osc message from sensor script. Syntax updated
v= (b[0]/12.0).to_i #scale reading and convert result to integer
puts v #print on screen
# puts scale(:g3,:major,num_octaves: 5).length #for debugging puts total no notes
if v < 36
#change the scale if you wish. (may need to change note range too)
nv=scale(:g3,:major,num_octaves: 5)[v] #get note pitch from c scale
#control the note and set the pitch and volume
control get(:k),note: nv,amp: 1,amp_slide: 0.04
else
#if v is too high then set volume to zero
control get(:k),amp: 0,amp_slide: 0.04
end
end
end #fx reverb
The final example is called ToF_ThereminTb303Frenzy.rb As the name suggests, this uses the :tb303 synth, but it is also more complex than the other two programs, as it incorporates a rhythm section and the notes also have their cutoff values varied.
#Sonic Pi theremin using time of flight sensor
#tb303 frenzy version!
#by Robin Newman, August 2018 updated August 2020
#experiment changing octs(v+25.8,2) to octs(v+47.9,1) line22
#may want to increase sample default amp to 3 on a Mac
#and decrease tb303 vol from 1.4 to 0.5 on a Mac
use_sample_defaults amp: 2 #for percussion samples
with_fx :reverb,room: 0.8,mix: 0.7 do #can try gverb as well
use_synth :tb303
#start long continous note at zero volume: controlled later on by k
k = play 0,sustain: 10000,amp: 0
set :k,k #save k pointer in time state as :k
live_loop :theremin do #start thermin loop
use_real_time
b = sync "/osc*/range" #wait for input from python script
v= (b[0]/8.0)#scale the received data as a float
puts v #print value on screen
if v<=48.2 #set limit for high note
cue :drums #send cue to :dr live loop ty sync percussion
control get(:k),note: octs(v+25.8,2).tick,amp: 1.4,amp_slide: 0.04,note_slide: 0.04,cutoff: 1.5*v+57,pan: 0.8*(-1)**look
else #if outside range then set note volume to zero
control get(:k),amp: 0,amp_slide: 0.04
end
end
live_loop :dr do
use_real_time
sync :drums #only run when theremin playing a note
tick
#use three samples with spread to give some funky rhythm
sample :bd_haus if spread(2,5).look
sample :elec_twip if !spread(5,8).look
sample :drum_cymbal_closed if spread(5,8).look
sleep 0.2 #wait then go back for next cue
end
end #fx reverb
A second live loop is incorporated which plays three percussion samples, using the spread function to produce some rhythm. The loop only plays when there is an active note being played, and this is controlled by the sync :drums command, with a corresponding :cue drums added to the theremin live_loop. The note reverts to the continuously variable variety used in the first example, and further interest is added by including a controlled cutoff value related to the notes frequency, and a changing pan setting for each successive note. With suitable hand waving some quite zany sounds can be produced hence the program name. The video also illustrates a small modification that can be made to the program.
Resources
The Sonic Pi programs and the osctheremin.py script can be downloaded from my gist site here
For reference the Pimoroni library site is here, although you can install it just by using pip3, but it does have an examples folder of scripts that can be used with the sensor. The sensor itself can be purchase from here
There is a video of the project in action on a Raspberry Pi here
rbnrpi 