When people think about communicating data they often gravitate towards graphics and interactive visualizations, thinking that these are the only options. But why stop at the sense of sight?!? There are many other methods of perception, each with its own strengths and weaknesses, that can be used to enhance and augment own ability to present data. In this workshop you will explore the tools, theory, and techniques of sonifying data (as well as little history of how we got to where we are today) for both creative and informational uses.
If you have ever wanted to break past the limits of the eye and explore alternative methods of visualizing data, this workshop is for you! You WILL learn how to trigger sounds based on data and user interaction, modulate audio (pitch, tempo, etc.), and "play" a CSV using p5.js (with a little D3.js thrown in). In this workshop you WON'T learn how to create synthesizers, work with waveforms, or process sonic textures unfortunately (there are other workshops for that). Basic programming experience (ideally with Javascript) is necessary but experience with p5.js or D3.js is not required (or assumed).
And please do not hesitate to reach out to me directly via email at [email protected] or over twitter @jonathandinu
it is also recommended to bring a pair of your own headphones
You will need:
OR
- Text Editor: I recommend Sublime Text
- A (modern) Web Browser: I recommend Google Chrome
- Using the loadTable() function, load the Housing Economic data in the Housing/data/housing_bubble.csv folder into your sketch.
- To make sure the data was loaded, print how many rows and columns are in the dataset as well as the first row of data. use the functions defined on the
Table
object: https://p5js.org/reference/#/p5.Table - In the
draw()
function, play a static note using theplayNote()
function that has been defined for each row in the CSV file. Pick any MIDI number and set a fixed duration. - Advance the index to 'play' the next row of the CSV.
Now that you can trigger a random note at a fixed duration for each data point, it is time to modulate the pitch (and anything else we want) based on the data! To do this however we need to first compute a sensible mapping from our data to MIDI notes.
- Find the minimum and maximum values for the
house_price_index
. You will use the p5min()
andmax()
functions. - Also we need to compute a sensible duration for each note. To do so we can set a fixed length for the entire 'song' and figure out how long each note needs to be. Determine the length of each note by dividing the
total_length
by the number of data points you have. Now that we have a proper duration and the extent (min/max) of ourhouse_price_index
column we can start modulating our notes. - Refactor the
playNote()
function to play a properly scaled MIDI note whose frequency corresponds to the value ofhouse_price_index
for the given row. You will need to use themap()
function: https://p5js.org/reference/#/p5/map - One oscillator is good, but two oscillators are gooder-er! Create a second voicing to sonify the
delinquency
column. You will copy much of what we have done with our first voicing:- Create an oscillator
- Determine the min/max of the columns
- Create a map() to convert fro ma data value -> a MIDI note
- Call the
playNote()
function passing in this new Oscillator and the min/max values for the column
- We can modulate other characteristic of the sound without creating an entirely new voicing. Module the
masterVolume()
of the sketch to correspond to thehouses_sold
column (you will need to create amap()
again). - And the last touch we want to put on this is a bit of author editorialization to express how we feel about the housing crisis. Play the sound clip loaded in the
preload()
function at an appropriate point in the sonification (maybe just as the housing market crashes). You can hard code this value in the index. - Congratulations! You just sonified the Housing Crisis... now its time for some fun.
- Play with other voicings and oscillators
- Modulate other characteristics of the sound (tempo, filters, etc.)
- Load in your own sounds to trigger (I like freesounds.org)
- Find a dataset on the Durham open data portal to sonify (it should have some element of time to it).
- JS for Cats
- Codeschool Interactive Tutorial
- My Javascript Resource List
- p5 Javascript resource list
- Git (and Github) for Poets
- Jonathan's Favorite Datasets