Skip to main content
GAMESOVER

Note for Note: A Song That Is Its Own Sheet Music

Music is math. Note for Note, a song that is its own sheet music

I believe music is math. Not in a vague, poster-on-the-band-room-wall way. In a literal way: pitch is frequency, rhythm is division, harmony is ratios, and a song is a very precise set of instructions for moving air. "Note for Note" is my attempt to make a song where you can see all of that at once.

The rule was simple and a little bit ridiculous. Every lyric is a solfege syllable, and every syllable names the exact note being sung. When the singer sings "do," you are hearing C. When they sing "sol," you are hearing G. The lyric sheet is the sheet music. If you can sing "do re mi," you can sight-read the whole song.

As a young boy growing up in a Northern Minnesota town, I was introduced to the song "Do-Re-Mi" from the Rodgers & Hammerstein's "The Sound of Music". The musical is from 1965 and my mom liked this movie so much that we ended up watching it whenever it was aired on TV. (This was obviously before cable TV, DVDs, the internet and Netflix.) In this musical number, Maria broke our her guitar, sang to the kids and taught then the basics of music by revealing the names of the notes as the lyrics (just like "Note for Note").

It so true that "when you know the notes to sing, you can sing most anything". Thank you Rodgers & Hammerstein and Julie Andrews.

Watch the video on YouTube

The song, by the numbers

  • 122 beats per minute, in C major, 4/4 time. One beat lasts 0.492 seconds. The song runs 80 bars and 321 beats.
  • 216 sung syllables. Held notes are written with hyphens, one segment per beat, so "so-ol" is two beats and "do-o-o-o" is four. Counting every held segment, that's 252 beats of singing.
  • Every bar adds up to exactly four beats. I checked all of them with a script, because a single miscounted bar breaks the rule.
  • The most-sung note is sol (G), 51 times. Then la (38), do (33), mi (28), ti (26), and re and fa (20 each). A pop chorus that lives on the fifth of the scale is about as classic as it gets.
  • The range is 26 semitones, from C3 at the bottom of the men's part to D5 at the top of the women's. That's two octaves plus a whole step.
  • Three voices: 98 syllables for the male voice, 82 for the female voice, and 36 where the male voice is doubled by a choir.
  • The longest note is a 5.3-second "do-o" that the female voice holds over the top while the men climb the scale underneath her.

Verse 2 is lyrically identicalal to verse 1. This is not due to a lack of effort. It follows a rule where the same melody requires the same lyrics. In an effort to not just repeat myself, I added variation by introducing a female vocalist. As a result, this song was transformed from a solo to a duet.

Where the math is hiding

A few favorite facts from the song itself:

  • Every semitone is the previous one times the twelfth root of 2, about 1.0595. Twelve of those multiplications double the frequency, which is why an octave sounds like "the same note, higher." Middle C is 261.63 Hz, and the high C the female voice holds is 523.25 Hz, exactly double.
  • The bridge moves to the relative minor by centering on la (A). Same seven notes, same key signature, different home base, and the mood flips from sunny to moody.
  • The chords at the end are counting. A major chord stacks 4 semitones and then 3. A minor chord stacks 3 and then 4. My first pass at the songbook page labeled the second chord "E," and it's really E minor (E, G, B), because the gap from E up to G is 3 semitones, not 4. Wrong math, wrong chord.

Seeing the math: 11 eras of sheet music

I designed the video to display the lyrics in the same way that musicians read musical notes. The words scroll from right to left across a staff. Each word is positioned on the specific line or space that corresponds to the sung pitch. The horizontal length of each word indicates the duration of the note. A colored line follows behind the text to track the singer's actual pitch, which allows viewers to see details like vibrato and pitch slides. Blue represents the male voice, crimson represents the female voice, and gold represents the male voice singing with the choir.

Then every section of the song jumps forward in time:

  1. 1820s ink manuscript. A composer's working sketch on stained, uneven paper, modeled on real Beethoven sketches, with hand-drawn staff lines and the title written at the top.
  2. 1870s engraving. The clean, precise look of a printed engraved plate.
  3. 1960s ditto sheet. Purple spirit-duplicator ink on cheap paper, the way a lot of us got our choir music, with the lyrics typed on a typewriter.
  4. 1990 notation program. Black-and-white pixels on a curved CRT screen, with a menu bar, a tool palette, and a little window counting measures and beats.
  5. 2010s tablet. A sheet-music reader app on a tablet held sideways, complete with the bezel, home button, and status bar (airplane mode on, because this is a rehearsal). A faint gray border pulses on the first beat of every bar, like the app's visual metronome.
  6. Rhythm game highway. The notes come down a fret-board highway at you, and the notation turns into a game.
  7. Sing-along game vocal track. Pitch bars scroll past, and you're scored on how well you match them.
  8. The techno break consists of four simultaneous music visualizers. These include the blocky shapes of a 1970s home visualizer, the spectrum bars of a late-90s media player, a MilkDrop-style preset captured live from projectM and synchronized to the kick drum within a frame, and an oscilloscope that displays the left and right channels as an X-Y figure.
  9. Modern recording software. The score sits under a multitrack arrange view, and below that is a pitch editor that shows every sung syllable as an orange blob tracing its real pitch. At one point a mouse cursor grabs the one note that was sung about half a semitone sharp and drags it onto the right row.
  10. Right-click, Add Effect, Stutter. The final note gets chopped into repeating slices while a mod wheel rises, the screen shakes in time with the slices, and then the whole screen tips back and falls away.
  11. A children's toy keyboard is shown at the bottom of the screen with a letter on each key and a songbook page above it, and the keys illuminate as the melody plays.

Two hundred years of reading music, in 2 minutes and 41 seconds.

The video - by the numbers

The video was as much a math problem as the song.

  • The scroll speed comes from the tempo. I gave every beat 200 pixels of paper. At 122 BPM that's 406.5 pixels per second, and at 60 frames per second it's 6.8 pixels per frame. So if any piece of the video is off by a single frame, you can see it jump 6.8 pixels.
  • 9,677 frames total, including a 2-second (120-frame) title card before the music starts. The song starts at exactly 2.000 seconds, and the delivery check confirms it to within one frame.
  • Every onset is snapped to a frame by the same formula, rounding the time times 60 up to the next whole frame. When one part of the video rounded and another part rounded up, they drifted apart. One formula everywhere fixed that.
  • The music was mapped out before a single frame was drawn. I matched every syllable to the lead-vocal MIDI and aligned it against the isolated vocal track. The pitch trace comes from the vocal itself, 60 samples a second. The detector also found the ornaments: 22 melismas, 20 stretches of vibrato, 2 trills, and 2 mordents, each drawn with the symbol its era would have used.
  • Each era renders as three layers: the paper, the words, and a glow mask. That's 33 layer files, plus 4 masks for the eras whose paper changes with a moving seam. DaVinci Resolve assembles them on 12 video tracks with 31 Fusion effects and 88 markers.
  • The finished render is checked against the source layers, pixel by pixel. There are 28 automated checks at the seams, moves, glows, and pulses. Each one has to match its own frame better than the frames on either side of it, which is how you prove it's frame accurate.
  • Over 350 automated tests hold the whole thing together across the song-data tools, the animation, and the Resolve build.

My favorite math bug

The pitch editor in era 9 was designed to identify the single note with the greatest tuning error. In the first instance, it selected a male "mi" at E3 (164.8 Hz) and measured it as 89 cents sharp. This value represents nearly a full semitone, which is a significant deviation for a clean vocal recording.

The audio was not out of tune. The female singer held a high C at 523.25 Hz. One third of that frequency is 174.4 Hz, which corresponds to F3. This is one semitone higher than the note the male singer was performing. The pitch tracker identified a subharmonic of the female voice rather than the male voice. I resolved this by ensuring the software does not select a note while another voice is singing simultaneously. These physics also affected her sustained note. The male singer's final C4 produces a first overtone at C5, which is the same frequency as her note. The recording does not provide enough data to distinguish between her note and his overtone. In that instance, I relied on my own hearing to make the adjustment.

Even the colors are math

A digital video stores color as brightness plus two color-difference signals, and there are two common standards for turning them back into red, green, and blue (RGB). The older one weights green at 0.587 of brightness, and the HD standard weights it at 0.7152. The layers were written with one set of numbers and read with the other, which shifted the sepia of the 1820s paper by a couple of levels. Nobody would have noticed, but it wasn't right. As a result, every layer was re-rendered with the HD numbers. The error on that sepia dropped from about 1.4 levels to 0.37.

Why bother?

The mathematical rules made everything easier to check. When the lyrics are the notes, there's exactly one right answer for every syllable: its pitch, its length, its place on the staff. The same goes for every frame of the video. Music is math, and math can be checked.

I rendered each portion of this video on my Dell laptop and assembled it using DaVinci Resolve Studio. In order to produce the highest quality of horizontal scrolling, I processed 1920x108 with 60 frames per second. Since this video was all math and didn't leverage any cloud-based generative video, my PC performed all of the video generation and it took a very long time. In the end, the final result was worth it.

"Note for Note" is out now on all streaming platforms: https://www.gamesovermusic.com/music/song/note-for-note

Watch the Video

Lyrics

do re mi do  mi fa so-ol
la sol fa sol mi do  re-e do-o
do re mi do  mi fa so-ol
la ti la sol fa re  re mi fa sol

fa sol la-a  sol la ti-i
la ti do-o  re do ti-i

sol sol la sol mi  do ti la so-ol
fa la sol mi  re mi do-o
sol sol la sol mi  do ti la so-ol
fa la sol ti  re ti do-o

do re mi do  mi fa so-ol
la sol fa sol mi do  re-e do-o
do re mi do  mi fa so-ol
la ti la sol fa re  re mi fa sol

fa sol la-a  sol la ti-i
la ti do-o  re do ti-i

sol sol la sol mi  do ti la so-ol
fa la sol mi  re mi do-o
sol sol la sol mi  do ti la so-ol
fa la sol ti  re ti do-o

la mi la ti  do-o ti la
sol mi sol la  ti-i-i-i
la mi la ti  do-o re do
ti la sol fa  so-o-o-ol

sol sol la sol mi  do ti la so-ol
fa la sol mi  re mi do-o
sol sol la sol mi  do ti la so-ol
fa la sol ti  re ti do-o

do re mi fa  sol la ti do
do-o-o-o