Machine Learning for Signal Processing - 2P2021

Assignment 1: Linear Algebra Refresher

Due date: March 14, 11:59 PM

Adapted from Bhiksha Raj.

Problem:

Below are links to pieces of music and recordings of several notes. You are required to transcribe the music.

For transcription, you will have to determine the note or set of notes being played at each time

Blowin’ in the wind

[This file] contains a recording of a harmonica piece rendering the song “Blowin’ in the wind”. Also given are a collection of notes, and an example of musical scale. Transcribe both the musical scale and the main song in terms of notes.

[wav] version of music_extract.mp3

Polyushka Polye

[This] is a recording of “Polyushka Polye”, played on the harmonica.

Below are a set of notes from a harmonica

Download the following matlab files:[stft.m]

Matlab Instructions

You can read a wav file into Matlab as follows:

[s,fs] = audioread('filename');
s = resample(s,16000,fs)

The recordings of the notes can be computed to a spectrum as follows:

spectrum = mean(abs(stft(s',2048,256,0,hann(2048))),2); 

‘spectrum’ will be a 1025 x 1 vector

The recordings of the complete music can be read just as you read the notes. To convert it to a spectrogram, do the following:

sft = stft(s',2048,256,0,hann(2048)); 
sphase = sft./abs(sft); smag = abs(sft); 

‘smag’ will be a 1025 x K matrix where K is the number of spectral vectors in the matrix. We will also need ‘sphase’ to reconstruct the signal later

Additional Info

Compute the spectrum for each of the notes. Compute the spectrogram matrix ‘smag’ for the music signal. This matrix is composed of K spectral vectors. Each vector represents 16 milliseconds of the signal.

You may find, projections, pseudo inverses, and dot products useful. If you know of any other techniques, you can use those too. Tricks like thresholding (setting all values of some variable that fall below a threshold to 0) might also help.

The output should be in the form of a matrix:

1 1 0 0 0 0 0 1 ...
0 0 0 1 1 0 1 1 ...
0 1 1 1 0 1 1 1 ...
...........................

Each row of the matrix represents one note. Hence there will be as many rows as you have notes in table 1.

Each column represents one of the columns in the spectrogram for the music. So if there are K vectors in the spectrogram, there will be K vectors in your output.

Each entry will denote if a note was found in that vector or not. For instance, if matrix entry (4,25) = 0, then the fourth note (d) was not found in the 25th spectral vector of the signal.

Synthesizing Audio

You can use the notes and the transcription matrix thus obtained to synthesize audio. Note that matrix multiplying the notes and the transcription will simply give you the magnitude spectrum. In order to create meaningful audio, you will need to use the phases as well. Once you have the phases included, you can use the stft to synthesize a signal from the matrix. Submit the synthesized audio along with the matrix.

The following problems are optional

Linear Algebra

Rotational Matrices:

A rotation in 3-D space is characterized by two angles. We will characterize them as a rotation along the $X-Y$ plane, and a rotation along the $Y-Z$ plane. Derive the equations that transform a vector $[x, y, z]^\top$ to a new vector $[x’, y’, z’]^\top$ by rotating it counterclockwise by angle $\theta$ along the $X-Y$ plane and by an angle $\delta$ along the $Y-Z$ plane. Represent this as a matrix transformation of the column vector $[x, y, z]^\top$ to the column vector $[x’, y’, z’]^\top$. The matrix that transforms the former into the latter is a rotation matrix.

Projecting Instrument Notes:

For this problem you will transform the harmonica notes of problem 1 to piano notes, by a matrix transform. The piano notes can be downloaded from here. Note that, in this case, you don’t know which piano notes correspond to which notes from the harmonica. There are 3 parts to this problem:

• Find the piano note corresponding to each note from the harmonica. The dot product is your friend.
• Find a transformation that converts the harmonica notes to piano notes. To do so, you must list the spectra for all harmonica notes as a matrix $H$. List the corresponding piano notes as a matrix $P$. There must be a one-to-one correspondence between the notes represented by the columns of $H$ and those represented by the columns of $P$. The desired transformation is a matrix $M$ such that $MH \approx P$. Provide the matrix $M$.
• Synthesize the music piece from Problem 1, using both the actual piano notes and those obtained by transforming the harmonica notes. Submit both synthesized recordings.