Monday, October 24, 2016

Audio classification with a convolutional neural network

Edited 20161120

All resources referenced in this post can be found here.

As the title says I have been experimenting with audio classification with a convolutional neural network. Input to the network is loudness adjusted spectrograms. The spectrograms cover 65-3500Hz for two seconds. The temporal resolution is 40ms and the frequency resolution is ~11Hz, 48x319 values.

The network looks like this:

80 6x250 (240ms x 2692Hz) filters that are supposed to capture overtone features (timbre)
ReLU
3x1 max pooling to keep the network size down
40 8x70 (960ms x 754Hz) filters that are supposed to capture longer temporal features
ReLU
Max pooling to one value

The classes are populated from the layer 2 outputs according to a weight vector for each class and then converted to probabilities using softmax. The network contains ~1.9 million trainable variables and performs ~377 million operations.

The loss function is the mean of the cross entropies between the expected probability distribution and the actual probability distribution for each example.

For more detailed information check out audio_classifier.py.

To test it all out I decided to train the network to recognize Nathan Englander's voice. To do this I used two classes. One class for Nathan's voice, which was trained on 50 minutes of Nathan's voice from three different recordings. Another class for everything else, which was trained on 5 hours of audio not containing Nathan's voice (music, sound from Youtube and movies, book readings, etc).

python audio_classifier.py --train 2 nathan_englander.raw other.raw

The network quickly achieved 100% accuracy on the training data, something like 10-20 hours on my i5-4460.

The following images show the performance on the test data. None of the test data is present in the training data. No person, except Nathan Englander, and no recording exists in both data sets.

python audio_classifier.py 2 nathan_englander_interview.raw

Interview with Nathan Englander
Nathan Englander reading a book
Interview with Ariana Grande
Dialog from the Ex Machina movie
Interview with Samuel L. Jackson
Skrillex - Bangarang
Will Self reading a book

No comments:

Post a Comment