"<div align=\"right\"><a href=\"https://people.epfl.ch/paolo.prandoni\">COM418 - Computers and Music</a></div>\n",
"<br />\n",
"\n",
"# I like DSP and I Feel Fine"
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 113,
"metadata": {
"slideshow": {
"slide_type": "-"
},
"tags": []
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"from IPython.display import Audio\n",
"from scipy import signal\n",
"from scipy.io import wavfile\n",
"import ipywidgets as widgets\n",
"\n",
"plt.rcParams['figure.figsize'] = 14, 4 \n",
"matplotlib.rcParams.update({'font.size': 14})"
]
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 114,
"metadata": {
"scrolled": true,
"slideshow": {
"slide_type": "-"
}
},
"outputs": [],
"source": [
"DEFAULT_SF = 24000 "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. \"I Feel Fine\" by the Beatles\n",
"\n",
"\n",
"<img src=\"beatles.png\" alt=\"Drawing\" style=\"float: left; width: 200px; margin: 20px 30px;\"/>\n",
" \n",
"\n",
" * recorded on October 18, 1964\n",
" * one of the first (if not the first) example of distortion via feedback\n",
" \n",
" \n",
"> _\"I defy anybody to find a record... unless it is some old blues record from 1922... that uses feedback that way. So I claim it for the Beatles. Before Hendrix, before The Who, before anybody. The first feedback on record.\"_ -- John Lennon\n",
"Note the big difference in spectral content between the undistorted and the distorted sound: since we know that linear filters cannot add frequency components, the system is clearly non linear!"
"If we look at the location of the main peak in the spectrum, we can see that its frequency is about 110Hz, which corresponds to the pitch of the guitar's A string. We can also see that the clean tone has only a few detectable overtones."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Simulating the guitar"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.1. The vibrating string\n",
"\n",
"We need to model the string as a input/output system:\n",
"\n",
" * input is the action on the string (picking and/or sound from the amp)\n",
" * output is a signal compatible with the physical properties of a vibrating string."
" Your browser does not support the audio element.\n",
" </audio>\n",
" "
],
"text/plain": [
"<IPython.lib.display.Audio object>"
]
},
"execution_count": 149,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Audio(s, rate=DEFAULT_SF)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.3. The fret buzz"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, we can see the nonlinearity as being the string hitting the fretboard as its vibration becomes wider and wider. The super fast hitting generates high frequencies. Visually, the nonlinearity looks like a hard clipping waveshaper, except that when the amplitude reaches one (that is, when the string reaches its maximum amplitude), the value of the output bouces back in the $[-1,1]$ range instead of staying at the value 1, as if the string was boucing back on the fretboard very quickly before going back to its normal vibration range."
"Although we have already studied the Karplus-Strong algorithm as an effective way to simulate a plucked sound, in this case we need a model that is closer to the actual physics of a guitar, since we'll need to drive the string oscillation in the feedback loop. \n",
"\n",
"In a guitar, the sound is generated by the oscillation of strings that are both under tension and fixed at both ends. Under these conditions, a displacement of the string from its rest position (i.e. the initial \"plucking\") will result in an oscillatory behavior in which the energy imparted by the plucking travels back and forth between the ends of the string in the form of standing waves. The natural modes of oscillation of a string are all multiples of the string's fundamental frequency, which is determined by its length, its mass and its tension (see, for instance, [here](http://www.phys.unsw.edu.au/jw/strings.html) for a detailed explanation). This image (courtesy of [Wikipedia](http://en.wikipedia.org/wiki/Vibrating_string)) shows a few oscillation modes on a string:\n",
"These vibrations are propagated to the body of an acoustic guitar and converted into sound pressure waves or, for an electric guitar, they are converted into an electrical waveform by the guitar's pickups."
"plt.title(\"Harmonics of the guitar extract\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plot_spectrum(x[10000:40000], fs, 1000)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Indeed we can see that the frequency content of the sound contains multiples of a fundamental frequency at 110Hz, which corresponds to the open A string on a standard-tuning guitar. \n",
"\n",
"From a signal processing point of view, the guitar string acts as a resonator resonating at several multiples of a fundamental frequency; this fundamental frequency determines the _pitch_ of the played note. In the digital domain, we know we can implement a resonator at a single frequency $\\omega_0$ with a second-order IIR of the form \n",
"i.e. by placing a pair of complex-conjugate poles close to the unit circle at an angle $\\pm\\omega_0$. A simple extension of this concept, which places poles at _all_ multiples of a fundamental frequency, is the **comb filter**. A comb filter of order $N$ has the transfer function\n",
"It is easy to see that the poles of the filters are at $z_k = \\rho e^{j\\frac{2\\pi}{N}k}$, except for $k=0$ where the zero cancels the pole. For example, here is the frequency response of $H(z) = 1/(1 - (0.99)^N z^{-N})$ for $N=9$:"
"plt.title(\"Harmonics of the generated guitar\")\n",
"plt.show()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. The amplifier"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the \"I Feel Fine\" setup, the volume of the amplifier remains constant; however, because of the feedback, the input will keep increasing and, at one point or another, any real-world amplifier will be driven into saturation. When that happens, the output is no longer a scaled version of the input but gets \"clipped\" to the maximum output level allowed by the amp. We can easily simulate this behavior with a simple memoryless clipping operatora \"hard clipper\" as we implemented in the Nonlinear Modelling notebook: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can easily check the characteristic of the amplifier simulator: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"While the response is linear between -0.9 and +0.9, it is important to remark that the clipping introduces a nonlinearity in the processing chain. In the case of linear systems, sinusoids are eigenfunctions and therefore a linear system can only alter a sinusoid by modifying its amplitude and phase. This is not the case with nonlinear systems, which can profoundly alter the spectrum of a signal by creating new frequencies. While these effects are very difficult to analyze mathematically, from the acoustic point of view nonlinear distortion can be very interesting, and \"I Feel Fine\" is just one example amongst countless others. \n",
"\n",
"It is instructive at this point to look at the spectrogram (i.e. the STFT) of the sound sample (figure obtained with a commercial audio spectrum analyzer); note how, indeed, the spectral content shows many more spectral lines after the nonlinearity of the amplifier comes into play.\n",
"The last piece of the processing chain is the acoustic channel that closes the feedback loop. The sound pressure waves generated by the loudspeaker of the amplifier travel through the air and eventually reach the vibrating string. For feedback to kick in, two things must happen:\n",
"\n",
"* the energy transfer from the pressure wave to the vibrating string should be non-negligible\n",
"* the phase of the vibrating string must be sufficiently aligned with the phase of the sound wave in order for the sound wave to \"feed\" the vibration.\n",
"\n",
"Sound travels in the air at about 340 meters per second and sound pressure decays with the reciprocal of the traveled distance. We can build an elementary acoustic channel simulation by neglecting everything except delay and attenuation. The output of the acoustic channel for a guitar-amplifier distance of $d$ meters will be therefore\n",
"\n",
"$$\n",
"\ty[n] = \\alpha x[n-M]\n",
"$$\n",
"\n",
"where $\\alpha = 1/d$ and $M$ is the propagation delay in samples; with an internal clock of $F_s$ Hz we have $M = \\lfloor d/(c F_s) \\rfloor$ where $c$ is the speed of sound."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Play it, Johnny"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"OK, we're ready to play. We will generate a few seconds of sound, one sample at a time, following these steps:\n",
"\n",
"* generate a guitar sample\n",
"* process it with the nonlinear amplifier\n",
"* feed it back to the guitar via the acoustic channel using a time-varying distance\n",
"\n",
"During the simulation, we will change the distance used in the feedback channel model to account for the fact that the guitar is first played at a distance from the amplifier, and then it is placed very close to it. In the first phase, the sound will simply be a decaying note and then the feedback will start moving the string back in full swing and drive the amp into saturation. We also need to introduce some coupling loss between the sound pressure waves emitted by the loudspeaker and the string, since air and wound steel have rather different impedences. \n",
"# the \"coupling loss\" between air and string is high. Let's say that\n",
"# it is about 80dBs\n",
"COUPLING_LOSS = 0.0001\n",
"\n",
"# John starts 3m away and then places the guitar basically against the amp\n",
"# after 1.5 seconds\n",
"START_DISTANCE = 3 \n",
"END_DISTANCE = 0.05\n",
"\n",
"N = int(fs * 5) # play for 5 seconds\n",
"y = np.zeros(N) \n",
"x = [1] # the initial plucking\n",
"# now we create each sample in a loop by processing the guitar sound\n",
"# thru the amp and then feeding back the attenuated and delayed sound\n",
"# to the guitar\n",
"for n in range(N):\n",
" y[n] = hit_fret(g.play(x), 1/0.8415)\n",
" x = [COUPLING_LOSS * f.get(y[n], START_DISTANCE if n < (1.5 * fs) else END_DISTANCE)]\n",
" \n",
"Audio(data=y, rate=fs, embed=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Pretty close, no? Of course the sound is not as rich as the original recording since\n",
"\n",
"* real guitars and real amplifiers are very complex physical system with many more types of nonlinearities; amongst others:\n",
" * the spectral content generated by the string varies with the amplitude of its oscillation\n",
" * the spectrum of the generated sound is not perfectly harmonic due to the physical size of the string\n",
" * the string may start touching the frets when driven into large oscillations\n",
" * the loudspeaker may introduce additional frequencies if driven too hard\n",
" * ...\n",
"* we have neglected the full frequency response of the amp both in linear and in nonlinear mode\n",
"* it's the BEATLES, man! How can DSP compete?\n",
"\n",
"Well, hope this was a fun and instructive foray into music and signal processing. You can now play with the parameters of the simulation and try to find alternative setups: \n",
"\n",
"* try to change the characteristic of the amp, maybe using a sigmoid (hyperbolic tangent)\n",
"* change the gain, the coupling loss or the frequency of the guitar\n",
"* change John's guitar's position and verify that feedback does not occur at all distances."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"fig, ax = plt.subplots(figsize=figsize)\n",
"\n",
"ax.magnitude_spectrum(y, Fs=fs, scale=\"dB\", label=\"Distorted sine FFT\")\n",