AM/FM Modeling of Harmonic Sounds

Demonstration: glissando

Download all the audio files from this page in a single ZIP archive (3MB).

This demo shows, how the technique is capable of representing the noisy character of a bowed string sound. With the help of fundamental frequency tracking over a wide range, a major part of the macro and micro-modulations of individual partials is preserved during glissando and vibrato. The spectrograms and audio examples of sounds reconstructed without a dedicated model for the background noise reveal that indeed, a significant part of the noise inherent in the vibrations of strings is carried within common IF and IA instantaneous parameters. Of course, the difference is somewhat masked with an additional model for the background noise (We use a 10-th order warped LPC model in this experiment). Nevertheless, even with background noise, there is an audible difference in quality.

For a comparison, we show also the sounds reconstructed from the spectral modeling synthesis technique (SMS) by X.Serra. Please note, that while the SMS model is harmonic, (in a sense that tracking is guided by the detected F0), it generates incoherent frequency trajectories that must be encoded individually. We also show a comparison with the bandwidth-enhanced sinusoidal model (LORIS) by K.Fitz which generates bandwidth-enhanced partials modulated by random noise. For fair comparison, we performed a partial selection operation ("distill" command of the LORIS software) that constraints the partials to harmonic multiples of the fundamental frequency.

Original sound (WAV file, 44.1kHz, 16bit, 360kB)

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from synthesis based on F0 + Harmonic Envelope subsampled 1:500
(A baseline incoherent heterodyne analysis, acting as a mock-up of a perfect harmonic sinusoidal model, without residual noise)

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from synthesis based on instantaneous F0 + Harmonic Envelope subsampled 1:500 + prototype signal. NOTE: no residual noise is modeled in this example.

Problems revealed in this example:

A part of high frequency content is not properly captured in the first stage of the sound, since only 18 partials are tracked. This lack is compensated by the higher background noise energy (see below), since noise is modeled upon the spectral difference between the original and the reconstructed signal. Note, that this noise energy is located at 15kHz and thus it is almost indistinguishable from tonal partials.

A filter with constant bandwidth is used throughout the signal. It is apparently too narrowband in the second stage of the sound, because the frequency distance between partials significantly increases due to the rising pitch. A more flexible solution should employ a time-varying filter.

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from synthesis based on F0 + Harmonic Envelope subsampled 1:500 + a noise residual modeled by 10-th order warped LPC (A mock-up of a harmonic sinusoidal + noise model)

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from synthesis based on instantaneous F0 + Harmonic Envelope subsampled 1:500 + prototype signal. As above, the noise residual is modeled by 10-th order WLPC.

Reconstructed sound (WAV file, 44.1kHz, 16bit, 395kB) obtained from the SMS technique with frame rate 88Hz (hop = 500 samples) and no residual noise. Please note the chaotic frequency variations of high-order partials which is caused by estimation problems at low SNR and tracking errors. The audible effect of these variations is reflected by the "warbling" sound.

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from the SMS (as above) + background noise modeled using 10-order WLPC. We used our own noise model in this example, since the traditional LPC-based model in the SMS software produced too much artifacts. Note that the artifacts are not masked by the noise.

Reconstructed sound (WAV file, 44.1kHz, 16bit, 360kB) obtained from the LORIS technique with bandwidth association region width of 200Hz and partials constrained to harmonic. Unfortunately, the harmonic restriction leads to very audible artifacts. The amount of noise is apparently over-estimated. This is the best result we could obtain from the LORIS software. If you know, how to obtain a better result for this sound, please tell us.