This change was a long time in the making ever since we obtained sample
rate awareness in the system. Now, each client has its own sample rate,
accessible via new IPC APIs, and the device sample rate is only
accessible via the management interface. AudioServer takes care of
resampling client streams into the device sample rate. Therefore, the
main improvement introduced with this commit is full responsiveness to
sample rate changes; all open audio programs will continue to play at
correct speed with the audio resampled to the new device rate.
The immediate benefits are manifold:
- Gets rid of the legacy hardware sample rate IPC message in the
non-managing client
- Removes duplicate resampling and sample index rescaling code
everywhere
- Avoids potential sample index scaling bugs in SoundPlayer (which have
happened many times before) and fixes a sample index scaling bug in
aplay
- Removes several FIXMEs
- Reduces amount of sample copying in all applications (especially
Piano, where this is critical), improving performance
- Reduces number of resampling users, making future API changes (which
will need to happen for correct resampling to be implemented) easier
I also threw in a simple race condition fix for Piano's audio player
loop.
For example, on https://xboygeniusx.bandcamp.com/album/the-record, a
song with a duration of 03:52 would actually complete in 03:33 on my
machine. This issue only affects Ladybird on Lagom; on Serenity, we
already take the entire 03:52 to play the song.
It's currently possible to seek to the total sample count of an audio
loader. We must limit seeking to one less than that count.
This mistake was duplicated in both AudioCodecPluginSerenity/Ladybird,
so the computation was moved to a helper in the base AudioCodecPlugin.
When the default audio device changes on the host, it's convenient to
automatically switch to that device rather than needing to reload the
page to update.
We are currently forcing audio to play with a sample size of 16 bits. We
are also feeding the output audio device a hard-set amount of samples
without considering the actual size of its sample buffer. This would
cause a wide array of issues when playing audio elements. On my Linux
machine, we would hear some cracking; on my macOS machine, audio was
quite garbled.
We now write samples of the size requested by the output audio device.
We also limit the samples we provide to the audio device to however many
bytes are available in its buffer.
The main thread in the WebContent process is often busy with layout and
running JavaScript. This can cause audio to sound jittery and crack. To
avoid this behavior, we now drive audio on a secondary thread.
Note: Browser on Serenity uses AudioServer, the connection for which is
already handled on a secondary thread within LibAudio. So this only
applies to Lagom.
Rather than using LibThreading, our hands are tied to QThread for now.
Internally, the Qt media objects use a QTimer, which is forbidden from
running on a thread that is not a QThread (the debug console is spammed
with messages pointing this out). Ideally, in the future AudioServer
will be able to run for non-Serenity platforms, and most of this can be
aligned with the Serenity implementation.
This creates (and installs upon WebContent startup) a platform plugin to
play audio data.
On Serenity, we use AudioServer to play audio over IPC. Unfortunately,
AudioServer is currently coupled with Serenity's audio devices, and thus
cannot be used in Ladybird on Lagom. Instead, we use a Qt audio device
to play the audio, which requires the Qt multimedia package.
While we use Qt to play the audio, note that we can still use LibAudio
to decode the audio data and retrieve samples - we simply send Qt the
raw PCM signals.
