Support libcanberra for audio notification
ClosedPublic

Authored by broulik on Jul 26 2018, 8:54 AM.

Details

Summary

This library is meant to play event sounds and is significantly lighter than using Phonon.
Based on a draft patch made by Harald Sitter

Test Plan

We probably lose the ability to play sound on Windows and Mac? How does it work there normally? (We could just make it do QApplication::beep() if a sound is configured :p)

  • Warning sounds in e.g. KMessageBox play just fine
  • KNotification events also play sound just fine
  • Deleting/closing a notification manually stops the sound
  • Changing "Notification sounds" in volume applet affects those sounds like it should
  • Verified that it works when threaded (hence the QMetaObject::invokeMethod in the callback handler)
  • LoopSound also work but probably won't create uninterrupted playback anymore. But I have seen only one user of that in lxr which is some playground dialer app..

I'm wondering if we can also get rid of the custom QStandardPaths lookup we have in there given libcanberra is meant to support xdg sound scheme spec?
findCanberra can probably move to ECM? Plasma-pa now has it, plasma-desktop used to have it, now we have it here, too.

Verified that without canberra it uses the Phonon one. Is there a CMake way to say "one of those two is required"? Right now you could build KNotifications without any sound support which wasn't possible before, despite the #ifdefs everywhere

Diff Detail

Repository
R289 KNotifications
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
broulik created this revision.Jul 26 2018, 8:54 AM
Restricted Application added a project: Frameworks. · View Herald TranscriptJul 26 2018, 8:54 AM
Restricted Application added a subscriber: kde-frameworks-devel. · View Herald Transcript
broulik requested review of this revision.Jul 26 2018, 8:54 AM

Canberra's primary advantage over phonon is that it can directly use pulseaudio/alsa, bypassing the entire sound system middleware madness and leveraging PA's builtin caching mechanism. i.e. if a sound notification is played a bunch of times it will be cached in-between, making subsequent playbacks much cheaper, until the sound gets pressured out of the cache. This is opposed to phonon, where a playback is a full state cycle, such that playing the same sound multiple times means reading&&piplining it over and over again, even when the raw output PCM didn't even change. Also full-disclosure: there was a proposal for implement this functionality in Phonon, but that was years ago and never materialized, so I think canberra is the way forward here.

We probably lose the ability to play sound on Windows and Mac? How does it work there normally? (We could just make it do QApplication::beep() if a sound is configured :p)

from IRC:

  • gstreamer is technically x-platform, whether one actually wants to attempt building and using it on these platforms is another question I suppose
  • one could write native plugins for canberra on windows/mac; or write a vlc plugin as libvlc is generally easier to use on windows I find (or we could ask videolan if they want to write it for us ;))

I'm wondering if we can also get rid of the custom QStandardPaths lookup we have in there given libcanberra is meant to support xdg sound scheme spec?

Probably yes. Three problems come to mind:

  • our sounds aren't actually in the form of a theme (yet!)
  • notification sounds may be paths to random files (this mostly only maters as far as which CA properties need setting)
  • we have no capability to actually set a theme from the UI, and our icon names are not really theme compliant (in that we have sounds that are in xdg-sound-theme, but have a different name)

All things consider what we could do as a first step is pack our sounds into a proper theme and adjust our file naming, then use that as "default" theme and whenever a notification sound is not an absolute path we'll let canberra resolve it against our theme.

We probably lose the ability to play sound on Windows and Mac?

Does libcanberra (still) require pulseaudio or can it work with different backends? It not, a move to using it exclusively for sound would get a big fat -1 from me. Pulseaudio is problematic on Mac and more alien than DBus; it simply shouldn't be required.
(I have gripes with it on Linux too; too often I get the sound notifications way after the event simply because the PA daemon had been swapped out or otherwise unavailable.)

How does it work there normally?

It works just fine with the gstreamer or libVLC backends. Building libVLC for 3rd party use is a bit of an adventure on Mac but I have made a MacPorts port for it.

(We could just make it do `QApplication::beep()` if a sound is configured :p)

I proposed that once as a fallback for misconfigured alerts and caused knee-jerk reactions in certain reviewers who remembered the days of the PC tweeter beep...

...

  • LoopSound also work but probably won't create uninterrupted playback anymore. But I have seen only one user of that in lxr which is some playground dialer app..

The Multimedia/Sound KCM has a few test buttons; do those work?

IIUC KNotifications already has the logic for finding sound theme files, so if a lighter sound backend is required, why not use a truly cross-platform library like PortAudio?

FWIW, using QSP for finding those theme files instead of some independent implementation has the advantage that they are expected in a QSP-compatible location. Or will have that advantage, the day a solution is found for working with the non-XDG QSP locations on Mac and MSWin.

Does libcanberra (still) require pulseaudio or can it work with different backends?

According to its website "It comes with several backends (ALSA, PulseAudio, OSS, GStreamer, null)"

  • gstreamer is technically x-platform, whether one actually wants to attempt building and using it on these platforms is another question I suppose

Gstreamer builds and functions on Mac, but AFAIK comes with a whole bunch of dependencies to install and run.

According to its website  "It comes with several backends (ALSA, PulseAudio, OSS, GStreamer, null)"

Which are all either Linux-specific or *nix-desktop oriented with ties to X11 and probably even GTk.

Justifying a change with loss of weight while at the same time introducing a dependency on an entire rival GUI middleware seems a bit awkward to me ;)

I didn't think of this before, but do you actually need a 3rd party library for playing alert sounds? Wouldn't QtMultimedia do the trick? I just checked: it's perfectly native on Mac, which means it almost has to be on MSWin too. It also provides control over the device to use (nice to be able to send notifications to another device than the one for audio playback; something that still doesn't work with Phonon on Mac).
I'd start looking into this myself if we weren't in the middle of an endless heatwave :-/

I didn't think of this before, but do you actually need a 3rd party library for playing alert sounds?

Canberra's primary advantage over phonon is that it can directly use pulseaudio/alsa, bypassing the entire sound system middleware madness and leveraging PA's builtin caching mechanism. i.e. if a sound notification is played a bunch of times it will be cached in-between, making subsequent playbacks much cheaper, until the sound gets pressured out of the cache.

According to its website  "It comes with several backends (ALSA, PulseAudio, OSS, GStreamer, null)"

Which are all either Linux-specific or *nix-desktop oriented with ties to X11 and probably even GTk.

GStreamer works on all major operating systems such as Linux, Android, Windows, Max OS X, iOS, as well as most BSDs, commercial Unixes, Solaris, and Symbian. It has been ported to a wide range of operating systems, processors and compilers. It runs on all major hardware architectures including x86, ARM, MIPS, SPARC and PowerPC, on 32-bit as well as 64-bit, and little endian or big endian.

https://gstreamer.freedesktop.org/features/

That's a nicety, not necessarily a requirement, at least not for playing notification sounds. Just how often do you get sound alerts at such high rates that caching becomes unavoidable, before you get mad and resolve the issue or yank the plug? I think that proper cross-platform support without requiring multiple 3rd-party libraries is more important than being able to leverage PA caching directly.

With QMM, playing audio can be as simple as

player = new QMediaPlayer;
player->setMedia(QUrl::fromLocalFile("/path/to/audio.whatever"));
player->play();

That doesn't look like "sound middleware madness" to me, is probably much easier than leveraging "sound lowerware" directly and the documentation suggests this class has buffering support.
The lower-level and audio-only QAudioOutput requires a bit more coding but I'd say it's still preferrable over introducing new external dependencies.

If not already the case, PA caching can probably be leveraged from QMediaPlayer, and if that's justifiable it shouldn't be hard to upstream such a modification.

GStreamer works on all major operating systems such as Linux, Android, Windows, Max OS X

I know it works on Mac (and MSWin) but that doesn't make it "native". I also know what it depends on (the basic MacPorts port:gstreamer1 will pull in GTk3). I'd say that's not even acceptable for a KDE requirement on any Unix variant; introducing such a dependency on Mac (or MSWin) is going against the current flow of making KDE a proper and as-native-as-possible citizen on those platforms.

Anyway, if using QtMultiMedia cannot be considered please make it a build option to use just QApplication::beep() for sound notifications so that packagers can decide whether or not they want to introduce dependencies on libcanberra and everything it needs. On Mac at least QApplication::beep() will trigger the user-selected system notification sound (which can but doesn't have to be a horrible beep).

Impressive work, Kai! However I rather like the idea of improving QTMultiMedia's sound handling features rather than working around the lack of them by using a different library.

broulik updated this revision to Diff 38512.Jul 26 2018, 2:49 PM
broulik retitled this revision from Port audio notification to libcanberra to Support libcanberra for audio notification.
broulik edited the test plan for this revision. (Show Details)
  • Keep Phonon as fallback when libcanberra isn't provided
rjvbb added a comment.Jul 26 2018, 4:07 PM

However I rather like the idea of improving QTMultiMedia's sound handling features rather than working around the lack of them by using a different library.

Thanks Nathan for rewording my argument better than I managed.

Is there a CMake way to say "one of those two is required"?

I'm not sure if this is still an actual open question but I think the usual way would be to make both optionan and then to check and raise an error if neither is found.

Impressive work, Kai! However I rather like the idea of improving QTMultiMedia's sound handling features rather than working around the lack of them by using a different library.

That's reinventing the wheel to a point. There is a perfectly reasonable abstraction layer already. It's libcanberra. A library specifically designed for notification sounds.

That's reinventing the wheel to a point.

Creating Qt after Gtk or vice-versa was also reinventing the wheel to a much larger extent - so is adding more and more OS features to Qt as has been going on for years.

libcanberra comes from "the other" GUI universe. My arguments against depending on it stand, I think, including the one about not increasing and probably reducing the number of dependencies.

Improving QtMultimedia with useful features (as far as that's even necessary!) may be reinventing the wheel but will have benefits for much more than just KNotifications (because, you know, *useful*).

I'd love to look into using QAudioOutput (esp. if someone a bit more familiar with audio programming is willing to provide guidance), but after summer. (Thinking beyond the issue at hand: phonon works just fine, but its backends have been bothering me for a while and a QtMultiMedia backend could be an interesting idea to investigate as an easier cross-platform option.)

Not using something because it's from a "rival GUI" is not a valid argument. FWIW plasma-pa already uses Glib and Canberra. QtMultimedia uses GStreamer as backend on Unix, so it pulls in glib anyway. The only advantage, if any, of using QtMultimedia is that it uses native APIs on Mac/Windows, but Canberra over GStreamer works there as well.

That's reinventing the wheel to a point.

Creating Qt after Gtk or vice-versa was also reinventing the wheel to a much larger extent - so is adding more and more OS features to Qt as has been going on for years.

libcanberra comes from "the other" GUI universe. My arguments against depending on it stand, I think, including the one about not increasing and probably reducing the number of dependencies.

Your arguments were against depending on gstreamer. Canberra is an abstraction. You could shove coreaudio or quicktime behind it if you wanted.

phonon works just fine

That's not true. If it worked fine, this patch would not exist, as I would not have apps freeze for seconds on teardown of Phonon objects, or latencies beyond all comparison

But Currently, libcanberra is tested on Linux only. 😢 Plasma-pa and plasma-whatever can use anything and be Linux-centric, KNotifications is a framework, isn't it?
++vote for QtMultimedia option...

Currently, libcanberra is tested on Linux only.

I restored the Phonon option for when canberra isn't available on the platform.

rjvbb added a comment.Jul 28 2018, 7:43 AM

[I didn't get to send this yesterday]

Not using something because it's from a "rival GUI" is not a valid argument.

It is IMVHO if "it" introduces a dependency on another GUI middleware. Libcanberra does that to the best of my knowledge.

Canberra over GStreamer works there as well.

Only if you accept dependencies that shouldn't be required if you want to keep things as native as possible. The fact that a plasma project already uses libcanberra shouldn't be used as an argument for using it in a *framework*. Plasma is exclusive to non-Apple Unix (and in fact almost to Linux), and it's probably irrealistic to think that one could construct a completely GTk-free distribution on such systems (= the library will probably be there anyway).
KNotifications is a Tier1 framework with a cross-platform vocation. For me that means that a solution with QtMultimedia should be preferred if anyway possible. Regardless what that component depends on itself... (Alexey beat me to it I see :))

NB: I'm not talking about glib. That's a platform-agnostic support library that Qt uses even on Mac (I don't know about MSWin).

You could shove coreaudio or quicktime behind it if you wanted.

*That* would be reinventing the wheel. Why would you want to do that in a Qt-based universe if Qt already has a component where this has been done?
Quicktime is dead, btw.

That's not true.

It is if you leave my remark in its (generic) context... And sound latencies can hardly be avoided with a 100% guarantee with asynchronous playback over an independent daemon process.

I restored the Phonon option for when canberra isn't available on the platform.

I haven't checked, but I'd appreciate if that could also be done through a cmake option. In packaging systems like MacPorts and HB it's perfectly possible to have libcanberra and/or pulseaudio installed for Gnome apps that cannot do without them, but still not want to let other software use them (opportunistic dependencies). Alternatively, CMake has a DISABLE* backdoor for the basic find_package(), not very elegant but maybe it can do the trick?

broulik abandoned this revision.Jul 30 2018, 7:57 AM

I haven't checked, but I'd appreciate if that could also be done through a cmake option. In packaging systems like MacPorts and HB it's perfectly possible to have libcanberra and/or pulseaudio installed for Gnome apps that cannot do without them, but still not want to let other software use them (opportunistic dependencies)

I don't really care about your artificially created problems. You *have* libcanberra but want to use it for one project and not the other. Sorry, but no. How about I cannot do without them like your Gnome apps? Then what. (Can't you override those variables externally using -D... anyway?)

But whatever, I'm out. Was an attempt to fixup the subpar experience we had with notification sounds and I'll happily continue to use use this patch locally but I'm tired of this.

(Oh btw it's not like I didn't try, I originally ported all of this to QtMultimedia but QAudioEffect which is for low-latency sound effects only supports WAV sounds and QMediaPlayer had significant overhead and initialization times as well which is why I eventually ended up using canberra)

rjvbb added a comment.Jul 30 2018, 8:20 AM
(Oh btw it's not like I didn't try, I originally ported all of this to QtMultimedia but `QAudioEffect` which is for low-latency sound effects only supports WAV sounds and `QMediaPlayer` had significant overhead and initialization times as well which is why I eventually ended up using canberra)

See, it would have been constructive to mention that earlier.

FWIW, I think we just want to play a file here, not juggle with audio effects.
https://stackoverflow.com/questions/41197576/how-to-play-mp3-file-using-qaudiooutput-and-qaudiodecoder.

Anyone else interested in the idea of having a QtMultiMedia Phonon backend that (a priori) uses the most low-level APIs from that Qt component?

rjvbb added a comment.Jul 30 2018, 8:31 AM
I don't really care about your artificially created problems. You *have* libcanberra but want to use it for one project and not the other. Sorry, but no.

Oh, and do you have to show ignorance? This is a very common problem in software packaging, part of what allows your distribution to warn you something you're trying to uninstall is a dependency for something you might want to keep installed.

As I said, yes, there is a CMake mechanism to disable specific find_package calls but it's rather confidential, ugly and useless with custom findfoo.cmake implementations that bypass find_package.

So, what shall we do with this? Either we take it or leve it, I won't be the one changing it to QtMultimedia as canberra is *the* way to go.

+1 for Canberra

+1 too; we can always work on upstream Qt improvements later.

cfeck added a subscriber: cfeck.Jul 31 2018, 1:19 PM

+1 for supporting libcanberra.

broulik reclaimed this revision.Jul 31 2018, 1:30 PM
rjvbb added a comment.Jul 31 2018, 1:55 PM

I have no other objections than the ones above, as long as there's a fallback to NOT using libcanberra which can be enforced without having to patch the code (= via CMAKE_DISABLE_FIND_PACKAGE_FOO or a dedicated option).

sitter requested changes to this revision.Jul 31 2018, 2:07 PM

Code looks mostly ok. As discussed on IRC my main concern is that the current code ignores the return values of just about every ca_* method. This is a bit problematic from a diagnostics POV as we'll have no way to figure out why things went wrong if they go wrong. I'd like to at least have some logging added with a helper method. Bonus points obviously for not exploding if e.g. ca_context_create failed.

suggestion:

bool validResult(int result)
{
if (result == 0) {
return true;
}

qCWarning(CATEGORYY) << ca_strerror(result));
return false;
}
CMakeLists.txt
87

I'd prefer if this was moved up, before finding Phonon, and then find phonon in the else branch of CANBERRA_FOUND.

The rationale is that (e.g.) all of neon's tech which auto detects missing cmake dependencies can only do it's thing automatically if only actually missing dependencies are reported as such. With the current structure both Phonon and Canberra would always be in the cmake summary, even though Phonon being missing is irrelevant if canberra was found. OTOH if only phonon is found we'd still want to raise a stink because canberra is our preferred choice.

src/notifybyaudio_canberra.cpp
113

Looks superfluous?

119

This may benefit from an explicit Qt::QueuedConnection. I am not sure foreign-thread detection is 100% reliable with C call chains. Specifying it certainly is more explicit about the intent when reading the code though.

This revision now requires changes to proceed.Jul 31 2018, 2:07 PM
broulik updated this revision to Diff 38841.Jul 31 2018, 2:10 PM
  • More error handling
broulik updated this revision to Diff 38842.Jul 31 2018, 2:15 PM

MOAAAR error handling

broulik updated this revision to Diff 38843.Jul 31 2018, 2:25 PM
  • Require Phonon so that a sound system is preferred (it is only searched for if Canberra isn't)
sitter accepted this revision.Jul 31 2018, 2:34 PM

Well, I still think the result handling is more complicated than it needs to be, but whatever. LGTM for landing after 5.49 tagging (expected August 4th IIRC).

https://cgit.kde.org/plasma-phone-components.git/plain/sounds/sitter/ohits.ogg

This revision is now accepted and ready to land.Jul 31 2018, 2:34 PM
rjvbb added inline comments.Jul 31 2018, 3:06 PM
CMakeLists.txt
87

Preferred choice on Linux and other non-Mac Unix variants (or even only Linux because as mentioned earlier, libcanberra is only tested there). It can't be the preferred choice when no native backend is available, IMHO.

Wouldn't the cleanest way to achieve that be the use of a USE_CANBERRA CMake option that defaults to ON on platforms where the dependency can be expected? Then you can make both backends hard dependencies.

src/notifybyaudio_canberra.cpp
119

That also corresponds to a Qt guideline.

apol added a subscriber: apol.EditedJul 31 2018, 3:10 PM
This comment has been deleted.
CMakeLists.txt
87

I'd say this is perfectly fine, already not optional. If you want to disable it, pass -DCMAKE_DISABLE_FIND_PACKAGE_Canberra=OFF.

I really don't see what the big fuss is all about.

rjvbb added a comment.Jul 31 2018, 3:46 PM

I really don't see what the big fuss is all about.

Did you read Harald's remark (about raising a neon fuss) to which I replied?!

dfaure added a comment.Aug 1 2018, 9:28 AM

Just FYI, canberra is already an optional dependency of kmix and plasma-pa. https://lxr.kde.org/search?_filestring=CMakeLists.txt&_string=canberra

rjvbb added a comment.Aug 1 2018, 11:45 AM

That was already mentioned :)

I see no problem with those components depending on something that is probably inevitable in a "linuxy desktop" environment.

Also FYI: I built the latest canberra (PA support disabled) and gstreamer1 on Mac and am not getting any sound out of canberra-gtk-play (a memory error instead).
Not that QtMultimedia is such an evident alternative (if QMediaPlayer has too much overhead): QAudioDecoder only works when you activate Qt's gstreamer support (and apply a simple patch so that the plugins are actually built) ...

This revision was automatically updated to reflect the committed changes.