Basic Music Player Integration

[danry25]

Between calls I'd like to have a radio stream playing for people to listen to. Here is an example radio stream I'd like to play.

When an inbound call comes in, the radio stream should be muted so that the person making said call/announcement can be heard. After the call has ended, the radio stream should be unmuted so that employees and patrons can resume listening.

I think integration with mpd would cover this need, although I am open to using a different package if it would work better or be easier to integrate.

[mk-fg]

So I've been looking into pjsua + pulseaudio, which would be kinda neat for integrating it with multiple sound streams from whatever sources (and mixed in whatever ways), but it's probably more trouble than it's worth.

pjsua doesn't have native support for pulse, i.e. using libpulse, connecting to pulse server(s), creating sinks and individual playback streams there, controlling whatever parameters - nothing of the sort, only works with sound devices through portaudio lib, which uses either ALSA or OSS on linux.

Strictly one playback_dev + one capture_dev from there at a time, so one can forget about feeding sound from these "proper" sources for e.g. #5.

But pjsua can also just play stuff from wav, and record to wav, and assuming it's not too smart about it, one can always put a fifo (as in mkfifo) in place of a wav (or - worst-case - a trivial FUSE layer to "emulate" wav or patch the smart bits out), and feed stuff there from whatever software audio mixer or player.

pj* itself has internal plumbing for audo - conference bridges/slots, so once audio is fed into pjsua, it can multiplex it to whatever callers via that.

mpd has "fifo" output plugin, so that's probably How It Should Be Done.
"mute" is easier to control by unplugging that wav input from conference slot, but mpd should provide nicer fade-out and whatever effects, so sending "pause" there might be better.

#5 in this case becomes "multiple wav-fifo inputs from multiple players in diff conference bridges", #4 is just a matter of running a script or a cron job to occasionally switch tracks/streams in players, and #6 can probably be based on any existing mpd webui, adding knobs to also control paging.py there.

[mk-fg]

@danry25

• You did also mention that some extra sounds should be played for each listen-only caller when they join - it's just one "greeting" sample for everyone, played just once on join (with music muted during that and unmuted afterwards), or is there something more to it?

• If listen-only caller joins during an announcement from a listen+speak caller, should the greeting sample still be played?

• Is this mental model of "paging" service correct:

  One big conference call with everyone's mic in it "muted" by the hub (hence I call these callers "listen-only"), occasionally joined by "announcer" - a special caller, identified by some whitelisted sip id, who doesn't have mic muted and speaks to everyone else.

  As mentioned here, music gets fed into this conference call on "hub", and that's the only thing callers can hear until "announcer" joins. Music then gets muted until "announcer" hangs up, unmuted afterwards.

  Sounds right? As noted before, whole concept is kinda new to me (phones, lol).

• Wrt announcements - are there proper terms for things in the scenario I've described above?

  • What's the term for passive "listen-only" callers in paging?
  • What's the right word for "announcer" there?
  • Special term for this "greeting" sound sample?
  • Special term for stuff that plays in-between?
  • Thing that everyone is calling - the "hub"?
  • Input audio stream (from caller to call) is "mic", or sip has special word for it?
  • Is there a name for policy ("whitelist" above) that diffs plebs who only listen from announcer in paging?
  • "music gets ...(fed?)... into a conference call"? ;)

  Mostly curious, don't want to make up my own incomprehensible language for all this.

• I wonder, why did you pick pjproject as a base for the thing?

  You did mention on irc that you've used/tried SFLPhone before (ring.cx now?), and that has pulse integration, what was the worst thing there?

  Also mostly just curious, pjproject looks fine, if maybe a bit rusty.

Cheers.

[mk-fg]

Actually, one more important thing to mention is that PortAudio has JACK support.
Never used that, but afaik should be similar to pulse.

Problem though is that whatever weird mesh of sinks and streams one can have in jack, there's still doesn't seem to be any way to configure more than one sound input from portaudio into pjsua, so not much point using that either, I guess.

[mk-fg]

And there's an extensive and fairly recent (portaudio v19, 2015) third-party patch for pulse support: https://build.opensuse.org/package/view_file/home:illuusio:portaudio/portaudio/portaudio-pulseaudio-v19_20150116.patch?expand=0

Kinda cool, but same as with previous point, doesn't really matter unless one can actually send multiple streams from pulse into pjsua, which doesn't seem to be the case.

Might hopefully help with segfaults runing pjsua without pasuspend here though...

[danry25]

@mk-fg Let me try to answer your questions (sorry for the delay by the way!)

You did also mention that some extra sounds should be played for each listen-only caller when they join - it's just one "greeting" sample for everyone, played just once on join (with music muted during that and unmuted afterwards), or is there something more to it?

When a person calls in to make an announcement, they hear nothing, and don't need to hear or listen to anything. The user just wants to make an announcement (speak an announcement) in the store (like "I need help"), the file played is played out the speaker outputs from the computer this script runs on, and it is usually a short tone to get the listeners attention.

If listen-only caller joins during an announcement from a listen+speak caller, should the greeting sample still be played?

No, there usually will never be more than 1 person calling at any time, and callers are calling to speak, not listen.

Is this mental model of "paging" service correct:

One big conference call with everyone's mic in it "muted" by the hub (hence I call these callers "listen-only"), occasionally joined by "announcer" - a special caller, identified by some whitelisted sip id, who doesn't have mic muted and speaks to everyone else.

Nope, not quite right, there is no whitelisting and usually only one caller, the caller only wants to announce over the speakers attached to the computer, not listen.

As mentioned here, music gets fed into this conference call on "hub", and that's the only thing callers can hear until "announcer" joins. Music then gets muted until "announcer" hangs up, unmuted afterwards.

Not quite right, see prior comments.

Sounds right? As noted before, whole concept is kinda new to me (phones, lol).

Yeah, telephones and stuff can be confusing and silly, and the concept of this is a bit hard to explain. :)

Wrt announcements - are there proper terms for things in the scenario I've described above?

For some of these there are proper terms, see below.

What's the term for passive "listen-only" callers in paging?

There is no term, people calling in do not listen, ever.

What's the right word for "announcer" there?

Caller

Special term for this "greeting" sound sample?

Paging Tone, Paging Chime or klaxon are all commonly used words to describe this sound

Special term for stuff that plays in-between?

Radio stream

Thing that everyone is calling - the "hub"?

The caller hub doesn't exist, and isn't needed for our use case.

Input audio stream (from caller to call) is "mic", or sip has special word for it?

Yep, Mic would be what you could call the input audio stream from a caller.

Is there a name for policy ("whitelist" above) that diffs plebs who only listen from announcer in paging?

Nope, since there is no whitelist there is no special name for it.
"music gets ...(fed?)... into a conference call"? ;)

Mostly curious, don't want to make up my own incomprehensible language for all this.

Definitely, I'm sure there is special terms for all this stuff that even i don't know, but I'd like to keep our terminology simple and understandable.

I wonder, why did you pick pjproject as a base for the thing?

@thefinn93 picked pjproject, we can change away from pjproject though if needed.

You did mention on irc that you've used/tried SFLPhone before (ring.cx now?), and that has pulse integration, what was the worst thing there?

SFLPhone with Rhythmbox playing background music was great, except every few hours SFLPhone would disconnect from the SIP Server and not reconnect until I restarted SFLPhone. SFLPhone is also depreciated/discontinued, and ring.cx doesn't work yet on Debian.

Also mostly just curious, pjproject looks fine, if maybe a bit rusty.

Yeah, pjproject does some very silly things, its tripped @thefinn93 and myself up a few times with its oddities!

Comment #2

Actually, one more important thing to mention is that PortAudio has JACK support.
Never used that, but afaik should be similar to pulse.

Problem though is that whatever weird mesh of sinks and streams one can have in jack, there's still doesn't seem to be any way to configure more than one sound input from portaudio into pjsua, so not much point using that either, I guess.

If we can just capture the output from PJSUA and have the output from PJSUA override audio on all output devices, I think thats all we need. I know PulseAudio can do this, but not sure if its better to use the experimental PulseAudio support you mentioned in Comment #3 or try to figure out how to do it with PortAudio.

If you have any other questions or if I confused you at all, please ask questions, I want you to understand what this script should do 100%.

Thanks!

[mk-fg]

I know PulseAudio can do this, but not sure if its better to use the experimental PulseAudio support you mentioned in Comment #3 or try to figure out how to do it with PortAudio.

As I also mentioned there (and in comment-1), and same with JACK, that support is pretty much useless in context of #5 plans - you can't have more that 1 input and 1 output from PortAudio in pjsua, and it might even be hard limitation of PortAudio (e.g. if it has no concept of separate audio streams).

So unless I'm misunderstanding something, don't really see that as an option.

And if we have just one in + one out stream going out from pjsua and portaudio, why'd we need pulse or jack there?
Whole point of these is in mixing, processing and controlling individual streams from/to lots of apps or whatever sources to whatever sinks, with just one in/out, it should work just as well as a pipe ;)

EDIT: To be fair, one reason it might be worth having pulse/jack in that sound pipeline is because it's easier to setup them as a "pipe" than alsa libs, but fairly sure it still counts as a gross misuse.

EDIT 2: Read first part of your message above, totally changing my mental model of the whole thing (it was WRONG, I thought people would be "listening" through calls, not output devices connected to the machine). I think pulse is indeed a viable, easy and otherwise very good option, will definitely test it here.

[mk-fg]

Striken-through whole previous comment - with corrections from the first part of your comment, using pulse definitely makes a lot of sense, will check how well that PortAudio patch works.

[mk-fg]

Ok, I did look into using pulse just now, and it's all kinds of bad, unfortunately.

---

In case someone else will be going down the same road:

• Current patch is at the URL above, or at https://build.opensuse.org/package/show/home:illuusio:portaudio/portaudio in general - download latest .src.rpm (different OpenSUSE have different versions, latest package is currently for 13.2), unpack, .patch is inside.

  Thread where the patch was introduced: http://music.columbia.edu/pipermail/portaudio/2014-February/015778.html

  It's mentioned in the other threads that author wanted to make a more public release (maybe merge upstream, not sure) once it's less beta, but guess it hasn't happened yet.

• Bulding thing with the patch:

  patch -p1 -N < /path/to/portaudio-pulseaudio-v19_20150116.patch
  
  aclocal --force
  autoreconf --force --install ||:
  libtoolize --force
  
  ln -s /usr/share/libtool/build-aux/depcomp .
  grep -qF pa_ringbuffer.o Makefile.in \
   || sed -i 's|\(/pa_trace\.o\)|\1 src/common/pa_ringbuffer.o|' Makefile.in
  
  ./configure --prefix=/usr --enable-cxx --disable-static \
   --with-alsa=no --with-jack=no --with-oss=no \
   --with-pulseaudio=yes PKG_CONFIG=pkg-config
  make
  

  Not sure why modern autoconf/automake (2.69/1.15) mess it up a bit, but there are lot of deprecation notices and warnings, so maybe because it's just old.

  pa_ringbuffer.o and PKG_CONFIG seem to be straight-up forgotten there.

  Building with other backends as well are fine, they'll all show up in pjsua as options for playback/capture device.

  No need to rebuild pjproject, unless it's built with internal portaudio, I guess.

• ./pjsip-apps/bin/pjsystest* option "10: View Devices" should show pulse devices, can be picked via lib.set_snd_dev(...) or pjsua --playback-dev ....

  Can be tested with pjsua --play-file test.wav (and cc 1 0 there), or ./pjmedia/bin/pjmedia-test*.

• Not a single "sound test" on my machine worked, tripped on stuff like this:

  • Assertion 'c->callback' failed at pulsecore/socket-client.c:126, function do_call(). Aborting.
  • Assertion 'c->defer_event == e' failed at pulsecore/socket-client.c:170, function connect_defer_cb(). Aborting.
  • Assertion 'o' failed at pulse/operation.c:133, function pa_operation_get_state(). Aborting.
  • Lots of warning: No playback frame available (your application is buggy and/or got xruns)
  • Good old plain segfault.

  All on diff runs of the same binaries.

  Running thing under gdb seem to slow it down enough to get warnings and segfaults instead of assertions, so I guess it's some kind of race, maybe because pjsua runs threads liberally and that pulse hostapi wasn't written to handle that.

  Random and plentiful errors suggest that this won't go away even if one happens to run it successfully once (moon phases matter!).

  Shouldn't be too surprising for a alpha/beta ad-hoc patch, never really used widely.

Conclusion is that I think it's not worth bothering with using portaudio-pulse in this state, likely won't be a viable option for months or years, and given that portaudio isn't under very active dev, I'd say it's not worth hoping for.

---

Ideally, that special pulse thing won't be needed, as portaudio can just play through pulse's ALSA compat module, and pulse will wrap it into some stream like "ALSA plug-in [KSP.x86_64]", but I can't get it to run here either - thing just segfaults, and it's actually what prompted the guy to write that pulseaudio hostapi thing in the first place, seem to be common issue, lots of hits, no solutions, maybe not 100% though.

There's plenty of quirks of ALSA API - e.g. stuff from http://0pointer.de/blog/projects/guide-to-sound-apis.html - which are likely what's causing this not to work through that emulation layer on my Arch, but if it works reliably on some Debian, guess maybe it's not that bad.

Not sure what should I do about that either, doesn't sound worth debugging, nor looks to be a candidate for "fixed soon (tm)" - there's no new pj media backends in dev that I can google, wikis all recommend pasuspend -- <portaudio app>, heh.

---

So that's it for pulseaudio in pjsua.

[mk-fg]

Also, @danry25

I did mention on irc that pjsua's don't seem to connect two accs you gave me, here's why (hope it's not a big deal to dox them like that):

00:08:49.504 pjsua_core.c .RX 923 bytes Response msg 404/INVITE/cseq=18063 (rdata0x7f112c006f48) from TCP 64.140.158.61:5060:
SIP/2.0 404 Not Found
Via: SIP/2.0/TCP 192.168.0.13:39601;rport=39601;branch=z9hG4bKPj1488b0a3-0338-444e-a43c-1a490fb86285;alias;received=188.226.62.174
Max-Forwards: 70
From: <sip:1009@sea2.callpipe.com>;tag=d8d0f1c8-4398-451e-8174-904dc13223e1
To: <sip:1008@sea2.callpipe.com>;tag=KaaZt26K31K2g
Call-ID: 03917437-ed94-4164-b8f9-69ece7e0a80c
CSeq: 18063 INVITE
User-Agent: FreeSWITCH-mod_sofia/1.4.19-10-1~64bit
Accept: application/sdp
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE
Supported: timer, path, replaces
Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer
Reason: Q.850;cause=3;text="NO_ROUTE_DESTINATION"
Content-Length: 0
Remote-Party-ID: "1008" <sip:1008@sea2.callpipe.com>;party=calling;privacy=off;screen=no


--end msg--

Gist is SIP/2.0 404 Not Found + Reason: Q.850;cause=3;text="NO_ROUTE_DESTINATION" - "This cause indicates that the called party cannot be reached because the network through which the call has been routed does not serve the destination desired. This cause is supported on a network dependent basis.".

Like I mentioned, both clients are online, regged just fine, can send messages, it's just calls that seem to be failing like that.

Looks like a server error, or am I doing something wrong there?

[mk-fg]

Actually, I think JACK might be an easier option, since it (a) has a native and old portaudio driver (b) doesn't need any extra stream mixer on top - does same stuff as pulse. Other "simple" option is pure (native, not some emu module) ALSA and dmix, I guess.

Not sure how easy it is to control stuff in these, barely heard of either, since pulse seem to be used everywhere where it's not alsa, but definitely sound like a better options, given the crap above.