Two way audio

[pergolafabio]

Hi @AlexxIT

Just a question, but is there a possibility to have two way audio enabled?

Thnx in advance

[danielducu]

I would definitely pay for that :)

[pergolafabio]

Yes, latest android app (beta) supports now microphone and speaker

[rbray89]

Technically this *may" be feasible, but it would likely depend on the method of backend audio. Either ONVIF 2-way would be assumed, or something like SIP would have to be used. Currently Home assistant is not aware of the audio sink, so it would have to be configured entirely in the component.

[pergolafabio]

Mine cameras are based on ISAPI, to start twoway audio, i need to send 2 commands (no onvif)
Synology is also using the same method for my hikvision door camera

would be a great feature, so we can chat with the HA app :-)

[rbray89]

WebRTC is also limited in supported codecs, which is why audio on some cameras doesn't work. This won't be an easy change to get it to work on devices that support 2way audio.

[pergolafabio]

Is it worth a try? Would be great!!
Mine audio codec is ulaw, it's the default audio protocol, most common used... One way audio already works now with current card...
But I have no idea how to enable the mic :-) and to send the ISAPi command from the card to active it on the stream...

[pergolafabio]

I have a trace how Synology is doing it, they just send 2 command to active the two-way audio

[AlexxIT]

RTSP is spec for camera stream translation. There is no spec for two way audio. Each company use their own tech. Usual it's some kind of P2P.

[pergolafabio]

yeah, mine does work with sending 2 commands, do you think its easy to integrate?

[rbray89]

@danielducu I don't think you understand the difficulty here.

1. vendors all do this differently, and it's not documented, so everything has to be reverse engineered.

2. webRTC only uses a few different audio codecs, so they'd almost certainly have to transcode things on the server.

[nanosonde]

If we rely on standards-based cameras, it could work though.

ONVIF Profile T: Chapter 8.12 Audio Output streaming: https://www.onvif.org/wp-content/uploads/2018/09/ONVIF_Profile_T_Specification_v1-0.pdf
More practical details: https://www.happytimesoft.com/knowledge/audio-back-channel.html
So basically a third stream towards the camera (besides video and audio from camera) is negotiated in the SDP of the server.
The audio out stream then just flows in the opposite direction of the other two RTP streams (sendonly vs. recvonly).

Video doorbell cameras mostly rely on two-way audio negotiated via SIP.
It did a PoC implemenation for Homebridge here: homebridge-plugins/homebridge-camera-ffmpeg#1253
Ok, this is not related to WebRTC directly.

Solutions for propriertary vendor solutions could be doable. Example neolink for Reolink cameras: thirtythreeforty/neolink#268

Concerning ISAPI I only found this: https://www.hikvision.com/content/dam/hikvision/vn/webinar/Thang3_Hikvision_Tich_Hop_He_Thong_Overview-of-3rd-Party-Integration.pdf

[pergolafabio]

I have the Hikvision, and also all ISAPI documentations, so if i can help
i also know the ISAPI urls that should be used to activcate twoway audio

[mattkerrison]

The Amcrest AD410 Doorbell supports ONVIF Profile T for 2-way audio.

The Dahua integration for Home Assistant hasn't been able to get it working yet but the dream would be WebRTC with 2 way audio, could finally get rid of the garbage Amcrest app