Skip to content

Google AEC rework: dynamic formats, legacy platform support#8919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cujomalainey merged 7 commits into from
Jun 21, 2024
Merged

Google AEC rework: dynamic formats, legacy platform support #8919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
df6881c
google_aec: Don't allocate giant blobs on the heap
andyross Mar 7, 2024
2b6dea1
google_aec: Fix input stream identification
andyross Mar 7, 2024
53c9fad
google_aec: Remove FLOAT_API configurability
andyross Mar 7, 2024
654d1c3
google_aec: Remove broken/unmaintained "IPC3" process code paths
andyross Mar 7, 2024
722a17b
google_aec: Intercept cross-pipeline trigger propagation
andyross Mar 7, 2024
47e61c6
google_aec: Clean up kconfig handling
andyross Mar 7, 2024
018cec3
google_aec: Rework prepare/process() to support dynamic formats
andyross Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions src/audio/google/Kconfig
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,7 +15,6 @@ config COMP_GOOGLE_HOTWORD_DETECT
config COMP_GOOGLE_RTC_AUDIO_PROCESSING
bool "Google Real Time Communication Audio processing"
select COMP_BLOB
select GOOGLE_RTC_AUDIO_PROCESSING_MOCK if COMP_STUBS
default n
help
Select for Google real-time communication audio processing. It
Expand All@@ -24,13 +23,7 @@ config COMP_GOOGLE_RTC_AUDIO_PROCESSING
This component takes raw microphones input and playback reference
and outputs an echo-free microphone signal.

@marcinszkudlinskimarcinszkudlinskiApr 15, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The internals of the AEC library are floating point already.

just a comment: from my measurements of the AEC binary I got - 16bit API used to be faster than 32bit.

@andyrossandyrossApr 23, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that maybe polluted by conversion overhead? Or maybe it's a cache footprint issue? My understanding is that the behavior is identical once you get across the API boundary. @lkoenig might have input.

@lkoeniglkoenigApr 26, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internally the int16 are converted back to float32. If the int16 is faster than the float32, we might need to look into it.

@lgirdwoodlgirdwoodApr 26, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@singalsu any comments here ?

@marcinszkudlinskimarcinszkudlinskiApr 29, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lkoenig I might have had an obsolete version of AEC binary, so my observations may be inaccurate.

@singalsusingalsuMay 3, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From AEC quality point of view I think 32 bit float is a good choice over 16 bit integer. The raw Mic signal level is typically low since there is headroom for very loud ambient noise.

config COMP_GOOGLE_RTC_USE_32_BIT_FLOAT_API
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
bool "Use 32bit API in Google Audio processing"
default n
help
Selects an API to be used in communication with the Google real-time
communication audio processing: 32bit float or 16bit integer
if COMP_GOOGLE_RTC_AUDIO_PROCESSING

config COMP_GOOGLE_RTC_AUDIO_PROCESSING_SAMPLE_RATE_HZ
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
Expand DownExpand Up@@ -61,10 +54,10 @@ config COMP_GOOGLE_RTC_AUDIO_REFERENCE_CHANNEL_MAX
retrieved at runtime, but channels higher than this number
are ignored

config COMP_GOOGLE_RTC_AUDIO_PROCESSING_MEMORY_BUFFER_SIZE_BYTES
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_MEMORY_BUFFER_SIZE_KB
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
int "Memory buffer size for Google Real Time Communication Audio processing"
default 200000
default 200
help
Sets the size of the memory buffer for the Google real-time
communication audio processing.
Expand All@@ -87,10 +80,12 @@ config COMP_GOOGLE_RTC_AUDIO_PROCESSING_MIC_HEADROOM_LINEAR

config GOOGLE_RTC_AUDIO_PROCESSING_MOCK
bool "Google Real Time Communication Audio processing mock"
default n
default y if COMP_STUBS
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
help
Mock Google real-time communication audio processing.
It allows for compilation check and basic audio flow checking.

endif # COMP_GOOGLE_RTC_AUDIO_PROCESSING

endmenu
Loading