Skip to content

Upgrade audiusd to bookworm, move off essentia - #10867

Merged
endline merged 3 commits into
mainfrom
endl/audiusd-bookworm
Dec 31, 2024
Merged

Upgrade audiusd to bookworm, move off essentia#10867
endline merged 3 commits into
mainfrom
endl/audiusd-bookworm

Conversation

@endline

@endlineendline commented Dec 30, 2024

Copy link
Copy Markdown
Contributor

WHAT THIS?

Reduces final image size by 67.3%

$ docker images | grep -E 'local|bookworm'
audius/audiusd bookworm bf31d0b1ae0d 3 minutes ago 696MB
audius/audiusd current 17ee18fa7d6d 7 days ago 2.13GB

Removes use of essentia in favor of aubio.

// bpm_analyzer.cpp
#include <aubio/aubio.h>

Yields about a 4-5x speed improvement of bpm analysis

# essentia
root@a9a45ea61f2b:/app# time analyze-bpm /dub.wav BPM: 119.993
Confidence: 3.36337
real 0m5.285s
user 0m4.934s
sys 0m0.348s
# aubio
root@5a64d8048973:/# time analyze-bpm /dub.wav
BPM: 120.794
real 0m1.644s
user 0m1.437s
sys 0m0.149s

Furthermore, essentia was breaking our ability to upgrade to debian:bookworm which is a requirement now that the container installed postgres was throwing this warning - which needs to be addressed.

2024-12-30 20:16:37.614 UTC [20871] WARNING: database "audius_creator_node" has a collation version mismatch
2024-12-30 20:16:37.614 UTC [20871] DETAIL: The database was created using collation version 2.36, but the operating system provides version 2.31.
2024-12-30 20:16:37.614 UTC [20871] HINT: Rebuild all objects in this database that use the default collation and run ALTER DATABASE audius_creator_node REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.

Lastly, I removed the now very out of date mediourm-cmd as I dont believe it adds actual confidence in the mediorum-tests. From here we should concentrate more on the mediorum-unittests and have more e2e functional style audiusd-tests.

TEST THIS

Tests performed using a local copy of the current prod debian:bullseye base image, compared the new debian:bookworm-slim base image.

Ill note that I REMOVED the confidence value being returned, as this is used nowhere in the mediorum code.

TEST A

# current bullseye base image
$ docker run --rm -ti --entrypoint /bin/bash -v /private/var/dub.wav:/dub.wav audius/audiusd:current
root@a9a45ea61f2b:/app# analyze-bpm /dub.wav BPM: 119.993
Confidence: 3.36337
root@a9a45ea61f2b:/app# analyze-key /dub.wav F minor

TEST B

# proposed bookworm-slim base image
$ docker run --rm -ti --entrypoint /bin/bash -v /private/var/dub.wav:/dub.wav audius/audiusd:bookworm
root@155073e2ea96:/# analyze-bpm /dub.wav BPM: 120.794
root@155073e2ea96:/# analyze-key /dub.wav F minor

TEST C
✅ Stage test: ran this image on all stage content nodes, bpm and key detected successfully and accurately.
✅ Also tested and verified premium track, creating a preview, then editing the preview.

Screenshot 2024-12-30 at 3 31 51 PM

@changeset-bot

changeset-botBot commented Dec 30, 2024

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 50d875a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@alecsavvy

Copy link
Copy Markdown
Contributor

Confidence doesn't appear to be used in discovery either https://github.com/AudiusProject/audius-protocol/pull/10734/files#diff-5ced548a17b4b93f7d78282969a1cc0f31137061ea9af5fe7249464a117097abR11-R12

@endline
endlineforce-pushed the endl/audiusd-bookworm branch 3 times, most recently from 1db0b5c to ecf7e20CompareDecember 30, 2024 22:58

@alecsavvyalecsavvy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noice

@endline
endlineforce-pushed the endl/audiusd-bookworm branch from 09ffafd to 50d875aCompareDecember 31, 2024 03:31
@endline
endline merged commit 9318508 into mainDec 31, 2024
@endline
endline deleted the endl/audiusd-bookworm branch December 31, 2024 03:48
audius-infra pushed a commit that referenced this pull request Dec 31, 2024
[9318508] Upgrade audiusd to bookworm, move off essentia (#10867) endline
[937a873] SLA rollup improvements, fixes (#10868) Danny
[a38e4b6] Fix search for old clients (#10869) Raymond Jacobson
[b555bc5] add dev to healthz (#10862) alecsavvy
[373ce1d] add custom legacy profile to core and mediorum (#10866) alecsavvy
[c0b91c5] fix ci: add back mediorum and core in legacy docker compose files (#10864) alecsavvy
[8ef27ef] Remove go pkg/infra (#10863) endline
[04efb42] audiusd in audius-compose (#10859) alecsavvy
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@endline@alecsavvy