Automatic annotation of syllables in bioacoustics recordings

This repository contains code for the fully unsupervised method for syllable annotation in bioacoustics recordings described in Identifying Birdsong Syllables without Labelled Data.

Getting started

This project uses python 3.12.

Code structure:

The note_sequencing folder contains all source code for running the automatic labelling method. We explain in the following section how to adapt the code to your own data.

note_sequencing/
├── README.md
├── cluster/: functions for clustering used in the split and merge steps
├── data_loader/ : contains dataset specific preparation (including conversion of raw audio to spectrograms) and dataloader creation code. ├── detect/: deconvolution (template matching)
├── preprocessing/: generic preprocessing helpers
├── templates/: functions to create templates from clusters
├── util/: functions to save results
├── viz/: visualization functions

configcontains an example YAML configuration file.

scripts contains an example main_pied_flycatcher.py script for running the method.

The code used for the paper was slightly different than the one we share here because of extra outputs to different functions for the case when we had labels with which to evaluate our method. We provide code specific to the Bengalese finches and Great tits datasets in the paper is in paper (in particular, preprocessing of the Bengalese finch song dataset) -- (note: this is in progress, and the repo will be updated). However, we chose to release a more minimalistic version of the code which is easily adaptable to new datasets.

We improved on the initial detection (with connected components) presented in the paper by using spectral gating which we recommend as the default method to use. The relevant code can be found in detect/spectral_detect.py An addition to the paper is the possibility to apply noisereduce on input audio as an additional denoising preprocessing step, which can be controlled by the parameter preprocessing.bandpass_filter in the YAML configuration file.

Data structure requirements and adapting the code to your dataset

• adapt the note_sequencing/data_loader/pied_flycatchers_dataset.pyfile to your dataset. This dataset file includes the preprocessing of raw audio to spectrograms. You should first chop the recordings into the length of windows that you wish to process. The method can take any length of recording as input (as long as it fits in the memory) but you might want shorter chunks for ease of visualization. Typically for birds, we would chunk into windows of 4-10s. All the recordings (or chunked recordings) should then be put in a single folder. The important thing is that you save processed raw audios into numpy spectrograms in individual files, and that you have a dataframe with a summary of the paths to those numpy files in the audio_pathcolumn of the dataframe (see dict_all in the process_data function). You should also output a combined_spectrograms.npy output file which is a numpy array containing all the spectrograms. This is useful for sped up processing. In order to run the spectral detection (as the initial detection step in detect/spectral_detect.py) you also need to have access to the .wav recordings (unprocessed) and have their paths in the raw_audio_path column of the dataframe. Make sure that the parameters to savespecfunction in the remove_spectral_components function in the spectral detection are the same as those used in the the dataset definition.

• Then you can write a main file like the one in scripts/main_pied_flycatchers.py - the only change here is that you need to import the correct dataset in the imports

• All the parameters can be defined in a config file following this one configs/pied_flycatchers.yaml

🐍 Python Version

This project uses Python 3.12.

📦 Package & Environment Management

This project uses poetry for package and environment management.

Clone the repository,install poetry and run poetry install

Running the method

Then run poetry run python scripts/main_pied_flycatchers.py --config "configs/pied_flycatchers.yaml"