SpatialScaper: a library to simulate and augment soundscapes for sound event localization and detection in realistic rooms.
Warning
SpatialScaper is still undergoing active development. We have done our due diligence to test that example_generation.py works as expected. However, please open an issue and describe any errors you encounter. Also, make sure to pull often, as we are actively adding more features. Note: You'll need 100GB of storage space to comfortably setup and run the DCASE Task 3 data generation pipeline.
Guides
- Requirements and Installation
- Preparing Sound Event Assets
- Preparing RIR Datasets
- Example data generation (for DCASE Task 3)
SpatialScaper is a python library to create synthetic audio mixtures suitable for DCASE Challenge Task 3.
To run the SpatialScaper library, manually setup your environment as follows.
The minimum environment requirements are Python >= 3.8. You could find the versions of other dependencies we use in setup.py.
git clone https://github.com/iranroman/SpatialScaper.git
cd SpatialScaper
pip install -e .Click for more details
conda create -n "ssenv" python=3.8
python3.8 -m venv "ssenv"
First we need to prepare sound event assets for soundscape synthesis. SpatialScaper works with any sound files that you wish to spatialize. You can get started using sound events from the FSD50K and FMA (music) dataset by using.
python scripts/prepare_fsd50k_fma.py --download_FSD --download_FMA --cleanupThe --cleanup argument deletes the original FSD50K and FMA zip files (to save space), keeping only the files needed to get started with SpatialScaper.
This creates a datasets/sound_event_datasets/FSD50K_FMA directory with a structure of sound event categories and files.
Attention: the first time setup takes some time ⏳, we recommend running under a screen or tmux session.
python scripts/prepare_rirs.py --cleanup
The --cleanup argument deletes the original RIR database zip files (to save space).
Attention: the first time setup takes some time ⏳, we recommend running under a screen or tmux session.
Note: stay tuned as we will soon release our A2B ambisonics encoder. In the meantime, refer to the table below to download the respective FOA sofa file for the METU, RSoANU, and DAGA datasets. Place alongside all other sofa files that prepare_rirs.py generates under SpatialScaper/datasets/rir_datasets/spatialscaper_RIRs.
| Dataset | URL |
|---|---|
| METU | Link |
| RSoANU | Link |
| DAGA | Link |
Full descriptions of available rooms
The available rooms for soundscape generation are as follows:
| Room Name | Description | Trajectory type | URL |
|---|---|---|---|
| metu | Classroom S05 at the METU Graduate School of Informatics on 23 January 2018. | Square | Link |
| arni | Arni variable acoustics room at the Acoustics Lab, Aalto University, Espoo, Finland. | Linear | Link |
| bomb_shelter | Large open space in underground bomb shelter, with plastic-coated floor and rock walls. Ventilation noise. | Circular | Link |
| gym | Large open gym space. Ambience of people using weights and gym equipment in adjacent rooms. | Circular | Link |
| pb132 | Small classroom with group work tables and carpet flooring. Ventilation noise. | Circular | Link |
| pc226 | Meeting room with hard floor and partially glass walls. Ventilation noise. | Circular | Link |
| sa203 | Lecture hall with inclined floor and rows of desks. Ventilation noise. | Linear | Link |
| sc203 | Small classroom with group work tables and carpet flooring. Ventilation noise. | Linear | Link |
| se203 | Large classroom with hard floor and rows of desks. Ventilation noise. | Linear | Link |
| tb103 | Lecture hall with inclined floor and rows of desks. Ventilation noise. | Linear | Link |
| tc352 | Meeting room with hard floor and partially glass walls. Ventilation noise. | Circular | Link |
| motus | Seminar room with configurable furniture, carpet tiles, and absorption wedges. | Sparse | Link |
| rsoanu | ANU School of Music Recording Studio with variable wall panels: wood or felt. | Rectangular | Link |
| daga | Small conference room with large wood table and carpet flooring. | Sparse | Link |
Note that SRIR directions and distances differ with the room. Possible azimuths span the whole range of
Below we present the example_generation.py. The example generates 20 soundscapes, 1 minute long each, using audio clips from FSD50K, spatialized in the gym room. These soundscapes are consistent with the DCASE Task 3 format.
Execute as:
python example_generation.pyimportnumpyasnpimportspatialscaperasssimportos# ConstantsNSCAPES=20# Number of soundscapes to generateFOREGROUND_DIR="datasets/sound_event_datasets/FSD50K_FMA"# Directory with FSD50K foreground sound filesRIR_DIR= (
"datasets/rir_datasets"# Directory containing Room Impulse Response (RIR) files
)
ROOM="bomb_shelter"# Initial room setting, change according to available rooms listed belowFORMAT="mic"# Output format specifierN_EVENTS_MEAN=15# Mean number of foreground events in a soundscapeN_EVENTS_STD=6# Standard deviation of the number of foreground eventsDURATION=60.0# Duration in seconds of each soundscape, customizable by the userSR=24000# SpatialScaper default sampling rate for the audio filesOUTPUT_DIR="output"# Directory to store the generated soundscapesREF_DB=-65# Reference decibel level for the background ambient noise. Try making this random too!# List of possible rooms to use for soundscape generation. Change 'ROOM' variable to one of these:# "metu", "arni","bomb_shelter", "gym", "pb132", "pc226", "sa203", "sc203", "se203", "tb103", "tc352"# Each room has a different Room Impulse Response (RIR) file associated with it, affecting the acoustic properties.# FSD50K sound classes that will be spatialized include:# 'femaleSpeech', 'maleSpeech', 'clapping', 'telephone', 'laughter',# 'domesticSounds', 'footsteps', 'doorCupboard', 'music',# 'musicInstrument', 'waterTap', 'bell', 'knock'.# These classes are sourced from the FSD50K dataset, and# are consistent with the DCASE SELD challenge classes.# Function to generate a soundscapedefgenerate_soundscape(index):
track_name=f"fold5_room1_mix{index+1:03d}"# Initialize Scaper. 'max_event_overlap' controls the maximum number of overlapping sound events.ssc=ss.Scaper(
DURATION,
FOREGROUND_DIR,
RIR_DIR,
FORMAT,
ROOM,
max_event_overlap=2,
speed_limit=2.0, # in meters per second
)
ssc.ref_db=REF_DB# static ambient noisessc.add_background()
# Add a random number of foreground events, based on the specified mean and standard deviation.n_events=int(np.random.normal(N_EVENTS_MEAN, N_EVENTS_STD))
n_events=n_eventsifn_events>0else1# n_events should be greater than zerofor_inrange(n_events):
ssc.add_event() # randomly choosing and spatializing an FSD50K sound eventaudiofile=os.path.join(OUTPUT_DIR, FORMAT, track_name)
labelfile=os.path.join(OUTPUT_DIR, "labels", track_name)
ssc.generate(audiofile, labelfile)
# Main loop for generating soundscapesforiscapeinrange(NSCAPES):
print(f"Generating soundscape: {iscape+1}/{NSCAPES}")
generate_soundscape(iscape)If you find our SpatialScaper library useful, please cite the following paper:
@inproceedings{roman2024spatial,
title={Spatial Scaper: a library to simulate and augment soundscapes for sound event localization and detection in realistic rooms},
author={Roman, Iran R and Ick, Christopher and Ding, Sivan and Roman, Adrian S and McFee, Brian and Bello, Juan P},
booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
year={2024},
organization={IEEE}
}
Also cite the RIR and sound event databases that SpatialScaper uses.
@dataset{politis_2022_6408611,
author = {Politis, Archontis and
Adavanne, Sharath and
Virtanen, Tuomas},
title = {{TAU Spatial Room Impulse Response Database (TAU-SRIR DB)}},
month = apr,
year = 2022,
publisher = {Zenodo},
doi = {10.5281/zenodo.6408611},
url = {https://doi.org/10.5281/zenodo.6408611},
}
@dataset{orhun_olgun_2019_2635758,
author = {Orhun Olgun and
Huseyin Hacihabiboglu},
title = {{METU SPARG Eigenmike em32 Acoustic Impulse
Response Dataset v0.1.0}},
month = apr,
year = 2019,
publisher = {Zenodo},
version = {0.1.0},
doi = {10.5281/zenodo.2635758},
url = {https://doi.org/10.5281/zenodo.2635758},
}
@article{mckenzie2021dataset,
title={Dataset of spatial room impulse responses in a variable acoustics room for six degrees-of-freedom rendering and analysis},
author={McKenzie, Thomas and McCormack, Leo and Hold, Christoph},
journal={arXiv preprint arXiv:2111.11882},
year={2021}
}
@article{fonseca2021fsd50k,
title={Fsd50k: an open dataset of human-labeled sound events},
author={Fonseca, Eduardo and Favory, Xavier and Pons, Jordi and Font, Frederic and Serra, Xavier},
journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
volume={30},
pages={829--852},
year={2021},
publisher={IEEE}
}
@article{defferrard2016fma,
title={FMA: A dataset for music analysis},
author={Defferrard, Micha{\"e}l and Benzi, Kirell and Vandergheynst, Pierre and Bresson, Xavier},
journal={arXiv preprint arXiv:1612.01840},
year={2016}
}
@article{gotz2021dataset,
title={A dataset of higher-order Ambisonic room impulse responses and 3D models measured in a room with varying furniture},
author={G{\"o}tz, Georg and Schlecht, Sebastian J and Pulkki, Ville},
journal={2021 Immersive and 3D Audio: from Architecture to Automotive (I3DA)},
pages={1--8},
year={2021},
publisher={IEEE}
}
@article{chesworth2024room,
title={Room Impulse Response Dataset of a Recording Studio with Variable Wall Paneling Measured Using a 32-Channel Spherical Microphone Array and a B-Format Microphone Array},
author={Chesworth, Grace and Bastine, Amy and Abhayapala, Thushara},
journal={Applied Sciences},
volume={14},
number={5},
pages={2095},
year={2024},
publisher={MDPI}
}
@article{schneiderwind2019data,
title={Data set: Eigenmike-DRIRs, KEMAR 45BA-BRIRs, RIRs and 360◦ pictures captured at five positions of a small conference room},
author={Schneiderwind, Christian and Neidhardt, Annika and Klein, F and Fichna, S},
journal={45th Annual Conference on Acoustics (DAGA), Rostock, Germany},
year={2019}
}