AI-Based Soundscape Analysis: Jointly Identifying Sound Sources and Predicting Annoyance

The AI-Soundscape is uploading....

Figure 1: The DCNN-CaF model with only Mel features.

Figure 2: The DCNN-CaF model with only RMS features.

Run models

1) Unzip the Dataset under the application folder

2) Unzip the pretrained_models under the application folder

3) Enter the application folder: cd application

1) Models for the joint ARP and SSC tasks

1.1) DCNN_CaF

python inference_ARP_SSC.py -model DCNN_CaF
----------------------------------------------------------------------------------------
Parameters num: 7.614459 M
ARP:
        MAE: 0.8372967121097868, RMSE: 1.0516823194115195
SSC:
        AUC: 0.9006453415552284

1.2) DNN

python inference_ARP_SSC.py -model DNN
----------------------------------------------------------------------------------------
Parameters num: 0.375769 M
ARP:
        MAE: 1.0421484866315518, RMSE: 1.3410438201986232
SSC:
        AUC: 0.8803192730739317

1.3) CNN

python inference_ARP_SSC.py -model CNN
----------------------------------------------------------------------------------------
Parameters num: 1.270811 M
ARP:
        MAE: 0.9336034988862009, RMSE: 1.1964973202106266
SSC:
        AUC: 0.8791363928144383

1.4) CNN_Transformer

python inference_ARP_SSC.py -model CNN_Transformer
----------------------------------------------------------------------------------------
Parameters num: 17.971227 M
ARP:
        MAE: 0.9509568487731644, RMSE: 1.1900119539356668
SSC:
        AUC: 0.8636173978248662

2) Models for the SSC task

2.1) DCNN_CaF_SSC

python inference_SSC.py -model DCNN_CaF_SSC
----------------------------------------------------------------------------------------
Parameters num: 4.961112 M
SSC:
        AUC: 0.922593503235641

2.2) YAMNet

python inference_ARP_SSC.py -model DNN
----------------------------------------------------------------------------------------
Parameters num: 3.231 M
SSC:
        AUC: 0.8727500810814068

2.3) PANN

python inference_ARP_SSC.py -model CNN
----------------------------------------------------------------------------------------
Parameters num: 79.72284 M
SSC:
        AUC: 0.903805115943463

2.4) AST

python inference_ARP_SSC.py -model CNN_Transformer
----------------------------------------------------------------------------------------
Parameters num: 86.207256 M
SSC:
        AUC: 0.8509725627182434

3) Computational overhead of models

Table 1: Details of the parameters and computational overhead of models used and proposed in the paper.

Visualization

Attention distributions of the cross-attention-based fusion module in the DCNN-CaF model. The 8 heads in the upper row come from MHA1, and the 8 heads in the lower row come from MHA2.

mitchellacoustics / ai-soundscape Goto Github PK

ai-soundscape's Introduction

AI-Based Soundscape Analysis: Jointly Identifying Sound Sources and Predicting Annoyance

Figure 1: The DCNN-CaF model with only Mel features.

Figure 2: The DCNN-CaF model with only RMS features.

Run models

1) Models for the joint ARP and SSC tasks

1.1) DCNN_CaF

1.2) DNN

1.3) CNN

1.4) CNN_Transformer

2) Models for the SSC task

2.1) DCNN_CaF_SSC

2.2) YAMNet

2.3) PANN

2.4) AST

3) Computational overhead of models

Table 1: Details of the parameters and computational overhead of models used and proposed in the paper.

Visualization

Fig. 1: (a) Demo 1.

Fig. 1: (b) Demo 2.

Fig. 1: (c) Demo 3.

Fig. 1: (d) Demo 4.

Fig. 1: (e) Demo 5.

To be continued...

ai-soundscape's People

Contributors

Recommend Projects

Recommend Topics

Recommend Org