Feature-level fusion in wireless acoustic sensor networks with graph attention networks for classification of domestic activities

Wei Wei, Matthias Hutsebaut-Buysse, Thomas Avé, Tom De Schepper, Kevin Mets

2nd Workshop on Data Fusion for Artificial Intelligence (DAFUSAI). October 19th, 2024 at ECAI, Santiago de Compostela

Abstract:

Wireless acoustic sensor networks (WASNs) can detect and monitor omnidirectional audio data. To increase the performance of the WASN, sensor fusion techniques are used to aggregate the data from sensors. In this work, we focus on feature-level fusion for the classification of domestic activities using a WASN. We found that the existing datasets may not reflect WASN in a real world scenario as the acoustic sensors are closely located to each other. We constructed a new dataset, corresponding with a sparse WASN deployment. Next, we propose a new feature-level fusion framework. Our framework is the first to use Graph Neural Networks (GNNs) for the classification of domestic activities using WASN, which accepts variable sensor inputs. Empirical results show that our framework outperforms decision-level fusion with restrained data transmission cost.