Auditory Stream Segregation in Auditory Scene Analysis with a Multi-Agent System

Abstract

We propose a novel approach to auditory stream segregation which extracts individual sounds (au-ditory stream) from a mixture of sounds in audi-tory scene analysis. The HBSS (Harmonic-Based Stream Segregation) system is designed and devel-oped by employing a multi-agent system. HBSS uses only harmonics as a clue to segregation and extracts auditory streams incrementally. When the tracer-generator agent detects a new sound, it spawns a tracer agent, which extracts an au-ditory stream by tracing its harmonic structure. The tracer sends a feedforward signal so that the generator and other tracers should not work on the same stream that is being traced. The qual-ity of segregation may be poor due to redun-dant and ghost tracers. HBSS copes with this problem by introducing monitor agents, which detect and eliminate redundant and ghost trac-ers. HBSS can segregate two streams from a mix-ture of man’s and woman’s speech. It is easy to resynthesize speech or sounds from the cor-responding streams. Additionally, HBSS can be easily extended by adding agents of a new capa-bility. HBSS can be considered as the first step to computational auditory scene analysis.