Computational Auditory Scene Analysis: Principles, Algorithms, and Applications

DeLiang Wang, Guy J. Brown

  • 出版商: IEEE
  • 出版日期: 2006-10-01
  • 售價: $5,130
  • 貴賓價: 9.5$4,874
  • 語言: 英文
  • 頁數: 395
  • 裝訂: Hardcover
  • ISBN: 0471741094
  • ISBN-13: 9780471741091
  • 相關分類: Algorithms-data-structures
  • 海外代購書籍(需單獨結帳)

買這商品的人也買了...

商品描述

Description

How can we engineer systems capable of "cocktail party" listening?

Human listeners are able to perceptually segregate one sound source from an acoustic mixture, such as a single voice from a mixture of other voices and music at a busy cocktail party. How can we engineer "machine listening" systems that achieve this perceptual feat?

Albert Bregman's book Auditory Scene Analysis, published in 1990, drew an analogy between the perception of auditory scenes and visual scenes, and described a coherent framework for understanding the perceptual organization of sound. His account has stimulated much interest in computational studies of hearing. Such studies are motivated in part by the demand for practical sound separation systems, which have many applications including noise-robust automatic speech recognition, hearing prostheses, and automatic music transcription. This emerging field has become known as computational auditory scene analysis (CASA).

Computational Auditory Scene Analysis: Principles, Algorithms, and Applications provides a comprehensive and coherent account of the state of the art in CASA, in terms of the underlying principles, the algorithms and system architectures that are employed, and the potential applications of this exciting new technology. With a Foreword by Bregman, its chapters are written by leading researchers and cover a wide range of topics including:

  • Estimation of multiple fundamental frequencies
  • Feature-based and model-based approaches to CASA
  • Sound separation based on spatial location
  • Processing for reverberant environments
  • Segregation of speech and musical signals
  • Automatic speech recognition in noisy environments
  • Neural and perceptual modeling of auditory organization

The text is written at a level that will be accessible to graduate students and researchers from related science and engineering disciplines. The extensive bibliography accompanying each chapter will also make this book a valuable reference source. A web site accompanying the text (www.casabook.org) features software tools and sound demonstrations.

 

Table of Contents

Foreword.

Preface.

Contributors.

Acronyms.

1. Fundamentals of Computational Auditory Scene Analysis (DeLiang Wang and Guy J. Brown).

1.1 Human Auditory Scene Analysis.

1.1.1 Structure and Function of the Auditory System.

1.1.2 Perceptual Organization of Simple Stimuli.

1.1.3 Perceptual Segregation of Speech from Other Sounds.

1.1.4 Perceptual Mechanisms.

1.2 Computational Auditory Scene Analysis (CASA).

1.2.1 What Is CASA?

1.2.2 What Is the Goal of CASA?

1.2.3 Why CASA?

1.3 Basics of CASA Systems.

1.3.1 System Architecture.

1.3.2 Cochleagram.

1.3.3 Correlogram.

1.3.4 Cross-Correlogram.

1.3.5 Time-Frequency Masks.

1.3.6 Resynthesis.

1.4 CASA Evaluation.

1.4.1 Evaluation Criteria.

1.4.2 Corpora.

1.5 Other Sound Separation Approaches.

1.6 A Brief History of CASA (Prior to 2000).

1.6.1 Monaural CASA Systems.

1.6.2 Binaural CASA Systems.

1.6.3 Neural CASA Models.

1.7 Conclusions 36

Acknowledgments.

References.

2. Multiple F0 Estimation (Alain de Cheveigné).

2.1 Introduction.

2.2 Signal Models.

2.3 Single-Voice F0 Estimation.

2.3.1 Spectral Approach.

2.3.2 Temporal Approach.

2.3.3 Spectrotemporal Approach.

2.4 Multiple-Voice F0 Estimation.

2.4.1 Spectral Approach.

2.4.2 Temporal Approach.

2.4.3 Spectrotemporal Approach.

2.5 Issues.

2.5.1 Spectral Resolution.

2.5.2 Temporal Resolution.

2.5.3 Spectrotemporal Resolution.

2.6 Other Sources of Information.

2.6.1 Temporal and Spectral Continuity.

2.6.2 Instrument Models.

2.6.3 Learning-Based Techniques.

2.7 Estimating the Number of Sources.

2.8 Evaluation.

2.9 Application Scenarios.

2.10 Conclusion.

Acknowledgments.

References.

3. Feature-Based Speech Segregation (DeLiang Wang).

3.1 Introduction.

3.2 Feature Extraction.

3.2.1 Pitch Detection.

3.2.2 Onset and Offset Detection.

3.2.3 Amplitude Modulation Extraction.

3.2.4 Frequency Modulation Detection.

3.3 Auditory Segmentation.

3.3.1 What Is the Goal of Auditory Segmentation?

3.3.2 Segmentation Based on Cross-Channel Correlation and Temporal Continuity.

3.3.3 Segmentation Based on Onset and Offset Analysis.

3.4 Simultaneous Grouping.

3.4.1 Voiced Speech Segregation.

3.4.2 Unvoiced Speech Segregation.

3.5 Sequential Grouping.

3.5.1 Spectrum-Based Sequential Grouping.

3.5.2 Pitch-Based Sequential Grouping.

3.5.3 Model-Based Sequential Grouping.

3.6 Discussion.

Acknowledgments.

References.

4. Model-Based Scene Analysis (Daniel P. W. Ellis).

4.1 Introduction.

4.2 Source Separation as Inference.

4.3 Hidden Markov Models.

4.4 Aspects of Model-Based Systems.

4.4.1 Constraints: Types and Representations.

4.4.2 Fitting Models.

4.4.3 Generating Output.

4.5 Discussion.

4.5.1 Unknown Interference.

4.5.2 Ambiguity and Adaptation.

4.5.3 Relations to Other Separation Approaches.

4.6 Conclusions.

References.

5. Binaural Sound Localization (Richard M. Stern, Guy J. Brown, and DeLiang Wang).

5.1 Introduction.

5.2 Physical and Physiological Mechanisms Underlying Auditory Localization.

5.2.1 Physical Cues.

5.2.2 Physiological Estimation of ITD and IID.

5.3 Spatial Perception of Single Sources.

5.3.1 Sensitivity to Differences in Interaural Time and Intensity.

5.3.2 Lateralization of Single Sources.

5.3.3 Localization of Single Sources.

5.3.4 The Precedence Effect.

5.4 Spatial Perception of Multiple Sources.

5.4.1 Localization of Multiple Sources.

5.4.2 Binaural Signal Detection.

5.5 Models of Binaural Perception.

5.5.1 Classical Models of Binaural Hearing.

5.5.2 Cross-Correlation-Based Models of Binaural Interaction.

5.5.3 Some Extensions to Cross-Correlation-Based Binaural Models.

5.6 Multisource Sound Localization.

5.6.1 Estimating Source Azimuth from Interaural Cross-Correlation.

5.6.2 Methods for Resolving Azimuth Ambiguity.

5.6.3 Localization of Moving Sources.

5.7 General Discussion.

Acknowledgments.

References.

6. Localization-Based Grouping (Albert S. Feng and Douglas L. Jones).

6.1 Introduction.

6.2 Classical Beamforming Techniques.

6.2.1 Fixed Beamforming Techniques.

6.2.2 Adaptive Beamforming Techniques.

6.2.3 Independent Component Analysis Techniques.

6.2.4 Other Localization-Based Techniques.

6.3 Location-Based Grouping Using Interaural Time Difference Cue.

6.4 Location-Based Grouping Using Interaural Intensity Difference Cue.

6.5 Location-Based Grouping Using Multiple Binaural Cues.

6.6 Discussion and Conclusions.

Acknowledgments.

References.

7. Reverberation (Guy J. Brown and Kalle J. Palomäki).

7.1 Introduction.

7.2 Effects of Reverberation on Listeners.

7.2.1 Speech Perception.

7.2.2 Sound Localization.

7.2.3 Source Separation and Signal Detection.

7.2.4 Distance Perception.

7.2.5 Auditory Spatial Impression.

7.3 Effects of Reverberation on Machines.

7.4 Mechanisms Underlying Robustness to Reverberation in Human Listeners.

7.4.1 The Role of Slow Temporal Modulations in Speech Perception.

7.4.2 The Binaural Advantage.

7.4.3 The Precedence Effect.

7.4.4 Perceptual Compensation for Spectral Envelope Distortion.

7.5 Reverberation-Robust Acoustic Processing.

7.5.1 Dereverberation.

7.5.2 Reverberation-Robust Acoustic Features.

7.5.3 Reverberation Masking.

7.6 CASA and Reverberation.

7.6.1 Systems Based on Directional Filtering.

7.6.2 CASA for Robust ASR in Reverberant Conditions.

7.6.3 Systems that Use Multiple Cues.

7.7 Discussion and Conclusions.

Acknowledgments.

References.

8. Analysis of Musical Audio Signals (Masataka Goto).

8.1 Introduction.

8.2 Music Scene Description.

8.2.1 Music Scene Descriptions.

8.2.2 Difficulties Associated with Musical Audio Signals.

8.3 Estimating Melody and Bass Lines.

8.3.1 PreFEst-front-end: Forming the Observed Probability Density Functions.

8.3.2 PreFEst-core: Estimating the F0’s Probability Density Function.

8.3.3 PreFEst-back-end: Sequential F0 Tracking by Multiple-Agent Architecture.

8.3.4 Other Methods.

8.4 Estimating Beat Structure.

8.4.1 Estimating Period and Phase.

8.4.2 Dealing with Ambiguity.

8.4.3 Using Musical Knowledge.

8.5 Estimating Chorus Sections and Repeated Sections.

8.5.1 Extracting Acoustic Features and Calculating Their Similarity.

8.5.2 Finding Repeated Sections.

8.5.3 Grouping Repeated Sections.

8.5.4 Detecting Modulated Repetition.

8.5.5 Selecting Chorus Sections.

8.5.6 Other Methods.

8.6 Discussion and Conclusions.

8.6.1 Importance.

8.6.2 Evaluation Issues.

8.6.3 Future Directions.

References.

9. Robust Automatic Speech Recognition (Jon Barker).

9.1 Introduction.

9.2 ASA and Speech Perception in Humans.

9.2.1 Speech Perception and Simultaneous Grouping.

9.2.2 Speech Perception and Sequential Grouping.

9.2.3 Speech Schemes.

9.2.4 Challenges to the ASA Account of Speech Perception.

9.2.5 Interim Summary.

9.3 Speech Recognition by Machine.

9.3.1 The Statistical Basis of ASR.

9.3.2 Traditional Approaches to Robust ASR.

9.3.3 CASA-Driven Approaches to ASR.

9.4 Primitive CASA and ASR.

9.4.1 Speech and Time-Frequency Masking.

9.4.2 The Missing-Data Approach to ASR.

9.4.3 Marginalization-Based Missing-Data ASR Systems.

9.4.4 Imputation-Based Missing-Data Solutions.

9.4.5 Estimating the Missing-Data Mask.

9.4.6 Difficulties with the Missing-Data Approach.

9.5 Model-Based CASA and ASR.

9.5.1 The Speech Fragment Decoding Framework.

9.5.2 Coupling Source Segregation and Recognition.

9.6 Discussion and Conclusions.

9.7 Concluding Remarks.

References.

10. Neural and Perceptual Modeling (Guy J. Brown and DeLiang Wang).

10.1 Introduction.

10.2 The Neural Basis of Auditory Grouping.

10.2.1 Theoretical Solutions to the Binding Problem.

10.2.2 Empirical Results on Binding and ASA.

10.3 Models of Individual Neurons.

10.3.1 Relaxation Oscillators.

10.3.2 Spike Oscillators.

10.3.3 A Model of a Specific Auditory Neuron.

10.4 Models of Specific Perceptual Phenomena.

10.4.1 Perceptual Streaming of Tone Sequences.

10.4.2 Perceptual Segregation of Concurrent Vowels with Different F0s.

10.5 The Oscillatory Correlation Framework for CASA.

10.5.1 Speech Segregation Based on Oscillatory Correlation.

10.6 Schema-Driven Grouping.

10.7 Discussion.

10.7.1 Temporal or Spatial Coding of Auditory Grouping.

10.7.2 Physiological Support for Neural Time Delays.

10.7.3 Convergence of Psychological, Physiological, and Computational Approaches.

10.7.4 Neural Models as a Framework for CASA.

10.7.5 The Role of Attention.

10.7.6 Schema-Based Organization.

Acknowledgments.

References.

Index.

商品描述(中文翻譯)

描述

「如何設計能夠進行『雞尾酒會』聆聽的系統?」人類聽眾能夠從聲音混合物中感知出單一聲源,例如在繁忙的雞尾酒會中從其他聲音和音樂中分辨出單一聲音。我們如何設計「機器聆聽」系統來實現這種感知能力?Albert Bregman於1990年出版的《Auditory Scene Analysis》一書,將聽覺場景的感知與視覺場景進行類比,並描述了對聲音感知組織的一個連貫框架。他的觀點激發了對聽覺計算研究的興趣。這些研究在一定程度上受到實用聲音分離系統的需求推動,這些系統具有許多應用,包括噪音響應自動語音識別、聽覺輔助裝置和自動音樂轉錄。這個新興領域被稱為計算聽覺場景分析(CASA)。

《Computational Auditory Scene Analysis: Principles, Algorithms, and Applications》提供了CASA領域的最新技術的原則、算法和系統架構的全面且連貫的介紹,以及這項令人興奮的新技術的潛在應用。該書由Bregman撰寫前言,各章節由領先的研究人員撰寫,涵蓋了多個主題,包括:

- 多個基本頻率的估計
- 基於特徵和模型的CASA方法
- 基於空間位置的聲音分離
- 用於混響環境的處理
- 語音和音樂信號的分離
- 噪音環境下的自動語音識別
- 聽覺組織的神經和感知建模

本書的內容適合研究相關科學和工程學科的研究生和研究人員閱讀。每章附帶的廣泛參考文獻也使本書成為一個有價值的參考資料來源。附帶的網站(www.casabook.org)提供軟件工具和聲音示範。

目錄

- 前言
- 前言
- 貢獻者
- 縮寫
- 第1章:計算聽覺場景分析的基礎(DeLiang Wang和Guy J. Brown)
- 1.1 人類聽覺場景分析
- 1.1.1 聽覺系統的結構和功能
- 1.1.2 簡單刺激的感知組織
- 1.1.3 語音與其他聲音的感知分離
- 1.1.4 感知機制
- 1.2 計算聽覺場景分析(CASA)
- 1.2.1 CASA是什麼?
- 1.2.2 CASA的目標是什麼?
- 1.2.3 為什麼需要CASA?
- 1.3 CASA系統的基礎
- 1.3.1 系統架構
- 1.3.2 耳蝸圖
- 1.3.3 相關圖
- 1.3.4 交叉相關圖
- 1.3.5 時間-頻率遮罩
- 1.3.6 重合成
- 1.4 CASA評估
- 1.4.1 評估標準
- 1.4.2 語料庫
- 1.5 其他聲音分析方法