Dereverberation in the stft and log mel frequency feature domains
•Als PPT, PDF herunterladen•
2 gefällt mir•2,281 views
Slides explaining a few algorithms for dereverberation, removing the effect of reverberation from far-field recordings.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (12)
Ähnlich wie Dereverberation in the stft and log mel frequency feature domains
Ähnlich wie Dereverberation in the stft and log mel frequency feature domains (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Dereverberation in the stft and log mel frequency feature domains
- 1. 1 April 2012 Dereverberation in the STFT and log mel-frequency feature domains Takuya Yoshioka
- 2. Dereverberation is necessary for many speech applications“ ”
- 3. 0 10 20 30 0.2 0.3 0.4 0.5 0.6 ASR (connected digit recognition) T60 in seconds Worderrorratein%
- 4. ASR (LVCSR using WSJ-20K) 0 20 40 60 80 100 Clean training +MLLR Multi-style training Worderrorratein%
- 5. Source separation T60=0.3 s T60=0.5 s 0 2 4 6 8 10 12SNRindB
- 6. And others… • Source localization • Adaptive beamforming • VAD
- 7. Dereverberation is necessary for many speech applications“ ”
- 8. Acoustic feature extraction process STFT | ・ |2 Mel FB Log compression DCT Δ, ΔΔ Microphone Decoder
- 9. Acoustic feature extraction process STFT | ・ |2 Mel FB Log compression DCT Δ, ΔΔ Microphone Decoder STFT coefficients Fully benefit from the use of microphone arrays
- 10. Acoustic feature extraction process STFT | ・ |2 Mel FB Log compression DCT Δ, ΔΔ Microphone Decoder Power spectra Easy to combine with noise suppressors
- 11. Acoustic feature extraction process STFT | ・ |2 Mel FB Log compression DCT Δ, ΔΔ Microphone Decoder Log mel-frequency features Efficient for reducing the acoustic mismatch between observations and training data
- 12. n : frame index ny : corrupted vector nx : clean vector nxˆ : estimate of xn Notations
- 13. Optimal estimation in the MMSE sense ∫= nn xxˆ ),,|(p 1nnYY,|X past yyx ndx
- 14. ∫= nn xxˆ ),,|(p 1nnYY,|X past yyx ndx ),,,|(p 11-nnnYX,|Y past yyxy )(p nX x × Clean speech modelReverberation model Generative approach (using Bayes rule)
- 15. ∫= nn xxˆ ),,|(p 1nnYY,|X past yyx ndx ),,,|(p 11-nnnYX,|Y past yyxy )(p nX x × Clean speech modelReverberation model Generative approach (using Bayes rule)
- 16. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain Linear prediction VTS
- 17. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 18. n : frame index ny : corrupted complex-valued spectrum (consisting of 257 bins) nx : clean complex-valued spectrum nxˆ : estimate of xn Notations
- 19. ∏= j X jn,jn,CNnX,nX )λ;0,(xf)Λ;(p x Clean STFT coefficients: normally distributed X Jn, X n,1 λ,...,λ X nP1,...,p X pn, σ,)(a = 2 p piωX pn, X nX jn, j ea1 σ λ ∑ − − = All-pole model No model Model Form Parameters Clean PSD
- 20. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 21. 1-source 1-microphone case: multi-step LP ∑≥ − ∗ += Δp jp,njp,jn,jn, ygxy 1,2,...njn, )(y = 1,2,...njn, )(x =
- 22. 1-source 1-microphone case: multi-step LP ∑≥ − ∗ += Δp jp,njp,jn,jn, ygxy ＋ 1,2,...njn, )(y = 1,2,...njn, )(x =
- 24. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 25. When model parameters are known jn,p jp,jn,jn, ygyx ∑ ∗ −= ˆˆ )ygyδ(x jn,p jp,jn,jn, ∑ ∗ +−= ˆ )Λ,Λ;y,y|(xp RXj1,jn,jn,YY,|X past ˆˆ Inverse filtering
- 26. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 27. ML for parameter estimation ∑∑= j n RXj1,j1,-njn,Y|YRX )Λ,Λ;y,y|(ylogp)Λ,L(Λ past
- 28. ML for parameter estimation ∑∑= j n RXj1,j1,-njn,Y|YRX )Λ,Λ;y,y|(ylogp)Λ,L(Λ past ∫ × )xygδ(y )Λ;y,,y,x|(yp jn,jn,p jp,jn, Rj1,j1,-njn,jn,YX,|Y past −−= ∑ ∗ ∏= j X jn,jn,CN nX,nX )λ;0,(xf )Λ;(p x
- 29. ML for parameter estimation ∑∑= j n RXj1,j1,-njn,Y|YRX )Λ,Λ;y,y|(ylogp)Λ,L(Λ past ∑∑ ∑ − ∗ − −−= j n X jn, 2 p jp,njp,jn,X jn, λ |ygy| )log(λ
- 30. ML for parameter estimation ∑∑= j n RXj1,j1,-njn,Y|YRX )Λ,Λ;y,y|(ylogp)Λ,L(Λ past ∑∑ ∑ − ∗ − −−= j n X jn, 2 p jp,njp,jn,X jn, λ |ygy| )log(λ ∑ ∑ − ∗ − = n X jn, 2 p jp,njp,jn, Λ jR, λ |ygy| argminΛ jR, ˆ ˆ If is knownX jn,λˆ
- 31. Iterative optimization Initializing ΛR Inverse filtering Updating ΛR Convergent? Updating ΛR RΛˆ RΛˆ XΛˆ
- 32. Why LP model for reverberation? Chain rule is applicable to derive the likelihood function
- 33. Drawback Non-minimum phase terms cannot be accurately modeled “ ”Solution: using extra microphones
- 34. Extensions • Integration with source separation • Integration with additive noise reduction • Adaptive inverse filtering – Using an RLS-like algorithm • Application to music signals – Using a clean source model accounting for strong harmonic structures • Exploiting prior knowledge on room properties
- 35. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 36. n : frame index ny : corrupted log mel-frequency feature (consisting of 24 coefficients) nx : clean log mel-frequency feature nxˆ : estimate of xn Notations
- 37. ∑= k X k X knNkXnX ),;(fπ)Λ;(p Σμxx Clean features: pre-trained GMM )Λk;|(p XnK|X xDenoted by
- 38. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 40. Reverberation model Early reflections Late reverberation H=nY +⋅ nX nR Direct sound
- 41. Reverberation model Early reflections Late reverberation ＊ Clean speech RIR > 50ms H=nY +⋅ nX nR Direct sound
- 43. Reverberation model Early reflections Late reverberation ),,( ))--exp(log(1 nn nnnn hrxg hxrhxy = +++= )),,(δ()Λ;,|(p nnnRnnnRX,|Y hrxgyrxy −= Direct sound
- 44. Reverberation model )),,(δ()Λ;,|(p nnnRnnnRX,|Y hrxgyrxy −= );( RR -nnNR11-nnY|R ,f)Λ;,,|(p past Σβyryyr += ∆ ∫×
- 45. Reverberation model ),;(f)Λk;,,,,|(p X|Y kn, X|Y kn,nNR11-nnnK,YX,|Y past Σμyyyxy ≈ ),,( ))(,,( R Δn X k X kn R Δn X k X|Y kn, hβyμg μxhβyμGμ ++ −+= − − R2R Δn X k X|Y kn, )),,(( ΣhβyμGIΣ +−= −
- 46. STFT domain Clean speech model Reverberation model Posterior distribution Parameter estimation Clean speech model Reverberation model Posterior distribution Parameter estimation Log mel-frequency feature domain
- 47. pastY|R p K|X pg KR,|Y pK,YX,|Y past p pastY|Y p K,Y|Y past p pastYY,|K p K,YY,|X past p K,YY,|R past p pastYY,|X p kπ Relationship among pdfs
- 48. Connected digit recognition • 1024-component GMM for VTS • Clean complex back-end defined in Aurora2 • Evaluation data set consisting of 4004 reverberant utterances – Simulated data – Impulse responses measured in a varechoic room – Speaker-microphone distance = 3.5 m – T60 = 0.2~0.6 sec
- 49. 0 5 10 15 20 25 30 35 0.2 0.3 0.4 0.5 0.6 Unprocessed Dereverberated Dereverberated (lower bound) Worderrorratein% T60 in seconds
- 50. Concluding remarks • Dereverberation can be performed in different domains • Reverberation model must accounts for the strong statistical dependencies between consecutive observation frames