Sslis

By: Khalid El-Darymli G0327887 S peech to S ign L anguage I nterpreter S ystem ( SSLIS ) Supervisor: Dr. Othman O. Khalifa International Islamic University Malaysia Kulliyyah of Engineering, ECE Dept.

OUTLINE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Problem Statement ,[object Object],[object Object],[object Object],! IS IT FAIR ?

RESEARCH GOAL AND OBJECTIVES ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Main Parts of the SSLIS Speech-Recognition Engine Sign Language Database Recognized Text ASL Translation Continuous Input Speech Recognized Text

Automatic Speech Recognition ( ASR ): ,[object Object],[object Object],SR Engine Recognized Text Input Voice

Sphinx 3.5 ,[object Object],[object Object],[object Object],[object Object],[object Object]

The Structure of SR Engine (LVCSR) Signal Processing AM P ( A 1 , …, A T | P 1 ,… , P k ) Dictionary P ( P 1 , P 2 , …, P k | W ) LM P ( W n | W 1 , …, W n-1 ) X={x 1 ,x 2 , …, x T } Hypothesis Evaluation Decoder P(X | W)*P(W) TRAINING DECODING Best Hypotheses H = {W 1 , W 2 , …, W k } W BEST Input Audio

SR ENGINE SPECS ,[object Object],0.97 Pre-Emphasis  6855.4976 Hz Higher Filter Frequency ( f h ) 133.33334 Hz Lower Filter Frequency ( f l ) 512 DFT Size 40 Number of Mel Filters 13 Number of Cepstra Mel FIlterbank Filterbank Type 0.025625 sec Window Length 100 frames/sec Frame Rate 16000.0 Hz Sampling Rate Default Value Parameter

KNOWLEDGE BASE ,[object Object],[object Object],[object Object],[object Object],Acoustic Model

DICTIONARY ,[object Object],[object Object],[object Object]

LM ,[object Object],[object Object],[object Object]

SIGN LANGUAGE ,[object Object],[object Object]

AMERICAN SIGN LANGUAGE ( ASL ) ,[object Object],[object Object],[object Object],[object Object]

ASL ALPHABETS ,[object Object],[object Object],[object Object],[object Object],[object Object],Aa Bb Cc Dd Ee Ff Gg Hh Ii Jj Kk Ll Mm Nn Oo Pp Qq Rr Ss Tt Uu Vv Ww Xx Yy Zz

SIGNED ENGLISH ( SE ) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

ASL vs. SE (an Example) It is alright if you have a lot ASL Translation SE Translation IT I S ALL RIGHT IF YOU HAVE A LOT

DEMONSTRATION OF THE ASL IN OUR SW A number of 2,600 ASL prerecorded video clips In case of nonbasic word, extract the basic word out of it Recognized Word (SR engine’s output) Is the basic word within the ASL database vocabulary? The American Manual Alphabet Only in case of a nonbasic input word, append some suitable marker Final Output None of the database contents matched the input basic word No Yes Fingerspelling of the original input word The equivalent ASL video clip of the input word, some marker could be appended

STRUCRURE AND FLOW OF SSLIS ,[object Object],Program Start ,[object Object],[object Object],[object Object],[object Object],[object Object],Has Exit been clicked? ,[object Object],[object Object],[object Object],[object Object],Is Program selected Run selected program Is selected program “ Live Decode”? Enable button to stop “Live Decode” Has stop button been clicked? Stop Program “ Live Decode” Show Program Output Wait for Event from Class Is “ Live Decode” running? Stop “ Live Decode” End Program Initialize Class Continue next slide NO YES NO YES YES NO NO YES YES NO

[object Object],Event raised Word lattice entry received Word lattice ending received Word lattice total received Class Class running in the background Wait for “ INFO” event Call program function “ AddWordLattice” Add word lattice entry to an appropriate table Display “Speech to Text” output Call program function “ AddWordLattice” Add word lattice entry to an appropriate table Live Decode starting received Word hypothesis entry received Total hypotheses entry received Total Frames entry received Call program function “ AddWordHypothesis” Add word hypothesis entry to an appropriate table Call program function “ AddTotalHypothesis” Add Total Hypotheses entry to an appropriate table Display Total Frames entry in appropriate position NO NO Display msg box to user to start decoding Live speech Press ENTER to start recording YES NO YES NO YES YES NO NO YES YES YES YES NO NO NO

PARAMETER TUNNING & ACCURACY MEASUREMENTS ,[object Object],[object Object],[object Object],TUNING THE PRUNING BEHAVIOUR:

TUNING LM RELATED PARAMETERS : ,[object Object],[object Object]

SSLIS CAPABILITIES ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Conclusions ,[object Object],[object Object],[object Object]

Shortcomings &Further Work ,[object Object],[object Object],[object Object]

References ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Thank You ,[object Object],[object Object]

Sslis

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Sslis

Ähnlich wie Sslis (20)

Mehr von kkkseld

Mehr von kkkseld (12)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Sslis