自然言語処理における構文解析器の構造と解釈
•
5 gefällt mir•1,674 views
JOIss2014 発表資料 https://github.com/zakuro9715/shino
Empfohlen
Weitere ähnliche Inhalte
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Empfohlen
Empfohlen (20)
自然言語処理における構文解析器の構造と解釈
- 1. ⮬↛ゝㄒฎ⌮䛻䛚䛡䜛 ᵓᩥゎᯒჾ䛾ᵓ㐀䛸ゎ㔘 Structure and Implementation of the Parsers for Natural Language Processing JOIss2014 @zakuro9715
- 2. ⮬ᕫ⤂
- 3. YuZakuro @zakuro9715 ᫂▼㧗ᑓ ༟⌫(㏆␥3) HN twitter Ꮫᰯ ㊃
- 4. ᡂᯝ≀
- 6. 䛸䛣䜝䛷
- 7. 䝇䝷䜲䝗䛾ᯛᩘ䛜
- 8. 101ᯛ
- 9. 䛒䜛䛾䛷ᛴ䛞┠䛷䜔䜚䜎䛩
- 10. What is Parser? ᵓᩥゎᯒჾ 䛳䛶䠛
- 11. ᵓᩥゎᯒჾ ● 䛘䜙䜜䛯䝖䞊䜽䞁ิ䜢ゎᯒ䛧䚸 ᩥ䛾ᵓ㐀䜢⾲䛩䝕䞊䝍ᵓ㐀(୰㛫⾲⌧)䛻ኚ 䛩䜛䚹
- 12. ᵓᩥゎᯒჾ ● 䛘䜙䜜䛯䝖䞊䜽䞁ิ䜢ゎᯒ䛧䚸 ᩥ䛾ᵓ㐀䜢⾲䛩䝕䞊䝍ᵓ㐀(୰㛫⾲⌧)䛻ኚ 䛩䜛䚹 構文木・AST・S式など
- 13. せ䛩䜛䛻…
- 14. I am You
- 15. ᩥ ྡモྃ ே⛠௦ྡモ I ືモྃ ືモ am ྡモྃ ᅛ᭷ྡモ You
- 16. Tips 1. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ 2. 䛔䜝䛔䜝䛺୰㛫⾲⌧
- 17. Tips 1. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ 2. 䛔䜝䛔䜝䛺୰㛫⾲⌧
- 18. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ ● ᵓᩥゎᯒ䛻䛛䛡䜛䛯䜑䛻䛿䚸ᩥ䜢䝖䞊䜽䞁 䛻ศゎ䛩䜛ᚲせ䛜䛒䜛 ● 䝖䞊䜽䞁䜈䛾ศゎ䛿Ꮠྃゎᯒჾ䛜⾜䛖
- 19. You are PRO ᩥᏐิ
- 20. [You, are, PRO] 䝖䞊䜽䞁ิ
- 21. 䛒䛺䛯䛿䝥䝻䛷䛩 ᩥᏐิ
- 22. [䛒䛺䛯, 䛿, 䝥䝻, 䛷䛩] 䝖䞊䜽䞁ิ
- 23. 䝥䝻 ᩥᏐิ
- 24. [䝥䝻] 䝖䞊䜽䞁ิ
- 25. Tips 1. ゎᯒᑐ㇟䛿䝖䞊䜽䞁ิ 2. 䛔䜝䛔䜝䛺୰㛫⾲⌧
- 26. S-Expression (S (NP (PeopleN (‘I’))) (VP (V (‘am’)) (NP (ProperN ('You')))))
- 27. Syntax Tree S NP PeopleN I VP V am NP ProperN You
- 28. 䛭䛾䛔䜝䛔䜝 ● Abstract Syntax Tree ● Reverse Polish Notation ● Polish Notation
- 29. Context Free Language ᩥ⬦⮬⏤ᩥἲ
- 30. ᩥ⬦⮬⏤ᩥἲ ● ᩥἲ䜢ᐃ⩏䛩䜛䛯䜑䛻⏝䛥䜜䜛ᩥἲ ● ṇつ⾲⌧䛷䛿ᐃ⩏䛷䛝䛺䛔ᩥἲ䜢ᐃ⩏䛷䛝䜛
- 31. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes the dog 䛾ᵝ䛺ᩥἲ䜢⏕ᡂ䛩䜛CFG
- 32. Implements of typical parsers ௦⾲ⓗ䛺ᵓᩥゎᯒჾ䛸䛭䛾ᐇ
- 33. ௦⾲ⓗ䛺ᵓᩥゎᯒჾ ● Recursive Descent Parser ● Shift-Reduce Parser
- 34. ௦⾲ⓗ䛺ᵓᩥゎᯒჾ ● Recursive Descent Parser ● Shift-Reduce Parser
- 35. ≉ᚩ ● ẚ㍑ⓗᐇ䛜ᐜ᫆䛷䛒䜛 ● 䜶䝷䞊䝯䝑䝉䞊䝆䜢⾲♧䛩䜛䛾䛜ᐜ᫆
- 37. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 38. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 39. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 40. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 41. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 42. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 43. S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes” a cat likes a dog
- 44. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 45. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 46. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 47. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 48. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 49. a cat likes a dog S NP N Det VP V -> -> -> -> -> -> NP VP Det N “cat” | “dog” “the” | “a” V NP “likes”
- 50. ௨ୗ␎
- 51. ᙅⅬ ● ᕥᖐ ● Ỵᐃ⬟ᛶ
- 52. ᕥᖐ ● ྑ䛾䜘䛖䛺CFG䜢⪃䛘䜛 䛣䛾CFG䛿௵ពಶ䛾”a”䜢୪䜉䛯ᩥᏐ ิ䜢⾲䛩䛜䚸䝟䞊䝇䛩䜛䛸↓㝈䝹䞊䝥 䛻㝗䜛 S -> S A | A A -> “a”
- 53. a a a a a S A -> -> S A | A “a”
- 54. a a a a a S A -> -> S A | A “a”
- 55. a a a a a S A -> -> S A | A “a”
- 56. a a a a a S A -> -> S A | A “a”
- 57. a a a a a S A -> -> S A | A “a”
- 58. a a a a a S A -> -> S A | A “a”
- 59. a a a a a S A -> -> S A | A “a”
- 60. a a a a a S A -> -> S A | A “a”
- 61. ゎỴ᪉ἲ ● ᩥἲ䜢௨ୗ䛾䜘䛖䛻ኚᙧ䛩䜛 S S_ A -> -> -> A S_ “” | A S_ “a”
- 62. Ỵᐃ⬟ᛶ ● ぢ䛶䛔䜛䝖䞊䜽䞁䛷ᵓᩥ䜢Ỵᐃ䛩䜛䛾䛷䚸ఝ䛯 ᵓᩥ䛜䛒䜛ሙྜ䛿Ỵᐃ䛷䛝䛺䛔䛣䛜䛒䜛 ● 䛯䛸䛘䜀ୗ䛾䜘䛖䛺ᵓᩥ䛿A䜢ぢ䛯Ⅼ䛷䛿Ỵ ᐃ䛷䛝䛺䛔 S A B C -> -> -> -> A B | A C “a” “b” “c”
- 63. ゎỴ᪉ἲ ● A䜢ぢ䛯Ⅼ䛷䛿Ỵᐃ䛷䛝䛺䛔䛾䛷䚸䜒䛖䜂䛸䛴 ඛ䜢ぢ䛶ุ᩿䛩䜛䚹 S A B C -> -> -> -> A B | A C “a” “b” “c”
- 64. ゎỴ᪉ἲ ● ඛㄞ䜏䜢ቑ䜔䛩䜋䛹ィ⟬㔞䛜ቑ䛘䛶䛔䛟䛯䜑⌧ ᐇⓗ䛷䛿䛺䛔
- 65. ௦⾲ⓗ䛺ᵓᩥゎᯒჾ ● Recursive Descent Parser ● Shift-Reduce Parser
- 66. ≉ᚩ ● LLἲ䛜ⱞᡭ䛸䛩䜛ᩥἲ䜢ゎᯒ䛷䛝䜛 ● 䜶䝷䞊䝯䝑䝉䞊䝆䜢⾲♧䛩䜛䛾䛜㞴䛧䛔
- 67. ཎ⌮ ● 䝖䞊䜽䞁䜢䝇䝍䝑䜽䛻✚䜏䛴䛴䚸䝇䝍䝑䜽䛾ୖ䛛䜙 䝹䞊䝹䜢㐺⏝䛥䛫䜛 ● 䝇䝍䝑䜽䛻✚䜐ືస䜢Shift, 䝹䞊䝹䜢㐺⏝䛩䜛 ືస䜢Reduce䛸䜆
- 68. a cat likes a dog
- 69. cat likes a dog a Shift
- 70. cat likes a dog Reduce Det
- 71. likes a dog Det Shift cat
- 72. likes a dog Reduce N Det
- 73. likes a dog Reduce NP
- 74. a dog NP Shift likes
- 75. a dog Reduce V NP
- 76. dog a NP Shift V
- 77. dog Reduce Det V NP
- 78. dog Det NP Shift V
- 79. Reduce N Det V NP
- 80. Reduce NP V NP
- 81. Reduce S
- 82. Recursive Descent Parser 䛾ᙅⅬ ● ᕥᖐ ● Ỵᐃ⬟ᛶ
- 83. ᕥᖐ ● ⮬᫂䛻Ⓨ⏕䛧䛺䛔
- 84. ᕥᖐ ● ᕥᖐ䛷↓㝈䝹䞊䝥䛻㝗䜛ཎᅉ䛿䚸ᩥἲ䛻ᑐ 䛧䛶䝖䞊䜽䞁䜢ᙜ䛶䛶䜖䛟䛛䜙 ● 䝖䞊䜽䞁䛻ᑐ䛧䛶ᩥἲ䜢ᙜ䛶䜛᪉ἲ䛷䛿Ⓨ⏕䛧 䛺䛔
- 85. Ỵᐃ⬟ᛶ ● Recursive Descent Parser䛸ྠᵝ䛾ၥ㢟䛜Ⓨ⏕ 䛧䛺䛔䛣䛸䛿᫂䜙䛛
- 86. Ỵᐃ⬟ᛶ ● Recursive Descent Parser 䛷䛿᭱ึ䛾䝖䞊䜽䞁 䜢Ⓨぢ䛧䛯Ⅼ䛷ᩥἲ䜢Ỵᐃ䛧䛯 ● 䝖䞊䜽䞁䜢ぢ䛶䚸㐺⏝䛷䛝䜛ᩥἲ䜢᥈䛩᪉ἲ䛷䛿 ྠᵝ䛾ၥ㢟䛿Ⓨ⏕䛧䛺䛔
- 87. LLἲ䛾ᙅⅬ䜢ඞ᭹
- 88. 䜔䛳䛯䛽
- 89. 䜔䛳䛯䛽 䛷䜒…
- 90. Shift-Reduce Parser 䛾ᙅⅬ ● Shift-Reduce⾪✺ ● Reduce-Reduce⾪✺
- 91. Shift-Reduce Parser 䛾ᙅⅬ ● Shift-Reduce⾪✺ ● Reduce-Reduce⾪✺
- 92. Shift-Reduce ⾪✺ ● ୗ䛾䜘䛖䛺ᩥἲ䛷䚸”a”䛜䝇䝍䝑䜽䛻䛒䜛䛸䛝 Reduce䛩䜛䛛Shift䛩䜛䛛䛜Ỵᐃ䛷䛝䛺䛔 S A B -> -> -> A | A B “a” “b”
- 93. ゎỴ᪉ἲ ● ඛㄞ䜏
- 94. Shift-Reduce Parser 䛾ᙅⅬ ● Shift-Reduce⾪✺ ● Reduce-Reduce⾪✺
- 95. Reduce-Reduce ⾪✺ ● ୗ䛾䜘䛖䛺ᩥἲ䛷䚸”a”䛜䝇䝍䝑䜽䛻䛒䜛䛸䛝䚸A1 䜢㐺⏝䛩䜛䛛A2䜢㐺⏝䛩䜛䛛Ỵᐃ䛷䛝䛺䛔 S A1 A2 -> -> -> A1 ‘x’ | A2 ‘y’ “a” “a”
- 96. ゎỴ᪉ἲ ● ඛㄞ䜏
- 97. Summary 䜎䛸䜑
- 98. ● Recursive Descent Parser 䛿ᐇ䛜⡆༢ ○ 䛯䛰䛧ᙅⅬ䜒ከ䛔 ● Shift-Reduce Parser 䛿ᗄ䛴䛛䛾ᙅⅬ䜢ゎᾘ ○ ᪂䛧䛔ᙅⅬ䛜…
- 99. ⤖ㄽ
- 100. ඛㄞ䜏
- 101. 䛤Ύ⫈䛒䜚䛜䛸䛖䛤䛦䛔䜎䛧䛯