生成語彙論の入門と、単語の表現学習、compositionalityの学習に関する入門です

1. 2014/10/16 PFIVZX^
¿Ó½Ǝ»:
—ÐŇǖĦ!5;D
ĉPreferred Infrastructure
Ù¦Ʋǁ (@unnonouno)

2. ƝLJŘ
Ù¦Ʋǁ (@unnonouno)
! ŪåŇĭ¸GdVfuSh€XGÁnj½Ǝ
! ĘƬĭ¸E/H05GKA'
! “‚ú߸ýŵġIBMôLjŎPFI
8

3. mrep5c}P1KA%-
! řĹƑƚ8›%3ÖĨåŇ4íƾ4A'
! pip install mrep 4S€f}
9

4. Ÿ§:Ú
! word2vecP†ù8ƄŇ:¿Ó½ƎŹK:;7%P%A
%-
:

5. ¿Ó½Ǝ:ÚPň3Ĉ0-!5
ƣƵ.5ăJM3LƄ
Ň:+)PrWf}
Į :†8ƅ%ñDJM
L:4;7
;

6. —ÐŇǖĦ
Generative Lexicon (GL)

7. ®:£Ć
! —ÐŇǖĦ813:ƴƄ7šʼn
! ¿Ó½Ǝ5Êů:Ð5:¨Ē
18EA%3Œ:Ú
OJ7,
=

8. The Generative Lexicon [Pustejovsky95]
! James PustejovskyĀdž
! ÄƸA4;ƟDA)Q4%
-
! TimeML:ĀdžŽ4EĐL
! „Å; ¿Ó813:ÚP
%Iƙ03-:40K
%-
ÊÌ5¥%
http://www.cs.brandeis.edu/~jamesp/

9. „Ň:„7J·¦é—:„ [·¦05]
! ·¦ǍƆ—ÐŇǖÊů
Ħ
! ®ƟQ4LǕƥ† ; _ ;
?

10. Æ8ħă8%-´ē
! ·¦ǍƆ—ÐŇǖÊůĦ[·¦05]:Û12
ƽ
! James Pustejovsky, Introduction to
Generative Lexicon. [Pustejovsky05]
! ě„ƲëGenerative Lexicon:NjÇ [ě„
06]
76

11. É‹:Êů813ă3BI
ƛŸ:8‹0-
ŒP8ƍ-
¥ƺ48‹0-
! ‰č8É‹5å03EĬî5%3:É‹5æŲ
5%3:É‹L
77

12. gR:Êů813ă3BI
Ps€V4Ǒ0-
Jš03«#
! wj5%3:gR5‹Ò'L”Ï5%3:gR:Ê
ůL
! é:É‹:ŗ5™I7!5ĸ!037

13. 78

14. ! ă3BL5ƄŇ;1:ƌ7LĎćP
â03L!58Ñ2
! ċOMœ8I03ÊůģO03L
! !:Êů:ģµ8;ęª:ĨƏÎ:I7E:
ĐL
! %Eƌ7LåŇ4Eǃ-ÓłL
79

15. —ÐŇǖĦ:ăœ
! 1:Ňǖ8ęª:ļŋPâ03L
! đǓ8Ī3+:ļŋJƦ§ƌ7L
Êů—Ð#M3L5ăL
7:
É‹
Ĭî:É‹
æŲ:É‹
$_

16. ŅÊ™ŸƌĺŇ5êĺŇ;ĔĖ#ML
! êĺŇ;Êů¢8ŒJ:¨ĒPâ03LÔ
çÇ
! -A-A™Ÿ870-Ň;™ŸƌĺŇ5%
3ĔĖ#ML
7;

17. Ù¦;7*!ÚÝ8ŰůPâ03L
Ī¹‡ƄŇGƣŇ~r}:ÊůPA
Ʊ7Ë݉ŏê5ă3L
%(¿þņ6™Êů
%(¿þ™.6ņÊů
7

18. Ňǖ:ĝŃļŋWU|Rļŋ5E:P
ăL
ŇPń­'LƠÎGł:ğ
1. ļÐWU|RConstitutive
! ƋēG£Ć76Œ4ˆÒ3LPā'
2. řĚWU|RFormal
! +MŒ7:‡ì:NjōPā'
3. ²¢WU|RTelic
! ÁĊG²¢Pā'
4. ƱWU|RAgentive
! +MP—Bˆ'¾ÿGÄƉPā'
7=

19. Œ‚ģ+.
7

20. ŗò:WU|Rļŋ
7?
1. ļÐ`Syǀ
k€g}
2. řĚŀKîÁ
nj
3. ²¢ŀLĄĩ
'L
4. ƱæB©3L
ÿL
@1AKN`KBC0
L4GM*!JDIBQS[^T

21. Œ4!%-úßĠÇ7:

22. ŔŌ:ò;ş
ŔŌ:ò;ngş
! ò;Ŋ¾'L5ƿǏ:Ƥø!!4;
²¢WU|RĐL
! +:-D…;«:đ:Êů.5øDŽ4
L
86

23. ƇőƿǏ:ƤøPįŁ'L5Y€f87L

24. ƭǎä4Dž3-Q.I Ɯö4
lhtS

25. 60/

26. ä4

27. Ɯö4

28. ƭǎ.Jä:†8LƜö:†.I
! Ɯö;ä8LE:5ƿǏ:ƤøPįŁ%3L
! ƶ8ärbg8703-Jƭǎ:^|
o;­J8ÀŪ
1Ƨ:åŇĭ¸5:PGK-9
87
^[]U5@CLASSICAOP

29. 7Q4WU|RļŋPăL:

30. ! ê:ƄŇ:Êů8¨%3é8•-I7ƿǏ:•
øūĢ'L
! +:-D-Eáơ#M-:I7¿Ó4Êů
ª0-K+0K7đĖ:Êů87L!5ĐL
! !:ƿǏ:•øJ³ï:đ†4:Êů—Ð#
ML:4;7
88

31. begin8;į×:Êů—Ð#ML
begin the meeting
begin a dance
begin the book
begin the movie
! +E+Ebegin:58;‹ƒGł4;

32. ! beginį×:ÊůPâ03L5ĈIKđǓ8Ī
3Êů—Ð#ML5ăL?Ū
89

33. ęÐco-composition
begin

34. the book
1. begin:žN8;łÒL;(.book;ł4
;7
,
2. bookPł5%3øDŽ%I`SqÍÈ
3. WU|RJúßPÂ03L
‰¶:—Ðq^P 5Ž=
8:

35. cf. ƿǏ:ŝģŶ
double x =

36. 10
1. double:žN8;double:ŞÒL;(.10;
double4;7
,
2. 10Pdouble:Ş5%3øDŽ%I
3. Vxf:œ¯:úßPÂ03L
‰¶:—Ðq^P13*,5Ž=
8;

37. ħă³;—ÐŇǖĦ8;ĴK31:ļŋL
ĝŃļŋ;—ÐŇǖĦ:41ļŋ:†:11
! Ƴļŋ
! łļŋ
! ĝŃļŋ
! Ňǖżƕļŋ
®;ėŷ%A'I
8

38. !!A4:A5D
! ĥŇǖ8;+:ċOMœ8¨'LúßPâ0
3LWU|Rļŋ
! ŹŴ:ƄŇ5śćªK:øDŽ478
!:WU|R:úßPċ03¥%ÊůP—Ð
'LęÐ
8=

39. ŪåŇĭ¸:†4—ÐŇǖĦ;6MJċOM
3-

40. ! åŇĭ¸½ƒ:«430÷
! ACL:«4300÷
8
ÊÌ5Ĥ7

41. /H05!M;‚ģ
! 11:Ňǖ4!MJ:úß
5 .
$./LHE,
@A M5BAM

42. +3
8?
45.
$.#/R%BFSY]W2M( $

43. WU|Rļŋ;‚ĨũdVfJ¾Ʈõ%-
! Web:íƾóČJ…’4ÿÐ%-m`€
Pܹ%3ǐˆ'Lœ¯ [Cimiano+07]
! ęĸǔ°5ðǒG¾ǒ:ƠÎPĝƯ8%3{
€V€X½ƎPƂ¹'Lœ¯ [ĻŖ+12]
5;ċ:EƴƄ4;7#+
96

44. Ĥ%ƖJ•7%3BL
! ƄŇ6:I7ƃ:ƄŇ5‰Ƽ8ċOML
5!58›'LüŭP±ųµ%3L
! 6:I7ƄŇ5ċOML8I03ƄŇ:
ÊůEģOL
97

45. !:Ú6!4
٤:
Statistical Semantics:Ú8ǃ3L
98

46. !!4‰°—ÐŇǖĦJŮM3
Êů:Ð:ÚP%A'
99

47. Statistical Semantics:#J
Statistical Semantics is the study of how the
statistical patterns of human word usage can be
used to figure out what people mean, at least to
a level sufficient for information access”(ACL
wiki)
! ŹŴƄŇ:ˆÓǔ°76:IJè¢7ÎŃJ+:Ň:
ÊůPăL
! ‹ƚ‘øGhz{}ibf76:’¯¹JML
9:

48. ƄŇ:rWf}¿Ó
! Ÿ§:Ú;ƄŚ:ƄŇ:ÊůP¿Ó%-
! ®§;ŹŴƄŇ5:¨Ē4ÊůģOLÚ7:
4/(J7Lo~]:Êů67
L:

49. 9;

50. —ÐŇǖĦ5Statistical Semantics:¨Ē
! —ÐŇǖĦ
! ĝ¡:đǓ8LƄŇ:ˆÓ:ÊůPă3L
!
5+:I7ģµPĸ!'-D:Ňǖ:ļ
ŋPă3L
! Statistical Semantics
! ŹŴƄŇ:‘Ɯ8I03‰Ɣ¢7+:ƄŇ:Êů
Pă3L
! ƀ±¢7ƄŇ:ˆÓ8›'LÊů8åſ%37
®J'LI
9

51. Principle of compositionality (Frege1892]
… the meaning of a complex expression is determined by
the meanings of its constituent expressions and the rules
used to combine them (wikipedia

52. )

53. 9=

54. ·Š#Q:{SgA5A03L
https://speakerdeck.com/mamoruk/a-tensor-based-factorization-model-of-semantic-compositionality
9

55. ¿ÓrWf}:Ð;ĵ%ĕIKƻĕķ
[Mitchell+08]
! ĵ%ĕGƻĕ76rWf}:Ðœ¯Pılj
! ÇƑ5:ƻĕMultiplicative¬0/F0-
9?

56. Statistical Semantics5:¨Ē
! Statistical SemanticsŸ§:£Ć;6/J
5å5#-(:rWf}PƮõ'LŎŠ
! Compositionality:ŎŠ;+:rWf}:
:ŎŠ
ĞœP™8øœØEL%¿Ó½Ǝ:ˆÓ
4ƁK‡KP•)3L
:6

57. Êů:ÐP‹ƚ4¿Ó'L (MV-RNN) [Socher
+12]
! ĥƄŇ8;rWf}5‹ƚ›Ī'L
! ŵƹ#ML5ƪ:‹ƚP¼’:rWf}8ƻ3
#J8žĶř7ģŶ f PŦż¢8‹
! RootA4ƗKľ'5đ:¿ÓrWf}õJML
:7

58. GGd€_}. [Cruys+13]
! ÆŇ¾ǒ²¢Ň:31Pd€_}Pċ03æ)
3L:.5ăL
:8

59. Dynamic Convolutional Neural Network
[Kalchbrenner+14]
! CNNPċ03ƄŇ:A5AKPæB‡ 3
! ļđÞ4;7Żǃ:ŜČõJML

60. :9

61. ƇőCNN5compositionality
! Ŀ~Sy4;ŧD3ijÏ¢ƄƷ7úß
! ˜Ă:~Sy87L8¶M3+MJæBO#K
IKƣƵ7Êů:LúßPûŭ%3L
::
Layer1
Layer3
Layer5
[Zelier+14]OP'

62. åŇĭ¸;ĸÔņ
4
nW^}Tb[¤‘ǂ
źřŸƑƄŇ–Ú
¿Ó

63. ƄŇo~]đ
‘ø'LæB‡ L
M- :;
M-

64. !:œŐ8—ÐŇǖĦ:ü•Pď)7
! —ÐŇǖĦ;CompositionalityP…’4űƘ%
3±ų2L-D8Ľ¸#M-¸Ħ5•M7

65. ! šƕ½Ǝ76:m{aSvZofĸ+7
®!+ĜŨ:ü•GŹŴ‘¦P•ą'
:

66. Ãà:Ú
ƣƵ.5ăJM3LƄ
Ň:+)PrWf}
Į :†8ƅ%ñDJM
L:4;7
:=

67. ęÐP¿ÓrWf}85K!C [Tsubaki+13]
companyM
JMrun
run companyM)
run
LOQ
! ƪƪ8ŕŢPſ@%3¥%rWf}PÿL
! 4-rWf}JÊů:Ð‹OML
:
run companyM
Mrun
company
LOQ
runM
JMcompany
Mcompany
'48

68. ƄŇ:Ð8›'L1:ŁÔ
! —ÐŇǖĦ
! åŇÓłPűƘ%3Awe}µ%I5'L
īB
! Ť:׸we}
! űƘ#M-åŇe`P‡’ń­'LI7we
}:ŬƘ
! ¿Ó½Ǝųšƕ½Ǝų
! ļÐ5¿Ó:½ƎP™8‹5%3L
!MJ:RSeR;ƞţ8¨Ē%3L
:?

69. A5D
! —ÐŇǖĦ5;

70. ! Ňǖ;8›'LÀƊº7üŭJđǓ8Ī3Ê
ů—Ð#ML5ăL
! Ň:Êů:Ð8¨'LŎŠ
! ƄŇ:¿ÓrWf}PÐ%3ƣ¿Ó:ÊůP
ÿL
! ¿Ó½Ǝpv8ŀ03®İ‹03L
åŇ½:ü•5åŇĭ¸:¥%’¯:ã
E05ÕC

71. ;6

72. ƓƢK5$A%-
;7

73. ħăđƫ (1/3)
! [Pustejovsky95] James Pustejovsky.
The Generative Lexicon. MIT Press.
! [·¦05] ·¦ǍƆ.
(4+$. N%ˆư
! [Pustejovsky05] James Pustejovsky.
Introduction to Generative Lexicon. Foundations of
SemanticsƩĺ´ē.
http://www.cs.brandeis.edu/~jamesp/classes/LING130/
! [ě„06] ě„Ʋë.
Generative Lexicon2. Ɛ:ğ†NJ̓:ŏÌƈ
;8

74. ħăđƫ(2/3)
! [Cimiano+07] Philipp Cimiano, Johanna Wenderoth.
Automatic Acquisition of Ranked Qualia Structures from the
Web. ACL2007
! [ĻŖ+12] ĻŖ˜ƨ, ·Šť, ě„Ʋë.
(4$
(!')

75. .
0. NLP2012.
! [Mitchell+08] Jeff Mitchell, Mirella Lapata.
Vector-based Models of Semantic Composition. ACL2008.
! [Socher+12] Richard Socher, Brody Huval, Christopher D.
Manning, Andrew Y. Ng.
Semantic Compositionality through Recursive Matrix-Vector
Spaces. EMNLP2012.
;9

76. ħăđƫ(3/3)
! [Cruys+13] Tim Van de Cruys, Thierry Poibeau, Anna Korhonen.
A Tensor-based Factorization Model of Semantic
Compositionality. NAACL2013.
! [Kalchbrenner+14] Nal Kalchbrenner, Edward Grefenstette, Phil
Blunsom.
A Convolutional Neural Network for Modelling Sentences.
ACL2014.
! [Zelier+14] Matthew D. Zeiler, Rob Fergus.
Visualizing and Understanding Convolutional Networks.
ECCV2014.
! [Tsubaki+13] Masashi Tsubaki, Kevin Duh, Masashi Shimbo,
Yuji Matsumoto.
Modeling and Learning Semantic Co-Compositionality through
Prototype Projections and Neural Networks. EMNLP2013
;: