A Data-Driven Question Generation Model for Educational Content - by Jack Wang
•
1 gefällt mir•310 views
Deep learning workshop at Houston machine learning meetup. Video at: https://youtu.be/8fmcn7ull8M?t=3
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie A Data-Driven Question Generation Model for Educational Content - by Jack Wang
Ähnlich wie A Data-Driven Question Generation Model for Educational Content - by Jack Wang (20)
Mehr von Yan Xu
Mehr von Yan Xu (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
A Data-Driven Question Generation Model for Educational Content - by Jack Wang
- 1. QG-Net: Data-Driven Question Generation Model for Educational Content Jack Wang June 25, 2018
- 6. [ J. D. Karpicke et. al., 2012 ] [ R. J. Hift. 2014 ] [ D. Rohrer et. al., 2010 ]
- 9. ?
- 14. … one of the positive contributions of deviance is that it fosters social change. Input text QG-Net
- 15. … one of the positive contributions of deviance is that it fosters social change. Input text QG-Net
- 16. … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? Input text QG-Net Generated question
- 17. … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? Input text QG-Net Generated question ✓ Fluent ✓ relevant
- 18. Input text QG-Net Generated question ✓ Generates fluent and relevant questions
- 19. Input text QG-Net Generated question ✓ Generates fluent and relevant questions
- 20. Input text QG-Net Generated question ✓ Generates fluent and relevant questions
- 21. Input text QG-Net Generated question ✓ Generates fluent and relevant questions ✓ Adaptive to texts from various subjects
- 22. QG-Net
- 23. QG-Net
- 24. QG-Net Reader
- 26. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 27. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 28. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 29. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 30. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 31. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 32. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 33. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 34. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 35. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 36. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 37. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 38. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 39. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 40. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change.
- 41. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 42. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 43. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 44. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 45. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 46. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 47. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 48. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 49. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 50. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 51. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 52. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 53. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? LSTM network LSTM network 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 54. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? LSTM network LSTM network Attention mechanism Copy mechanism 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 55. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? LSTM network LSTM network Attention mechanism Copy mechanism 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation 1. D. Bahdanau et. al., 2014 2. A. See et. al., 2017 1 2
- 56. QG-Net Reader Generator … one of the positive contributions of deviance is that it fosters social change. What is one of the positive contributions of deviance? LSTM network LSTM network Attention mechanism Copy mechanism ✓ Fluent and relevant questions ✓ Adaptive to texts from various subjects 0100010101 0100110100 1010001010 0111010101 01010101... Intermediate representation
- 57. Experiment overview Source sentence (from Wikipedia) Answer (selected from text) Generated question[ ] ● QG-Net learns to generate questions using SQuAD
- 58. Experiment overview Source sentence (from Wikipedia) Answer (selected from text) Generated question[ ] ● QG-Net learns to generate questions using SQuAD Nikola Tesla was a Serbian American inventor, …
- 59. Experiment overview Source sentence (from Wikipedia) Answer (selected from text) Generated question[ ] ● QG-Net learns to generate questions using SQuAD Nikola Tesla was a Serbian American inventor, … Serbian American
- 60. Experiment overview Source sentence (from Wikipedia) Answer (selected from text) Generated question[ ] ● QG-Net learns to generate questions using SQuAD Nikola Tesla was a Serbian American inventor, … Serbian American What is Tesla’s nationality?
- 61. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks
- 62. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks
- 63. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks
- 64. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks
- 65. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks ○ Pre-processed the textbook texts to the same format as SQuAD
- 66. Experiment overview ● QG-Net learns to generate questions using SQuAD ● We test it on three OpenStax textbooks ○ Pre-processed the textbook texts to the same format as SQuAD ○ Compared to 2 of the best existing models (baseline11, baseline 22) 1. M. Heilman, 2010 2. X. Du et. al., 2017
- 67. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people’s reactions to inequality were moderated by class differences and rates of social mobility, ...
- 68. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ...
- 69. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by?
- 70. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,?
- 71. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,?
- 72. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,? Baseline 2: Along with education, race, race, and race, what other groups were affected by the Kinsey scale?
- 73. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,? Baseline 2: Along with education, race, race, and race, what other groups were affected by the Kinsey scale?
- 74. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,? Baseline 2: Along with education, race, race, and race, what other groups were affected by the Kinsey scale?
- 75. Qualitative evaluation result #1 (Sociology) Weber noted that different groups were affected differently based on education, race and gender, and that people's’ reactions to inequality were moderated by class differences and rates of social mobility, ... QG-Net: Along with education and race, what did Weber believe different groups were affected by? Baseline 1: What was education, race, and,? Baseline 2: Along with education, race, race, and race, what other groups were affected by the Kinsey scale? Chapter 12.1 Chapter 1.3
- 76. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson.
- 77. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. Chapter 10.4
- 78. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. Chapter 10.4
- 79. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Chapter 10.4
- 80. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Baseline 1: What did and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfil the vision of a white nation? Chapter 10.4
- 81. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Baseline 1: What did and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfil the vision of a white nation? Chapter 10.4
- 82. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Baseline 1: What did and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfil the vision of a white nation? Baseline 2: What was the name of the act that caused a white nation to become a white nation? Chapter 10.4
- 83. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Baseline 1: What did and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfil the vision of a white nation? Baseline 2: What was the name of the act that caused a white nation to become a white nation? Chapter 10.4
- 84. Qualitative evaluation result #2 (History) The 1830 Indian Removal Act and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfilled the vision of a white nation and became one of the identifying characteristics of the age of Jackson. QG-Net: What act became one of the identifying characteristics of the age of Jackson? Baseline 1: What did and subsequent displacement of the Creek, Choctaw, Chickasaw, Seminole, and Cherokee tribes of the southeast fulfil the vision of a white nation? Baseline 2: What was the name of the act that caused a white nation to become a white nation? QG-Net: ✓ Generates fluent and relevant questions ✓ Is adaptive to text from various subjects
- 85. Human evaluation experiment ● We sample 150 sentences from each textbook ● We use the three models to generate questions ● Human evaluators evaluate the quality of generated questions ○ Fluency ○ relevance ● Human evaluators pick which question(s) could have been generated by human ○ Can choose none of them
- 86. Human evaluation results Baseline 1 Baseline 2 OS = OpenStax
- 87. Human evaluation results Baseline 1 Baseline 2
- 88. Human evaluation results Baseline 1 Baseline 2 QG-Net: ✓ Generates superior questions in terms of both fluency and relevance ✓ Works well for inputs from various subjects
- 89. Human evaluation results Baseline 1 Baseline 2
- 90. Human evaluation results Baseline 1 Baseline 2 Perceptually, QG-Net generates questions of superior quality than those generated by baseline models.
- 92. Input text QG-Net Generated question ✓ Generates fluent and relevant questions ✓ Adaptive to texts from various subjects
- 96. Or
- 97. Acknowledgements Rich Baraniuk Andrew Lan Phil Grimaldi Drew Waters Weili Nie John and Ann Doerr Arthur & Carlyse Ciocca Charitable Foundation
- 98. Questions? QG-Net is on Github! https://github.com/moonlightlane/QG-Net jzwang@rice.edu Input text QG-Net Generated question
- 99. Appendix
- 100. Appendix
- 101. Appendix