Software Security Education at Scale
•Download as PPTX, PDF•
1 like•335 views
The document discusses research into scaling software security education online to help retrain professionals in the workforce. It describes flipping a university class into online video lectures and using the videos to create an online course. The research questions examine why software engineers enrolled, how their performance compared to university students, and feedback on the online format. Challenges discussed include technical issues, varying participant backgrounds, and underestimating time commitments. Improving future courses involves addressing these challenges, iterating based on lessons learned, and exploring additional online platforms.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (13)
Similar to Software Security Education at Scale
Similar to Software Security Education at Scale (20)
More from Chris Theisen
More from Chris Theisen (6)
Recently uploaded
Recently uploaded (20)
Software Security Education at Scale
- 1. Software Security Education At Scale Chris Theisen, Laurie Williams, Emerson Murphy-Hill, Kevin Oliver {crtheise, lawilli3, emurph3, kevin_oliver}@ncsu.edu North Carolina State University National Science Foundation Grant Number 4900-1318428.
- 2. Introduction • Cisco 2014 Annual Security Report: Worldwide shortage of 1 million security professionals • Educating students is no longer enough! • How do we help retrain people who are currently in the workforce? 2Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 3. Solution: Online Coursework 3Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 4. “Flipping” A University Class • Students watch video lectures, listen to Silver Bullet Podcast before the class takes place, take a quiz • Class time devoted to exercises, discussion, etc. • Videos can then be reused for online course 4Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 5. 5
- 6. 6
- 7. Research Questions • RQ1: Why did software engineers sign up for the online course? • RQ2: How do software engineers in the online course perform on quiz and test questions relative to university students being taught in an on-campus setting? • RQ3: How well does the online course format work for software engineering professionals? What could be improved on for future courses? 7Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 8. Research Questions • RQ1: Why did software engineers sign up for the online course? • RQ2: How do software engineers in the online course perform on quiz and test questions relative to university students being taught in an on-campus setting? • RQ3: How well does the online course format work for software engineering professionals? What could be improved on for future courses? 8Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 9. Time Commitment 9Introduction | Methodology | Lessons Learned | Next Steps | Conclusion • For students: even asking 2-3 hours a week is a lot. • Specific assignment deadlines should be relaxed. • For instructors: Take your first guess, double it. • Video editing, message boards, technical problems, email, language barriers, etc…
- 10. Technical Issues 10Introduction | Methodology | Lessons Learned | Next Steps | Conclusion • Issues with Course Builder – Quizzes stopped working night before we launched – Slow response times of the site itself – Fixed in latest version • Peer review project had to be scrapped – Should have required it be complete before course launched
- 11. Consider Your Audience 11Introduction | Methodology | Lessons Learned | Next Steps | Conclusion • Wider spread of participants means… – Can’t assume background knowledge • Participants included: – Administrative assistant working with sec. professionals – High school teacher teaching a CS class with minimal background
- 12. Discussion Video 12Introduction | Methodology | Lessons Learned | Next Steps | Conclusion
- 13. Iterate and Improve 13Introduction | Methodology | Lessons Learned | Next Steps | Conclusion • Rerun the online course (tentatively this fall) • OpenEDx, new Google Course Builder…? • Better idea of what works/what doesn’t for videos • Professionally shot videos for lectures/discussion
- 14. 14
- 15. Conclusion 15Introduction | Methodology | Lessons Learned | Next Steps | Conclusion @crtheisen crtheise@ncsu.edu theisencr.github.io
Editor's Notes
- Michael Brown, CEO of Symantec, says that shortfall could increase up to 1.5 million by 2019.
- Online classwork and MOOCs have emerged as one way to train busy professionals. Typically taken at your own pace or at a relaxed pace compared to usual coursework Can take the courses from your couch, no brick-and-mortar requirements
- “Flipping” a course The question; how do the two courses compare? Exact same for both offerings for online and in-person
- The syllabus for the course. Course is about security management and prevention, with an introduction to specific types of exploits More about prevention than exploitation Not a crypto class
- Google Coursebuilder, running on google app engine (circa late 2014, been updated since) Quizzes via Google Forms (built in quiz functionality broke, more on that later) Navigate via next page Embedded videos and quizzes
- Going to focus on RQ3 for this talk, preview RQ1 and RQ2: RQ1: Variety of reasons, slight bias toward retraining/filling in gaps Interesting participants: high school teacher who was dropped into teaching a computer science course, and administrative assistant who works with security professionals RQ2: 450 people signed up online, 60 finished. 120 signed up for NCSU course, 115 finished. Compared both sets of students on common multiple choice; online students performed about 10% worse than the brick-and-mortar students
- Going to focus on RQ3 for this talk, preview RQ1 and RQ2: RQ1: Variety of reasons, slight bias toward retraining/filling in gaps Interesting participants: high school teacher who was dropped into teaching a computer science course, and administrative assistant who works with security professionals RQ2: 450 people signed up online, 60 finished. 120 signed up for NCSU course, 115 finished. Compared both sets of students on common multiple choice; online students performed about 10% worse than the brick-and-mortar students
- Most frequently quoted reason for dropping out: not enough time to complete. Even though we specifically set out to set the bar as low as we could! By relaxing assignment deadlines, we helped improve retention. Estimation of effort is always hard. We were warned it would take more time than we thought, tried to overestimate, STILL wasn’t enough. Death by a thousand cuts: so many individual things adds up to a lot of time. Also means that divide and conquer could work well.
- One of the biggest timesinks was dealing with pop up technical issues. Quizzes weren’t retaining scores the night before we launched, slowness on App Engine was a constant issue (Some of this is apparently resolved in the newer version, but scalability testing before your launch is important, even for a smaller course). We had a peer review project component being run by another group, but group didn’t finish until right before the week we were going to launch it; launch had a ton of problems ended up having to scrap it. Not a good look from a PR perspective, plus a huge headache
- Interesting participants: high school teacher who was dropped into teaching a computer science course, and administrative assistant who works with security professionals How do we consider these folks when designing our lectures and assignments? Can’t make the same assumptions about prior knowledge
- (Video starts automatically, plays silently, I talk over it and explain what’s going on, since we can’t guarantee sound in presentations) One of the things that worked great: videos about current events in software security! Example topics: walking through breaches, how they happened, how they could have been prevented, what they’re doing now Discussed Heartbleed, Home Depot breach, the White House breach We always had something to discuss! This was the most well received part of the course, quote: “I felt like the discussion videos made it more of a personal experience”
- So what’s next? We’re running the course again this fall. Moving to OpenEDx or the new iteration of Coursebuilder We have a better idea on what works/what doesn’t and will incorporate the lessons learned into the new course Videos will be professionally shot, rather than by us.
- Course is also running on-demand on DigitalChalk Can take it for a certificate or just for knowledge Targeted towards corporate group buys of the course
- Here’s my contact info, thanks for coming, any questions?