1. Evaluation of a Flipped Classroom
Format for an Introductory Level
Marketing Class
George Krueger
Assistant Professor
University of Wisconsin--Platteville
3. Flipped Class Concept
The research is mixed on learning outcome results. Some studies
show a positive impact on quantitative students results and
qualitative student perceptions. Others show a negative impact or no
difference.
Marketing is an application-based subject and largely qualitative in
evaluating student results. Would a flipped class result in a more
positive student outcomes?
4. Purpose
The purpose of this study was to evaluate the
flipped classroom methodology in an
introductory level course.
The professor has taught the course for several
years in a conventional lecture format.
5. What is “Flipped”
There are numerous ways to define flipped as a
teaching methodology.
• Video presentations
• Pre-assigned readings
• Assignments to be completed outside of class and
then reviewed during class
• Outside of class online discussions
• Preparation for student presentations.
• Online quizzes
6. Online Quizzes
• Flipped classroom 10 quizzes at 50 points &
due the Monday night BEFORE the
Tuesday/Thursday classes
• Control: 10 quizzes at 20 points due the
Thursday before Friday class
7. Allocation of Class Time
Flipped:
• Student presentations
• In-depth discussions
Control
• More lecture, less projects and discussion
9. The Classes
• BSAD 2630 Introduction to Marketing
• Section 02: Tuesdays and Thursdays from 9:30 to 11:45
• Section 03: Monday, Wednesday, Friday from 9:00 to
9:52.
• The courses used the same textbook, in-class
instruction materials and followed the same course
content.
• Students in each class was 35 at the start of the
semester.
• The students were predominantly business majors
along with business minors. Very few were marketing
emphasis.
10. Online Quizzes
• The quizzes were administered online in D2L.
• Multiple choice questions were loaded
directly from the textbook testbank.
• Quiz questions were randomized for each
student.
• Open book/open notes. The quizzes covered
the chapters that were assigned for each
week.
11. Flipped Group Projects
Each group focused on one company and
applied the subject matter in discussion and
presentation formats.
– Oakley Sunglasses
– Gatorade
– Canon Digital Photography
– Monster Beverages
– Blackberry
– UnderArmour
12. Control Group Projects
• Sunday Ticket—Direct TV
• Netflix
• Kwik Trip Convenience Stores
• Mountain Dew KickStart
• Domino’s
14. Exams
• 3 exams administered during the semester.
• Study guides were provided and the professor
emphasized key items that would be on the
exam.
• Exams were similar but not exactly the same
for the two classes.
• Approximately 60% multiple choice and 40%
short answer/essay. The exams were returned
to the students.
15. Final Exam
• Comprehensive exam covering the entire
semester.
• Exams were exactly the same for the 2 groups.
• 40% multiple choice and 60% application of
terms related to a case on Subway
Restaurant’s marketing.
• The case subject was given to the students
along with a study guide before the exam.
16. Final Individual Project
• An individual marketing plan project was
assigned to the students.
• The student to selects a product or service to
market and write a marketing plan for their
selection.
• Outline and grading rubric was provided to
each class.
• Intended to be a capstone application of the
marketing concepts.
17. Summary of Grading
Control Class Flipped Class
Online Quizzes 200 500
Group Projects 100 100
Exams 300 300
Final Exam 100 100
Final Individual Project 100 100
Class Participation 100 100
Total Points 900 1200
19. Online Quizzes
• Conventional class took the quizzes after the
course material had been covered in class and
averaged 77.4%.
• Flipped class took the quizzes before the
lecture they only trailed the conventional class
by .7 at 76.7%.
• No statistical difference between the two
measurements.
20. Exams
The results for the exams were not statistically
significant. There were no trends apparent in
the data.
Exam 1 Exam 2 Exam 3 Final Exam
Flipped 71.6% 79.0% 77.8% 68.2%
Conventional 73.5% 76.6% 75.0% 72.5%
• The observations of the Final Exam answers
for the Flipped group was deemed poor by the
professor.
21. Final Individual Project
• The flipped class scored 80.8% on the
individual marketing project while the
conventional class scored 83.2%. There was
no statistical difference.
22. Overall Class Grades
• The final class percentage achievement for the
flipped classroom was 78.9% and the
conventional class was 77.7%. No statistical
difference.
23. Pre and Post Survey Results
Flipped Flipped Control Control
Question Average
Pre
Average
Post
Average Pre Average Post
This class has met my expectations
so far for content delivery
4.1 4.1 4.3 4.1
This class is taught like most classes
that I have taken
2.7 3.2 3.1 3.2
I am satisfied with the way this
course is being taught
4.2 4.0 4.0 3.7
I believe that I understand the
content that has been covered so far
in the class
3.8 3.8 3.8 3.7
I believe that I can apply the content
that has been covered so far in the
class
4.0 4.0 4.0 3.9
I believe that the learning objectives
are clear
4.0 3.9 4.0 3.9
I am satisfied with my learning so far
in this class
3.8 3.9 3.8 3.8
I believe that the amount of
homework in the course is
2.1 1.9 2.0 1.9
I believe the difficulty of the
homework is
1.9 1.9 2.0 1.8
24. Student Evaluations
Question Flipped
Fall 13
Control
F 13
Sp 13 Fall 12 Fall 12 Fall 11 Fall 11
3. Uses examples and
illustrations that clarify the
material.
4.857 4.524 4.367 4.308 4.519 4.548 4.31
4. Explains course material
clearly.
4.5 4.05 3.967 3.8 3.926 4.097 4.179
7. Incorporates current
material into the course.
4.821 4.619 4.533 4.692 4.519 4.452 4.621
9. Encourages classroom
participation from students.
4.857 4.476 4.4 4.28 4.481 4.677 4.483
10. Presentations/
discussions enhance the
text.
4.571 4.238 4.367 4.038 4.296 4.258 4.138
15. Uses class time for
student discussions,
presentations or questions.
4.821 4.238 4.3 4.292 4.296 4.613 4.448
Mean 4.738 4.394 4.361 4.278 4.51 4.498 4.42
Responses 28 21 30 26 27 31 29
25.
26. Student Comments
Flipped
Positives
14 Entertaining/enthusiasm/engaging
6 Good knowledge
6 Liked group presentations/discussions
Improvements
9 More help on projects/expectations/assignments
2 Dislike online quizzes
4 Go through study guides & reviews
1 Less Powerpoint by Professor
1 Some student groups were not good presenters
Control
Positives
14 Entertaining/enthusiasm/engaging
5 Good knowledge/examples
1 Liked group presentations/discussions
1 Professor is concerned about students (cares)
Improvements
7 More help on projects/expectations/assignments
6 Dislike online quizzes
1 Go through study guides & reviews
3 Lectures can be confusing at times
1 Stay on syllabus
1 More activities
28. They Are Students!
• Several times during the semester the
professor had to intervene in the group
presentations to correct mistakes and add in
omitted subject matter. In addition, some of
the presentations were very poor and
students attention was lacking.
29. Unique Challenge
• The flipped format was a challenge with 35
students. Individual attention and a large
number of groups can be difficult to manage
in a classroom environment. When the
professor is focusing on a group of 6 students
the other 29 students can be disengaged from
the class.
31. Another Observation
• As noted in the research, not having a base of
knowledge led to visible frustration from some
students as observed by the professor. Often
when the subject matter was enhanced in the
presentations by the professor the response
from the students was something like, “how
are we supposed to know that?”
32. Interesting….
• The flipped format required more work
outside of class by the students and this was
perceived by the professor to be an issue
during the semester. However, this perception
did not manifest itself in the evaluations.
33. Interesting….
• Some students expressed their preference to
be presented the information in lecture
format first with explanation by the professor.
They were then more comfortable and
effective with the group work and projects.
35. Conclusions
• The flipped classroom methodology utilizing
the key tools of online quizzes taken before
classroom coverage of the subject matter,
student preparation for presentations outside
of class, student presentations and a much
higher level of group work in class did not
yield any statistical differences in student
outcomes as measured by grades when
compared the control class of lecture and
class discussion format.
36. Conclusions
• The students did not perceive a significant
level of difference in the way the classes were
taught from their previous experiences.
37. Conclusions
• The flipped classroom did create a higher level
of satisfaction for the students numerically
and did appear to engage the students more
actively as measured by student evaluation
scores.
38. As a Result of This Study…
• Attempted to use the “best of both worlds” in
the Spring 2014 for Introduction to Marketing
and it worked very well.
• More focused lecture covering key areas.
• More focus was put on class presentations
and projects applying the subject matter to
their group projects.
39. Follow-Up Study
• Lessons Learned
– 1. To take the best of both worlds and utilize
highly focused lectures on key marketing concepts
during the Tuesday class session and then have the
students apply the concepts in a weekly discussion
format during the Thursday class session.
– 2. Utilize the student’s projects throughout the
semester in the context of lecture and application of
marketing concepts.
41. Grading
Summary of Grading
Lessons Learned
Class Spring 2014
Flipped Class Fall
2013
Online Quizzes 200 500
Group Projects 100 100
Exams 400 300
Final Exam 100 100
Final Individual Project—
Marketing Plan
100 100
Class Participation 100 100
Total Points 1000 1200
42. Table 2. Average scores for the fall 2013 Flipped class and spring 2014 Lessons Learned class.
Fall 2013 Flipped
n = 33
Lessons Learned
n = 31
t-test results
Mean Std Dev Mean Std Dev t p
Online Quizzes 77% 8.6 78% 16.5 0.344 .37
Total Exams 76% 9.2 76% 9.8 0.035 .49
Final Exam 68% 9.5 77% 16.1 2.62 .005**
Marketing Plan Project 81% 13.6 88% 13.5 2.02 .024*
*Significant at the .05 level, **Significant at the .01 level
43. Table 3. Student evaluations from fall 2011to spring 2014.
Question Lessons
Learned
Sp 2014
Flipped
Fall 13
Control
F 13
Sp 13 Fall 12 Fall 12 Fall 11 Fall 11
3. Uses examples and
illustrations that clarify
the material.
4.45 4.857 4.524 4.367 4.308 4.519 4.548 4.31
4. Explains course
material clearly.
3.69 4.5 4.05 3.967 3.8 3.926 4.097 4.179
7. Incorporates current
material into the
course.
4.61 4.821 4.619 4.533 4.692 4.519 4.452 4.621
9. Encourages
classroom
participation from
students.
4.66 4.857 4.476 4.4 4.28 4.481 4.677 4.483
10. Presentations/
discussions enhance
the text.
4.03 4.571 4.238 4.367 4.038 4.296 4.258 4.138
15. Uses class time for
student discussions,
presentations or
questions.
4.64 4.821 4.238 4.3 4.292 4.296 4.613 4.448
Mean (all responses) 4.39 4.738 4.394 4.361 4.278 4.51 4.498 4.42
Responses 29 28 21 30 26 27 31 29
44. Table 4. Categorical assessment of student comments including number of responses (n).
Done Well
n 2013 Flipped n Lessons Learned
14 Entertaining/enthusiasm/engaging 20 Entertaining/enthusiasm/engaging
6 Good knowledge 9 Good knowledge/examples
6 Liked group presentations/discussions 4 Liked group presentations/discussions
Improvements
n 2013 Flipped n Lessons Learned
9 More help on
projects/expectations/assignments
8 More help on projects/expectations/
assignments
2 Dislike online quizzes 5 Dislike online quizzes
4 Go through study guides & reviews 4 Go through study guides & reviews
1 Less PowerPoint by professor
1 Some student groups were not good
presenters
47. Observations
• Improved engagement of students
• More consistent engagement
• Basic knowledge improvement—fewer
mistakes in subject matter concepts
• Improved insight from the student—”why”
• Samsung/Apple—excellent compare and
contrast examples
• They really hate the online quizzes
48. Conclusions
• Improved learning outcomes as measured by
final exam and individual marketing project
scores
• Reverted back to the mean on student
evaluations
