Research seminar for Educational Sciences, Lecture 6 by prof. Dr. Chang Zhu

1. 13/12/2013
1
Research Seminar for
Educational Sciences
Prof. Dr. Chang Zhu
Department of Educational Sciences
Overview
• Quantitative research
• Qualitative research
• Questionnaires
• Interviews and observations
• Sampling
2

2. 13/12/2013
2
• Make questions clear: word questions as clearly
as possible
• Keep questions short
• Collect demographic information based on the
need of the study
3
Questionnaire
Questionnaire
• Include only items that are relevant
• Define or explain them if some terms are
not familiar for the participants.
• Use examples if item format is unusual.
4

3. 13/12/2013
3
Questionnaire
• Avoid leading questions: A leading
question is one that suggests a certain
response (either literally or by
implication) or contains a hidden
premise.
• Avoid ambiguous, and define or explain
them if some terms are not familiar for
the participants.
5
Questionnaire
• Cover letter or brief introductory or
directory words
Why this questionnaire
How the data will help the respondent and
the field in general
How to respond
Endorsement or approval by certain
admin/organization if applicable
6

4. 13/12/2013
4
Questionnaire
I am happy to work in this school.
I am happy in my job as a teacher.
There is no better job than to be a teacher.
If I could choose again, then I would change the teacher
profession for another job. (R)
I like to be in my class.
I don't like to be in my class. (R)
I want to remain in the education for my job.
7
An example: well-being of teacher
Reliability
• E.g. measuring your weight 4 times a day
(is it reliable?)
• Testing your statistics knowledge 3 times a
week…
• Asking the well-being of a student with 5
different questions….
8

5. 13/12/2013
5
Questionnaire
• Pilot testing
A smaller sample
Understanding of the questions
Identify problems if any
Provide suggestions for modification
9
Questionnaire
• Questionnaires (written)
o Larger sample
By mail
Via Email
online
• In-person question-and-answer (oral)
o Smaller sample
Telephone
In person
10

6. 13/12/2013
6
Questionnaire
• Questionnaires
Deadlines
Follow up if possible (a reminder)
Consider the convenience of respondents (e.g.
return envelope)
• Ethics
Anonymity, confidentiality
No harm, not threatening questions
11
Online survey
1
2

7. 13/12/2013
7
Online survey
1
3
Online survey
1
4

8. 13/12/2013
8
Online survey tools
• LimeSurvey
• https://www.vub.ac.be/survey/index.php?lang
=en
• Survey Monkey
• http://www.surveymonkey.com
• Qualtrics
• https://www.qualtrics.com/
• VUB/ULB Rekencentrum (rekencentrum@vub.ac.be)
• OSuCre
1
5
1
6

9. 13/12/2013
9
Interview
Individual
Group
Focus group interview
– several individuals
– ensure all have their say
– individuals can interact, respond
– possibly lead to a shared understanding..
17
Interview
• Understand themes of the world from the
subject’s own perspectives
• Capture the experiences and lived meanings of
the subjects’ everyday world (from their own
perspectives and in their own words)
18

10. 13/12/2013
10
Interview
• Can obtain data that cannot acquire
from observation alone; gather in-depth
data about the experiences, views,
feelings of participants; explore reasons
• Can be structured, unstructured or semi-
structured
19
Interview questions
• Don’t ask two things in one question. E.g.
do you find online communication a
problem and the teacher should give more
guidance?
• Don’t ask Confusing or wordy questions
• Avoid unrelated questions
• Avoid Yes-No Answers.
20

11. 13/12/2013
11
Interview questions
Interview questions
o E.g. ‘Can you tell me about…’
‘what do you think of…’
‘can you describe…’
‘can you say something more about…’
‘do you mean that….’
21
Interview
Interviewing
• Setting the stage, briefing
• Encourage the interviewee to describe their
point of views
• Follow up questions, clarifications
• If needed, summarize or round off (and ask
confirmation)
22

12. 13/12/2013
12
Interviewing
– listen more
– don’t interrupt
– avoid leading questions; ask open-ended
questions
– keep participants focused; follow up on
what they say
– do not judge or debate
23
Interview
Collecting data from interviews
– take notes during the interview
– audio- or video-taping the interview
– write notes after the interview
– transcription of recording; write down
the data, subjects, and participant name
or codes
24
Interview

13. 13/12/2013
13
• Some data can be observed in natural setting
• More objective information than self-reports
• Recording observations
• Field notes (description and understanding of the
research setting and participants, as detail as
possible)
• Recorded data and filed notes can be organized,
categorized and analyzed.
25
Observation
Observation
Observational notes
• descriptive notes: detailed notes about
what occurred, what the observer sees,
hears, the actual setting, etc.
• reflective notes: about the observer’s
thoughts, impressions, personal reactions,
experiences
26

14. 13/12/2013
14
Validity
Validity in qualitative research
• descriptive: factual accuracy
• interpretative: are the meanings or words of
participants correctly interpreted?
• theoretical: how is the studied phenomenon
relate to a broader theory?
• evaluative: does the researcher report in an
objective, unbiased way?
27
Validity
Strategies for ensuring the validity
– talk little, listen a lot
– record accurately
– begin writing early
– report fully
– write accurately
28

15. 13/12/2013
15
A short discussion
• What kind of research method you would use for
your master thesis research?
• How are you going to collect your data?
• What would be your target sample?
• How many sample participants to reach?
2
9
Sampling
A. Defining a population
B. Selecting a sample
30

16. 13/12/2013
16
Population
• What is your population of interest?
• To whom do you want to generalize your
results?
– All doctors
– All university students in Flanders
– All secondary school pupils
– Women aged 25-45 years
– Other
• Can you sample the entire population?
3
Sampling
• A sample is “a smaller (but hopefully representative)
collection of units from a population used to
determine truths about that population” (Field, 2005)
• Why sample?
– Resources (time, money) and workload
– Gives results with known accuracy that can be
calculated mathematically
• The sampling frame is the list from which the
potential respondents are drawn
– Registrar’s office
– Class rosters
– …..
3

17. 13/12/2013
17
Sampling
• 3 factors that influence sample
representativeness
• Sampling procedure
• Sample size
• Participation (response)
• When might you sample the entire population?
• When your population is very small
• When you have extensive resources
• When you don’t expect a very high response
3
Sampling
34

18. 13/12/2013
18
Sampling
3
5
TARGET POPULATION
STUDY POPULATION
SAMPLE
The sampling process…
POPULATION
SAMPLE
INFERENCE

19. 13/12/2013
19
Sampling
Selecting a random sample
• Simple random sampling
• Systematic sampling
• Stratified sampling
• Cluster sampling
37
• Simple random sampling
38

20. 13/12/2013
20
Sampling
• Systematic sampling
3
9
Sampling
4
0
• Stratified sampling

21. 13/12/2013
21
Sampling
• Cluster sampling
41
Sampling
C. Selecting a non-random sample
• Convenience sampling
• Purposive sampling
• Quota sampling
D. Qualitative sampling
• Representative
• Often purposive
• Criterion sampling
42

22. 13/12/2013
22
Sampling
Important to bear in mind:
• Is your sample representative of your target
population?
• Are the research results generalizable to your
target population?
• The degree to which the selected sample
represents the population is the degree to
which the research results are generalizable
to the population.
43
Sampling
Sample size:
• In quantitative studies, samples should be as
large as possible; the larger the sample, the
more generalizable the results will be…
• In qualitative studies, samples usually
smaller, depending on the type of research
44

23. 13/12/2013
23
Rules of thumb for determining
the sample size...
For smaller samples (N ‹ 100): Survey the entire
population.
The larger the population size, the smaller the
percentage of the population required to get a
representative sample
If the population size is around 1500: 20%
should be sampled.
If the population size is around 500: 50%
should be sampled.
Beyond a certain point (N = 5000): a sample
size of min. 400
Rules of thumb for determining
the sample size...

24. 13/12/2013
24
Sample size
Sample size:
• No universally minimum sample sizes
• Some suggestions for quantitative studies:
if Population=<100, n=<100
if P=500, n~250
if P=1500, n~300
if P>5000, n~400-500 or more
47
Sample size
Sample size:
• No hard rules
• Some suggestions for Qualitative studies:
Can be 1- 60/70 participants
interviews: 15-30 or more, depending on population
size, should be representative of the sub-groups/
multiple contexts…
Case studies: 1-several, depending research needs
48

25. 13/12/2013
25
Sample size
Sample size:
• Two general indicators to determine
whether a sample size is sufficient:
Representativeness
Redundancy of information (data
saturation) (in Qualitative research)
49
Sample size
Sample size:
• Avoid sampling error (occurs in the random
selection procedure)
• Be aware of sampling bias (eg. use of non-
random sampling..) (should be noted and
described in the research report)
50

26. 13/12/2013
26
Task 1st semester
• Submit your research paper or the literature
review paper by 31 Jan. 5pm in Pointcarre
Assignment
5
1