Evaluating beyond format

1. MODULE 4
EVALUATING BEYOND
FORMAT

2. Other attributes of a research
paper beyond the format:
INTRODUCTION
• 1. Is the general purpose of the study clear?

3. 2. Is the study significant?
The potential value/s of the study .

4. 3. Will it make a practical or
theoretical contribution?
• Pragmatic potential benefit/s of the study.
• New knowledge?
• New processes?
• Etc.

5. 4. Is the introduction well
organized and clear?
• The organization of the introduction must
effectively establish the framework upon
which the researcher proposes to undertake
the study.

6. REVIEW OF RELATED
LITERATURE
• 1. Is the review comprehensive and up to
date?

7. 2. Is there an emphasis on
primary resources?

8. 3. Is there a critical review or a
review of findings?
• Advantages
• Disadvantages
• Limitations

9. 4. Is the review well-organized?

10. 5. Does the review clearly relate
previous studies to the current
research problem?

11. 6. Does the review help establish
the importance of the study?

12. RESEARCH PROBLEM,
QUESTIONS OR HYPOTHESIS
1. Is the problem or hypothesis clear
and concise?

13. 2. What kind of research is being
carried out?
• Qualitative
• quantitative

14. METHODOLOGY: SAMPLES/
PARTICIPANTS
• 1. Is the population described adequately?

15. 2. Is the sample clearly
described?

16. 3. Is the method of selection of
the sample clear?

17. 4. Could the method of selection
affect the result?

18. 5. Are subjects likely to be
motivated to give biased
responses?

19. METHODOLOGY:
INSTRUMENTS
• 1. Is evidence for validity and reliability
clearly presented and adequate?
• Qualitative – matter of degree than absolute
state.
• Quantitative –
– Stability
– Equivalence
– Internal consistency

20. 2. Is there a clear description of
how the instrument was
administered?
• What instructions and guidelines were given?
• What was the nature of the text?
• What was the degree of difficulty?
• Was there interview?
• Focus-group discussion?
• Who conducted the interviews/FGDs?

21. 3. Is it likely that subjects would
fake their responses?

22. 4. Are interviewers and
observers trained?

23. METHODOLOGY:
PROCEDURES
• 1. Are there clear weaknesses in the
design of the study?

24. 2. Are the procedures for
collecting information described
fully?
a. kinds of written questions
b. kinds of semi-structured
interview questions.

25. 3. Is it likely that the researcher
is biased?
• Use help of colleagues to triangulate
problems.

26. RESULTS
• 1. Are the findings presented clearly?
• 2. Is there appropriate use of tables, charts,
and figures?

27. 3. Is the number of subjects
taken into consideration when
presenting the results?

28. 4. Is there sufficient descriptive
information to interpret the
results?

29. 5. Do illustrative quotes and
specific instances accompany the
results?

30. DISCUSSIONS AND
CONCLUSIONS
• 1. Is the discussion based on the research
problem and results, or is there a tendency
to discuss unrelated material or ideas?

31. 2. Is there an adequate
interpretation of the findings?

32. 3. Is the interpretation separate
from the results?
4. Are the results discussed in
relation to previous studies?

33. • 5. Are limitations due to methodology
included in the discussion?
• 6. Are the conclusions clearly stated and
based on the results and the discussion?
• 7. Are the conclusions reasonable?

34. • 8. What is the external validity of the
study?
• 9. What factors would affect the external
validity?
– Does the sampling capture variation in a
diverse enough manner? (Profiling)
– Is the context fully described so that the reader
can judge transferability to other settings?
– How well have the concepts been extracted
from the data?

35. END OF MODULE 4
BY: Jocelyn M. Gallegos