1. Innovative Design & Manufacturing Research Centre
University of Bath
“World-leading research in engineering
design and manufacture.”
IdMRC Social Research Methods
Autumn Lecture-Workshop Series
2. ESQUEMAS DE DISEÑO DE INVESTIGACIÒN
• Enfoque cuantitativo
• Enfoque cualitativo
• Enfoque utilizando métodos mixtos
• Elementos del esquema:
• Supuestos filosóficos (Qué constituye el conocimiento)
• Procedimientos de investigación (Estrategia de búsqueda)
• Los métodos:
• Procedimientos detallados de recolección de datos o
información
• Estrategia de análisis de la información
• Reporte de los hallazgos
3. Esquemas: Para su propuesta de un diseño de investigación
contéstese cuatro preguntas:
• ¿Cuál posición epistemológica?. Teoría del conocimiento que
subyace en la perspectiva teórica que Ud. ostente. (ej.:
Objetivismo, subjetivismo)
• ¿Cuál Perspectiva teórica? Sustento filosófico detrás de la
metodología a utilizar.
• ¿Qué metodología? Estrategia o plan de acción que
conectará los métodos que utilizará y concatenará los
hallazgos. (ej.: experimental, survey, etnografía, etc.)
• ¿Cuáles métodos? Técnicas y procedimientos que se
propone utilizar. (ej.: cuestionario, entrevista, grupo focal,)
4. Orden de las preguntas
¿Qué métodos utilizaré?
¿Cuál metodología define el uso de esos métodos?
¿Desde cuál perspectiva teórica empezamos para aplicar la
metodología correcta o más adecuada?
¿ Qué posición epistemológica sustenta la perspectiva teórica
seleccionada?
¿Qué conocemos?
Lo ontológico. Sobre la naturaleza de la realidad.
¿Cómo lo conocemos?.
Lo epistemológico. Sobre la naturaleza de la relación investigador-objeto.
¿Con cuáles valores?. Lo axiológico. Sobre los valores
¿Cómo lo escribimos?. Lo retórico. Sobre el análisis del discurso
¿Con cuál proceso lo estudiamos?. Lo metodológico
5. Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. stat analysis
evaluation data reduction
(post)modernism cognitive mapping
Social-constructivism interpretative meth
document analysis
content analysis
conversation anal.
6. ESQUEMAS PARA LA PROPUESTA DE DISEÑO DE INVESTIGACIÓN
CUALITATIVA, CONSTRUCTIVISTA PROPUESTA CUANTITATIVA
• Introduction • Introduction
– Statement of the problem (including – Statement of the problem
existing literature about the problem)
– Purpose of he study
– Purpose of the study
– Theoretical perspective
– The research questions
– Research question or hypotheses
– Delimitations and limitations
– Definition of terms
• Procedures
– Delimitations and limitations
– Characteristics of qualitative research
• Review of the literature
(optional)
• Methods
– Qualitative research strategy
– Type of research design
– Role of the researcher
– Sample, population and participants
– Data collection procedures
– Data collection instruments, variables,
– Data analysis procedures
and materials
– Strategies for validating findings
– Data analysis procedures
– Narrative structure
• Anticipated ethical issues in the study
• Anticipated ethical issues
• Preliminary studies or pilot tests
• Significance of the study
• Significance of the study
• Preliminary pilot findings
• Appendixes: Instruments, timeline, and
• Expected outcomes proposed budget
• Appendices: interview questions,
observational forms, timeline, and proposed
budget
6/24
7. PROPUESTA DE FORMATO MIXTO
• Introduction
• Statement of the problem
• Purpose of the study (include both quantitative and quantitative
• Statements and a relational for mixing methods)
• Research questions(include both qualitative and quantitative)
• Review of the literature (separate section, if quantitative)
• Procedures or methods
• Characteristics of mixed methods research
• Type of mixed methods design (including decision involved in its choice)
• Visual model and procedures of the design
• Data collection procedures
• Types of data
• Sampling strategy
• Data analysis and validity procedures
• Report presentation structure
• Role of the researcher
• Potential ethical issue
• Significance of the study
• Preliminary pilot findings
• Expected outcomes
• Appendixes: instruments or protocols, outline for chapters, and proposed budge
8. Science
• Aim?
• When is knowledge scientific knowledge?
• Criteria?
• Knowledge sources?
• When is research scientific research?
Henri Christiaans
9. Science
• Realism
What we observe is real
• Instrumentalism
What we observe doesn’t need to be real
• Social constructivism
Theories only get meaning through social and political
context
10. What is Knowledge?
• Justified true belief (Plato’s Theaetetus)
• The Greeks classify knowledge into 2 types:
• Doxa (believed to be true)
• Episteme (known to be true)
• Doxa Epistime
• Through Scientific process of inquiry
• How do we know what we know?
• Define knowledge alternatively
• Supported by evidence (usually empirical)
• Conceive knowledge claims in a probabilistic sense
• Knowledge is a matter of societal acceptance
11. How is Knowledge Acquired?
• Role of science, where science is a convention, related to
societal norms, expectations, values, etc.
• Thus, is science equals any scholarly attempt at acquiring
knowledge
• Science requires conventions to be followed
12. How is Knowledge Acquired?
• Role of science, where science is a convention, related to
societal norms, expectations, values, etc.
• Thus, is science equals any scholarly attempt at acquiring
knowledge
• Science requires conventions to be followed
13. Knowledge in design
• Implicit prioritisation of the (language-based mode of)
propositional knowledge (justified true beliefs) seems to
exclude certain kinds or formats of knowledge associated
with practice, which are often called practical, experiential,
personal, or tacit knowledge and which evade verbal
articulation.
15. Science based on empirism
Empirism:
Knowledge derived from how the world is experienced.
Scientific statements are controlled by and derived from our
experiences and observations. en
Scientific theories developed and tested by experiments and
observations through empirical methods
16. Questions to be asked
1. Which methods do we plan to use?
2. Which methodology defines the use of methods?
3. Which theoretical perspective do we start from in order to
apply the right methodology?
4. Which epistemology feeds this theoretical perspective?
17. Ontology
1. A systematic account of Existence. Nature of the world
around us.
2. (From philosophy) An explicit formal specification of how to
represent the objects, concepts and other entities that are
assumed to exist in some area of interest and the
relationships that hold among them.
3. The hierarchical structuring of knowledge about things by
subcategorising them according to their essential (or at least
relevant and/or cognitive) qualities.
18. Epistemology and ontology
The way of understanding and interpreting how we
know what we know.
Particular methodologies tend to entail (subscribe to)
particular epistemologies and, in their turn, particular
forms of ontology
19. Ontology in Computing Terms
• For AI systems, what "exists" is that which can be
represented.
• We can describe the ontology of a program by defining a set
of representational terms. Definitions associate the names of
entities in the universe of discourse (e.g. classes, relations,
functions or other objects) with human-readable text
describing what the names mean, and formal axioms that
constrain the interpretation and well-formed use of these
terms. Formally, an ontology is the statement of a
logical theory.
• A set of agents that share the same ontology will be able to
communicate about a domain of discourse without
necessarily operating on a globally shared theory. The idea of
ontological commitment is based on the Knowledge-Level
perspective.
20. Epistemology
• From the Greek words episteme (knowledge) and logos
(word/speech) is the branch of philosophy that deals with the
nature, origin and scope of knowledge.
• Refers to our theory of knowledge, in particular, how we
acquire knowledge (Hirschheim, 1992).
21. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. stat analysis
evaluation data reduction
(post)modernism cognitive mapping
Social- interpretative meth
constructivism document analysis
content analysis
conversation anal.
Crotty, 1998
22. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. stat analysis
evaluation data reduction
(post)modernism cognitive mapping
Social- interpretative meth
constructivism document analysis
content analysis
conversation anal.
23. Theoretical perspective
Philosophical point of view which feeds the methodology and
offers a context for the process and the logics, and gives our
criteria a basis.
Cultural differences play a role
24. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretivism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. statistical analysis
evaluation data reduction
(post)modernism cognitive mapping
Social- interpretative meth
constructivism document analysis
content analysis
conversation anal.
Crotty, 1998
26. Interpretivism
Interpretivism rests upon idealism:
•the world is interpreted through the mind; e.g., classificatory
schemes of species;
•the social world cannot be described without investigating how
people use language and symbols to construct what social practices;
i.e., understand their experience;
•the social world becomes the creation of the purposeful actions of
conscious agents; and
•no social explanation was complete unless it could adequately
describe the role of meanings in human actions
•Actions are not governed by discrete patterns of cause and effect
(as in positivism), but by rules that social actors use to interpret the
world
27. Positivist Science
• 5 Pillars
• Unity of scientific method
• Causal Relationships
• Empiricism
• Science and its process is Value-Free
• Foundation of science is based on logic and maths
28. Ontology of Positivism
• Realism
• Universe comprised of objectively given, immutable
objects and structures, existing as empirical
entities, on their own, independent of the
observer’s appreciation of them.
• Contrasts with relativism or instrumentalism, where
reality is a subjective construction of the mind, thus
varying with different languages and cultures.
• While hugely successful in physical sciences, it is
not as successful for social science.
29. Anti-Positivism
• Latter part of 19th century
• Man as an actor could not be studied through the methods of
natural sciences that focus on establishing general laws. In
the cultural sphere man is free (Burrell and Morgan, 1979)
30. Post-Positivism
• Based on the concept of critical realism, that
there is a real world out there independent of our
perception of it and that the objective of science is
to try and understand it,
• combined with triangulation, i.e., the recognition
that observations and measurements are inherently
imperfect and hence the need to measure
phenomena in many ways.
• The post-positivist epistemology regards the
acquisition of knowledge as a process that is more
than mere deduction. Knowledge is acquired
through both deduction and induction.
31. Simon versus Schon
Designer Designer
objective Analysis Objective Analysis subjective Interpretation
design design Design Task design
Problem Solution (= problem + Solution
situation+ teime)
07-06-12
Rational Solving Problem Paradigm Reflection in Action Paradigm
Rationalist Root Constructivist Root
POSITIVISM PHENOMENOLOGY
32. Methodology
Our strategy and action plans, the design process which
defines what specific methods we will choose
33. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. statistical analysis
evaluation data reduction
(post)modernism cognitive mapping
Social- interpretative meth
constructivism document analysis
content analysis
conversation anal.
Crotty, 1998
34. Types of Research
Analytical Historical Philosophical
Literature study Meta-analysis
-------------------------------------------------------------------------------------------------------
Descriptive Survey (questionnaire, interview)
Case study Task analysis
Document analysis Correlation anal.
Observation Etnographics
-------------------------------------------------------------------------------------------------------
Explorative Survey Correlational
Case study Experimental
-------------------------------------------------------------------------------------------------------
Experimental Pre-experimental
True-experimental
Quasi-experimental
-------------------------------------------------------------------------------------------------------
36. Types of research methods
empirical
participatory
quantitative
inductive prescriptive
idiographic nomothetic
descriptive deductive
unbiased
qualitative
rational
37. Fundamental Research: the Empirical cycle
induction
describing/ hypotheses generalising
interpreting
modelling
Explaining/
interpreting
knowledge problem theory
modelling
specifying
evaluation deduction
prediction
evaluating
testing
testing
‘t Hart c.s.
38. Practice oriented Research: The regulative cycle
describing/ diagnosis
generalising
interpreting
modelling
designing
plan
problem from practice
(problem solving)
deciding
evaluating
intervention
evaluation action-process supporting
observing
process evaluation
‘t Hart c.s.
40. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. stat analysis
evaluation data reduction
(post)modernism cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Crotty, 1998
41. Qualitative Positivist Research versus
Non-Qualitative Positivist Research
QPR Methods Non-QPR Methods
Field experiment Math Modeling (analytical
modeling)
Lab experiment Group feedback
Free simulation experiment Participative research
Experimental simulation Case study
Adaptive experiment Philosophical research
Field study
Opinion research
Archival research
Table 1. QPR versus Non-QPR Methods (Click on the method for its definition)
41
42. Type of Research, General Research Approaches,
Data Collection Techniques, & Data Analysis Techniques
45. Research background
Epistemology Theoretical Methodology Methods
perspective
objectivism experimental scaling
positivism
questionnaires
subjectivism Interpretativism descriptive
observation
symbolic survey interview
interactionism ethnography focus group
heuristic case study
phenomenology narratives
hermeneutics action research ethnographic
feminism discourse anal. statistic. analysis
evaluation data reduction
(post)modernism cognitive mapping
interpretative meth
document analysis
content analysis
conversation anal.
Crotty, 1998
46. Definitions
Research’ = the systematic inquiry to the end of gaining new
knowledge
‘researcher’ = a person who pursues research (e.g., in
design).
ractice’ = professional practice (e.g., in design) or to
processes usually used in professional practice to produce
professional work for any purpose other than the (deliberate)
acquisition of knowledge.
Practitioner’ = anyone who works in professional practice.
47. Design Knowledge
Process (design methodology)
product people
designers
48. Design knowledge
• Design knowledge resides firstly in people: in designers especially.
Therefore, we study human ability - of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design
49. Design knowledge
• Design knowledge resides firstly in people: in designers especially.
Therefore, we study of human ability - of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design.
• Design knowledge resides secondly in processes: in the tactics and
strategies of designing. A major area of design research is
methodology: the study of the processes of design, and the
development and application of techniques which aid the designer.
50. Design knowledge
• Design knowledge resides firstly in people: in designers especially.
Therefore, we study of human ability - of how people design. This
suggests, for example, empirical studies of design behaviour, but it
also includes theoretical deliberation and reflection on the nature of
design ability. It also relates strongly to considerations of how people
learn to design
• Design knowledge resides secondly in processes: in the tactics and
strategies of designing. A major area of design research is
methodology: the study of the processes of design, and the
development and application of techniques which aid the designer.
• The product dimension asks for forms and materials, and finishes
with the embodiment of design attributes: both the intentional world
(teleological and functional –wishes and needs–) in relation with the
principal, partial and elementary function and the man’s connection
with the systemic formal and material part (structure, organization,
parts and connections).
51. Design Research
Terry Love’s view:
• Design Research is dominated by two contradicting incompatible
approaches:
• Scientific: design can be completely understood
• Interpretive: design is an ‘intuitive’ activity, dependent on creativity,
and scientifically inaccessible
• The approaches are epistemologically and practically contradictory in
that scientific empiricism and interpretivistic exploration regard each
other’s central assumptions as invalid.
• Empirical scientific research specifically excludes subjective
reporting as reliable evidence.
• Interpretive approaches deny that the scientific empirical approach
addresses the central target of design research – the human
internal creative design activities
52. Design Research
Scientific Interpretive
Theoretical Scientific, usually based on physics Interpretive, focusing on
individuals’ experiences, their
perspective
construction of understanding,
perceptions and interpretation of
reality. Often centres on
individual creativity and
subjective perceptions relating to
being creative.
Focus Empirical realities of the design Experiences of designers and
processes, design objects, design other design constituents. Tries
brief and contexts. to identify form of internal
The core concept of ‘design’ is creative design activities from
defined in terms of these activities. observation of externalities.
Typically defines design in terms
of creativity, art, individual genius
and socio-cultural influences
53. Design Research
Scientific Interpretive
View of Design Design is a process. Intuitive, involving hidden aspects of
May or may not include human subjective thinking and
creativity. affective activity.
View of ‘Something, or a Human internal activity that results in
specification for something, ideas for new, unusual, highly valued,
creativity
is “created”’. never before created things, emerging
‘magically’ from the genius of
designers.
Creation can be achieved
mechanically, by Focus on ‘individual creativity’
automation or intuitively. attributed to specific ‘designers’ and
socio-cultural influences.
54. Design Research
Scientific Interpretive
Data collection Similar to physics and Drawn from various qualitative
natural sciences. traditions, e.g. anthropology,
ethnography, history, includes self
reporting data collection.
Analysis Similar to physics and Drawn from various qualitative
natural sciences. traditions, e.g. anthropology,
methods
ethnography, history, includes
reflective analysis of self reports and
self perception.
Knowledge Discipline specific empirical Tacit and embodied skills of designers
information (along with) and users.
focus
elicited representations of Culturally-determined knowledge.
tacit information and data Embedded meanings.
that designers use.
55. Scientific Interpretive
Strengths 1. Techniques to investigate 1. Focus on human considerations,
phenomena in ways that are such as the human creative aspects of
transparent, repeatable, testable, design, and how users and other
and verifiable. interpret designed outcomes.
2. Research methods are 2. Interpretive methods give space for
expressed in a formal language designers and users to explain, in their
that enables precise critique of own words, and from their own
the data collection techniques, perspectives, how they design and use
methods of analysis, processes designed outcomes and how they
that lead to abstractions, and the communicate with others about
theory abstractions and designs.
conclusions. 3. Interpretive methods also allow
3. Correspondence between exploration of opinions of users about
characteristics of phenomena cultural aspects of particular designs.
and the formal defined symbolic 4. The interpretive approach can be
language of concepts and extended to draw strength from the
operations in which use of large data sets by which
mathematically theories and correlations and measures of
representations of the confidence in them can be established
phenomena are expressed. between individuals’ ‘stories’ and the
phenomena being studied.
56. Scientific Interpretive
Weaknesses Scientific empirical method does Main weakness is lack of reliability of
not adequately address human individuals’ evidence, perceptions and
subjective, interpretive and interpretations i.e. lack of correlation
experiential phenomena except between what people say and reality.
via physiological substrates.
Evidence of this problem in studies of
e.g. witness testimony, reliability of
memory, relationships between
reported thoughts and physiological
evidence, influence of subconscious
‘thinking’, mental illusions and
delusions in normal people.
‘False consciousness’: people’s
representations of themselves are
inaccurate or simply wrong.
Extends to individuals descriptions of
processes, and the social activities
that they undertake.
57. Scientific Interpretive
Contradictions There is an incompatibility There is tension between interpretive
between scientific modelling of approaches that focus on experiential
design process and inclusion of subjective phenomenological aspects
a process element ‘create a of human creative design activity and
new solution’ as a subjective the frequent shift of emphasis onto
human activity. aspects of design and creative activity
that are more accessible empirically
using a physical science approach.
Claims that all sub-fields of
design are incommensurate as
they use different knowledge There is an epistemological
(and that the broader field of inconsistency in claims that Design
design is fundamentally exists of itself as a phenomena
fragmented) is at odds with capable of creative agency and action.
scientific representation of
designers working across
disciplines and in multi-cross-
and trans-disciplinary teams.
58. Design Research
Love’s proposal:
a unified basis for design theory bridging these two
incompatible approaches.
Advantages
• It provides a coherent epistemological basis for new theories
• It recasts prior research and theory within a justified
integrated framework with a clear epistemology and ontology.
• This in turn provides the basis for developing a design field.
59. Foundations for a unified basis
• Designs (i.e. the specification for creating or doing something)
• Designed outcomes (after they are manufactured/actualised)
• Design activity
• Design processes
• The skills of designers
• The role of design activity
• Cognitive design processes
• Behaviour of designers as individuals and in social groups
• Combinations of the above
61. Deductive logic of quantitative research
Researcher tests or verifies a theory
Researcher tests hypotheses
or research questions
Researcher defines and operationalizes
variables derived from the theory
Researcher measures or observes
variables using an instrument
to obtain scores
Creswell, 2003
62. Inductive logic of qualitative research
Generalizations or theories
to past experiences and literature
Researcher looks for broad patterns.
Generalizations or Theories from
Themes or Categories
Researcher analyze data to
form themes or categories
Researcher asks open-ended questions
of participants or records field-notes
Researcher gathers information
e.g. interviews, observations Creswell, 2003
63. Qualitative vs Quantitative
Quantitative Qualitative
Purpose General Laws Unique/Individual case
Test Hypotheses Understanding
Predict behavior Meanings/Intentions
Perspective Outsider-Objective Insider-Subjective
Procedures Structured Unstructured
formal measures open ended measures
probability samples judgement samples
statistical analysis interpretation of data
64. Qualitative Research
Triangulation
By using several data collecting methods – field
notes, interviews, narratives – a complete picture
of the phenomenon can be provided
67. Interpretation:
observation of discourse
J (reading) pack is firmly attached to the bike positioning of the
backpack was alright fact that the centre of gravity of the
backpack is placed rather far to the back of the bike (inaudible)
I do we have any … em...
J there's a problem with potholes .. the backpack tends to slide up
and down which adversely influences stability I guess when you
hit bumps
I isn't that in the negative?
J mm yeah well the product was considered ugly well that's
solvable (laughter) we can fix that one if nothing else ... it takes a
while to get used to cycling with this weight; mistakes are made
attaching the fastening device to the bike so it has to be easy to
attach
K with only one yeah gotta be fool proof so that's part of our
J yeah that should be in our spec
K functional spec
68. The role of interpretation
Gap between objects and our representations, in
3 forms ('methodological horrors', Woolgar '88):
1. Indexicality
2. Inconcludability
3. Reflexivity
Atomen en electronen als voorbeeld Sociaal constr: objectiviteit niet belangrijk
Despite of continued criticism, the definition of knowledge as ‘justified true belief’ has remained the prevailing definition, and Niedderer (2007) has shown that this understanding of propositional knowledge is implicit in the definition of research because of additional requirements such as the textual/written presentation of an intellectual position (proposition, thesis – ‘true belief’), because of the logic of verification and defence of this intellectual position through argument and evidence (justification), and the requirement for generalisability/transferability and explicit and unambiguous communication.
Therefore, I will try to bring structure, first to link the the relevant questions which have to be asked before we start the actual study.
Definitions associate the names of entities in the universe of discourse (e.g. classes, relations, functions or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms.
Modelling is not a research method, it’s a tool. Testing the tool is about research
Reality. Ho do we know what we know?
Colombia: Oslo: research on design education, describing cases from different countries: emancipation: making people aware of design issues.
Surveys or interviews are not always the right techniques to answer our specific research questions.
Modelling is not a research method, it’s a tool. Testing the tool is about research
First resides in people, especially designers. So, it’s obvious to study design ability/expertise + how they learn. Second, it resides I the process: tactics and strategies, and the techniques they use: methodology. Third, design knowledge is in the product: in one system relating the three dimensions Man – Artefact – Ambiance.
Difficulty in research with interpreting the results of a study. Even in ‘hard’ sciences. Example: Geologists have found fossiles in several layers of the surface of the earth. This is the pattern they find.
Three groups of geologists (b, c and d) come up with an explanatory model for the findings of the geologists (a). Different interpretations can be based on the same empirical data.
For qualitative data such as ‘thinking aloud’ protocol, the interpretation is even harder.