Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Metaphors as Measurement Models: A Study of "Love is a Rose
Ähnlich wie Metaphors as Measurement Models: A Study of "Love is a Rose (20)
Metaphors as Measurement Models: A Study of "Love is a Rose
- 1. Metaphor as Measurement and Vice Versa A Study of the Metaphor “Love is a Rose” William P. Fisher, Jr. University of California, Berkeley International Meeting of the Psychometric Society 9-13 July 2012 Lincoln, Nebraska
- 2. Two Preliminary Points • On the distinction between the literal and the metaphorical… • Metaphor is necessary in discourse as it is the means by which new things come into language.
- 3. Overview • Metaphors as construct models • Models as construct metaphors • A study of “Love is a rose” • Implications • Directions for future research
- 4. "What I cannot create, I do not understand." From Richard Feynman's Caltech classroom blackboard at the time of his death. Hawking, S. W. (2001). The universe in a nutshell. New York: Bantam Books, p. 83.
- 5. Constructs Recreated from Theory • Irvine, Dunn, Anderson, 1990 – British Army Recruitment Battery R > .70 • Embretson, 1998: Abstract Reasoning Test R > .70 • Stenner & Smith, 1982: Knox Cube Test R > .90 • Fischer, 1973: Elementary calculus test R > .85 • Stenner, et al, 1983: Peabody Vocab Test R > .80 • Stenner, et al, 1997: Reading tests R > .90 • Bejar, et al, 2003: Math tests R > .85 • Fisher, 2008: Physical function surveys R > .90
- 6. Criteria for Laboratory Synthesis and Demonstrated Understanding • Data fit a model has the form of a multiplicative scientific law. • A linear unit is defined by the invariance of the estimates across subsamples. • A predictive theory explains a significant portion of the variance in the item location estimates. • The metaphor-model informs a distributed metrology system for point-of-use applications.
- 7. "Every metaphor is the tip of a submerged model." Black, M. (1962). Models and metaphors. Ithaca, New York: Cornell University Press, p. 30. And so might it also be that every model elaborates an often unnoticed metaphor? Perhaps there would be some value in expanded use of a concept of heuristic fiction, fictional truth, or guiding ideal.
- 8. Models as Metaphors “To take a parallel from elementary physics: A ‘mathematical pendulum’ is defined as ‘a heavy point, swinging frictionless on a weightless string in vacuum.’ A contraption like that was never seen; thus as a model for the motion of a real pendulum it is ‘unrealistic’. “ Rasch (1973/2011, p. 1309)
- 9. Models as Metaphors • George Box (1979, p. 202): – “All models are wrong, but some are useful.” • Georg Rasch (1960, pp. 37-38): – “That the model is not true is certainly correct, no models are—not even the Newtonian laws…. Models should not be true, but it is important that they are applicable.” • Also see Rasch (1973/2011): – “All statistical models are wrong!”
- 10. “Love is a rose”
- 11. Love is a Rose Study • 68 entailments, such as – Love is beautiful. Love is thorny. – Love can be bought in a store. – Love grows in the ground. – Love is given to a special person. – Love fades. Love needs sunlight. – Love has leaves. Love inspires passion.
- 12. Construct Hypotheses • Before data were gathered, entailments were divided into three groups • Most likely to be rated TRUE • Rating UNDECIDABLE • Most likely to be rated UNTRUE
- 13. Three survey forms • 33-36 items each – 20 items in common – 11-14 items unique to each form • On each form, items were selected to – Span the full expected calibration range – Represent equally the three hypothetical groups
- 14. 36 total respondents • Subset of original 44 – Selected for completion & cooperative responses – Locations • 15 in Moline, Illinois • 21 in Chicago, Illinois – Sex • 18 Female • 18 Male – Age • Overall average 39.2 • No significant differences by location or sex
- 15. Model-Metaphor Multiplicative Form of Rasch Rating Scale Model Lrqs = Or / Vq / Es where • L is the rate at which love is a rose in the interaction of person r with entailment q at the s level of fictional truth, and L is equal to • the love-rose experience O of person r divided by • the love-rose unity V of entailment q and divided by • the level of love-rose fictional truth E of rating s.
- 17. 15-20 responses per item. R = .94 R disattenuated = 1.00
- 18. 17 or 18 responses per item for each group.
- 19. 35 or 36 responses per item.
- 20. F = .08, 3 df, p = .97
- 24. Romantic entailments of “love is a rose.”
- 27. Concrete entailments of “Love is a rose”
- 29. Conclusions • The model has the form of a multiplicative scientific law. • A linear unit is defined by the invariance of the estimates across subsamples. • A predictive theory explains about 80% of the variance in the item location estimates. • The metaphor itself embodies a distributed metrology system for point-of-use expressions of a measure of romantic love.
- 30. "The greatest thing by far is to be a master of metaphor.” Aristotle
Hinweis der Redaktion
- Before moving into the main part of the presentation, there are two preliminary points to be made, one on the distinction between metaphorical and literal statements, and the other on the role of metaphor in science. First, the symposium abstract sets up the issue of validity in terms of understanding psychological measurement either literally or metaphorically, or perhaps both. But over the last 30 years, since the publication of Lakoff and Johnson’s Metaphors We Live By, experimental studies applying conceptual metaphor theory have shown metaphor to play a necessary, nonarbitrary, and constitutive role in structuring experience (Lakoff and Johnson, 1980; Gibbs, 2008, 2011; Ortony, 1993). Neuro-imaging studies in cognitive linguistics have shown that simple metaphorical and literal statements are comprehended in the same way (Kintsch, 2008; Mashal and Faust, 2010). The pervasiveness of metaphor in everyday discourse was documented in another study showing that English speakers invoke a metaphor every 25 words (Graesser, Long, and Mio, 1989). More recently, a series of five experiments conducted at Stanford University in 2010 (Thibodeau and Boroditsky, 2011) concluded that “metaphors exert an influence over people’s reasoning by instantiating frame-consistent knowledge structures, and inviting structurally-consistent inferences.” This influence was shown to persist over time as people seek out information conforming to a metaphor’s bias, and as they ignore or downplay information that does not conform, even when they are presented with lists of ideas that included information from outside the metaphoric frame. And despite the fairly large effect sizes and high levels of statistical significance obtained in the experiments showing the strong influence of a metaphor on reasoning, participants in the study overwhelmingly denied the influence of the metaphor on their choices and decisions when its presence was made known to them. Metaphor’s ubiquitous presence thus not only shapes perceptions and choices, it does so covertly. Thus it happened that John Locke, like others before and since, famously used finely crafted metaphors and rhetoric to defend his thesis against their use and in favor of strictly literal language.
- Metaphoric process is the means by which new things come into language Discovery and invention of new effects or phenomena require new vocabulary, grammar, references. Metaphor structures learning as the means of connecting the new and never-before-seen with the old and familiar. Where analogy relates known factors in a pattern, metaphor creates new meanings by interactively re-organizing what is known in terms of the emergent knowledge.Second, quite apart from its role in everyday discourse, metaphor also plays a necessary role in the development of new understandings. Persuasive rhetorical devices, including metaphor, are chosen, deliberately or not, from among those available when the means of creating the desired end state or decision are already in hand. But the situation is different when the goal is a new understanding sufficiently informed to enable the consistent reproduction of a never-before seen effect or scientific construct. In this case, a new vocabulary and a new language are needed to be able to refer to, connect with, recreate, and use the new things being brought into the world. Gibbs, R. W., Jr. (Ed.). (2008). Cambridge handbook of metaphor and thought. New York: Cambridge University Press.Gibbs, R. W., Jr. (2011). Evaluating conceptual metaphor theory. Discourse Processes, 48, 529-562.Graesser, A., Long, D., & Mio, J. (1989). What are the cognitive and conceptual components of humorous texts? Poetics, 18, 143-164.Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, Illinois: University of Chicago Press.Ortony, A. (Ed.). (1993). Metaphor and thought (2d edition). Cambridge: Cambridge University Press.Thibodeau, P. H., & Boroditsky, L. (2011, February). Metaphors we think with: The role of metaphor in reasoning. PLoS One, 6(2), e16782.
- Metaphor is the means by which things come into words. Concepts begin as metaphors, both within science and outside of it. The present study was designed with the intention of testing the hypothesis that a metaphor’s system of associated commonplaces or entailments are invariantly structured in the understandings of people familiar with the metaphor. This research
- This statement from Richard Feynman frames the perspective on validity in psychometrics that I would like to expand upon today. Holding that all concepts, including those associated with measurement, begin as metaphors does not imply a lax or weak conception of validity. On the contrary, the capacity to recreate or synthesize the construct in the laboratory is a fundamental criterion that I think must be met before claims to understand a construct enough to make valid statements about it are warranted.------------------------SKIPFollowing Messick and others, I agree that construct validity—showing that the thing measured is what is supposed to be measured—precedes or informs all other forms of validity, such as content, predictive, concurrent, consequential, etc. Being able to repeatedly reproduce an invariant construct with the expected properties—across different samples of persons and across different samples of items—demonstrates one level of theoretical understanding of the construct. But a whole new level of theoretical understanding—one that, I believe, more closely approximates Feynman’s sense of creative understanding—is obtained when not only can the construct be reproduced across samples, but predicted calibrations can be produced from theory for entirely new items. Steps in this direction are taken when items with the same theoretical properties calibrate to the same locations when administered to separate samples, and when the theoretical properties and predicted locations in fact conform to observed values. A study of the metaphor “love is a rose” was undertaken with the purpose of exploring these possibilities.
- There is a significant body of work in this area. This is not a comprehensive list. Much interesting work in the area of cognitive diagnostics was presented at the 2012 IMPS in Lincoln, Nebraska.
- That is, the question asked is, does a metaphor function as an implicit model of a construct? If its entailments are drawn out and made explicit, can questions about them be presented in a way that will allow responses from people familiar with the metaphor to form frame-consistent knowledge structures, and to provide evidence of inferences that are structurally consistent with that frame? In short, is there a model submerged beneath the “love is a rose” metaphor? If so, can it be expressed mathematically? Can data be gathered to conform to it? Can an implicit measure of the weight of meaning be made explicit? Can the implied linguistic instrument be calibrated?Now, the obvious challenge to positive answers to these questions is that metaphors are plainly not true. Obviously, love is not a rose. Surely, man is not a wolf. Nature is not really a book written in the language of mathematics. And the universe is not a giant clockworks. But models are no more true than metaphors…
- Neither do we have any triangles that perfectly fit the Pythagorean theorem, any circles that are really described by πr2, or any molecules that fully obey the Combined Gas Law. As Butterfield put it in his classic history of science, the "law of inertia is not the kind of thing you would discover by mere photographic methods of observation—it required a different kind of thinking-cap, a transposition in the mind of the scientist himself; for we do not actually see ordinary objects continuing their rectilinear motion in that kind of empty space....“"...we do not in real life have perfectly spherical balls moving on perfectly smooth horizontal planes—the trick lay in the fact that it occurred to Galileo to imagine these.“Heidegger is not typically thought of as a philosopher of science, but he was trained in physics and was competent enough in mathematics to serve on dissertation committees for the mathematics department at his university. He noted that Newton's first law "...speaks of a body...which is left to itself. Where do we find it? There is no such body. There is also no experiment which could ever bring such a body to direct perception. But modern science…is supposed to be based upon experience. Instead, is has such a law at its apex. This law speaks of a thing that does not exist. It demands a fundamental representation of things which contradict the ordinary." "The mathematical is based on such a claim, i.e., the application of a determination of the thing which is not experientially derived from the thing and yet lies at the base of every determination of the things, making them possible and making room for them. Such a fundamental conception of things is neither arbitrary nor self-evident. Therefore, it required a long controversy to bring it into [p. 90] power. It required a change in the mode of approach to things along with the achievement of a new manner of thought.”Butterfield, H. (1957). The origins of modern science. New York: The Free Press, pp. 16-17.Heidegger, M. (1967). What is a thing? (W. B. Barton, Jr. & V. Deutsch, Trans.). South Bend, Indiana: Regnery/Gateway, pp. 89-90.Rasch, G. (1973/2011, Spring). All statistical models are wrong! Comments on a paper presented by Per Martin-Löf, at the Conference on Foundational Questions in Statistical Inference, Aarhus, Denmark, May 7-12, 1973. Rasch Measurement Transactions, 24(4), 1309 [http://www.rasch.org/rmt/rmt244.pdf].
- The point of course is that models are not meant to be true. Their value lies in being useful as heuristic guides in practical applications. In this context, the common objection that metaphors obviously cannot be true, and so have no place in science, appears in need of revision. Perhaps models and metaphors have more in common than they are typically perceived to have.
- Metaphors are everywhere in our thoughts, words, and deeds, from mathematical models to everyday language to music and poetry. Roses, for instance, symbolize romantic love in a way that allows various of their primary features to inform lovingexpressions and behaviors.Of course not every feature of roses provides positive support to the metaphor “love is a rose.” Conceptual metaphor theory and cognitive linguistics tend to focus only on the positive entailments to the exclusion of the negative. Philosophers since Parmenides, on the other hand, have noted the interweaving what is with what is not. Psychometricians, furthermore, will immediately understand the value of variation in revealing latent structure.
- A metaphor is the tip of a submerged model in the sense that a large number of unstated implications follow from it. Black referred to these implications as the system of associated commonplaces; Lakoff and Johnson call them entailments. The history of the “love is a rose” metaphor is entwined with the emergence of romantic love during the 12th century in southern France. Immersion in this history was useful for the way the implied system of associated commonplaces emerged from between the lines of the more overt issues in play.A 6-point rating scale offered levels of fictional truth1 Absolutely True 2 Very True 3 Mostly True4 Mostly Untrue 5 Very Untrue 6 Absolutely UntrueMostly Untrue and Very Untrue turned out to be indistinguishable to respondents and so were combined into a single category.
- Thibodeau and Boroditsky (2011) say the results of their experiments support the idea that metaphors instantiate frame-consistent knowledge structures and invite structurally consistent inferences. The “love is a rose” research asks whether people familiar with this particular metaphor will rate its entailments in a way that provides evidence of (a) a knowledge structure consistent with the larger frame of reference and (b) of inferences consistent with that structure. Data analyses focused on evaluating the fit of the data to a unidimensional Rasch model, testing statistical hypotheses using ANOVA, and regressing theoretical item component features on the empirical estimates.Variation in the perceived truth of the metaphorical entailments will be consistent. Entailments will vary from those most supportive of the popular meaning of the metaphor to those least supportive. Subsamples of respondents will invariantly order the entailments Subsamples of entailments will invariantly order the respondents Dividing entailments into groups expected to vary in this way will be useful in developing a predictive theory of the construct.
- Hypotheses. And so we might ask if an analogy could hold between the mathematical pendulum and a mathematical rose? Entailments of the “love is a rose” metaphor were interpreted as an implicit theory of the measured construct. The lawful pattern of observations predicted by this theory was provisionally modeled as Lrqs = Or / Vq / Eswhere the rate L at which love is a rose in the interaction of person r with entailment q at the s level of fictional truth is equal to the love-rose experience O of person r divided by the fictional truth V of entailment q and divided by the additional fictional truth E of rating s. This initial and provisional hypothesis is tested in a two-phase experimental process. First, the observed data must be evaluated for conformity with a probabilistic Rasch model implementing the stated law. Second, the theory of the construct implied by the entailments must explain the variation they exhibit, and must enable some nonrandom degree of precision in those predictions.
- No extreme perfect scores for either items or persons.Mean person measure is within an error of mean item estimate.Four items had mean square fit statistics around 2.0 but were retained for the following analyses.Separation ReliabilityPersons .88Items .92
- This scatter plot explores the question of whether different subsamples of respondents experience the metaphor in the same way, such that the entailments are invariantly ordered. Again, items are centered at 50 and each logit is 10 units for both samples, but no items or transition thresholds were anchored.
- Again, items are centered at 50 and each logit is 10 units for both samples, but no items or transition thresholds were anchored.The rose model of love is again shown to not be entirely true, as discrepancies persist in the contrasting opinions of men vs women.Item 58 (Love has parts that make good tea) tends to be rated more nearly Absolutely Untrue among women than among men. The item has marginal mean square fit statistics in both groups (1.3 to 1.5). The significance of these disagreements on its truth may be related to its extreme location toward the bottom end of the scale.Item 17 (Love withers) tends to be rated more true among men than women. Though the men are in agreement about this stronger degree of truth, with mean square fit statistics of about 1.0, there is little consensus among women as to its truth, as their mean square fit statistics are about 1.6.
- Do different subsets of entailments provoke structurally-consistent inferences, as claimed by Thibodeau and Boroditsky (2011)? The disattenuated correlation of 1.1 indicates some degree of nonrandom error.
- Items were selected for use as links or for distribution across the three forms on the basis of the expected groups of True, Undecidable, and Untrue. In a single analysis simultaneously estimating all of the item locations, the original logit estimates were multiplied by 10 and centered at 50. Though there is some variation in the shapes of the distributions, each group of items spans about the same 3.5-logit range from 35 to 70 and is centered on 50 to within a tenth of a logit.
- Comparing the items by their a priori theoretical groups of True, Undecidable, and Untrue using ANOVA gives an F of 41.3, with 2 df, and p < .001.
- The three theoretical groups are consistently separated within each of the three forms and the linking items. Fs range from 4 to 26, with 2 df, and p from .000 to .04. Conversely, each of the three theoretical groups maintains a consistent profile across the forms, with the exception being the Untrue items on Form 2, which calibrate lower, but not higher, than expected. F for the True and Undecidable groups across the four groups of items are 0.3 and 0.4, with 3 df, and p from .75 to .80. For the Untrue group, F is 3.4, and p is .04.
- Further theory development exercises Post hoc theory expansion Entailments categorized in six groups Romance (1) >> Plant needs (2) Goddesses (3) >> Plant characteristics (4) Plant parts (5) >> Concrete forms (6) Groups ordered from more to less fulfilling of the fictional truth of the metaphor
- ImplicationsVariation in perceived truth of metaphoric entailments is consistent instantiating frame-consistent knowledge structures, and inviting structurally-consistent inferences.Dividing entailments into groups based on their distinctive properties enables theoretical control over the construct. Entailments appear to be virtually adaptively administered in daily practice by those seeking to express a measure of romantic love. The bank of potential entailments may be infinite.The metaphor models and provides a predictive theory that implicitly guides behavior and decision makingModels are metaphors and metaphors are models.In both cases, abstract ideals are heuristic fictions governing a dialogue between data and theory mediated by instruments.ConclusionsThis very small sample study identified and defined an invariant unit of measurement across subsamples of respondents and items. established that the “love is a rose” metaphor is understood by the sample studied to constitute an invariant conceptual law. elaborated a predictive theory of love informed by the properties of roses accounting for 80% of the observed variance in the item location estimates.Directions for Future ResearchThe viral spread of the culture of romantic love following its introduction around 1150 in southern France is an early example of how social networks embody both a medium and a message.The ubiquitous and uniform distribution of metaphors within a culture provides a useful clue as to the importance metrological traceability and reference standards should have in psychological measurement theory and practice.
- These results suggest there is nothing negative or disparaging in saying that psychological measurement is metaphorical, and neither is there anything negative or disparaging in saying that physical measurement is a metaphor. Every metaphor may well be the tip of a submerged model, as Black suggested, and every model may be an elaboration of a usually unnoticed metaphor. If that is the case, then Aristotle’s maxim is more true than ever.--------------------There is further a reasonable possibility that we may be able to come to understand metaphors well enough to create their effects from the theories they imply. It may be possible to write new items, assign them calibration values from theory, and to generate measures from those values without having to gather data for the purpose of estimating item locations and generating person measures from them.Possibilities of this kind have, of course, been explored for quite some time, and have in some cases been brought into practical application, in the works of G. Fischer, Embretson, Bejar, Stone, Stenner, Green, and others.These piecemeal efforts will someday be brought together into a coherent and focused research paradigm. Heidegger (1967, pp. 89-90) speaks of the long controversy associated with the process by which unrealistic mathematical models, such as Galileo’s, grew to fulfill their scientific potential and exert their power in society. Following along the trail blazed by Galileo and Newton, Rasch’s models contradict ordinary perception, assert ideals contrary to experience, and demand both new approaches to things and the achievement of new manners of thought. Any number of other kinds of models might be said to do so as well. To prove any modeling approach’s value, it will be necessary to demonstrate capacities for extraordinary new degrees of productivity in practical application.I foresee two ways in which extraordinary new degrees of productivity are at our doorstep. One involves the efficiencies obtained when items can be selected, calibrated, and used to produce measures on the fly from theory, with no need for engaging the expenses of item writing or of data collection, storage, and analysis. A feasibility study of this approach in mathematics was presented by Bejar and colleagues (2003), and has been a practical reality in reading measurement for several years (Stenner, et al., 2006). The study reported here suggests that the metaphorical process fundamental to the emergence of new concepts in language is itself a manifestation of this potential for creative adaptation. The efficiency gains to be made were suggested during a discussion of this topic at the February 2012 Pearson Global Research conference in Fremantle, Australia: the construction, calibration, administration, and scoring of an item used in a traditional paper and pencil high stakes educational test costs over US$3,000; the same item constructed, calibrated, administered, and scored electronically from theory costs a fraction of a cent.The second way in which new degrees of productivity are well within our grasp involves metrology. Social studies of science have shown that the amazing productivity of science has far less to do with the nature of its constructs, its instruments, or its methods than it does with the way collective intelligence is supported and nurtured by metrological networks of instruments traceable to universally uniform reference standard units (Latour, 1987, 2005; Ihde, 1991; Wise, 1995; Schaffer, 1992; Fisher, 2000, 2005, 2009). Anyone with the least acquaintance with economics understands that markets depend on common currencies for the exchange of value. Ineradicable inefficiencies are the unavoidable consequence when no resources are invested in supplying every decision maker in a field of research and practice with properly calibrated tools. Rasch’s models have been controversial since their introduction over 50 years ago (Wright, 1977, 1984; Andrich, 1989, 2002, 2004; Divgi, 1989; Goldstein, 1979; Fisher, 1994). Perhaps the time is approaching when available expertise, circumstances, necessity, and chance will conspire to cause researchers to put on the “different kind of thinking-cap” Butterfield describes Galileo and Newton as doing. In the same way that mechanics "...only became manageable when in a certain sense it had been 'geometrised” (Butterfield, 1957, p. 96), perhaps psychology will also be seen in retrospect to have become more securely manageable only after its constructs were similarly treated as purely geometrical.