SlideShare ist ein Scribd-Unternehmen logo
1 von 10
Downloaden Sie, um offline zu lesen
Practical Applications of
                       Do-It-Yourself Citation Analysis
                                        Steve Black
                              Serials and Reference Librarian
                           The College of Saint Rose, Albany, NY
June 11, 2012

Rationale for using citation analysis:
       The study of patterns and frequencies of citations is an objective,
quantitative way to measure the impact of journals, authors, institutions, nations,
etc. For librarians, citation analysis can help one identify journals for supporting
research, building collections, or submitting papers for publication.

Impact factor (simple version):
        cites to articles published in last 2 years ÷
        number of articles published in last 2 years
Full definition at
http://thomsonreuters.com/products_services/science/free/essays/impact_factor/

Critiques of relying on impact factors include:
(See bibliography)
      Data errors, both in the citations themselves and in how they are compiled
      by the citation indexes
      Matthew effect, coined by Robert Merton after the passage in Matthew
      (13:12), ―for unto every one that hath shall be given, and he shall have
      abundance: but from him that hath not shall be taken away even that which
      he hath.‖ The more attention a work gets, the more attention it gets, skewing
      citations to papers that are notable for being noted, not necessarily because
      they’re the highest quality or the most important.
      Impact vs. quality vs. importance. Impact is how often cited. Quality is the
      caliber of the work—elegance of method, thoroughness of literature review,
      quality of writing, etc. Importance is contribution to a line of inquiry,
      regardless of how narrow. For various reasons high quality and very
      important papers may not be cited much.
      Global vs. local impact. Citation in publications may poorly correlate with
      the needs of local patrons.
                                          1
Basic DIY Citation Analysis Method
•     Select a target population, e.g.
      –     journals in a sub-discipline
      –     group of researchers
      –     subject(s) or keyword(s)
•     Select a sample that represents the target population
•     Compile works cited in the chosen sample
•     Sort and count the works cited


Case study based on author’s ―Frequently Cited Journals in Forensic Psychology,‖
Psychological Reports, v.110, no.1, (2012): 276-282.


Tools used:
•     WorldCat (to identify most widely held journals)
•     PsycINFO via EBSCOhost (to identify works cited)
•     RefWorks (to organize works cited)
•     Excel (to count and rank cited journals)


Notes on these tools:
            Our ability to use WorldCat holdings as a measure of importance may
            fade over time as fewer libraries subscribe to individual titles that
            appear as holdings in OCLC
            PsycINFO is particularly well suited to the task because all works
            cited are included in the database whether the article is in full text or
            not, and are in a format that is easily exported. If you do a similar
            project, seek out data sources that allow easy downloading of works
            cited.
            RefWorks is only one of many citation management tools that can be
            used for organizing citations.
            Excel has a COUNTIF function that could be used to count times
            cited, but it will undercount if there are any variations in spelling,
            punctuation, or spacing. It’s more accurate to count by highlighting
            the titles and noting the count of highlighted rows.



                                         2
Ranks of cited journals by single issues of Law and Human Behavior

Title cited in LHB 2011                         frequency all
                                                          Allrank FebRank AprilRank JunRank AugRank OctRank DecRank
Law and Human Behavior                                154         1      1         1       1       1       1       1
Criminal Justice and Behavior                          45       2.5    9.5         6       2       2     9.5
Journal of Applied Psychology                          45       2.5      2       4.5               3     4.5       4
Behavioral Sciences and the Law                        40         4                3       6       4       2      16
Journal of Personality and Social Psychology           35         5   16.5       7.5       3     8.5     9.5    11.5
Psychological Assessment                               32         6      3                13     8.5     11        4
Applied Cognitive Psychology                           29         7      4      17.5      13     12     13.5       2
American Psychologist                                  26       8.5   16.5       9.5             8.5     9.5     8.5
Psychiatric Services                                   26       8.5                2            16.5       6
Journal of Abnormal Psychology                         21      10.5      5                13             9.5    11.5
Psychology, Public Policy, and Law                     21      10.5              4.5                     4.5      16
Journal of Consulting and Clinical Psychology          18        12                      8.5     5.5              21
Personality and Individual Differences                 15      13.5                        4                      16
Psychological Bulletin                                 15      13.5             11.5      13     8.5
Assessment                                             14      15.5    6.5               8.5                      21
Psychological Review                                   14      15.5                                        3      21
Journal of Applied Social Psychology                   13        18   16.5      17.5      10
Journal of Experimental Psychology: Applied            13        18                                     16.5     8.5
Psychological Science                                  13        18                             16.5            11.5
Psychology, Crime and Law                              12        20                                                4
International journal of law and psychiatry            11      21.5   16.5                              13.5
Legal and Criminological Psychology                    11      21.5             17.5      13                      16




Tips on assigning ranks
•      In order to test reliability, the number of ranked items in each group must be
equal. In this case December had only 22 titles with 2 or more times cited, so the
top 22 ranked journals in each group were compared.
•      Handle ties by averaging, e.g. tied for 2 & 3 are each ranked 2.5.
•      Assign bottom rank to titles with no match, e.g. if n=22, unmatched titles are
ranked 22.




Formula for Spearman’s rho rank correlation:

NOTE: This is the simple version of rsthat assumes no ties in ranks. See bibliography under
―Technical‖ for sources of the more complex formula. In practice the two formulas return very
similar resultseven with some ties between rankings.


                                                            3
rs correlations among issues of LHB
                          2011 Feb Apr June Aug Oct                  Dec
                     2011  1
                     Feb 0.24    1
                     Apr 0.47 -0.34    1
                     June 0.26 -0.05 -0.20  1
                     Aug 0.63 0.05 0.40 0.15     1
                      Oct 0.59 -0.05 0.46 -0.47 0.11   1
                     Dec 0.23 0.25 -0.66 -0.42 -0.03 -0.02            1

Weak, scattered correlations indicate very low reliability of ranks at the single
issue level, which for this case n=250-350.

rs correlation of LHB (2011) vs. ranking published in Psychological Reports
                                                LHB       Psych
                                                2011     Reports       D      D2
title                                           rank       rank
Law and Human Behavior                            1          1         0        0
Criminal Justice and Behavior                    2.5         3       -0.5     0.25
Journal of Applied Psychology                    2.5         5       -2.5     6.25
Behavioral Sciences and the Law                   4          2         2        4
Journal of Personality and Social Psychology      5          6        -1        1
Psychological Assessment                          6         12        -6       36
Applied Cognitive Psychology                      7          9        -2        4
American Psychologist                            8.5        14       -5.5    30.25
Psychiatric Services                             8.5        20      -11.5   132.25
Journal of Abnormal Psychology                  10.5        22      -11.5   132.25
Psychology, Public Policy, and Law              10.5        10        0.5     0.25
Journal of Consulting and Clinical Psychology    12         16        -4       16
Personality and Individual Differences          13.5        13        0.5     0.25
Psychological Bulletin                          13.5        22       -8.5    72.25
Assessment                                      15.5        22       -6.5    42.25
Psychological Review                            15.5        22       -6.5    42.25
Journal of Applied Social Psychology             18         21        -3        9
Journal of Experimental Psychology: Applied      18         22        -4       16
Psychological Science                            18         22        -4       16
Psychology, Crime and Law                        20         11         9       81
International journal of law and psychiatry     21.5        22       -0.5     0.25
Legal and Criminological Psychology             21.5        15        6.5    42.25
                                                       sum of D squared           684
rs=1-(6*684)/22*(222-1)
= 1 – (4104/10626)
= 1- .37
rs= .63

                                                4
ROUGH rule of thumb for reliability of ranking by sample size

Sample                                  approximate           expected correlation with population
                                        sample size
                                        (works cited)*
one issue of journal                    n=250                 very low
one volume of journal                   n=1500                moderate
several volumes of 3-5 journals         n>10,000              strong
*Of course the number of works cited in a volume or issue will vary depending on the journal




Measure variation of individual titles’ frequency of citation with

Coefficient of Variation = standard deviation ÷ mean


                             Times Cited in 2011 Law and Human Behavior

                                                                                               Coefficient of
                         Feb        Apr      June     Aug      Oct     Dec     σX     mean
                                                                                               Variation
Law and Human
                             28        30       17       25      27      27     4.55 25.67           18%
Behavior
Journal of Applied
                             13         9       0        8        8       7     4.23 7.50            56%
Psychology
Psychology, Public
                               1        9       0        0        8       3     4.04 3.50           115%
Policy, and Law

       In general, titles further down ranked lists have higher coefficients of
variation. A higher coefficient of variation means less reliability of the ranking—
it’s more likely that a different sample will result in a much different rank.




                                                        5
Statistical analysis of this case suggests that so long as the sampled journals validly
represent the topic,a sample of
•      n<1,000 works cited can indicate the top journal (if any)
•      n>1,000 can generate a rough indication of leading journals
•      n>10,000 can create a useful ranked list

CAUTIONARY NOTE
       Even with n>10,000, different samples will yield different ranks, especially
further down the list, so a journal’s rank must ALWAYS be taken as an
approximation of its ―true‖ ranking. Since rankings reflect the complex reasons
researchers cite one thing or another, there can never be a static, definitive ranking
for journals in any topic. Similar statistical analysis I’ve done onjournals in
communication disorders suggests that a third of movement in rankings over time
is due to random variation.


Tips for publication
•     Don’t be too parochial or narrow—editors must think their readers will be
      interested.
•     BUT choose something not already in Journal Citation Reports.
•     Be very thorough with your lit review before gathering data (search
      disciplinary databases as well as the library literature).
•     Gather citations and test sample before nailing down your method.
•     Take the time and energy to ensure sample is
      o representative
      o clearly defined
      o robust
      o valid on its face
•     Group data and run statistics by journal volume
•     Consider submitting to a journal in the topic area




                                           6
PRACTICAL APPLICATIONS OF DIY CITATION ANALYSIS
What’s the top journal on this topic?
–     Compile cites from 10-20 articles on the topic to identify it (or find
there’s not one).

What does my library need to support a proposed new major?
–     Compile cites from 1-2 volumes of 2-4 journals to flag titles not
already available.

Which journals are REALLY used by that herd of cats in Dept. X?
–   Analyze publications & dissertations from last decade.

OMG I’m up for tenure soon!
–    Identify a topic and do a thorough analysis of 2-3 volumes of 4-6
journals; include reliability tests.




                                     7
CITATION ANALYSIS: A SELECTIVE ANNOTATED BIBLIOGRAPHY
CLASSICS
S. C. Bradford, “Sources of Information on Specific Subjects,” Engineering 137 (1934): 85-86.
        This study of the concentration of citations to literature in applied geophysics and
        lubrication is the original source of Bradford’s Law of Distribution.

Eugene Garfield, “Citation Analysis as a Tool in Journal Evaluation,” Science 178 (1972): 471-
479.
      Garfield explains and argues for the Institute for Scientific Information’s Science Citation
      Index.

Robert K. Merton, The Sociology of Science: Theoretical and Empirical Investigations (Chicago:
University of Chicago Press, 1973).
       Includes the famous sociologist’s description of how the Matthew Effect impacts
       scientific output.

OVERVIEWS
Linda C. Smith, “Citation analysis,” Library Trends 30 (1981): 83-106.
       One of several important articles in a special issue of Library Trends devoted to
       bibliometricsin the early years of the “serials crisis,” when many libraries were first
       seriously confronted with having to decide which journals to cut.

Thomas E. Nisonger, “Chapter 5, The Application of Citation Analysis to Serials Collection
Management,” Management of Serials in Libraries (Englewood, CO: Libraries Unlimited, 1998):
121-156.
      A well-organized presentation of major issues and a thorough bibliography.

Thomas E. Nisonger, “Journals in the Core Collection: Definition, Identification, and
Applications,” Serials Librarian 51, no. 3/4 (2007): 51-73.
       Summarizes ten methods for creating lists of core journals and discusses applications of
       core lists.

CRITICAL ANALYSES
Michael H. MacRoberts and Barbara R. MacRoberts, "Problems of Citation Analysis: A Critical
Review," Journal of the American Society for Information Science 40, no. 5 (1989): 342-349.
       A thorough and well organized critique with specific emphasis on Science Citation Index.

Per O. Seglen, “Why the impact factor of journals should not be used for evaluating research,”
British Medical Journal 314 (1997): 498-502.
        Concise summary of the problems associated with the use of journal impact factors.



                                                 8
Maurice B. Line, "Changes in Rank Lists of Serials Over Time: Interlending versus Citation Data,"
College & Research Libraries 46, no. 1 (1985): 77-79.
Robert N. Broadus, "A Proposed Method for Eliminating Titles from Periodical Subscription
Lists," College & Research Libraries 46, no. 1 (1985): 30-35.
Maurice B. Line, "Use of Citation Data for Periodicals Control in Libraries: A Response to
Broadus," College & Research Libraries 46, no. 1 (1985): 36-37.
         These 3 articles comprise a dialogue between Broadus and Line on the validity of
         citation data as a tool for collection development in an era of cost-cutting.

R.E. Rice, Christine L. Borgman, Diane Bednarski, and P.J. Hart, “Journal-to-Journal Citation
Data: Issues of Validity and Reliability,” Scientometrics 15, no. 3 (1989): 257-282.
        Reviews issues of validity and reliability, discusses causes of measurement errors, and
        concludes with suggestions for how to reduce measurement errors.

Ben R. Martin, "The Use of Multiple Indicators in the Assessment of Basic Research,"
Scientometrics 36, no. 3 (1996): 343-362.
       Defines quality, importance, and impact and stresses the importance of respecting each.

Gordon and Breach Science v. American Institute of Physics and American Physical Society,
http://barschall.stanford.edu.
       This web site contains a thorough and well organized treatment of the Gordon and
       Breach case against Henry H. Barschall and the publishers of his studies. Barschall used
       citation counts and subscription costs to create rankings of physics journals. Gordon &
       Breach titles were shown to be among the poorest values in physics, and they sued the
       publisher for false advertising. The court records and related documents do an excellent
       job of presenting the issues surrounding applications of citation analysis.

METHODS
Steve Black, “Using Citation Analysis to Pursue a Core Collection of Journals for Communication
Disorders,” Library Resources & Technical Services 45, no. 1 (2001): 3-9.
       Includes a basic method for do-it-yourself citation analysis.

Steve Black, “How Much do Core Journals Change over a Decade?” Library Resources &
Technical Services 56, no.2 (2012): 80-93.
       Describes methods for correlating ranked lists over time.

 Jeffery D. Kushkowski, Kristin H. Gerhard and Cynthia Dobson, “A Method for Building Core
Journal Lists in Interdisciplinary Subject Areas,” Journal of Documentation 54, no. 4 (1998):
477-88.
        Describes a Simple Index Method for ranking journals based on results of subject or
        keyword searches in relevant databases.




                                                9
Daniela Rosenstreich and Ben Wooliscroft, “Measuring the impact of accounting journals using
Google Scholar and the g-index,” British Accounting Review 41 (2009): 227-239.
       Valuable for its treatment of Google Scholar, a table summarizing common criticisms of
       citation-based journal rankings, and comparisons of ranking methods. (Also a good
       example of how an important paper can be published in an unexpected place!)

Chris Piotrowski, “Top cited journals in forensic psychology: An analysis of the psychological
literature, “American Journal of Forensic Psychology 30, no. 2 (2012): 29-37.
        An example of using keyword searches to rank journals. Piotrowski’s method yields a
        very different ranked list from my list published in Psychological Reports.

TECHNICAL
Thomson Reuters, “The Thomson Reuters Impact Factor,”
http://thomsonreuters.com/products_services/science/free/essays/impact_factor.
        An overview by Eugene Garfield that includes the formula for calculating impact factor,
        the rationale for using it, caveats and cautions.

Stephen J. Bensman, "Probability Distributions in Library and Information Science: A Historical
and Practitioner Viewpoint,” Journal of the American Society for Information Science and
Technology 51, no. 9 (2000): 816-833.
       Argues that parametric statistics based on Poisson distribution are incapable of
       accurately modeling patterns of journal citations.

Sidney Siegel, "Nonparametric Statistics," American Statistician 11, no. 3 (1957): 13-19.
       An authoritative, readable description of when and why to use various statistical
       methods including Spearman’s rho.

Maurice Kendall and Jean Gibbons, Rank Correlation Methods. 5th ed. (New York: Oxford
University Press, 1990).
       Complete descriptions, formulas, and proofs of Spearman’s rho and Kendall’s tau (a
       calculation based simply on whether items go up or down in rank, disregarding the
       degree of change).



                                        Steve Black
                                     blacks@strose.edu
                                      (518) 458-5494




                                               10

Weitere ähnliche Inhalte

Ähnlich wie Practical applications of citation analysis-handout

DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docx
DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docxDataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docx
DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docxrandyburney60861
 
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docx
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docxEthical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docx
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docxAlleneMcclendon878
 
Explore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxExplore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxelbanglis
 
Explore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxExplore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxnealwaters20034
 
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxWritten AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxouldparis
 
PSY 550 Annotated Bibliography Assignment Overview A.docx
 PSY 550 Annotated Bibliography Assignment Overview A.docx PSY 550 Annotated Bibliography Assignment Overview A.docx
PSY 550 Annotated Bibliography Assignment Overview A.docxaryan532920
 
Week 2 Lecture Systematic Reviews II.pptx
Week 2 Lecture Systematic Reviews II.pptxWeek 2 Lecture Systematic Reviews II.pptx
Week 2 Lecture Systematic Reviews II.pptxBrianKyalo7
 
PSY 250 STUDY Education Counseling / psy250study.com
PSY 250 STUDY Education Counseling / psy250study.comPSY 250 STUDY Education Counseling / psy250study.com
PSY 250 STUDY Education Counseling / psy250study.comkopiko77
 
October 30, 2018 Joanna, Your second submissio.docx
October 30, 2018 Joanna,  Your second submissio.docxOctober 30, 2018 Joanna,  Your second submissio.docx
October 30, 2018 Joanna, Your second submissio.docxtarifarmarie
 
Feedback for Professor on week 1 discussionMental health and ill
Feedback for Professor on week 1 discussionMental health and illFeedback for Professor on week 1 discussionMental health and ill
Feedback for Professor on week 1 discussionMental health and illalisondakintxt
 
APA Guide to Preparing Manuscripts
APA Guide to Preparing Manuscripts APA Guide to Preparing Manuscripts
APA Guide to Preparing Manuscripts Sidra Akhtar
 
Somatic Symptom and Related DisordersPrior to beginning work o.docx
Somatic Symptom and Related DisordersPrior to beginning work o.docxSomatic Symptom and Related DisordersPrior to beginning work o.docx
Somatic Symptom and Related DisordersPrior to beginning work o.docxrronald3
 
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docx
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docxAssignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docx
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docxdaniatrappit
 
Discussion 6Initial post due Thursday July, 1, 2021 no later t
Discussion 6Initial post due Thursday July, 1, 2021 no later tDiscussion 6Initial post due Thursday July, 1, 2021 no later t
Discussion 6Initial post due Thursday July, 1, 2021 no later tLyndonPelletier761
 
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxWritten AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxericbrooks84875
 
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docxaulasnilda
 
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docxRAJU852744
 
PSY 215 Tracks and Topics for Final Project For the .docx
PSY 215 Tracks and Topics for Final Project For the .docxPSY 215 Tracks and Topics for Final Project For the .docx
PSY 215 Tracks and Topics for Final Project For the .docxpotmanandrea
 
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docxPSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx4934bk
 

Ähnlich wie Practical applications of citation analysis-handout (20)

DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docx
DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docxDataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docx
DataGenderAgeSupervisorTelecommuteCoworkersHappinessEngagementOver.docx
 
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docx
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docxEthical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docx
Ethical Use of AssessmentYOU ARE GOING TO WRITE THE PAPER AS T.docx
 
Explore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxExplore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docx
 
Explore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docxExplore the Issue PapersYou will choose a topic from the Complet.docx
Explore the Issue PapersYou will choose a topic from the Complet.docx
 
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxWritten AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
 
PSY 550 Annotated Bibliography Assignment Overview A.docx
 PSY 550 Annotated Bibliography Assignment Overview A.docx PSY 550 Annotated Bibliography Assignment Overview A.docx
PSY 550 Annotated Bibliography Assignment Overview A.docx
 
Week 2 Lecture Systematic Reviews II.pptx
Week 2 Lecture Systematic Reviews II.pptxWeek 2 Lecture Systematic Reviews II.pptx
Week 2 Lecture Systematic Reviews II.pptx
 
PSY 250 STUDY Education Counseling / psy250study.com
PSY 250 STUDY Education Counseling / psy250study.comPSY 250 STUDY Education Counseling / psy250study.com
PSY 250 STUDY Education Counseling / psy250study.com
 
October 30, 2018 Joanna, Your second submissio.docx
October 30, 2018 Joanna,  Your second submissio.docxOctober 30, 2018 Joanna,  Your second submissio.docx
October 30, 2018 Joanna, Your second submissio.docx
 
Feedback for Professor on week 1 discussionMental health and ill
Feedback for Professor on week 1 discussionMental health and illFeedback for Professor on week 1 discussionMental health and ill
Feedback for Professor on week 1 discussionMental health and ill
 
APA Guide to Preparing Manuscripts
APA Guide to Preparing Manuscripts APA Guide to Preparing Manuscripts
APA Guide to Preparing Manuscripts
 
Somatic Symptom and Related DisordersPrior to beginning work o.docx
Somatic Symptom and Related DisordersPrior to beginning work o.docxSomatic Symptom and Related DisordersPrior to beginning work o.docx
Somatic Symptom and Related DisordersPrior to beginning work o.docx
 
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docx
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docxAssignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docx
Assignment 1 LASA 2 The Forensic Psychology BookletPotential stu.docx
 
Discussion 6Initial post due Thursday July, 1, 2021 no later t
Discussion 6Initial post due Thursday July, 1, 2021 no later tDiscussion 6Initial post due Thursday July, 1, 2021 no later t
Discussion 6Initial post due Thursday July, 1, 2021 no later t
 
Psych 555 week 2 dq 2
Psych 555 week 2 dq 2Psych 555 week 2 dq 2
Psych 555 week 2 dq 2
 
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docxWritten AssignmentCounseling Psychology vs. Clinical Psychology.docx
Written AssignmentCounseling Psychology vs. Clinical Psychology.docx
 
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
 
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx1Running head REPLACE WITH SHORT TITLEPAGE  PAGE .docx
1Running head REPLACE WITH SHORT TITLEPAGE PAGE .docx
 
PSY 215 Tracks and Topics for Final Project For the .docx
PSY 215 Tracks and Topics for Final Project For the .docxPSY 215 Tracks and Topics for Final Project For the .docx
PSY 215 Tracks and Topics for Final Project For the .docx
 
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docxPSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx
PSY DEVELOPMENTAL PSYCHOLOGY ARTICLE REVIEW INSTRUCTIONS This assignment i.docx
 

Mehr von NASIG

Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...
Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...
Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...NASIG
 
The Serial Cohort: A Confederacy of Catalogers
The Serial Cohort: A Confederacy of CatalogersThe Serial Cohort: A Confederacy of Catalogers
The Serial Cohort: A Confederacy of CatalogersNASIG
 
Calculating how much your University spends on Open Access and what to do abo...
Calculating how much your University spends on Open Access and what to do abo...Calculating how much your University spends on Open Access and what to do abo...
Calculating how much your University spends on Open Access and what to do abo...NASIG
 
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...NASIG
 
Analyzing workflows and improving communication across departments
Analyzing workflows and improving communication across departments Analyzing workflows and improving communication across departments
Analyzing workflows and improving communication across departments NASIG
 
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...NASIG
 
Access to Supplemental Journal Article Materials
Access to Supplemental Journal Article Materials Access to Supplemental Journal Article Materials
Access to Supplemental Journal Article Materials NASIG
 
Communications and context: strategies for onboarding new e-resources librari...
Communications and context: strategies for onboarding new e-resources librari...Communications and context: strategies for onboarding new e-resources librari...
Communications and context: strategies for onboarding new e-resources librari...NASIG
 
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...NASIG
 
Bloomsbury digital resources
Bloomsbury digital resourcesBloomsbury digital resources
Bloomsbury digital resourcesNASIG
 
Web accessibility in the institutional repository crafting user centered sub...
Web accessibility in the institutional repository  crafting user centered sub...Web accessibility in the institutional repository  crafting user centered sub...
Web accessibility in the institutional repository crafting user centered sub...NASIG
 
Linked Data at Smithsonian Libraries
Linked Data at Smithsonian Libraries Linked Data at Smithsonian Libraries
Linked Data at Smithsonian Libraries NASIG
 
Walk this way: Online content platform migration experiences and collaboration
Walk this way: Online content platform migration experiences and collaboration Walk this way: Online content platform migration experiences and collaboration
Walk this way: Online content platform migration experiences and collaboration NASIG
 
Read & Publish – What It Takes to Implement a Seamless Model?
Read & Publish – What It Takes to Implement a Seamless Model?Read & Publish – What It Takes to Implement a Seamless Model?
Read & Publish – What It Takes to Implement a Seamless Model?NASIG
 
Mapping Domain Knowledge for Leading and Managing Change
Mapping Domain Knowledge for Leading and Managing ChangeMapping Domain Knowledge for Leading and Managing Change
Mapping Domain Knowledge for Leading and Managing ChangeNASIG
 
When to hold them when to fold them: reassessing big deals in 2020
When to hold them when to fold them: reassessing big deals in 2020When to hold them when to fold them: reassessing big deals in 2020
When to hold them when to fold them: reassessing big deals in 2020NASIG
 
Getting on the Same Page: Aligning ERM and LIbGuides Content
Getting on the Same Page: Aligning ERM and LIbGuides ContentGetting on the Same Page: Aligning ERM and LIbGuides Content
Getting on the Same Page: Aligning ERM and LIbGuides ContentNASIG
 
A multi-institutional model for advancing open access journals and reclaiming...
A multi-institutional model for advancing open access journals and reclaiming...A multi-institutional model for advancing open access journals and reclaiming...
A multi-institutional model for advancing open access journals and reclaiming...NASIG
 
Knowledge Bases: The Heart of Resource Management
Knowledge Bases: The Heart of Resource ManagementKnowledge Bases: The Heart of Resource Management
Knowledge Bases: The Heart of Resource ManagementNASIG
 
Practical approaches to linked data
Practical approaches to linked dataPractical approaches to linked data
Practical approaches to linked dataNASIG
 

Mehr von NASIG (20)

Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...
Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...
Ctrl + Alt + Repeat: Strategies for Regaining Authority Control after a Migra...
 
The Serial Cohort: A Confederacy of Catalogers
The Serial Cohort: A Confederacy of CatalogersThe Serial Cohort: A Confederacy of Catalogers
The Serial Cohort: A Confederacy of Catalogers
 
Calculating how much your University spends on Open Access and what to do abo...
Calculating how much your University spends on Open Access and what to do abo...Calculating how much your University spends on Open Access and what to do abo...
Calculating how much your University spends on Open Access and what to do abo...
 
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...
Measure Twice and Cut Once: How a Budget Cut Impacted Subscription Renewals f...
 
Analyzing workflows and improving communication across departments
Analyzing workflows and improving communication across departments Analyzing workflows and improving communication across departments
Analyzing workflows and improving communication across departments
 
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...
Supporting Students: OER and Textbook Affordability Initiatives at a Mid-Size...
 
Access to Supplemental Journal Article Materials
Access to Supplemental Journal Article Materials Access to Supplemental Journal Article Materials
Access to Supplemental Journal Article Materials
 
Communications and context: strategies for onboarding new e-resources librari...
Communications and context: strategies for onboarding new e-resources librari...Communications and context: strategies for onboarding new e-resources librari...
Communications and context: strategies for onboarding new e-resources librari...
 
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...
Full Text Coverage Ratios: A Simple Method of Article-Level Collections Analy...
 
Bloomsbury digital resources
Bloomsbury digital resourcesBloomsbury digital resources
Bloomsbury digital resources
 
Web accessibility in the institutional repository crafting user centered sub...
Web accessibility in the institutional repository  crafting user centered sub...Web accessibility in the institutional repository  crafting user centered sub...
Web accessibility in the institutional repository crafting user centered sub...
 
Linked Data at Smithsonian Libraries
Linked Data at Smithsonian Libraries Linked Data at Smithsonian Libraries
Linked Data at Smithsonian Libraries
 
Walk this way: Online content platform migration experiences and collaboration
Walk this way: Online content platform migration experiences and collaboration Walk this way: Online content platform migration experiences and collaboration
Walk this way: Online content platform migration experiences and collaboration
 
Read & Publish – What It Takes to Implement a Seamless Model?
Read & Publish – What It Takes to Implement a Seamless Model?Read & Publish – What It Takes to Implement a Seamless Model?
Read & Publish – What It Takes to Implement a Seamless Model?
 
Mapping Domain Knowledge for Leading and Managing Change
Mapping Domain Knowledge for Leading and Managing ChangeMapping Domain Knowledge for Leading and Managing Change
Mapping Domain Knowledge for Leading and Managing Change
 
When to hold them when to fold them: reassessing big deals in 2020
When to hold them when to fold them: reassessing big deals in 2020When to hold them when to fold them: reassessing big deals in 2020
When to hold them when to fold them: reassessing big deals in 2020
 
Getting on the Same Page: Aligning ERM and LIbGuides Content
Getting on the Same Page: Aligning ERM and LIbGuides ContentGetting on the Same Page: Aligning ERM and LIbGuides Content
Getting on the Same Page: Aligning ERM and LIbGuides Content
 
A multi-institutional model for advancing open access journals and reclaiming...
A multi-institutional model for advancing open access journals and reclaiming...A multi-institutional model for advancing open access journals and reclaiming...
A multi-institutional model for advancing open access journals and reclaiming...
 
Knowledge Bases: The Heart of Resource Management
Knowledge Bases: The Heart of Resource ManagementKnowledge Bases: The Heart of Resource Management
Knowledge Bases: The Heart of Resource Management
 
Practical approaches to linked data
Practical approaches to linked dataPractical approaches to linked data
Practical approaches to linked data
 

Kürzlich hochgeladen

UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICESayali Powar
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxDr. Santhosh Kumar. N
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxSaurabhParmar42
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptxmary850239
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfMohonDas
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17Celine George
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 

Kürzlich hochgeladen (20)

UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICE
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
M-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptxM-2- General Reactions of amino acids.pptx
M-2- General Reactions of amino acids.pptx
 
CAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptxCAULIFLOWER BREEDING 1 Parmar pptx
CAULIFLOWER BREEDING 1 Parmar pptx
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
3.21.24 The Origins of Black Power.pptx
3.21.24  The Origins of Black Power.pptx3.21.24  The Origins of Black Power.pptx
3.21.24 The Origins of Black Power.pptx
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
HED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdfHED Office Sohayok Exam Question Solution 2023.pdf
HED Office Sohayok Exam Question Solution 2023.pdf
 
How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17How to Show Error_Warning Messages in Odoo 17
How to Show Error_Warning Messages in Odoo 17
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 

Practical applications of citation analysis-handout

  • 1. Practical Applications of Do-It-Yourself Citation Analysis Steve Black Serials and Reference Librarian The College of Saint Rose, Albany, NY June 11, 2012 Rationale for using citation analysis: The study of patterns and frequencies of citations is an objective, quantitative way to measure the impact of journals, authors, institutions, nations, etc. For librarians, citation analysis can help one identify journals for supporting research, building collections, or submitting papers for publication. Impact factor (simple version): cites to articles published in last 2 years ÷ number of articles published in last 2 years Full definition at http://thomsonreuters.com/products_services/science/free/essays/impact_factor/ Critiques of relying on impact factors include: (See bibliography) Data errors, both in the citations themselves and in how they are compiled by the citation indexes Matthew effect, coined by Robert Merton after the passage in Matthew (13:12), ―for unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath.‖ The more attention a work gets, the more attention it gets, skewing citations to papers that are notable for being noted, not necessarily because they’re the highest quality or the most important. Impact vs. quality vs. importance. Impact is how often cited. Quality is the caliber of the work—elegance of method, thoroughness of literature review, quality of writing, etc. Importance is contribution to a line of inquiry, regardless of how narrow. For various reasons high quality and very important papers may not be cited much. Global vs. local impact. Citation in publications may poorly correlate with the needs of local patrons. 1
  • 2. Basic DIY Citation Analysis Method • Select a target population, e.g. – journals in a sub-discipline – group of researchers – subject(s) or keyword(s) • Select a sample that represents the target population • Compile works cited in the chosen sample • Sort and count the works cited Case study based on author’s ―Frequently Cited Journals in Forensic Psychology,‖ Psychological Reports, v.110, no.1, (2012): 276-282. Tools used: • WorldCat (to identify most widely held journals) • PsycINFO via EBSCOhost (to identify works cited) • RefWorks (to organize works cited) • Excel (to count and rank cited journals) Notes on these tools: Our ability to use WorldCat holdings as a measure of importance may fade over time as fewer libraries subscribe to individual titles that appear as holdings in OCLC PsycINFO is particularly well suited to the task because all works cited are included in the database whether the article is in full text or not, and are in a format that is easily exported. If you do a similar project, seek out data sources that allow easy downloading of works cited. RefWorks is only one of many citation management tools that can be used for organizing citations. Excel has a COUNTIF function that could be used to count times cited, but it will undercount if there are any variations in spelling, punctuation, or spacing. It’s more accurate to count by highlighting the titles and noting the count of highlighted rows. 2
  • 3. Ranks of cited journals by single issues of Law and Human Behavior Title cited in LHB 2011 frequency all Allrank FebRank AprilRank JunRank AugRank OctRank DecRank Law and Human Behavior 154 1 1 1 1 1 1 1 Criminal Justice and Behavior 45 2.5 9.5 6 2 2 9.5 Journal of Applied Psychology 45 2.5 2 4.5 3 4.5 4 Behavioral Sciences and the Law 40 4 3 6 4 2 16 Journal of Personality and Social Psychology 35 5 16.5 7.5 3 8.5 9.5 11.5 Psychological Assessment 32 6 3 13 8.5 11 4 Applied Cognitive Psychology 29 7 4 17.5 13 12 13.5 2 American Psychologist 26 8.5 16.5 9.5 8.5 9.5 8.5 Psychiatric Services 26 8.5 2 16.5 6 Journal of Abnormal Psychology 21 10.5 5 13 9.5 11.5 Psychology, Public Policy, and Law 21 10.5 4.5 4.5 16 Journal of Consulting and Clinical Psychology 18 12 8.5 5.5 21 Personality and Individual Differences 15 13.5 4 16 Psychological Bulletin 15 13.5 11.5 13 8.5 Assessment 14 15.5 6.5 8.5 21 Psychological Review 14 15.5 3 21 Journal of Applied Social Psychology 13 18 16.5 17.5 10 Journal of Experimental Psychology: Applied 13 18 16.5 8.5 Psychological Science 13 18 16.5 11.5 Psychology, Crime and Law 12 20 4 International journal of law and psychiatry 11 21.5 16.5 13.5 Legal and Criminological Psychology 11 21.5 17.5 13 16 Tips on assigning ranks • In order to test reliability, the number of ranked items in each group must be equal. In this case December had only 22 titles with 2 or more times cited, so the top 22 ranked journals in each group were compared. • Handle ties by averaging, e.g. tied for 2 & 3 are each ranked 2.5. • Assign bottom rank to titles with no match, e.g. if n=22, unmatched titles are ranked 22. Formula for Spearman’s rho rank correlation: NOTE: This is the simple version of rsthat assumes no ties in ranks. See bibliography under ―Technical‖ for sources of the more complex formula. In practice the two formulas return very similar resultseven with some ties between rankings. 3
  • 4. rs correlations among issues of LHB 2011 Feb Apr June Aug Oct Dec 2011 1 Feb 0.24 1 Apr 0.47 -0.34 1 June 0.26 -0.05 -0.20 1 Aug 0.63 0.05 0.40 0.15 1 Oct 0.59 -0.05 0.46 -0.47 0.11 1 Dec 0.23 0.25 -0.66 -0.42 -0.03 -0.02 1 Weak, scattered correlations indicate very low reliability of ranks at the single issue level, which for this case n=250-350. rs correlation of LHB (2011) vs. ranking published in Psychological Reports LHB Psych 2011 Reports D D2 title rank rank Law and Human Behavior 1 1 0 0 Criminal Justice and Behavior 2.5 3 -0.5 0.25 Journal of Applied Psychology 2.5 5 -2.5 6.25 Behavioral Sciences and the Law 4 2 2 4 Journal of Personality and Social Psychology 5 6 -1 1 Psychological Assessment 6 12 -6 36 Applied Cognitive Psychology 7 9 -2 4 American Psychologist 8.5 14 -5.5 30.25 Psychiatric Services 8.5 20 -11.5 132.25 Journal of Abnormal Psychology 10.5 22 -11.5 132.25 Psychology, Public Policy, and Law 10.5 10 0.5 0.25 Journal of Consulting and Clinical Psychology 12 16 -4 16 Personality and Individual Differences 13.5 13 0.5 0.25 Psychological Bulletin 13.5 22 -8.5 72.25 Assessment 15.5 22 -6.5 42.25 Psychological Review 15.5 22 -6.5 42.25 Journal of Applied Social Psychology 18 21 -3 9 Journal of Experimental Psychology: Applied 18 22 -4 16 Psychological Science 18 22 -4 16 Psychology, Crime and Law 20 11 9 81 International journal of law and psychiatry 21.5 22 -0.5 0.25 Legal and Criminological Psychology 21.5 15 6.5 42.25 sum of D squared 684 rs=1-(6*684)/22*(222-1) = 1 – (4104/10626) = 1- .37 rs= .63 4
  • 5. ROUGH rule of thumb for reliability of ranking by sample size Sample approximate expected correlation with population sample size (works cited)* one issue of journal n=250 very low one volume of journal n=1500 moderate several volumes of 3-5 journals n>10,000 strong *Of course the number of works cited in a volume or issue will vary depending on the journal Measure variation of individual titles’ frequency of citation with Coefficient of Variation = standard deviation ÷ mean Times Cited in 2011 Law and Human Behavior Coefficient of Feb Apr June Aug Oct Dec σX mean Variation Law and Human 28 30 17 25 27 27 4.55 25.67 18% Behavior Journal of Applied 13 9 0 8 8 7 4.23 7.50 56% Psychology Psychology, Public 1 9 0 0 8 3 4.04 3.50 115% Policy, and Law In general, titles further down ranked lists have higher coefficients of variation. A higher coefficient of variation means less reliability of the ranking— it’s more likely that a different sample will result in a much different rank. 5
  • 6. Statistical analysis of this case suggests that so long as the sampled journals validly represent the topic,a sample of • n<1,000 works cited can indicate the top journal (if any) • n>1,000 can generate a rough indication of leading journals • n>10,000 can create a useful ranked list CAUTIONARY NOTE Even with n>10,000, different samples will yield different ranks, especially further down the list, so a journal’s rank must ALWAYS be taken as an approximation of its ―true‖ ranking. Since rankings reflect the complex reasons researchers cite one thing or another, there can never be a static, definitive ranking for journals in any topic. Similar statistical analysis I’ve done onjournals in communication disorders suggests that a third of movement in rankings over time is due to random variation. Tips for publication • Don’t be too parochial or narrow—editors must think their readers will be interested. • BUT choose something not already in Journal Citation Reports. • Be very thorough with your lit review before gathering data (search disciplinary databases as well as the library literature). • Gather citations and test sample before nailing down your method. • Take the time and energy to ensure sample is o representative o clearly defined o robust o valid on its face • Group data and run statistics by journal volume • Consider submitting to a journal in the topic area 6
  • 7. PRACTICAL APPLICATIONS OF DIY CITATION ANALYSIS What’s the top journal on this topic? – Compile cites from 10-20 articles on the topic to identify it (or find there’s not one). What does my library need to support a proposed new major? – Compile cites from 1-2 volumes of 2-4 journals to flag titles not already available. Which journals are REALLY used by that herd of cats in Dept. X? – Analyze publications & dissertations from last decade. OMG I’m up for tenure soon! – Identify a topic and do a thorough analysis of 2-3 volumes of 4-6 journals; include reliability tests. 7
  • 8. CITATION ANALYSIS: A SELECTIVE ANNOTATED BIBLIOGRAPHY CLASSICS S. C. Bradford, “Sources of Information on Specific Subjects,” Engineering 137 (1934): 85-86. This study of the concentration of citations to literature in applied geophysics and lubrication is the original source of Bradford’s Law of Distribution. Eugene Garfield, “Citation Analysis as a Tool in Journal Evaluation,” Science 178 (1972): 471- 479. Garfield explains and argues for the Institute for Scientific Information’s Science Citation Index. Robert K. Merton, The Sociology of Science: Theoretical and Empirical Investigations (Chicago: University of Chicago Press, 1973). Includes the famous sociologist’s description of how the Matthew Effect impacts scientific output. OVERVIEWS Linda C. Smith, “Citation analysis,” Library Trends 30 (1981): 83-106. One of several important articles in a special issue of Library Trends devoted to bibliometricsin the early years of the “serials crisis,” when many libraries were first seriously confronted with having to decide which journals to cut. Thomas E. Nisonger, “Chapter 5, The Application of Citation Analysis to Serials Collection Management,” Management of Serials in Libraries (Englewood, CO: Libraries Unlimited, 1998): 121-156. A well-organized presentation of major issues and a thorough bibliography. Thomas E. Nisonger, “Journals in the Core Collection: Definition, Identification, and Applications,” Serials Librarian 51, no. 3/4 (2007): 51-73. Summarizes ten methods for creating lists of core journals and discusses applications of core lists. CRITICAL ANALYSES Michael H. MacRoberts and Barbara R. MacRoberts, "Problems of Citation Analysis: A Critical Review," Journal of the American Society for Information Science 40, no. 5 (1989): 342-349. A thorough and well organized critique with specific emphasis on Science Citation Index. Per O. Seglen, “Why the impact factor of journals should not be used for evaluating research,” British Medical Journal 314 (1997): 498-502. Concise summary of the problems associated with the use of journal impact factors. 8
  • 9. Maurice B. Line, "Changes in Rank Lists of Serials Over Time: Interlending versus Citation Data," College & Research Libraries 46, no. 1 (1985): 77-79. Robert N. Broadus, "A Proposed Method for Eliminating Titles from Periodical Subscription Lists," College & Research Libraries 46, no. 1 (1985): 30-35. Maurice B. Line, "Use of Citation Data for Periodicals Control in Libraries: A Response to Broadus," College & Research Libraries 46, no. 1 (1985): 36-37. These 3 articles comprise a dialogue between Broadus and Line on the validity of citation data as a tool for collection development in an era of cost-cutting. R.E. Rice, Christine L. Borgman, Diane Bednarski, and P.J. Hart, “Journal-to-Journal Citation Data: Issues of Validity and Reliability,” Scientometrics 15, no. 3 (1989): 257-282. Reviews issues of validity and reliability, discusses causes of measurement errors, and concludes with suggestions for how to reduce measurement errors. Ben R. Martin, "The Use of Multiple Indicators in the Assessment of Basic Research," Scientometrics 36, no. 3 (1996): 343-362. Defines quality, importance, and impact and stresses the importance of respecting each. Gordon and Breach Science v. American Institute of Physics and American Physical Society, http://barschall.stanford.edu. This web site contains a thorough and well organized treatment of the Gordon and Breach case against Henry H. Barschall and the publishers of his studies. Barschall used citation counts and subscription costs to create rankings of physics journals. Gordon & Breach titles were shown to be among the poorest values in physics, and they sued the publisher for false advertising. The court records and related documents do an excellent job of presenting the issues surrounding applications of citation analysis. METHODS Steve Black, “Using Citation Analysis to Pursue a Core Collection of Journals for Communication Disorders,” Library Resources & Technical Services 45, no. 1 (2001): 3-9. Includes a basic method for do-it-yourself citation analysis. Steve Black, “How Much do Core Journals Change over a Decade?” Library Resources & Technical Services 56, no.2 (2012): 80-93. Describes methods for correlating ranked lists over time. Jeffery D. Kushkowski, Kristin H. Gerhard and Cynthia Dobson, “A Method for Building Core Journal Lists in Interdisciplinary Subject Areas,” Journal of Documentation 54, no. 4 (1998): 477-88. Describes a Simple Index Method for ranking journals based on results of subject or keyword searches in relevant databases. 9
  • 10. Daniela Rosenstreich and Ben Wooliscroft, “Measuring the impact of accounting journals using Google Scholar and the g-index,” British Accounting Review 41 (2009): 227-239. Valuable for its treatment of Google Scholar, a table summarizing common criticisms of citation-based journal rankings, and comparisons of ranking methods. (Also a good example of how an important paper can be published in an unexpected place!) Chris Piotrowski, “Top cited journals in forensic psychology: An analysis of the psychological literature, “American Journal of Forensic Psychology 30, no. 2 (2012): 29-37. An example of using keyword searches to rank journals. Piotrowski’s method yields a very different ranked list from my list published in Psychological Reports. TECHNICAL Thomson Reuters, “The Thomson Reuters Impact Factor,” http://thomsonreuters.com/products_services/science/free/essays/impact_factor. An overview by Eugene Garfield that includes the formula for calculating impact factor, the rationale for using it, caveats and cautions. Stephen J. Bensman, "Probability Distributions in Library and Information Science: A Historical and Practitioner Viewpoint,” Journal of the American Society for Information Science and Technology 51, no. 9 (2000): 816-833. Argues that parametric statistics based on Poisson distribution are incapable of accurately modeling patterns of journal citations. Sidney Siegel, "Nonparametric Statistics," American Statistician 11, no. 3 (1957): 13-19. An authoritative, readable description of when and why to use various statistical methods including Spearman’s rho. Maurice Kendall and Jean Gibbons, Rank Correlation Methods. 5th ed. (New York: Oxford University Press, 1990). Complete descriptions, formulas, and proofs of Spearman’s rho and Kendall’s tau (a calculation based simply on whether items go up or down in rank, disregarding the degree of change). Steve Black blacks@strose.edu (518) 458-5494 10