Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Information retrieval in systematic reviews: A case study of the crime prevention literature 
American Society of Criminol...
Overview 
•Systematic reviews and systematic searching 
•Stages in a search 
–Structuring the research question 
–Choosing...
Systematic reviews 
•Systematic review: “a review of research literature using systematic and explicit, accountable method...
Publication bias 
•Publication bias ~ selection (sampling) bias 
•Strong positive effects are likely to be (Alderson & Gre...
Stages in a search 
From Hammerstrømet. al., 2011 
Campbell Collaboration 
1 
•Structure the research question 
2 
•Choose...
MISSION: 
“to identify the best available evidence on approaches to reducing crime (and the potential savings to the polic...
Creating a reference data set 
•Harvesting references from known lists of systematic reviews in Crime Prevention and Crimi...
Choosing sources 
•Crime prevention knowledge base very fragmented 
–Wide-ranging and multi-disciplinary sources selected ...
Create search strategy for databases -keywords 
•Research question broken down into concepts 
•Keywords and synonyms gener...
Create search strategy for databases –controlled vocabulary 
•Controlled vocabulary -manual assignment of descriptors 
•Th...
Create search strategy for databases –building the search syntax 
•The most powerful types of searches apply a search synt...
Review and revise results 
•Prudent to test (pilot) search terms before finalising the search syntax 
•‘Review’ search ter...
Review and revise results 
•We performed sensitivity analysis on each term 
•Also restricted searches by: 
–Date (post-197...
Database 
Unique 
N relevant 
Hit rate (%) 
Database records 
Applied Social Sciences Index and Abstracts (ASSIA) 
416 
22...
Results of database searches –database overlap 
Database 
TOTAL 
Unique 
with ASSIA 
with CJA 
with CJP 
with ERIC 
with I...
Identified studies by source and type 
Typeofpublication 
N 
% 
Bookchapter 
18 
5.5 
Book 
12 
3.6 
Dissertation 
14 
4.3...
Summary 
•High quality search will involve a substantial time involvement 
•Minimising bias = wide range of sources and se...
Thank youBowers, K., Tompson, L. & Johnson, S. (in press) Implementing information science in policing: mapping the eviden...
Final number of included studies = 325 
Our systematic search flowchart
Nächste SlideShare
Wird geladen in …5
×

Information retrieval in systematic reviews: a case study of the crime prevention literature

563 Aufrufe

Veröffentlicht am

Presentation given at the 2014 American Society of Criminology in San Francisco.

Veröffentlicht in: Präsentationen & Vorträge
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Information retrieval in systematic reviews: a case study of the crime prevention literature

  1. 1. Information retrieval in systematic reviews: A case study of the crime prevention literature American Society of Criminology, 19 November 2014 Lisa Tompson and Jyoti Belur UCL Department of Security and Crime Science
  2. 2. Overview •Systematic reviews and systematic searching •Stages in a search –Structuring the research question –Choosing sources –Creating search strategies –Review and refine results •Findings
  3. 3. Systematic reviews •Systematic review: “a review of research literature using systematic and explicit, accountable methods” (Gough, Oliver and Thomas, 2012:2) •A transparent search strategy is a distinctive feature of systematic reviews •Searching for studies ~ data collection •Overarching principle is to MINIMISE BIAS •Synthesising studies not found systematically risks producing unreliable or unrepresentative results
  4. 4. Publication bias •Publication bias ~ selection (sampling) bias •Strong positive effects are likely to be (Alderson & Green, 2002): –Published more (publication bias) –Published more rapidly (time lag bias) –Cited more often (citation bias) –Published in multiple outlets (multiple publication bias) –Published in English (language bias)
  5. 5. Stages in a search From Hammerstrømet. al., 2011 Campbell Collaboration 1 •Structure the research question 2 •Choose databases/sources 3 •Create search strategies for the selected sources 4 •Review results and revise search strategies if necessary 5 •Process references 6 •Log and report the search 7 •Update the search if necessary
  6. 6. MISSION: “to identify the best available evidence on approaches to reducing crime (and the potential savings to the police service, their crime reduction partners and the public)” Structuring our research question Findings from systematic review or meta-analyses Broadly defined ‘crime prevention’ Overall aim was to search for evaluations of interventions in all relevant fields that might have a crime prevention outcome
  7. 7. Creating a reference data set •Harvesting references from known lists of systematic reviews in Crime Prevention and Criminal Justice –Thus, we performed backwards reference checkingat the beginning of our search to generate known studies –Three lists; one known at beginning, two found serendipitously •We used these to help us: –Scope out what journals/sources would be the most appropriate to search –To generate search terms (keywords and controlled vocabulary) –Test the ways in which we refined each search strategy
  8. 8. Choosing sources •Crime prevention knowledge base very fragmented –Wide-ranging and multi-disciplinary sources selected •15 electronic databases: –Criminal justice specific –Social sciences –Multi-disciplinary –Grey literature •Other sources: –UK National Police Library –Hand searches –Reference checking –Contacting experts –Grey literature specialist
  9. 9. Create search strategy for databases -keywords •Research question broken down into concepts •Keywords and synonyms generated for each concept –diversity of expression important to capture •OR combines search terms that are representing the same concept •AND combinesterms that are representing different concepts
  10. 10. Create search strategy for databases –controlled vocabulary •Controlled vocabulary -manual assignment of descriptors •Thesauri available in some databases –this organises controlled vocabulary –Can ‘explode’ a term to search for subordinate terms •Thesauri are though database-specific
  11. 11. Create search strategy for databases –building the search syntax •The most powerful types of searches apply a search syntax –Boolean operators: OR, AND (notAND NOT) 1)Natural language terms for crime types 2)Controlled vocabulary terms for crime types 4)Natural language terms for prevention outcome 5)Controlled vocabulary terms for prevention outcome 8)Natural language terms for evidence synthesis research design 9)Controlled vocabulary terms for evidence synthesis research design 3) #1 OR #2 6) #4 OR #5 7) #3 AND #6 10) #8 OR #9 11) #7 AND #10
  12. 12. Review and revise results •Prudent to test (pilot) search terms before finalising the search syntax •‘Review’ search term generated over 3 million hits in SCOPUS –Other imprecise terms were ‘drugs’ and ‘disorder •To increase the precision of our keywords we incorporated speech marks, wildcards and proximity operators into our search terms Speech marks Wildcards Proximityoperators “calls for service” “bodily harm” Crim* offen?e “firearm? NEAR/5 offender?” systematic* PRE/2 review
  13. 13. Review and revise results •We performed sensitivity analysis on each term •Also restricted searches by: –Date (post-1975) –Language (English) –Document type –Subject headings –Keywords (in multi-disciplinary databases –used two researchers) •We empirically tested the results against the ‘reference data set’ of known studies
  14. 14. Database Unique N relevant Hit rate (%) Database records Applied Social Sciences Index and Abstracts (ASSIA) 416 22 5.3 Criminal Justice Abstracts (CJA) 1,319 77 5.8 Almost 500,000 Criminal Justice Periodicals (CJP) 412 28 6.8 CINCH 117 0 0 Education Resources Information Centre (ERIC) 458 10 2.1 Over 1,000,000 International Bibliography of the Social Sciences (IBSS) 190 12 6.3 Over 2,628,800 National Criminal Justice Reference Service (NCJRS) 2,389 54 2.3 Over 200,000 ProQuest theses and dissertations 598 3 0.5 Almost 3,000,000 PsycEXTRA 347 7 2.0 Over 315,000 PsycINFO 457 84 18.4 Over 3.5 million SCOPUS 2,635 91 3.5 Over 53,000,000 Social Policy and Practice 334 30 8.9 Over 300,000 Social Sciences Full Texts 78 3 3.8 Sociological Abstracts 927 18 1.9 Over 955,030 Web of Knowledge 1,408 32 2.3 Over 60,000,000 Total 12,085 Results of database searches –individual results studies found in individual databases
  15. 15. Results of database searches –database overlap Database TOTAL Unique with ASSIA with CJA with CJP with ERIC with IBSS with NCJRS with PsycEXTRA with PsycINFO with SCOPUS with SP&P with SA with T & d with WoK ASSIA 15 0 N/A 9 4 0 4 4 0 6 11 2 0 0 2 CJA 27 15 0 N/A 2 1 0 0 0 6 9 1 1 0 0 CJP 11 1 1 9 N/A 1 1 2 0 3 8 1 2 0 0 ERIC 5 2 1 1 0 N/A 1 1 0 1 0 0 0 0 0 IBSS 3 0 1 3 0 0 N/A 0 0 1 3 1 0 0 2 NCJRS 30 17 1 9 2 2 0 N/A 1 5 9 5 1 0 2 PsycEXTRA 3 2 0 0 0 1 0 0 N/A N/A 0 0 0 0 0 PsycINFO 16 14 0 2 1 0 0 1 N/A N/A 0 0 0 0 1 SCOPUS 32 15 1 4 0 0 0 1 1 3 N/A 2 1 0 7 SP&P 12 12 0 0 0 0 0 0 0 0 0 N/A 0 0 0 SA 7 1 0 5 4 0 3 4 0 3 4 1 N/A 0 1 T & D 3 0 0 0 2 0 0 0 0 0 0 0 3 N/A 0 WoK 10 5 0 0 0 0 0 0 0 1 3 2 0 0 N/A 84 unique studies out of a total of 174 studies •‘Unique’ column refers to studies not retrieved from other databases –Just under half (84 of 174 studies) unique –Three databases have no unique records –suggesting redundancy
  16. 16. Identified studies by source and type Typeofpublication N % Bookchapter 18 5.5 Book 12 3.6 Dissertation 14 4.3 Journalarticle 197 60.1 Report 87 26.5 Total 328
  17. 17. Summary •High quality search will involve a substantial time involvement •Minimising bias = wide range of sources and search tactics –Search terms need to capture the diversity of expression •Not a linear process –Piloting search terms is important –Empirically testing against a reference dataset useful for verification •Database specific conventions –tailor search strategy to each one •Grey literature vital in crime prevention
  18. 18. Thank youBowers, K., Tompson, L. & Johnson, S. (in press) Implementing information science in policing: mapping the evidence base. Policing: A Journal of Policy and Practice. Tompson, L. and Belur, J. (due to be submitted) Information retrieval in systematic reviews; A case study of the crime prevention literature. Lisa Tompsonl.tompson@ucl.ac.uk
  19. 19. Final number of included studies = 325 Our systematic search flowchart

×