The slides for the workshop Improving efficiency and confidence in systematic literature searching by Wichor Bramer and Gerdien de Jonge at #Eahil2015 #researchminded in Edinburgh.

1. Improving efficiency and
confidence in systematic
literature searching
Wichor M. Bramer - Gerdien B. de Jonge
Erasmus MC - Rotterdam the Netherlands
EAHIL+ICAHIS+ICLC – Edinburgh (UK) – June 11nd / 12th 2015

3. (512959)
44 or/18-43 (7140484)
45 random$.tw. (892218)
46 factorial$.tw. (23405)
47 crossover$.tw. (49675)
48 cross over$.tw. (22262)
49 cross-over$.tw. (22262)
50 placebo$.tw. (203743)
51 (doubl$ adj blind$).tw. (147609)
52 (singl$ adj blind$).tw. (14595)
53 assign$.tw. (241607)
54 allocat$.tw. (84725)
55 volunteer$.tw. (181433)
56 Crossover Procedure/ (39305)
57 double-blind procedure.tw. (219)
58 Randomized Controlled Trial/ (347009)
59 Single Blind Procedure/ (18444)
60 or/45-59 (1428276)
61 (animal/ or nonhuman/) not human/ (4623513)
62 60 not 61 (1262127)
63 7 and 12 and 17 and 44 and 62 (952)
64 (201203* or 201204* or 201205* or 201206* or 201207* or 201208* or 201209*
or 201210* or 201011* or 201212* or 2013* or 2014*).dd. (3247786)
65 63 and 64 (263)
66 limit 65 to embase (204)

4. 0%
25%
50%
75%
100%
0 10 20 30 40 50
PercentageofSystematicReviews
Hours needed to create search strategy
Meta-analysis (N=148)
Hausner, 2015 (13); Saleh, 2014 (17); Lyon, 2014 (9); Gann, 2013 (47); Allen, 1999 (19) [1-5]
and unpublished data from London Health Sciences Centre in Canada (43)

5. 0%
25%
50%
75%
100%
0 10 20 30 40 50
PercentageofSystematicReviews
Hours needed to create search strategy
Meta-analysis (N=148)
Bramer (N=258)
Hausner, 2015 (13); Saleh, 2014 (17); Lyon, 2014 (9); Gann, 2013 (47); Allen, 1999 (19) [1-5]
and unpublished data from London Health Sciences Centre in Canada (43)

6. How well do they perform?
Benchmark This study
# terms 46 54
# databases 3 7
# hits 1399 2224
# includes 19 25
precision 1.2% 2.0%
Time needed 960 min 70 min
Benckmark: 93 systematic reviews from 2014 with an acknowledgement or co-authorship
of a medical information specialist from another Dutch academic hospital.

8. Systematic searching
1. Examining the research question (practicum)
2. Identifying key terms (practicum)
a. Thesaurus terms
b. Free text terms
3. Database / interface choice (discussion)
4. Creating a basic search strategy (practicum)
5. Optimizing the basic strategy (practicum)
6. Translating the strategy between databases
(demonstration)
7. Increasing specificity or sensitivity (demonstration)
8. Filters (discussion)
9. Finding errors (demonstration)

9. Which articles can answer the question?
Facilitators and barriers in preconception care
 evaluation studies of preconception care
Predictive factors for oronasal fistulas in children
treated for clef lip
 follow up studies of children treated for clef
lip Study types not as exclusive filters
but additionally added in elements

10. Elements : Bias
Very specific characteristics
Duration of breastfeeding and intelligence
Judgements
Poor family functioning, harsh parenting and
bullying behavior
Why a bias?

11. Elements : Duplicates
Pre-eclampsia AND pregnant AND women
Lichtenstein therapy AND inguinal hernias

12. Which elements to use:
Does eating Broccoli help in the prevention of cancer
generalspecific
important
unimportant
Broccoli Cancer
Prevention
Broccoli CancerPrevention

13. Which elements to use:
your own research question
generalspecific
important
unimportant
PRACTICUM
Describe the articles you want to find
Determine elements
Plot them in the graph
Discuss with your neighbour

14. Identifying terms
Two major thesauri:
MeSH (Medline)
Emtree (Embase)
Three interfaces:
PubMed (MeSH)
Ovid (MeSH and Emtree)
Embase.com (Emtree)

15. Identifying thesaurus terms
One element  Multiple thesaurus terms (OR)
Multiple elements  One thesaurus term
One element  Multiple thesaurus terms (AND)

16. No thesaurus terms found
Truncate the search terms with * when searching
MeSH database in PubMed
tear vs. tear*
• Simple search in PubReminer
• Use [tt] in PubMed to find translations
• Use Google, Wikipedia etc. to find alternative
terms
If all this fails:
Use only free text words and see what you get

17. Identifying free text words
Optimal sensitivity only by additionally searching in
title / abstract:
• Relevant, non-inverted terms from synonyms /
entry terms
• Relevant words / phrases from narrower terms

18. Add for yourself
- Abbreviations
- Also partial abbreviations: Hip OA / Sentinel LN
- Alternative spelling
- Randomization / Randomisation
- Pediatric / Paediatric
- Tumor / Tumour

19. PRACTICUM (5 min) – Finding keywords
Using the MeSH interface from PubMed
Try to find as much search terms as possible for
Kidney dialysis

20. Renal Dialysis [MeSH]
Renal Dialy*
Hemodialy*
Extracorporeal Dialy*
Hemodiafiltration*
Peritoneal Dialy*
Kidneys, Artificial [mesh]
Artificial Kidney*
Blood Dialy*
Acetate Free Biofiltrat*
CAPD
Renal Replacement Therapy[mesh:noexp]
Renal Replace*
Kidney Replace*
Haemodialy*
Haemodiafiltrat*
(Dialysis [mesh] AND (Kidney
Diseases [mesh] OR Kidney
[mesh]))
Kidney dialy*

21. Database / interface choice

22. Advice for start database
1. embase.com if available
2. Otherwise embase with proximity
(preferably Ovid)
3. No embase? Medline with proximity
4. No proximity interface available? PubMed
– more complicated
– will miss relevant articles
– or get much more noise

23. Creating a basic search strategy

24. Embase.com Ovid EBSCOHost PubMed
Boolean AND, OR, NOT
Thesaurus '…'/exp exp "…"/ DE "…+" "…"[mh]
Title /
abstract
():ab,ti ().ab,ti. TI() OR AB() [tiab]
Proximity (() NEAR/n ()) (() ADJn ()) (() Nn ()) (() AND ())
truncated phrase*
Standard syntax
Extended version (20 syntaxes in 7 databases) as supplement in handouts
DEMO

25. Creating a basic search strategy
General rules:
1. Work in a word document, not the interface
2. Create single line search strategies

26. Single line approach or set numbers?
Well structured?
#1 brassica/exp
#2 brassica*:ab,ti
#3 broccoli:ab,ti
#4 #1 OR #2 OR #3
#5 neoplasm/exp
#6 neoplas*:ab,ti
#7 cancer*:ab,ti
#8 tumor*:ab,ti
#9 #5 OR #6 OR #7 OR #8
#10 #4 AND #9
#11 [animals]/lim
#12 [humans]/lim
#13 #11 NOT #12
#14 #10 NOT #13
Unstructured?
(brassica/exp OR (brassica* OR
broccoli):ab,ti) AND (neoplasm/exp
OR (neoplas* OR cancer* OR
tumor*):ab,ti) NOT ([animals]/lim
NOT [humans]/lim)

27. Creating a basic search strategy
General rules:
1. Work in a word document, not the interface
2. Create single line search strategies
3. Type code before adding the words
4. After opening parentheses, immediately close them
5. Never remove parentheses, only add
6. Prepare proximity statement for reuse
7. Copy terms from thesaurus instead of typing them
8. Use arrow keys and Ctrl-shortcuts
9. Use ab,ti to create the strategy, final syntax can be broad
PRACTICUM

28. Optimizing search strategy
Search temporarily per element:
(thesaurus) NOT (title/abstract)
(title/abstract) NOT (thesaurus)
Remove all field codes (automatic term mapping)
Check retrieved relevant articles for missed terms
DEMO + PRACTICUM

29. Translating the strategy between
databases
Embase.com (Embase and Medline)
Thorough optimization of thesaurus terms and title / abstract terms in search strategy
Macro and edit thesaurus
Medline (Ovid)
MeSH terms in PubMed / Optimization
Add extra terms Remove thesaurus
Cochrane library
Macro
Web-of-science
Macro
Scopus
Macro
PubMed
publisher
[sb]
Macro and
edit
thesaurus
CINAHL
(EBSCO)
edit
thesaurus
PsycINFO
(OvidSP)
Macro and edit
Google Scholar
Macro and edit
Lilacs / Scielo / ProQuest /
Clinical trials etc

30. Translation
Ctrl-H: find and replace
special tip:
^p for linebreak
^t for tab
Macros: bit.ly/emcmacros

31. Considerations
Explode a broad term or not?
Start exploded, when many irrelevant
results appear, be more specific
Truncate a term or not?
Truncate as much as possible, hardly noise
Phrases or proximity?
Always proximity (when available in
interface), easier and more sensitive
How broad should proximity be?
Start with 3, increase with 3 and
evaluate newly retrieved articles

32. • Remove too general terms or too broad
thesaurus terms.
• Do not explode thesaurus terms
• Add an extra element
• (mesh) AND (subheadings) 
(mesh/subheading)
• Narrow proximity
• Separate elements  proximity
• Remove truncation (longer wordstem)
• (I OR C)  (I) AND (C)
• Add Filters for study type?
date?
language?
Only if you can afford missing relevant refs:
• For the most important element: use major
and title instead of keyword and title abstract
Increase Sensitivity (always)
• Drop less important elements
• Add extra words (thesaurus NOT tiab)
• Generalize specific elements (broader
thesaurus terms)
• Explode thesaurus terms
• (mesh/subheading)  (mesh) AND
(subheading)
• Broaden proximity
• Proximity  separate words (AND)
• Add truncation (shorten wordstem)
• (I) AND (C)  (I OR C)
• Temporarily broaden or drop a
very specific element to optimize the
others
• Save relevant hits on clipboard to check
sensitivity of more specific searches
Increase Specificity (if too many)

33. Filters
Add filters only if they are important elements in
your research question
(top right corner of graph)
Where to find good filters to use?
Translating between databases?

34. Check for errors
How do you know an error occurred?
Sometimes: More or less hits than expected
Too often: NOT!
Check your query for mistakes
Use Ctrl-F to search for ' AND '
Scan or search for Automatic Term Mapping
(such as .mp. OR [all fields])

35. Successful search?
1. Coverage of accessible databases
2. Features of search engine of used interfaces
3. Quality of indexing
4. Presence and quality of abstract
5. Quality of search and experience of searcher

36. Success factors / Time savers
• Optimization techniques
• Macros for syntax translation
• Direct feedback by researcher
• Database order and interface choice
• Single-line search strategies
• Create query in Word and paste in database
• Experience (be pro-active)
• Having two screens on your computer
• Computer literacy
keyboard short cuts (supplement 3)

37. Quality is the norm, not speed!
Not a blueprint for speed but for
confidence by standardization
Speed can only be a result of experience
By gaining speed more customers can be helped
and overall review quality will improve

38. Contact details
E-mail: w.bramer@erasmusmc.nl
g.dejonge@erasmusmc.nl
info.mb@erasmusmc.nl
Website: www.erasmusmc.nl/medbib
Twitter: @wichor
LinkedIn: nl.linkedin.com/in/wichor
Slideshare: www.slideshare.net/wichor

39. Further reading:
• Bramer WM, Giustini D, Kramer BM, Anderson PF. The comparative recall of
Google Scholar versus PubMed in identical searches for biomedical
systematic reviews: a review of searches used in systematic reviews. Syst
Rev. 2013 Dec 23;2:115.
• Bramer WM, Holland L, Mollema J, Hannon T, Bekhuis T. Removing duplicates
in retrieval sets from electronic databases: comparing the efficiency and
accuracy of the Bramer-method with other methods and software packages
(Eahil conference, Rome)
• Embase webinar : Systematic searching with Wichor Bramer (25 feb 2015)
• Bramer WM, Pols DHJ, Bindels PJE, Bohnen AM. A fast, reliable and objective
method for creating thematic search filters in bibliographic databases [in
process]
• Work in progress on coverage, retrieval and relevance ranking of the major
databases for systematic reviews
(>450 review searches; >100 finished; >4500 included references)
• Two articles in progress about this methodology (how to, and how well)

40. What to do with subheadings
Separate element
(broccoli/de OR (broccoli* OR
bassica*):ab,ti) AND (neoplasm/exp
OR (cancer* OR neoplas*):ab,ti) AND
(prevention:lnk OR prevention/exp
OR (prevent*):ab,ti)
Combined element
(broccoli/de OR (broccoli* OR
bassica*):ab,ti) AND
(neoplasms/exp/dm_pc OR ‘cancer
prevention’/de OR ((cancer OR
neoplasm) NEAR/3 (prevent*)):ab,ti)
Sensitive
 More hits : 345 (101 unique)
 Less chance of missing relevant
articles
 Recommended for start
 For SRs highly recommended
Specific
 Less hits : 257 (13 unique)
 Higher chance of missing relevant
articles
 To reduce the number of hits
 Use proximity in free text

41. Why start in embase?
Source: Unpublished research Bramer (5048 included references from 111 reviews)

42. Relevance ranking

43. Which databases to use
Database Used in % of our SRs
Embase.com 100%
Medline (Ovid) 99%
(PubMed 1%)
Cochrane central 98%
Web-of-Science 95%
Google Scholar 92%
PubMed (publisher[sb]) 91%
Scopus 36%
Cinahl 25%
PsycINFO 19%
Others (proquest, lilacs, scielo,
sportdiscus, eric, amed etc)
9%

44. Unique articles per database
embase 509
Web-of-Science 244
medline 195
Google Scholar 139
Other databases 92
Source: own observation on 5048 included references of 111
systematic reviews