SlideShare ist ein Scribd-Unternehmen logo
1 von 64
Academic incentives and science
Gerry Carter
DO ACADEMIC
INCENTIVES
HURT SCIENCE ?
YES.
1.YES
2. Maybe- what’s the argument?
3. Huh, what are you talking about?
4. NO
What is science?
• Wikipedia: systematic enterprise that builds and
organizes knowledge in the form of testable explanations
and predictions
• Dictionary: systematic knowledge of the physical world
gained through observation and experimentation
What is science?
• Wikipedia: systematic enterprise that builds and organizes
knowledge in the form of testable explanations and predictions
• Dictionary: systematic knowledge of the physical world gained
through observation and experimentation
• the most rigorous way of gaining new knowledge about
nature (whatever that may be), and the sum of that
knowledge
• When you are adhering to the highest standards of logic and
evidence, you are thinking scientifically.(And when you’re not,
you’re not.)*
*Author: Sam Harris
science
• Rigor
• Falsifiability
• Repeatability
• Measures of uncertainty
What is academia?
• Wikipedia: the internationally recognized establishment
of professional scholars and students, usually centered
around colleges and universities, who are engaged in
higher education and research.
• the environment or community concerned with the pursuit
of research, education, and scholarship.
• the “job” of being a scholar at a university and the
community of those scholars
What is academia?
• Wikipedia: the internationally recognized establishment
of professional scholars and students, usually centered
around colleges and universities, who are engaged in
higher education and research.
• the environment or community concerned with the pursuit
of research, education, and scholarship.
• the “job” of being a scholar at a university
• By “academia”, I don’t mean teaching (vs research).
• And I don’t mean the humanities
• I mean the aspects that affect scientific research
How is scientific success different from
academic success?
What is success?
Science
= being correct, falsifiable, and repeatable
Academia
= having influence and impact, being hired, getting funding
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities,
and academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
My argument
1. Incentive structures of science are philosophical.
Examples: double-blind design, replication, and peer-review.
They are typically designed to aid scientific goals.
2. Incentives structures in academia are economic.
Examples: tenure, h-index, and journal impact factors. They
are not designed to meet scientific goals. They emerge as
byproducts of human goals and economic competition within
and among social groups.
(Economic competition exists between publishing companies, between
individuals, between labs, between departments, between universities, and
academia vs other state-funded institutions).
3. Therefore, many academic incentives  bad science.
4. Question: How can we design better academic incentive
structures so that they serve the interests of science?
Science and academia
have many shared incentives
To be a good scientist and a good academic…
• Do good research
• Make important discoveries
• Publish good papers
• Produce public knowledge
• Teach science to students well
As a reviewer, when do you reject a paper
for not enough significance?
As a reviewer, when do you reject a paper
for not enough significance?
What makes a paper good enough?
Science: good enough to be useful evidence for readers
Academia: good enough to be an accolade for the writer
The academic filter in biology
• 16,000 new US Biology PhD students/ year
• Of those recruits, 63% stay until they graduate
• Average time to degree = 7 years
• Of those graduates, 70% get a postdoc
• Of those postdocs, 30% do 1+ more postdoc
• Average postdoc time = ? but more than 4 years
• Within 6 years, 15% of postdocs get a tenure-track job
• We have a 6-7% chance of getting the job we were trained for.
Source: ascb.org “Where will a biology PhD take you”
Mentors: old vs. young
BUT…
…”Not exactly the sort of approach likely to reap
accolades today.”
“I doubt that I would fare particularly well in today’s
academic environment.”
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Replicate. Replicate. Replicate… if you must
Divergent interests (1 of 2)
To be a good scientist… To be a good academic…
Be skeptical of your results “Sell” your results
Interpret conclusions carefully Highlight/exaggerate importance
Publish negative results Publish “strategically”
Ignore social prestige Use impact factors to make writing
decisions
Challenge authority Cite authority. Make friends.
Replicate. Replicate. Replicate… if you must
Novel exciting results are less likely to be
true. Double-check them.
Publish novel exciting results before you
get scooped.
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Masicampoa & Lalande
2012 Quart J Exp Psych
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Long-term goals for important truths Short-term results with high impact
Divergent interests (2 of 2)
To be a good scientist… To be a good academic…
P=0.06 P=0.04
Help others replicate and find flaws in
your work
Have a “territory”
Be clear Sound smart
“Science is a marathon, not a sprint” Funded work must be “transformative”
Natural history is crucial for biological
understanding
Natural history is “low impact”
Long-term goals for important truths Short-term results with high impact
Build a consensus. Hot controversial topics are higher impact
What is academic impact?
• Influence on your field
• Usually involves counting citations, regardless of whether
you are being cited for being right or being wrong.
• Impact is measured at the level of journals (impact factor)
and authors (h-index)
• But actual scientific progress made by articles, methods,
hypotheses, theories, and often teams of researchers
What is academic impact?
• A fundamental problem:
• Over time, scientific validity and importance of an article
both increases and decreases.
• Measures of total impact can only increase.
• Papers that are influentially wrong have higher impact
metrics than papers that are correct.
A survey
From Daniel Dennett:
• (A) You solve the major problem of your choice so
conclusively that there is nothing left to say (thanks
to you, part of the field closes down forever, and you
get a footnote in history).
• (B) You write an article or book of such tantalizing
perplexity and controversy that it stays on the
required reading list for centuries to come.
Which would you choose?
Philosophers and Scientists
From Daniel Dennett:
• (A) You solve the major problem of your choice so
conclusively that there is nothing left to say (thanks
to you, part of the field closes down forever, and you
get a footnote in history).
• (B) You write an article or book of such tantalizing
perplexity and controversy that it stays on the
required reading list for centuries to come.
Scientists choose A. Philosophers often choose B.
Some of the most influential scholars
said little that was correct
Jacques Derrida
“Brilliant and controversial, Jacques Derrida is one
of the most important thinkers of our time, his
considerable body of work having ineradicably
altered the landscape of thought in the 20th and
21st centuries.”
• Wikipedia: best known for developing a form of semiotic analysis known as
deconstruction
• one of the major figures associated with post-structuralism and postmodern
philosophy
• published more than 40 books, hundreds of essays
• significant influence upon the humanities and social sciences, including—in
addition to philosophy and literature—law, anthropology, historiography, linguistics,
sociolinguistics, psychoanalysis, political theory, feminism, and gay and lesbian
studies…. also influenced architecture (in the form of deconstructivism), music, art,
and art critics. Derrida was said to "leave behind a legacy of himself as the
'originator' of deconstruction.”
Deconstruction denies the possibility of a "pure presence": "the present or presence of
sense to a full and primordial intuition".[13][14] It thus denies the possibility of essential
or intrinsic and stable meaning and the unmediated access to "reality”.
Dictionary: a method of critical analysis of philosophical and literary language that
emphasizes the internal workings of language and conceptual systems, the relational
quality of meaning, and the assumptions implicit in forms of expression.
“Pluralistic ignorance”
• no one actually believes, but everyone believes that
everyone else believes
The Emperor’s
New Clothes
The anti-academic culture of science
Richard Feynman on his Nobel Prize:
http://www.liveleak.com/view?i=e5a_1185599458
How should we change academia?
Towards a better academia…
1. Make publishing trivial. Make
peer-review rigorous.
2. Encourage replication.
3. Decentralize funding, rather than
centralized, trickle-down funding
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of impact factors, and then journals.
Journals make money from nothing
Journals make money from nothing
Journals are brands
Journals are brands
In theory, the whole point of a brand is quality control. But in practice,
brands manipulate consumers into seeing high quality regardless of
actual quality.
Journals are brands
In theory, the whole point of a brand is quality control. But in practice,
brands manipulate consumers into seeing high quality regardless of
actual quality.
Journal brands serve as third-party assessment, so that readers “can
judge an article by its cover” and can outsource their evaluation of papers
to 1 editor and 2+ reviewers. But this is better accomplished by a rating
system, like we use for judging movies and books.
Public can read
Publishing
Data
Conclusions
Paper Overhyped
bullshitPicked up
by media
This is how it should work.
“Experimental
treatments affected
gut biota (p=0.01)
and were correlated
with longevity
(p=0.03)”
“Gut microbiota is
linked to longevity
in male mice”
“Ten reasons why
your microbiome is
the key to health
and long life”
Public can
read
Public can read
Publishing
Data
Conclusions
Paper Overhyped
bullshitPicked up
by media
This is how it should work.
But this is how it often works…
SPIN
Data Journal.
University
access only.
Overhyped
bullshit
SPIN
Slightly
hyped
paperWritten for
high
impact
SPIN
Journal demands
brevity and certainty
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and
their reviews).
• Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
This is already basically how ArXive and PLOS ONE work.
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and their
reviews). Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
• Publish negative results and datasets.
Publishing
• Make publishing trivial. Make peer-review rigorous.
Article-level metrics.
• Get rid of journals, impact factors, and academic publishing as a
business.
• Central repository of open-access, peer-reviewed articles (and their
reviews). Magazines can then draw stories from those articles with
varying degrees of rigor (Nature, New Scientist, Fox News).
• Publish negative results and datasets.
• Published peer reviews. Make them objective and open access.
Focus on soundness of results, not subjective importance (which
can be assessed over time). Have standardized peer review
checklists.
Replication
• Encourage replication.
• Open data.
• Metrics to identify what is worth replicating.*
• Publish replications.
A replication crisis
Social psychology
Psychology
Biomedicine
Funding
• Decentralize funding
• Rather than centralized, trickle-down funding
• One idea: most labs receive baseline funding. Some percentage of
all funds must be invested in another lab.
Bollen et al. 2014. From funding agencies to scientific agency:
collective allocation of science funding as an alternative to peer
review. EMBO Reports.
Scientists spend about 40% of their time
chasing grants (2007 U.S. Govt study)
experimental physicist at Columbia University did the calculations:
some grants have a net negative value: they do not pay for the time
that applicants and peer reviewers spend on them.
(Dr. No Money: The Broken Science Funding System. 2011. Scientific American.)
Funding
• Decentralize funding
• Rather than centralized, trickle-down funding
• Use peer-review-like “voting” for funding.
• One idea: most labs receive baseline funding. 30% of all funds
must be invested in another lab justified by merit or promise.
.
Bollen et al. 2014. From funding agencies to scientific agency: collective
allocation of science funding as an alternative to peer review. EMBO Reports
In summary
• Academic incentives can erode good scientific practices.
• We can change academic practices. However, there
needs to be a cooperative consensus, because this is a
public goods dilemma. Like a boycott, it’s only cost-
effective, if everyone does it.
• It would be useful to enact policy changes that offset the
poor incentive structures of academia.
What do you think?
Thanks for the discussion.

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

How to conduct systematic literature review
How to conduct systematic literature reviewHow to conduct systematic literature review
How to conduct systematic literature review
 
Scientific writing anita goel du
Scientific writing   anita goel duScientific writing   anita goel du
Scientific writing anita goel du
 
5.1 identifying-a-research-problem-1
5.1 identifying-a-research-problem-15.1 identifying-a-research-problem-1
5.1 identifying-a-research-problem-1
 
(Research Methodology)
(Research Methodology)(Research Methodology)
(Research Methodology)
 
Public Understanding of Science - Lecture 2 #SciCommLSU
Public Understanding of Science - Lecture 2 #SciCommLSUPublic Understanding of Science - Lecture 2 #SciCommLSU
Public Understanding of Science - Lecture 2 #SciCommLSU
 
Citation indexing
Citation indexingCitation indexing
Citation indexing
 
Use ICT in Research
Use ICT in ResearchUse ICT in Research
Use ICT in Research
 
How to write a research paper?
How to write a research paper?How to write a research paper?
How to write a research paper?
 
How to write and publish good quality research paper
How to write and publish good quality research paperHow to write and publish good quality research paper
How to write and publish good quality research paper
 
Publication ethics: Definitions, Introduction and Importance
Publication ethics: Definitions, Introduction and ImportancePublication ethics: Definitions, Introduction and Importance
Publication ethics: Definitions, Introduction and Importance
 
Research philosophy by zewde alemayehu tilahun
Research philosophy by zewde alemayehu tilahunResearch philosophy by zewde alemayehu tilahun
Research philosophy by zewde alemayehu tilahun
 
Research Methodology-01: Review of Research Literature
Research Methodology-01: Review of Research LiteratureResearch Methodology-01: Review of Research Literature
Research Methodology-01: Review of Research Literature
 
Research Publications in Scopus
Research Publications in ScopusResearch Publications in Scopus
Research Publications in Scopus
 
Sources of knowledge
Sources of knowledgeSources of knowledge
Sources of knowledge
 
Needs and significance of teaching in physical science
Needs and significance of teaching in physical scienceNeeds and significance of teaching in physical science
Needs and significance of teaching in physical science
 
A2 Thomas Kuhn & Scientific Paradigms
A2 Thomas Kuhn & Scientific ParadigmsA2 Thomas Kuhn & Scientific Paradigms
A2 Thomas Kuhn & Scientific Paradigms
 
Bibliometric study
Bibliometric studyBibliometric study
Bibliometric study
 
Research proposal writing
Research proposal  writingResearch proposal  writing
Research proposal writing
 
RESEARCH PARADIGMS WORLD VIEWS
RESEARCH PARADIGMS WORLD VIEWSRESEARCH PARADIGMS WORLD VIEWS
RESEARCH PARADIGMS WORLD VIEWS
 
Guide on how to write Result and Discussion.pptx
Guide on how to write Result and Discussion.pptxGuide on how to write Result and Discussion.pptx
Guide on how to write Result and Discussion.pptx
 

Ähnlich wie Academia

AGU15 & Impact.tech talk
AGU15 & Impact.tech talkAGU15 & Impact.tech talk
AGU15 & Impact.tech talk
microryza
 
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docxDue Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
jacksnathalie
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
study help
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
study help
 
Purpose, theory and policy for higher education
Purpose, theory and policy for higher educationPurpose, theory and policy for higher education
Purpose, theory and policy for higher education
George Roberts
 
Overview In this you will select an issue or.docx
Overview In this you will select an issue or.docxOverview In this you will select an issue or.docx
Overview In this you will select an issue or.docx
bkbk37
 
How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...
TESDARegion5NATCAC
 
High level management theory 陳茂盛--1031113
High level management theory   陳茂盛--1031113High level management theory   陳茂盛--1031113
High level management theory 陳茂盛--1031113
茂盛 陳
 

Ähnlich wie Academia (20)

Academic Research Essay
Academic Research EssayAcademic Research Essay
Academic Research Essay
 
Writing for publication… Some tentative ideas
Writing for publication… Some tentative ideasWriting for publication… Some tentative ideas
Writing for publication… Some tentative ideas
 
AGU15 & Impact.tech talk
AGU15 & Impact.tech talkAGU15 & Impact.tech talk
AGU15 & Impact.tech talk
 
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docxDue Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
Due Jan 6Week 8Using Figure 1.2 in Ch. 1 of Exploring Resear.docx
 
NIH Neuroethics Meeting
NIH Neuroethics MeetingNIH Neuroethics Meeting
NIH Neuroethics Meeting
 
Covering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSUCovering Scientific Research #SciCommLSU
Covering Scientific Research #SciCommLSU
 
Critical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptxCritical-Thinking-Learn-Higher-1.pptx
Critical-Thinking-Learn-Higher-1.pptx
 
Pedagogical research: getting published
Pedagogical research: getting publishedPedagogical research: getting published
Pedagogical research: getting published
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
 
Select an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdfSelect an issue or event in wellness and explore it.pdf
Select an issue or event in wellness and explore it.pdf
 
Purpose, theory and policy for higher education
Purpose, theory and policy for higher educationPurpose, theory and policy for higher education
Purpose, theory and policy for higher education
 
Essay Topics On Religion.pdf
Essay Topics On Religion.pdfEssay Topics On Religion.pdf
Essay Topics On Religion.pdf
 
Overview In this you will select an issue or.docx
Overview In this you will select an issue or.docxOverview In this you will select an issue or.docx
Overview In this you will select an issue or.docx
 
2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust2022 - Fostering Strategic Science Communication related to Trust
2022 - Fostering Strategic Science Communication related to Trust
 
Lesson-1-3.pptx
Lesson-1-3.pptxLesson-1-3.pptx
Lesson-1-3.pptx
 
How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...How to develop student creativity by robert j. sternberg, wendy melissa willi...
How to develop student creativity by robert j. sternberg, wendy melissa willi...
 
Letters to a young(er) scholar: On (alternatives in) publishing
Letters to a young(er) scholar:  On (alternatives in) publishingLetters to a young(er) scholar:  On (alternatives in) publishing
Letters to a young(er) scholar: On (alternatives in) publishing
 
Narrative Literature Reviews
Narrative Literature ReviewsNarrative Literature Reviews
Narrative Literature Reviews
 
High level management theory 陳茂盛--1031113
High level management theory   陳茂盛--1031113High level management theory   陳茂盛--1031113
High level management theory 陳茂盛--1031113
 
An Introduction To Educational Research
An Introduction To Educational ResearchAn Introduction To Educational Research
An Introduction To Educational Research
 

Kürzlich hochgeladen

1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
QucHHunhnh
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
ciinovamais
 

Kürzlich hochgeladen (20)

1029 - Danh muc Sach Giao Khoa 10 . pdf
1029 -  Danh muc Sach Giao Khoa 10 . pdf1029 -  Danh muc Sach Giao Khoa 10 . pdf
1029 - Danh muc Sach Giao Khoa 10 . pdf
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
Grant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy ConsultingGrant Readiness 101 TechSoup and Remy Consulting
Grant Readiness 101 TechSoup and Remy Consulting
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdf
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpin
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 
Disha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdfDisha NEET Physics Guide for classes 11 and 12.pdf
Disha NEET Physics Guide for classes 11 and 12.pdf
 
Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17
 
General AI for Medical Educators April 2024
General AI for Medical Educators April 2024General AI for Medical Educators April 2024
General AI for Medical Educators April 2024
 
Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3Q4-W6-Restating Informational Text Grade 3
Q4-W6-Restating Informational Text Grade 3
 
Advance Mobile Application Development class 07
Advance Mobile Application Development class 07Advance Mobile Application Development class 07
Advance Mobile Application Development class 07
 
Unit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptxUnit-IV- Pharma. Marketing Channels.pptx
Unit-IV- Pharma. Marketing Channels.pptx
 
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
Mattingly "AI & Prompt Design: Structured Data, Assistants, & RAG"
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
IGNOU MSCCFT and PGDCFT Exam Question Pattern: MCFT003 Counselling and Family...
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptxSOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
SOCIAL AND HISTORICAL CONTEXT - LFTVD.pptx
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 

Academia

  • 1. Academic incentives and science Gerry Carter
  • 4. 1.YES 2. Maybe- what’s the argument? 3. Huh, what are you talking about? 4. NO
  • 5. What is science? • Wikipedia: systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions • Dictionary: systematic knowledge of the physical world gained through observation and experimentation
  • 6. What is science? • Wikipedia: systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions • Dictionary: systematic knowledge of the physical world gained through observation and experimentation • the most rigorous way of gaining new knowledge about nature (whatever that may be), and the sum of that knowledge • When you are adhering to the highest standards of logic and evidence, you are thinking scientifically.(And when you’re not, you’re not.)* *Author: Sam Harris
  • 7. science • Rigor • Falsifiability • Repeatability • Measures of uncertainty
  • 8. What is academia? • Wikipedia: the internationally recognized establishment of professional scholars and students, usually centered around colleges and universities, who are engaged in higher education and research. • the environment or community concerned with the pursuit of research, education, and scholarship. • the “job” of being a scholar at a university and the community of those scholars
  • 9. What is academia? • Wikipedia: the internationally recognized establishment of professional scholars and students, usually centered around colleges and universities, who are engaged in higher education and research. • the environment or community concerned with the pursuit of research, education, and scholarship. • the “job” of being a scholar at a university • By “academia”, I don’t mean teaching (vs research). • And I don’t mean the humanities • I mean the aspects that affect scientific research
  • 10. How is scientific success different from academic success?
  • 11. What is success? Science = being correct, falsifiable, and repeatable Academia = having influence and impact, being hired, getting funding
  • 12. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 13. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 14. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 15. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 16. My argument 1. Incentive structures of science are philosophical. Examples: double-blind design, replication, and peer-review. They are typically designed to aid scientific goals. 2. Incentives structures in academia are economic. Examples: tenure, h-index, and journal impact factors. They are not designed to meet scientific goals. They emerge as byproducts of human goals and economic competition within and among social groups. (Economic competition exists between publishing companies, between individuals, between labs, between departments, between universities, and academia vs other state-funded institutions). 3. Therefore, many academic incentives  bad science. 4. Question: How can we design better academic incentive structures so that they serve the interests of science?
  • 17. Science and academia have many shared incentives To be a good scientist and a good academic… • Do good research • Make important discoveries • Publish good papers • Produce public knowledge • Teach science to students well
  • 18. As a reviewer, when do you reject a paper for not enough significance?
  • 19. As a reviewer, when do you reject a paper for not enough significance? What makes a paper good enough? Science: good enough to be useful evidence for readers Academia: good enough to be an accolade for the writer
  • 20. The academic filter in biology • 16,000 new US Biology PhD students/ year • Of those recruits, 63% stay until they graduate • Average time to degree = 7 years • Of those graduates, 70% get a postdoc • Of those postdocs, 30% do 1+ more postdoc • Average postdoc time = ? but more than 4 years • Within 6 years, 15% of postdocs get a tenure-track job • We have a 6-7% chance of getting the job we were trained for. Source: ascb.org “Where will a biology PhD take you”
  • 21. Mentors: old vs. young BUT… …”Not exactly the sort of approach likely to reap accolades today.” “I doubt that I would fare particularly well in today’s academic environment.”
  • 22. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results
  • 23. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance
  • 24. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically”
  • 25. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions
  • 26. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends.
  • 27. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends. Replicate. Replicate. Replicate… if you must
  • 28. Divergent interests (1 of 2) To be a good scientist… To be a good academic… Be skeptical of your results “Sell” your results Interpret conclusions carefully Highlight/exaggerate importance Publish negative results Publish “strategically” Ignore social prestige Use impact factors to make writing decisions Challenge authority Cite authority. Make friends. Replicate. Replicate. Replicate… if you must Novel exciting results are less likely to be true. Double-check them. Publish novel exciting results before you get scooped.
  • 29. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Masicampoa & Lalande 2012 Quart J Exp Psych
  • 30. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory”
  • 31. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart
  • 32. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative”
  • 33. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact”
  • 34. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact” Long-term goals for important truths Short-term results with high impact
  • 35. Divergent interests (2 of 2) To be a good scientist… To be a good academic… P=0.06 P=0.04 Help others replicate and find flaws in your work Have a “territory” Be clear Sound smart “Science is a marathon, not a sprint” Funded work must be “transformative” Natural history is crucial for biological understanding Natural history is “low impact” Long-term goals for important truths Short-term results with high impact Build a consensus. Hot controversial topics are higher impact
  • 36. What is academic impact? • Influence on your field • Usually involves counting citations, regardless of whether you are being cited for being right or being wrong. • Impact is measured at the level of journals (impact factor) and authors (h-index) • But actual scientific progress made by articles, methods, hypotheses, theories, and often teams of researchers
  • 37. What is academic impact? • A fundamental problem: • Over time, scientific validity and importance of an article both increases and decreases. • Measures of total impact can only increase. • Papers that are influentially wrong have higher impact metrics than papers that are correct.
  • 38. A survey From Daniel Dennett: • (A) You solve the major problem of your choice so conclusively that there is nothing left to say (thanks to you, part of the field closes down forever, and you get a footnote in history). • (B) You write an article or book of such tantalizing perplexity and controversy that it stays on the required reading list for centuries to come. Which would you choose?
  • 39. Philosophers and Scientists From Daniel Dennett: • (A) You solve the major problem of your choice so conclusively that there is nothing left to say (thanks to you, part of the field closes down forever, and you get a footnote in history). • (B) You write an article or book of such tantalizing perplexity and controversy that it stays on the required reading list for centuries to come. Scientists choose A. Philosophers often choose B.
  • 40. Some of the most influential scholars said little that was correct
  • 41. Jacques Derrida “Brilliant and controversial, Jacques Derrida is one of the most important thinkers of our time, his considerable body of work having ineradicably altered the landscape of thought in the 20th and 21st centuries.” • Wikipedia: best known for developing a form of semiotic analysis known as deconstruction • one of the major figures associated with post-structuralism and postmodern philosophy • published more than 40 books, hundreds of essays • significant influence upon the humanities and social sciences, including—in addition to philosophy and literature—law, anthropology, historiography, linguistics, sociolinguistics, psychoanalysis, political theory, feminism, and gay and lesbian studies…. also influenced architecture (in the form of deconstructivism), music, art, and art critics. Derrida was said to "leave behind a legacy of himself as the 'originator' of deconstruction.”
  • 42. Deconstruction denies the possibility of a "pure presence": "the present or presence of sense to a full and primordial intuition".[13][14] It thus denies the possibility of essential or intrinsic and stable meaning and the unmediated access to "reality”. Dictionary: a method of critical analysis of philosophical and literary language that emphasizes the internal workings of language and conceptual systems, the relational quality of meaning, and the assumptions implicit in forms of expression.
  • 43. “Pluralistic ignorance” • no one actually believes, but everyone believes that everyone else believes The Emperor’s New Clothes
  • 44. The anti-academic culture of science Richard Feynman on his Nobel Prize: http://www.liveleak.com/view?i=e5a_1185599458
  • 45. How should we change academia?
  • 46. Towards a better academia… 1. Make publishing trivial. Make peer-review rigorous. 2. Encourage replication. 3. Decentralize funding, rather than centralized, trickle-down funding
  • 47. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of impact factors, and then journals.
  • 48. Journals make money from nothing
  • 49. Journals make money from nothing
  • 51. Journals are brands In theory, the whole point of a brand is quality control. But in practice, brands manipulate consumers into seeing high quality regardless of actual quality.
  • 52. Journals are brands In theory, the whole point of a brand is quality control. But in practice, brands manipulate consumers into seeing high quality regardless of actual quality. Journal brands serve as third-party assessment, so that readers “can judge an article by its cover” and can outsource their evaluation of papers to 1 editor and 2+ reviewers. But this is better accomplished by a rating system, like we use for judging movies and books.
  • 53. Public can read Publishing Data Conclusions Paper Overhyped bullshitPicked up by media This is how it should work. “Experimental treatments affected gut biota (p=0.01) and were correlated with longevity (p=0.03)” “Gut microbiota is linked to longevity in male mice” “Ten reasons why your microbiome is the key to health and long life”
  • 54. Public can read Public can read Publishing Data Conclusions Paper Overhyped bullshitPicked up by media This is how it should work. But this is how it often works… SPIN Data Journal. University access only. Overhyped bullshit SPIN Slightly hyped paperWritten for high impact SPIN Journal demands brevity and certainty
  • 55. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). • Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). This is already basically how ArXive and PLOS ONE work.
  • 56. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). • Publish negative results and datasets.
  • 57. Publishing • Make publishing trivial. Make peer-review rigorous. Article-level metrics. • Get rid of journals, impact factors, and academic publishing as a business. • Central repository of open-access, peer-reviewed articles (and their reviews). Magazines can then draw stories from those articles with varying degrees of rigor (Nature, New Scientist, Fox News). • Publish negative results and datasets. • Published peer reviews. Make them objective and open access. Focus on soundness of results, not subjective importance (which can be assessed over time). Have standardized peer review checklists.
  • 58. Replication • Encourage replication. • Open data. • Metrics to identify what is worth replicating.* • Publish replications.
  • 59. A replication crisis Social psychology Psychology Biomedicine
  • 60. Funding • Decentralize funding • Rather than centralized, trickle-down funding • One idea: most labs receive baseline funding. Some percentage of all funds must be invested in another lab. Bollen et al. 2014. From funding agencies to scientific agency: collective allocation of science funding as an alternative to peer review. EMBO Reports.
  • 61. Scientists spend about 40% of their time chasing grants (2007 U.S. Govt study) experimental physicist at Columbia University did the calculations: some grants have a net negative value: they do not pay for the time that applicants and peer reviewers spend on them. (Dr. No Money: The Broken Science Funding System. 2011. Scientific American.)
  • 62. Funding • Decentralize funding • Rather than centralized, trickle-down funding • Use peer-review-like “voting” for funding. • One idea: most labs receive baseline funding. 30% of all funds must be invested in another lab justified by merit or promise. . Bollen et al. 2014. From funding agencies to scientific agency: collective allocation of science funding as an alternative to peer review. EMBO Reports
  • 63. In summary • Academic incentives can erode good scientific practices. • We can change academic practices. However, there needs to be a cooperative consensus, because this is a public goods dilemma. Like a boycott, it’s only cost- effective, if everyone does it. • It would be useful to enact policy changes that offset the poor incentive structures of academia.
  • 64. What do you think? Thanks for the discussion.