1. CACAO
Community
Assessment of
Community
Annotation with
Ontologies
i.e. Undergrads doing functional annotation using Gene Ontology

2. Basic problem for Community
Annotation
• People are enthusiastic in theory but too busy in
practice
• Our approach: annotation as a teaching activity
provides incentives to participate (for some)

3. Student curation focused on IEA
validation
• Without much experience with GO, students can still
– look for experiments to support IEA or ISS annotations
– deprecate incorrect annotations

4. CACAO 0.1
• 20 students volunteered
• 16 students completed it ~ 2.5 weeks later
• Mini GO camp
– meeting 1: Survey/GO overview
– meeting 2: annotate 2 papers as group
• Paper 1: everyone
• Paper 2: divided into teams
• Each team got a list of E. coli genes; annotation on EcoliWiki
• meeting 3: Survey/Evaluate annotations/announce winners
– Peer review by competition = Teams steal points by correcting
annotations

5. Results
• Got many more annotations
than we expected
• Quality was not bad
• Survey provided insight into
student strategies
– Conversion of non-GO
functional annotations
– new annotations

6. Lessons/Refinements
• Students loved the competition
• Need to tweak the training
– More on finding appropriate papers
• Need multiple rounds
– Students did not use the mentors enough
• More time for challenges
• Better wiki tools for mentors/judges to track student
annotation

7. Scoreboard extension

8. Scoreboard extension

9. CACAO 1.0
• Recruiting participants for Fall 2010
– ASM, ASMCUE, AgBase, …
– Not just E. coli!!
• We will offer
– Content related to GO/genomics/function
– Support for assessment
• Rubrics, surveys etc.
– Plans for possible publications
• Incentives
– Credit for innovative teaching on their campuses
– NSF broader impacts
– inter-institution teams for recruiting