Some complementary thoughts to accompany a recent Drug Discov. Today paper (http://goo.gl/SolB6Q) on the positioning of crowdsourcing, and the need to ask the right crowd, the right questions. Presented at the 125th Annual Drug, Chemical and Associated Technologies (DCAT) Meeting in NYC, NY March 18th 2015

1. Crowdsourcing:
Bringing Outside, In
David C. Thompson
@dcthmpsn

6. http://www.netflixprize.com/

7. http://fold.it/portal/

8. http://www.kaggle.com/c/bioresponse
Drug Discov. Today. 2013 May;18(9‐10):472‐8. doi: 10.1016/j.drudis.2013.01.002

9. log loss= 

N
i
iiii yyyy
N 1
)ˆ1log()1()ˆlog(
1
Performance as a function of time
796 players
703 teams
8841 entries
55 forum topics, 409 posts
Drug Discov. Today. 2013 May;18(9‐10):472‐8. doi: 10.1016/j.drudis.2013.01.002

10. https://www.kaggle.com/c/MerckActivity
http://www.nytimes.com/2012/11/24/science/scientists‐see‐advances‐in‐deep‐learning‐a‐part‐of‐artificial‐intelligence.html

11. When doesn’t it work?
• If the solution holders are accessible to you.
• When the availability of a crowd of competent solvers is limited.
• If you do not have an internal framework established to evaluate the
solution(s).
• If the suspected cost (money/time) of implementation of the solution(s) is
too high.
• If safeguards are not in place to prevent malicious or ill‐aligned
participation from the crowd ‐ e.g. the submission of erroneous ‘red
balloon’ locations during the DARPA challenge.
• If the solution(s) are embedded in processes containing lengthy feedback
loops.
Drug Discov. Today. 2015 Jan 27. pii: S1359‐6446(15)00033‐1. doi: 10.1016/j.drudis.2015.01.011

12. What are we trying to do again?
Internalize
external
innovation

13. Getting the fit right
Weak
Partial
Strong
Abstract coupling constant
Drug Discov. Today. 2015 Jan 27. pii: S1359‐6446(15)00033‐1. doi: 10.1016/j.drudis.2015.01.011

14. Strong
• “Our program provides postdoctoral
scholars with a unique opportunity to
perform innovative fundamental
research in a pharmaceutical setting”
[1]
• “Trainees pursue their research and
career training in a culture steeped in
the translation of basic science
discovery into difference‐making
medicines for patients of all ages and
geographies.” [2]
• “Generate innovative science
resulting in high‐quality external
publications” [3]
[1] http://postdoc.nibr.com/, accessed December 18th 2014
[2] http://pfizercareers.com/university‐relations/postdoc, accessed December 18th 2014
[3] http://www.merck.com/research/fellow/home.html, accessed December 18th 2014

15. Partial ‐ Consortia
“We need an evidence‐
based approach ‐ a
science of collaboration ‐
to evaluate and inform the
evolving multi‐stakeholder
collaboration environment
in biomedical innovation.”
[4]
[4] Papadaki, M. and Hirsch, G. (2013) Curing Consortium Fatigue, Sci. Transl. Med. 5,
200fs35.

16. Partial ‐ Academic collaboration
and Open Innovation
Connections between
industry and academic
organizations are well
studied [5, 6] and there
exists a spectrum of ‘one‐
off’ relationships with
external collaborators,
through to an engagement
of ‘Open Innovation’
programs.
[5] Hugget, B. (2013) Academic‐industry partnerships 2012, Nature Biotechnology 31, 383.
[6] Frye, S. et al. (2011) US academic drug discovery, Nat. Rev. Drug Discov. 10, 409–410.

17. Weak
• A loosely connected network of
individuals who, through internet
enabled services and community,
can be engaged in a full spectrum
of activities ranging from micro‐
task through to multi‐hour
research project.
• As was described, such actors
may associate with skills‐based
communities that have no
connection with the
pharmaceutical industry, or
healthcare.

18. Some concluding remarks
If you are intent on trying to ‘internalize external
innovation’, please think about the following:
• Are you asking the right ‘crowd’, the right
questions?
• Are you attempting to coordinate the asking
of questions and your use of ‘crowds’?
– This is the same type of problem as ‘consortium
fatigue’ and has no solution; any attempt is a good
one[7]
[7] Rittel, HWJ and Webber, MW. (1973) Dilemmas in a General Theory of Planning,
Policy Sciences 4 155‐169.