We believe that the quality and efficiency of all phases of the clinical and translational research (CTR) process can potentially be increased by using digital practices and tools in open and networked contexts. However, most CT researchers lack the training to take advantage of the benefits that the Internet and the social Web provide. Standardized training in digital practices and tools (Digital Scholarship) to conduct CTR has not been formalized through structured curriculum, learning approaches, and evaluation. Our overall goal is to develop a robust curriculum to train CTR researchers in digital scholarship. Here we present preliminary data from a qualitative study that describes the range of key stakeholders’ perspectives on the need to: (A) formalize educational efforts in digital scholarship among CTR trainees; and (B) develop an educational framework that defines core competencies, methods, and evaluation methods. What we learned so far in a nutshell: 1. There was consistent support for a structured program to train CTR scientists to develop competency in digital research practices and approaches. 2. Participants considered all existing CTR and additional suggested competency areas relevant. 3. New training areas suggested by participants: network analysis, citizen science, crowdsourcing, machine learning, digital health training, computing, data management and licensing, among others. 4. A education program focused on digital scholarship should include a step-wise approach to meet different research and training goals. allowing attendees to increase their awareness through flexible online learning modules and lectures as well as specialized in-person and hands-on experience, also including consultations, coaching, and networking. 4. The development of a clear definition for “digital scholarship” in the context of CTR, program goals and evaluation metrics represents challenges that require further exploration. 5. Next steps: To establish consensus among the focus groups attendees regarding necessary components of a CTR-focused curriculum through a follow-up questionnaire. Presented at Translational Science 2018 conference in Washington, DC on April 20, 2018.

2. Perspectives on Increasing Competency in Using
Digital Practices and Approaches to Enhance
Clinical Translational Research
Preliminary Results of a Qualitative Study
Reuter K*, Simpson K, Le N, Bluthenthal RN, Patino CM
*Speaker: Assistant Professor of Clinical Preventive Medicine, Department of Preventive Medicine, Keck
School of Medicine of USC; Director of Digital Innovation and Communication, Southern California Clinical
and Translational Science Institute (SC CTSI), University of Southern California (USC)

3. Disclosure
The speaker and co-authors have no actual or potential financial or
other conflict of interest in relation to this research presentation.

4. The Vision

5. “Digital Scholarship”
• Term was originally coined to characterize professional
practices among scholars linked to digital environments and
tools [1,2].
• Defined generally as the use of digital evidence, methods of
inquiry, research, publication and preservation to achieve
scholarly and research goals [3].

6. Research Objectives
Overall goal: to develop a robust curriculum to train
CTR researchers in digital scholarship.
Describe the range of key stakeholders’ perspectives on
the need to:
(A) formalize educational efforts in digital
scholarship among CTR trainees; and
(B) develop an educational framework that
defines core competencies, methods, and
evaluation methods.

7. Target Audiences
• Digital Scholarship Experts
• KL2/TL1 Educators and Trainees
• SC CTSI administrators and staff

8. Methods
Participants: Recruitment via email (June-November, 2017) using
purposive and snowball sampling methods
Methods: Online focus groups using a semi-structured, open-
ended interview guide via Google Hangouts and conference call
interface
Analysis/Coding: Initial “a priori” codes and definitions based on
the interview guide and “Master’s CTR Core Competency Areas”
[4]; framework approach [5]; inter-rater reliability used to evaluate
consistency between two coders (kappa=0.68)
Software: Atlas.ti (Version 8.1.3)

9. Results: “Digital Scholarship” Definition
Broad definition was provided:
“Digital scholars can see the
options made available to
them, their students, and their
research by new, digital
technologies. They have a
rationale for taking the time
to learn these technologies
and for applying them to their
scholarship.”

10. Results: “Digital Scholarship” Definition

11. Results: “Digital Scholarship” Definition
“I think that’s a very succinct definition because technology is forever evolving. I
think the idea of standing still and not looking for fresher ways of working, even if
you use a (…) technology to solve a tiny part of your process, then it’s a useful
kind of idea. I think with technology you can’t stand still if you want to engage with
it. You have to be prepared to get things wrong. And you have to be prepared to
experiment. So I think that that definition you’ve got kind of captures [this] (…).”

12. Existing CTR Competency Areas
“Master’s CTR Core Competency Areas” guidelines developed by an
NIH Education Working Group [4]
 Literature review
 Literature critique
 Study design
 Research implementation
 Sources of error
 Statistical approaches
 Biomedical informatics
 Regulatory support
 Responsible conduct of research
 Scientific communication
 Cultural diversity
 Translational teamwork
 Leadership
 Cross-disciplinary training
 Community engagement

13. Competency Areas

14. Competency Areas
“I think there’s something digital that can run through all of these.”
“The idea of taking every aspect of the competency and finding out if there are
digital tools that relate to each aspect of those competencies I think would be
highly beneficial.”

15. Basic vs. Advanced Competency Level
"On a very basic level, it seems to me that you should know something about
how digital methods apply in the context of all of these competencies on a very
basic level. Maybe knowing how to become advanced in those areas is the
advanced section that can be self-study if designed correctly.”

16. Educational Didactics

17. Educational Didactics
“We try to create (…) spaces for them to do that work so that they’re not making
horrific mistakes in public, but that really enable them to do the hands-on
exploring. I’m providing the guidance in how to use [digital tools] appropriately
and effectively.”
“Connect people who are using the same tools so that they can help each
other.”
“Where you meet with someone, you talk about your research and your needs,
and they then identify the types of resources that would be useful for you and
here’s ways to get that information. ‘Cause I think it is very individual in terms of
what people need to learn or what they need for their research projects.”

18. Evaluation Metrics
“I think the harder part is how do you measure whether a participant now
using this digital scholarship has actually enhanced their work? How do you
quantify that?”

19. Conclusions
Consistent support for structured Digital Scholarship program for CTR scientists
All existing CTR and additional suggested competency areas relevant
New training areas suggested by participants: network analysis, citizen science,
crowdsourcing, machine learning, digital health training, computing, data
management and licensing, among others.
Establish consensus regarding necessary components of a CTR-focused
curriculum
Areas of clarification: definition for “digital scholarship” in the context of CTR,
program goals, and evaluation metrics
A education program should meet different research and training goals (i.e.,
flexible online learning, lectures, specialized in-person and hands-on experience,
consultations, coaching, networking)

20. Acknowledgement
This work was supported by the grant UL1TR001855 from the National Center
for Advancing Translational Science (NCATS) of the U.S. National Institutes of
Health. The content is solely the responsibility of the authors and does not
necessarily represent the official views of the National Institutes of Health.

21. Katja Reuter: katja.reuter@usc.edu
Twitter: @dmsci
Cecilia M. Patino-Sutton: patinosu@med.usc.edu
Twitter: @@ceciliapatinoMD
Contact Us
...if you’re interested in collaborating on
the project.
Explore the Digital Scholar Program page
and its resources online:
https://sc-ctsi.org/training-education/digital-
scholar-program

22. References
1. Ayers, E. (2004). Doing Scholarship on the Web: Ten Years of Triumphs —
and a Disappointment. Journal of Scholarly Publishing, 35(3), 143–147.
http://doi.org/10.3138/jsp.35.3.143.
2. Andersen, D., & Trinkle, D. (2004). Valuing digital scholarship in the
tenure, promotion, and review process - A survey of academic historians.
In D. Andersen (Ed.), Digital Scholarship in the Tenure, Promotion and
Review Process (pp. 61–77). New York: M.E. Sharpe.
3. Rumsey, A. S. (2011). New-Model Scholarly Communication: Road Map for
Change. Web link: http://uvasci.org/institutes-2003-2011/SCI-9-Road-Map-
for-Change.pdf.
4. CTSA consortium. Core Competencies for Clinical and Translational
Research. 2015. Web link: https://clic-ctsa.org/resources/education/core-
competencies.
5. Ritchie, J & Spencer, L (1994) Qualitative data analysis for applied policy
research Analysing Qualitative Data 173 – 194 Bryman A Burgess RG
London Routledge.