Alzheimer's disease (AD) is recognized as a public health crisis worldwide (IADRP, 2013). AD is a complex neurodegenerative disease and the leading cause of dementia among the elderly people (Evans et al., 1989). Currently, there are approximately five million AD cases in the United States and about 35 million cases worldwide (Alzheimer's Disease International, 2009. The focus of this case study is on the Uniform Data Set (UDS), a longitudinal database on Alzheimer’s patients, and the 29 Alzheimer’s Disease Centers (ADC) that submit their data to the UDS and actively collaborate in the ongoing maintenance, development and research utilization of the database. The ADCs are based in major medical institutions across the United States. They have a multi-decade track record of collaborative research and a networked and virtual approach to the scientific study of AD. The central coordinating mechanism for the ADCs and the UDS is the National Alzheimer’s Coordinating Center (NACC), which is located at the University of Washington. The NACC coordinates data collection and supports collaborative research among the ADCs.

1. The Alzheimer’s Disease Research Network and the Uniform Data Set1
Douglas R. Austrom2, Ph.D., Betty Barrett3, Ph.D., Elizabeth Merck4, Pamela A. Posey4, D.B.A., Bert
Painter5, Ramkrishnan V. Tenkasi6, Ph.D.
INTRODUCTION
Alzheimer's disease (AD) is recognized as a public health crisis worldwide (IADRP, 2013). AD is a
complex neurodegenerative disease and the leading cause of dementia among the elderly
people (Evans et al., 1989). Currently, there are approximately five million AD cases in the
United States and about 35 million cases worldwide (Alzheimer's Disease International, 2009.
The focus of this case study is on the Uniform Data Set (UDS), a longitudinal database on
Alzheimer’s patients, and the 29 Alzheimer’s Disease Centers (ADC) that submit their data to
the UDS and actively collaborate in the ongoing maintenance, development and research
utilization of the database. The ADCs are based in major medical institutions across the United
States. They have a multi-decade track record of collaborative research and a networked and
virtual approach to the scientific study of AD. The central coordinating mechanism for the ADCs
and the UDS is the National Alzheimer’s Coordinating Center (NACC), which is located at the
University of Washington. The NACC coordinates data collection and supports collaborative
research among the ADCs.
Interviews were conducted with individuals from the NACC, the NIA, and the ADCs about the
virtual work processes, the challenges facing participants in thes e virtual environments, and
about the coordination mechanisms used to resolve those problems in order to co-create
knowledge. These data were then analyzed to determine what coordinating mechanisms were
used to deal with and overcome the barriers to knowledge development and dissemination
that exist in virtual organizations.
This research site and project is one of three research sites included in our study of virtual R&D.
We used an exploratory case study methodology and a sociotechnical systems analysis
framework to assess the factors that contribute to the effectiveness virtual R&D projects. This
case study focused on two main research questions:
1. What distinguishes effective deliberations and forums from those that are ineffective in
virtual R&D projects?
2. What are the major variances or knowledge barriers in virtual R&D projects?
1 Thi s material is based upon work supported by the National Science Foundation under grant number NSF OCI 09-43237. Any
opinions, findings and conclusions or recommendations in th is material are those of the author(s) and do not necessarily
reflect the views of the National Science Foundation.
2 Indiana University
3 Mas sachusetts Institute of Technology
4 Sociotechnical Systems Roundtable
5 Modern Times Productions
6 Benedictine University

2. SCAN OF THE ALZHEIMER’S DISEASE RESEARCH NETWORK
National Institute of Aging and the Alzheimer’s Centers
In 1984, leaders at the National Institute of Aging (NIA) recognized that Alzheimer’s Disease and
related dementias were catastrophic diseases (Butler, 2005) and decided to create a national,
interdisciplinary research program focused on the causes and the course of Alzheimer’s disease.
With remarkable foresight, they believed that a network of centers “could create the necessary
infrastructure to promote longitudinal clinical-pathological studies; integrate basic and clinical
research; standardize clinical assessment tools, methods, and clinical trials; and establish
national data banks to share resources for clinical, neuropathological, and genetic studies”
(National Institute of Aging, 2005).
The NIA originally funded five Alzheimer’s Disease Research Centers (ADRC) in 1984 and five
additional centers in 1985. In 1986 the centers collaborated to establish the Alzheimer’s
Disease Patient Registries (ADPR) and the Consortium to Establish a Registry for Alzheimer’s
Disease (CERAD). CERAD standardized the definition, assessment, and profile of AD and the
criteria for the diagnosis of AD that are still widely used today.
The scientists and clinicians at these 10 ADRCs provided the foundation of a network that today
reaches across the country and includes ADCs at 29 major medical institutions (see figure 1 for
the geographic distribution of these centers).
Figure 1. Geographic Distribution of the 29 NIA-Funded Alzheimer’s Disease Centers
The ADCs conduct basic research on the mechanisms of AD and translational research to
improve diagnosis of the disease and improve care for AD patients. In addition, they support

3. the patients and their families with the actual diagnosis, medical management of the disease,
and information about the disease and available services and resources.
The ADCs are a multidisciplinary mix of clinicians, researchers, and administrators; for example,
neuropathologists, neurologists, neuropsychologists, geriatricians, geriatric psychiatrists,
radiology and imaging technicians, nurse practitioners, research technicians, social workers,
clinical psychologists, epidemiologists, biostatisticians, data base managers, and other
computer specialists. While the ADCs are required as a condition of funding to have five
common cores -- Clinical, Neuropathology, Data, Education, and Administration -- this does not
preclude them from establishing other core clinical or research areas such as neuroimaging and
genetics.
National Centers and Initiatives
In addition to the ADCs, the AD research network includes several NIA-funded centers: the
National Cell Repository for Alzheimer’s Disease located at Indiana University, the Alzheimer's
Disease Education and Referral Center in Bethesda, Maryland, coordinates and the National
Alzheimer’s Coordinating Center located at the University of Washington. The history of these
centers is intertwined with that of the ADCs.
In 1989, the NIA created the National Cell Repository for Alzheimer’s Disease (NCRAD), a
genetics cell repository. NCRAD banks DNA and cells, builds a database of family histories and
medical records, and provides researchers with cell lines and DNA samples from normal control
subjects and people with well-documented AD.
The U.S. Congress created the Alzheimer's Disease Education and Referral (ADEAR) Center in
1990 to compile and disseminate “current, comprehensive, unbiased” information concerning
AD for health professionals, people with AD and their families, and the public (NIA, 2013). In
addition to providing answers to specific questions about AD, free publications about the
disease -- diagnosis, related disorders, risk factors, treatment, caregiving tips, home safety
tips—and training materials, the ADEAR Center can provide information on ongoing clinical
trials and referrals to the ADCs and other local services.
Collaboration on multi-site clinical trials for the cholinesterase inhibitor, tacrine, led to the
creation of the Alzheimer’s Disease Cooperative Study (ADCS) in 1991. The ADCS coordinates a
consortium of ADCs and affiliated organizations that conduct large clinical trials on promising
compounds designed to improve cognitive functioning, slow the rate of decline, delay the onset
of the disease.
In 1997, the NIA established the Minimum Data Set (MDC), which was originally housed at Rush
Presbyterian Hospital in Chicago to compile data on patients and control subjects enrolled in the
ADCs. The MDS included brief clinical and demographic information on over 74,000 patients and
control subjects enrolled in the ADCs. While still valuable, the data were assembled
retrospectively and were primarily cross-sectional rather than longitudinal and minimal by
design. Also, the ADCs did not follow a standardized protocol when these data were originally

4. collected, even though the data were reported to NACC in defined data elements and the missing
data rate was as high as 30%.
National Alzheimer’s Coordinating Center (NACC) and the Uniform Data Set (UDS)
Key individuals at the NIA and in the larger AD research community recognized the need for a
common database with longitudinal data. They believed that this could be best accomplished
by creating a coordinating center for the ADCs. National Alzheimer’s Coordinating Center
(NACC) was established at the University of Washington by the NIA in 1999 to facilitate
collaborative research among the ADCs (NACC website, 2010).
The MDS was transferred to the NACC and the first task of the center was to reduce the error
rate in the MDS to less than two percent. In 2002, the MDS was expanded to include
neuropathological data from autopsies. In collaboration with neuropathology researchers, a
new data form was put together that linked clinical diagnoses and pathology findings.
In the early 2000’s, the NIA created an External Advisory Committee to make recommendations
on the future of ADC program. They recommended expanding and standardizing clinical data
collected at the ADCs and stored at the NACC. The NIA formed an ADC Clinical Task Force to
plan this expansion, define an expanded data set, and standardize clinical evaluation for all
ADCs. The ADC Clinical Task Force, working closely with the NACC, met regularly for three years
to define the content of the new Uniform Data Set (UDS), develop common data collection
templates, and construct a relational database. The ADCs agreed upon the UDS in early 2005.
The NACC piloted the standardized data collection templates in the summer of 2005 and the
common protocols were formally implemented that September.
The ADCs enroll subjects in various ways: referrals from clinicians, self-referrals by patients
themselves or concerned family members, active recruitment through community
organizations, and volunteers who wish to serve as healthy controls. Subjects visit the ADCs
annually and are assessed by clinicians, neuro-psychologists, and other ADC research personnel.
As many as 18 standardized forms are used and 725 data points are collected at each visit
including socio-demographics, family history, dementia history, neurological exam findings,
functional status, neuropsychological test results, clinical diagnosis, and imaging tests.
ADC research assistants enter the data from these standardized forms in the ADC’s local
database. Data managers at the ADCs monitor the quality of the local data before submitting it
electronically to the NACC each month. Data base managers at the NACC also conduct data
checks on the data they receive from the Centers before adding it to the UDS. The NACC makes
payments of $29,000 per year to the ADCs to cover the costs associated with submitting their
data.
A primary goal of the UDS is to accumulate a large-scale pool of data which all qualified
scientists can use to discover better diagnostic criteria and treatment options for patients
suffering from AD and related neurodegenerative diseases. Table 1 provides a description of
the data available from the NACC As of June 2013, there are 28,444 subjects in the UDS

5. longitudinal database, 74,397 subjects from the original Minimum Data Set, and 13,279
subjects from the Neuropatholgy Data Set.
Table 1. Description of the Data Available from the NACC as of June 20137
Minimum Data Set
(MDS)
Uniform Data Set (UDS)
(LONGITUDINAL)
Neuropathology
Data Set (NP)
Years covered 1984 - 2005 Sept. 2005 - present 1984 - present
Study subjects
Enrollees followed at ADCs
(with & without dementia)
Enrollees followed at ADCs
(with or without dementia)
Subjects who died
and underwent
autopsy
Approx. # of
subjects*
74,397 28,444 13,279
Approx. # of
variables
67 725 85
Method of
data collection
Mainly abstracted
retroactively from ADC
medical records
Collected prospectively by
clinicians, neuro-psychologists,
and other
ADC research personnel,
using up to 18
standardized forms at each
visit.
Standardized
neuropathology
form, completed by
neuropathologist
Time period
covered for
each subject
Mainly status on last ADC
visit; some variables also
capture initial-visit status
Initial visit and each annual
follow-up visit, plus
milestones such as
death or dropout
Status of brain at
autopsy
Topics covered
Demographics, cognitive
status, clinical dementia
diagnosis, selected clinical
manifestations, comorbid
conditions, MMSE score,
vital status, primary
neuropathological
diagnosis (if died and had
brain autopsy)
Socio-demographics on
subject and informant,
family history, dementia
history, neurological exam
findings, functional status,
neuropsychological test
results, clinical diagnosis,
whether imaging testing
done, ApoE genotype
Demographics, date
of death, primary and
secondary
neuropathological
diagnoses,
presence/absence of
neuropathological
features of most
major dementias,
APOE genotype, brain
weights
The NACC also has limited funds to support collaborative research projects that involve at least
three ADCs as well as research projects proposed by junior investigators that use the UDS or
Neuropathology data.
7 NACC, 2013, https://www.alz.washington.edu/WEB/data-descript.html

6. Any researcher can request access to the NACC’s database for basic and translational research.
The requests are reviewed by a methods group to make sure that they are legitimate. NAC’s
staff will also format data if needed, and occasionally give analytic support. The NACC received
more than 700 requests in 2010. Since its inception, the NACC has been instrumental in 590
publications as of June 2013:8 191 by external researchers using NACC data, 100 by NACC
personnel, 173 from NACC funded projects, 34 based on NACC-funded secondary analysis, 72
from NACC–ADC collaboration in external studies, and 20 that relied on indirect NACC support.
The UDS project and the NACC have received some acclaim in the popular press as an exemplar
of research collaboration (Kolata, 2010).
Figure 2. Structure of the ADC Committees
While the central role of the NACC is enabling coordination and collaboration among the 29
Alzheimer’s centers through the ongoing maintenance, development and research utilization of
the UDS, the NACC also organizes the face-to-face (F2F) meetings of the ADC core directors and
ADC steering committees. These meetings are held twice a year in conjunction with the annual
conferences of American Neurological Association and the American Academy of Neurology. The
structure of the ADC committees is shown in Figure 2. The members of the executive committee
and the steering committees are elected. The NACC Director and the NIA’s Program Director are
ex-officio members of the Date Core steering committee. The Clinical Task Force is comprised of
five members who are elected by the members of the Clinical Core Steering Committee, six
members who are appointed by the NIA and NACC, and three members who serve as liaison to
other programs.
Major Research Initiatives Involving the ADCs
This collaborative infrastructure has provided the foundation for several major research
programs that either directly or indirectly involve the ADC’s, the UDS, and the NACC.
8 NACC June 2013. https://www.alz.washington.edu/cgi -
bin/broker64?_service=naccnew9&_program=naccwww.pubrep1.sas&TYPEF=DISPLAYIDS

7. The Alzheimer’s Disease Genetics Consortium (ADGC) Genome Wide Association Study
(GWAS) was formed in 2003 to collaboratively use the collective resources of the AD research
community to identify variability in genes that influence susceptibility to AD. The ADGC is a
partnership between the NACC, NCRAD, the ADCs, and the Children's Hospital of Philadelphia
(CHOP). NACC identifies the subjects from each ADC who are eligible for the GWAS, tracks the
samples that have been sent to NCRAD for DNA banking, and reimburses the ADCs for their
costs. NCRAD sends DNA for genotyping to CHOP, which sends the genotype data to the ADGC.
Finally, NACC provides the ADGC with the HIPAA research-limited phenotypic data, which it
pairs with the genotype data.
Because AD is such a complex disease with significant phenotypic heterogeneity, it is a greater
challenge for researchers to identify the genetic variants associated with AD necessitating a
combination of larger studies and novel approaches such as GWAS. (University of Pennsylvania
School of Medicine, 2010).
The Alzheimer’s Disease Neuro-Imaging Initiative (ADNI) was launched the following year with
approximately 50 research sites in the United States and Canada. It is a public-private
partnership with the NIA and the National Institute of Bioimaging and Bioengineering, 22
companies, and two Foundations including the Foundation for the National Institutes of Health
(FNIH), which administers non-government sources of funds such as the Alzheimer's
Association. ADNI data (images, cerebrospinal fluid, and blood samples) are stored
anonymously in a central repository and all qualified researchers can access the electronically
stored images and test results as soon as data are available.
The success of these collaborative multi-center research programs in the United States has
provided a model and infrastructure that has been scaled internationally. To that end, North
American ADNI (NA-ADNI) served as a founding member of World Wide ADNI (WW-ADNI), a
collaborative effort of scientists from around the world. It is now the umbrella organization for
neuroimaging initiatives being conducted by NA-ADNI, European ADNI (2005), Japan ADNI
(2007), Australian ADNI (2006), Taiwan ADNI (2011), Korea ADNI, China ADNI (2011), and
Argentina ADNI (2012). Brazil ADNI (2014) is a future site. Since WW-ADNI was beyond the
funding mandate of the NIA, the Alzheimer’s Association, a major charitable organization
dedicated to addressing AD, provided some of the initial funding for this global initiative.
In line with NA-ADNI, the goals of WW-ADNI are to better understand the progression of mild
cognitive impairment and Alzheimer's disease, to standardize data collection methods so that
data from all sites can be combined and provide a worldwide picture this disease. WW-ADNI is
unique because the clinical, neuropsychological, imaging, and biological data are all available to
the scientific community at no cost so that scientists around the globe can use this information
for their own research purposes.
The Dominantly Inherited Alzheimer Network (DIAN) is another international partnership with
10 research institutions in the United States, United Kingdom, and Australia researching a rare
form of AD caused by a gene mutation. DIAN was established in 2008 with a grant from the NIA.

8. It has received additional funding from foundations, charitable donations, and the German
Center for Neurodegenerative Diseases. The DIAN coordinating center is located at Washington
University in St. Louis, which is also an ADC site.
Figure 3 provides a high level map of the AD research network and the tightly coupled
relationships between the ADCs, the centers and projects, and these collaborative research
programs.
Figure 3. Mapping the Alzheimer’s Disease Research Network
Our focus, based on our initial scan and mapping of the AD research network/system, is the
Uniform Data Set (UDS) project. It has played a central role in the emergence of this far-reaching
collaborative research enterprise yet is sufficiently bounded so that we could
meaningfully gather data on the key deliberations, both virtual and face-to-face, involved in
creating it, implementing it, revising it, and ensuring timely and accurate data submissions from
the ADC’s. It also involved a high degree of virtuality, multi-disciplinary activity, and varying
levels of complexity and uncertainty in the work to be done and the processes required to
complete the work.

9. These initial interviews also helped us confirm where to locate the UDS project on the six-stage
R&D continuum9 that we have been using in our research on virtual R&D projects. As shown in
Figure 4, the six stages are demarcated by the degree of task uncertainty and complexity.
Figure 4: R&D Continuum10 with Location of Case Study Projects
Based on
R
1
R
2
D
1
D
4
D
2
D
3
the initial scan and the results of the interviews, we concluded that creating, implementing, and
subsequently revising the UDS involved a degree of uncertainty characteristic of the initial
development stages, or D1 and D2 on the R&D continuum. Once implemented, the activities
involved in maintaining the UDS and ensuring timely submissions of accurate data were more
routinized and akin to the start-up and scale-up stages (D3 and D4) on the R&D continuum.
We used the differences in the degree of complexity to sort the major deliberations that the
interviewees identified as either the periodic deliberations involved in designing, implementing,
9 R&D has been characterized as an intrinsic learning system (Purser et al., 1992) with multiple stages. Each stage
is defined by the degree to which participants do or do not know the “what” (objective) or the “how” (method
or means) of their knowledge development and synthesizing activities. These stages form a developmental
continuum that ranges from projects with high uncertainty in which participants don’t know what is the final or
real objective in concrete terms and don’t know how to operationalize it – to projects with low uncertainty in
which participants know what they need to achieve and also know how to achieve it operationally.
10 Carolyn Ordowich (personal communication, March 26, 2009) adapted the R&D continuum from a research
portfolio model originally developed at Bell Laboratories (Mashey, 2008; Revkin, 2008).

10. and revising the UDS or the ongoing deliberations needed to ensure the quality and timeliness
of the data submitted to the UDS.
Major Periodic Deliberations to Design, Implement, and Revise the UDS
The NACC’s mission is to encourage and support more effective collaboration among ADCs
across the United States. The success of the UDS has been a critical contributor to achieving
that mission, both symbolically and practically.
The UDS is the result of innumerable deliberations that have occurred since 1992, and probably
many years before the planning of the UDS was formally initiated. As noted above, the process of
designing and then launching the UDS took three years and required considerable effort on the
part of numerous individuals from the ADCs, the NACC, and the NIA. This work was entrusted to
the Clinical Task Force and various sub-committees comprised of subject matter experts on the
main elements of the UDS.
The key deliberations, participants, values divergence and knowledge barriers, and the
coordinating mechanisms involved in designing, implementing, and revising/upgrading the UDS are
summarized in Table 2.
While the NACC and UDS have facilitated greater collaboration in the field of AD research, the task
was fraught with challenges. Some of the ADC scientists disagree with the fundamental premise
that standardized data collection and a uniform data base approaches is a more effective and
productive way to conduct research on AD. It is not clear whether this constitutes a knowledge
barrier due to lack of common frame of reference or due to divergent values .
One person we interviewed observed that the UDS approach violated what he understood about
the scientific method; namely that hypotheses and research questions should determine the
measures and data to be collected, and not the other way around. Another interviewee described
the disagreement this way: The trade off is the perceived value of data from tens of thousands of
people collected uniformly versus the value of 29 centers pursuing individual and local and maybe
smaller consortium expertise (Personal interview, 2011).
It is understandable that the scientists at the ADCs placed a high value on autonomy and the
discretion to conduct research on AD as they deemed most appropriate. As some of the most
eminent scientists in the field of AD researchers – demonstrated, in fact, by their ability to secure
NIA funding for their ADCs -- they undoubtedly believe that they deserve this latitude.
There is also the question of the cost of the UDS and the NACC, especially when research funds are
limited. Creating an integrated system that allows collection, distribution, and shared use of the
data is also expensive, and not everyone believes that it is the best use of limited research
funds. One center director that we interviewed commented on the fact that more than one grant
was necessary to create and maintain the UDS.

11. Table 2. Major Periodic Deliberations to Design and Revise the UDS
Topics and Issues
Participants
Values Divergence/
Knowledge Barriers
Coordinating
Mechanisms
Fundamental premise of
the UDS
ADC Directors and
Scientists, NIA, NACC PI
Autonomy and research
independence versus
standardized methods
and measures
Lack of shared frame of
reference
Transcendent purpose,
Threat of ADC not being
renewed
Latitude to pursue
personal research
agenda as well as UDS
Design, build support,
and agree for the key
elements of the UDS:
 Standardized clinical
evaluation protocol for
assessing and
diagnosing AD and
other dementias
 Common clinical data
points
 Common data
collection forms
 Relational database
Clinical Task Force,
discipline-specific sub-committees
that solicit
input and feedback from
ADC researchers, NIA
PM, and NACC PI
Lack of a common frame
of reference (competing
theories on the
mechanisms of AD)
Failure to use knowledge
from all disciplines
Multiple opportunities to
provide input and
influence decisions
Transcendent purpose,
Credible and influential
network builders and
leaders
ADC steering
committees
NACC
NIA
Revise UDS to replace
proprietary instruments
with non-proprietary
instruments; anticipate
and incorporate
scientific breakthroughs
Clinical Task Force plus
discipline-specific sub-committees
that solicit
input and feedback from
Center staff
Sunk costs in
longitudinal data already
collected versus rising
licensing costs and lost
data due to inability to
extend licenses to non-
ADCs
Failure to use knowledge
from all disciplines
Multiple opportunities to
provide input and
influence decisions;
Transcendent purpose;
Credible and influential
network builders, peer
leaders; ADC steering
committees; NACC; NIA
These deliberations occurred in informal discussions with peers at their own ADC and other
ADCs and with individuals from the NACC and the NIA and in more formal settings such as the
semi-annual directors meetings as well as numerous scientific meetings. The forums occurred
both face-to-face (F2F) and virtually. There were three factors that allowed a workable
resolution of the divergent values in this deliberation; first, the compelling mission that the AD
researchers share, the cure and eradication of this devastating disease; second, the latitude to
pursue an ADC-specific research agenda along with the common agenda of the UDS; and third
the financial incentives of continued funding for the ADC
Countless deliberations were also required to design, build support, and agree on the key
elements of the UDS which included a standardized clinical evaluation protocol for assessing
and diagnosing AD and other dementias, a common set of clinical data points, common data
collection forms, and a relational database for the UDS.

12. This was arguably the most challenging set of deliberations. Reaching agreement on the value
of common standards in most decentralized organizations is conceptually a straightforward
task. But in practice, people are only supportive of common standards and common
approaches if the chosen standards resemble their current practices. The scientists and ADCs
had all invested considerable, time, money, and professional credibility to conduct the clinical
evaluations and gather the data in ways that best supported their research agenda and theories
on the mechanisms of AD. Designing, building support, and reaching agreement on which
clinical evaluation protocol to use, which clinical data points to collect, how to configure the
data collection forms, and how to construct the optimal relational data base to satisfy ADCs
required an iterative process of building commitment to, or at least, compliance with the UDS.
The Clinical Task Force surveyed the ADCs to find out what protocols, data points, and
measures they were using at that time. They sought people’s input and feedback throughout
the process. In this set of deliberations, the knowledge barriers were due to divergent values,
lack of common frame of reference, the failure to use the full range of knowledge resident in
the ADC network. For example, some of the disciplines such as data managers did not believe
that their input was fully utilized in the final design of the UDS.
The UDS is evolving, and these changes are constrained by the choices made at the outset. The
early decisions shaped the current form of the data set, and the well-established longitudinal
database made it difficult to make alterations to the UDS. But several of the instruments used
in the UDS are proprietary and their use is governed by agreements that have become quite
restrictive; in particular, the license fees for these instruments have increased dramatically.
Here is how one interviewee described this challenge (Personal interview, 2011):
Almost all sites would use a Mini Mental State at some in their characterization of
patients. Well, it turns out that many of these instruments that we use, particularly the
neuropsychological instruments, have been licensed by commercial firms. So we had to enter
into licensing agreements. Actually NACC, the National Alzheimer’s Coordinating Center, is the
one that developed these licensing agreements. And the companies varied in their willingness
to do this. But some of them were really very restrictive. They’d had a bad experience with one
university or another, and they didn’t want them to be part of our licensing agreement. But in
the end of it all, it is that we do have blanket licensing agreements for all Alzheimer’s Disease
Centers to use the uniform data set in ADC participants. If you’re not in one of the ADCs, the
licensing agreement doesn’t extend to you; you would have to go to the individual companies
and develop your own licensing agreement with each company that has an instrument in the
UDS.
It’s a tremendous problem. It turns out that the licensing fees keep going up each year. It’s
absurd to us – I’m sorry, but this is a real problem for us. It’s absurd for us because many – I
think I mentioned the first centers began over 25 years ago, and we tried to adhere to using
the same instruments that they’ve been using for 25 years, so we didn’t ask to throw all of
their longitudinal data sets out of the window. So we’re using in many instances, forms or
versions of these cognitive tests that have been subsequently revised. So the forms we use are
no longer being manufactured, except for us, and the companies – we’re the only users and
they keep raising the rates on us, even though they’re selling other versions, newer versions of

13. these forms. So it’s been a real problem. Many people have asked for access to th e UDS; we’re
able to give them the parts, the components, that are not licensed, but most of the cognitive
tests are licensed, so it’s a real, real problem. And talk about evolution, we have a current task
force, primarily made of neuropsychologists, that is trying to develop a new
neuropsychological battery for the UDS that would be comprised of tests that are unlicensed,
so that we could then disseminate the entire UDS without these restrictive licensing
agreements.
One researcher remarked that standardization is also holding back change. The concern
raised is that changing the items or definitions of elements in the items can damage long-term
studies by creating uncertainty about the meaning of the data. Another concern raised was the
cost of change in terms of the reorganization of the data set, the forms used to gather the data,
and staff needed to make the changes.
The Clinical Task Force and subject area sub-committees conducted a lengthy set of
deliberations to develop and pilot a non-proprietary battery of instruments. Most of these
deliberations were conducted virtually using email, teleconferences, and occasionally
videoconferences. They also met in person at the Directors meetings. They experienced a
similar knowledge barrier revising the UDS as they did developing the original UDS; namely, the
failure to use the knowledge and expertise of all of the disciplines involved in the ADCs. For
example, one interviewee commented that they did not believe that any data managers from
the ADCs were included in the sub-committees.
But the need to anticipate scientific developments and modify the UDS accordingly is an
ongoing challenge. In a field as vitally important as AD research, there is unrelenting pressure
to stay on the forefront of scientific discoveries and technological advances. As one
interviewee stated (Personal interview, 2011),
. . . it’s a hard row to hoe because a lot of that literature is in computing journals and
engineering journals and linguistic journals, and it’s hard to get the mainstream
biomedical community to sort of turn the battleship around. It’s an interesting time
we live in. But I think one of the amazing transformations that’s going to happen is
that hopefully with the emphasis on the electronic media we will begin to realize that
there’s a tremendous power in aggregating data electronically, but actually the
electronic capture of data doesn’t just have to occur in a clinic. It can actually occur
anywhere at any time.
These anticipatory deliberations can conceivably involve all member of the AD Research
network; especially the formal and informal network leaders. These deliberations have
occurred and will continue to occur both within the formal structure and the informal influence
network.

14. Major Ongoing Deliberations to Maintain the UDS
The major ongoing deliberations required to maintain the UDS and ensure data quality and
timely submissions to the UDS are summarized in Table 3.
Implementing the original UDS as well as revised versions of the UDS involved deliberations to
educate ADC personnel regarding conducting standardized clinical evaluations, collecting the
common clinical data points, using the common data collection forms, and inputting the data
into the UDS’s relational database. The Clinical Task Force and the NACC developed detailed
user manuals for these activities which they made available to the ADCs on their website.
While most of these deliberations were conducted virtually, there were discussions regarding
implementation and revisions with the relevant Core Directors at the semi -annual Director’s
Meetings. The NACC also created an online bulletin board that the ADCs could use to submit
questions and get answers on data collection issues. In the early days of the UDS
implementation, there was considerable traffic on the bulletin board about how to deal with
various scenarios. If necessary, the NACC forwarded the question to the Clinical Task Force for
a decision.
Because they have been collecting UDS data for several years, the bulletin board is relatively
quiet; that is, until the NACC introduces minor modifications to the UDS. The number of
questions posted on the online bulletin board is contingent on the complexity of the
modifications. We were informed in our interviews that the decisions to make minor
modifications to the UDS are made centrally by the NACC and the Clinical Task Force and then
communicated by group emails to the ADCs. However, the group emails may not be sent to all
parties impacted by the decision. For example, if it is a clinical issue, data core staff may not be
copied on the changes. This may lead to downstream problems when the data manager
attempts to enter the modified data formats.
Attempting to implement and maintain a uniform data set across 29 geographically dispersed,
stand-alone ADCs is challenging. More specifically, while the Clinical Task Force and the NACC
standardized the outputs with the common data forms, they have not standardized the local
processes. The lack of standardization in how different types of expertise are integrated and
deployed across the ADCs was an issue that surfaced during our interviews.
Each ADC has its own unique clinical processes for assessing and diagnosing AD and collecting
and managing the data. The data is collected and analyzed at the ADCs by multidisciplinary
groups of people: clinicians, neurologists, statisticians, data managers, neuropsychologists,
nurses and research staff. The Centers have differing professional categories and
responsibilities for their staff. For example, some centers include the statisticians and data
managers in the team and others view them as support staff only leading to varying degrees of
involvement in deciding how best to code the items for the database. One statistician described
their role this way: “I’m the outsider looking in, I guess.” By comparison, the data manager at
another ADC meets with the clinical staff twice a week to review patient charts from which the
data to be submitted to the UDS is generated.

15. Table 3. Major Ongoing Deliberations to Maintain the UDS
Topics and Issues
Participants
Forums and Degree
of Virtuality
Values Divergence/
Knowledge Barriers
Coordinating
Mechanisms
Implement UDS and
major revisions to
the UDS
NACC staff, Clinical
Task Force, ADC
clinical core and
data core staff
Online users
manuals and
bulletin board for
questions, emails,
and F2F updates at
Directors meetings
Lack of knowledge
due to new or
updated standards
Well-defined
procedures, online
manuals, F2F
updates and online
Implement minor
modifications to the
UDS
NACC staff, Clinical
Task Force, ADC
clinical core and
data core staff
Emails and online
bulletin board
Failure to share
knowledge with all
relevant parties
NACC specifies
requirements
Consistency of
diagnosis, accuracy
and completeness of
data forms
ADC clinical core
staff and data core
staff
Email, F2F, hard
copies of the data
forms
Lack of common
frame of reference
due to diversity of
disciplines and ADC-specific
processes
NACC provides
specific
requirements
Quality and
timeliness of data
submitted to ADC by
ADCs
ADC staff (data
mgmt. staff, clinical
staff, Core and
Center Directors),
NACC staff and PI,
and NIA PM
Primarily virtual
(EDI, monthly
reports, emails, and
telephone if an issue
needs to be
escalated
Familiarity of staff
with data forms;
Failure to share
data/knowledge in
timely fashion due
to differing
priorities
NACC uses error
tracking
procedures;
ADC’s commitment
to data integrity
While decisions about team inclusion are made in each center, the NACC and UDS might benefit
from some analysis of the tasks that are vital to ensuring consistency of the data set. In this
situation, statisticians and data managers make decisions about the definitions that are core to
interpretation of the data.
Also, the degree of co-location differs from center to center. At some ADCs, staff members are
co-located while at other ADCs they are in separate departments that are housed in different
buildings and located on other areas of the campus. The degree of geographic dispersion
affected both the amount of interaction and the modes of interaction.
These factors impact the reliability of the UDS. To mitigate these risks to the UDS, both the
ADCs and the NACC have evolved a process that they follow each month to ensure data quality.
The data core staff at the ADCs conduct quality checks on their data before they submit it via
electronic data interchange to the NACC each month. Once they receive the data from the
ADC, NACC’s data managers and programmers conduct a standardized set of quality checks and
then distribute a report to the Center Directors, Clinical Core Directors, and the Data Core
Director. These reports will typically be shared with other Center staff such as the data core
manager, clinical core leader, scheduling assistants, and so on. If the NACC discovers any data
discrepancies, they are listed in these reports to the ADCs. ADC staff will compare the data

16. they submitted with what the NACC reports and then email questions to the programmers and
data managers at NACC.
The NACC reports also include questions regarding what they consider to be inconsistencies in
the data. One example involved how a Center’s clinicians classified subjects as depres sed.
When asked if a subject exhibited depressive symptoms, the clinicians said yes. When asked in
a later question if the patient was depressed, the clinicians indicated that s/he was not. The
data managers asked the clinicians for clarification and were told that the subjects had signs of
depression, but they were not comfortable diagnosing the subject as clinically depressed
because they did not know if the person had a history of clinically diagnosed depression.
Here is how one of the interviewees described another virtual deliberation that their ADC had
with the NACC (Personal interview, 2011): “We have been back and forth with them (NACC) on
how they clarify follow-up rates and how they count who is being followed in what time
window. The time window is where we really, really struggled with how they ended up with
different rates reported from how we thought they should be. But ultimately their decision is
the one we have to follow.”
The issue of follow-up rates was particularly relevant because the ADCs must report a specific
number of patients enrolled in their Center’s data set each month. Failure to do so signals
potential problems and is an important consideration in future funding. There were probably
several reasons that an ADC might be tardy in subject follow-up or timeliness of their data
submission: the disagreement on the calculation of follow up rates, the amount of effort
required to be fully up-to-date, and conflicting priorities for how they wanted to spend their
time. The original deliberations occurred initially with email requests from the NACC to the
ADCs. If the ADCs failed to comply or were slow to comply, the deliberations escalated to
telephone calls from the NACC’s Director to the Center’s Director. If this did not produce the
desired results, it would be followed by telephone calls from the NIA’s Program Manager.
Given the threat of losing center funding, subject follow-up and timeliness of data submissions
are less problematic now than when the UDS was originally implemented.
DISCUSSION AND CONCLUSIONS
This case provides illustrative insights into the complexity of deliberations involved in virtual R&D
projects. It is generally acknowledged that the UDS, and by extension the deliberations involved in
creating it, implementing it, maintaining it and revising it, have been successful. An August 2010,
New York Times article reveals that while scientists may have had initial concerns about such a
collaborative effort, they have started to recognize the benefits. One researcher quoted in the
article noted “it’s not science the way most of us have practiced it in our careers. But we all
realized that we would never get biomarkers unless all of us parked our egos and intellectual
property noses outside the door and agreed that all of our data would be public
immediately.” This attitude has prevailed and the Alzheimer’s project has become a model for
other disease states such as Parkinson’s disease.

17. Similarly, the following observation from our interviews attests to the success of the UDS
(Personal interviews, 2011): “I think the metric of success is the extent to which the data we’re
collecting is getting used and advancing scientific productivity. We look at how many different
proposals have come forward to use the data and how many publications have come out that
have used the data inside of the grant. I think that’s the main reason we’re doing this. It is not
just collecting data for collecting data sake, but actually to answer the questions. The first
couple of years were just about getting the data collected in a format that would be useable,
and now we’re beginning to see the fruits of that labor, if you will.”
Coordinating Mechanisms
The effectiveness of the UDS deliberations is due in part to the coordinating mechanisms that
provided the context for the key stakeholders to make the important trade-offs on a reasoned
and ongoing basis. The theory of organizational information processing and mutual meaning
making (Galbraith, 1974; Boland and Tenkasi, 1995; Malhotra & Majchrzak, 2004; Weick, 1995,
1979; Daft & Lengel, 1986) suggests that the structural mechanisms for coordination must
provide the means to address the level of task uncertainty (what) and the complexity of
achieving the task (how) in order to effectively co-create knowledge (Nonaka, 1994; Boland and
Tenkasi, 1995). Sabherwal (2003) condensed several typologies of coordinating mechanisms
into a continuum of four main types: informal mutual adjustment, formal mutual adjustment,
plans, and standards. Figure 5 superimposes these four coordinating mechanisms onto the
R&D continuum in light of the levels of task complexity and uncertainty.
The key stakeholders in the AD research network generally knew what to do to create,
implement, and subsequently revise the UDS. How to do it, especially how to build support for
the UDS given the divergent values and priorities of the 29 geographically dispersed ADCs, was
more challenging and complicated (stages D1 and D2 on the R&D continuum). The coordinating
mechanisms for these deliberations relied heavily on formal mutual adjustment and to a lesser,
but no less important, degree on informal mutual agreement. The key structural elements of
the formal mutual adjustment included the national coordinating structure (NACC), the ADC
steering committees, the semi-annual Directors meetings, the Clinical Task Force and its sub-committees,
the AD center model, and the NIA.
Arguably the most powerful coordinating mechanism has been the compelling mission that is
shared by the entire AD research community: eradicating Alzheimer’s disease. This
underscores both the informal and informal coordinating mechanisms and is the basis for
coordination of efforts requiring shared goals and shared plans.

18. Figure 5. Coordinating Mechanisms Across the R&D Continuum
The NACC has played a particularly important structural role in the formal mutual adjustment
involved in creating the UDS. The NACC has provided the requisite infrastructure for effective
and efficient deliberations within this virtual inter-organizational domain. In so doing, it serves
the key functions of a referent organization, which Trist (1983) identified as the coordination of
relationships and activities, appreciation of emergent trends and issues, and infrastructure
support including providing resources, sharing information, and conducting special projects. As
mentioned above, the NACC organizes the semi-annual, F2F meetings of the ADC core directors
and ADC steering committees. It provides funding to the ADCs to offset the cost of submitting
their data to the UDS, it funds collaborative research projects, and it assists scientists seeking to
use the UDS in their research.
The distinctions between informal and formal mutual adjustment are not so much binary, as
incremental or additive, in that the necessary conditions for effective informal mutual
adjustment such as trust and respect, also provide a positive foundation for more formal
mechanisms of mutual adjustment. The foundation for informal mutual adjustment among the
ADCs and the NACC was a dense network of collegial relationships among participants in the AD
research network that had developed over several years of interacting with each other at
conferences and symposiums, in professional organizations on editorial boards, and on review
panels for the NIA and other funding agencies. In addition, many of the ADC core directors and
center directors serve on the Advisory Boards of two to three other ADCs, much like the
interlocking directorates in the private sector.
This level of F2F interaction has certainly served to reduce some of the complexity in the
working relationships that might otherwise have adversely affected the efficacy of the
deliberations in this virtual R&D project; that is, the members of the teams tasked with creating

19. the UDS were familiar with each other and had already developed socially derived norms of
interaction.
Further, the participants who were elected or selected to serve on the formal coordinating
mechanisms – the Clinical Task Force the working sub-committees, and the ADC Steering
Committees are generally highly regarded members of the AD research network who possessed
considerable source credibility based on their acknowledged expertise and perceived
trustworthiness (Gilbert et al, 1998). An interviewee described the qualities of these individuals
as follows:
The task forces were comprised of people that were well recognized as being leaders and
reasonable people. I think that those are the two characteristics of the people on the
task forces; they’re not only leaders, they’re politically astute. They’re open. They’re “just
reasonable” is the best way to describe them. They’re not difficult to deal with. Because
they commanded confidence from the various other members that were involved in
contributing data, they’ve been successful in driving each other forward.
Another informal and formal mechanism for mutual adjustment was the presence of several very
effective network leaders and boundary spanners from the NIA, the NACC and the ADCs. These
network leaders or boundary spanners serve a ‘reticulist’ (Friend et al., 1974) role that has been
critical to the creation and implementation of the UDS from vision to inception. Alter and Hage
(1993) describe reticulists as “individuals who engage in networking tasks and employ methods of
co-ordination and task integration across organizational boundaries” (p. 46). Similarly, Webb
(1991) describes reticulists as “individuals who are especially sensitive to and skilled in bridging
interests, professions and organizations” (p. 231). Both are apt descriptions of the key network
leaders in the AD research network who have provided an effective balance of diplomacy,
negotiation, persuasion (Rhodes, 1999), and when necessary confrontation to ensure the success
of this project and the AD research network.
Once the UDS had been established, the NACC and the ADCs generally knew what to do and how
to do it both conceptually and operationally (stages D3 and D4 on the R&D continuum, Figure 4).
The tasks and deliberations required to maintain the UDS have become more routinized and are
less uncertain and less complex. These deliberations are coordinated with plans and standards
including specific requirements, delivery schedules for data submissions, and financial incentives.
In fact, coordination by standards is the fundamental premise of the UDS with its standardized
clinical evaluation protocol for assessing and diagnosing AD and other dementias, common data
points, standardized data collection forms with detailed coding manuals, and error tracking
procedures. And with regards to the UDS, the relationship between the NACC and the ADCs, while
still collegial, has generally become more hierarchical and prescriptive. As the interviewee stated
above: “… ultimately their decision is the one we have to follow.”
The findings from this case study on coordinating mechanisms support Sabherwal’s (2003)
assertion that more informal, communications-oriented coordinating mechanisms are suitable
when uncertainty is greater, and that the more impersonal and formal, control-oriented

20. mechanisms are a better fit when the level of task uncertainty is lower.
Virtuality and Communication
The nature of communication, whether F2F or technologically mediated and the choice of forums
for the deliberations, were also impacted by the levels of task complexity and uncertainty. The
deliberations to create, implement and revise the UDS relied on high levels of input and
participation directly for the members of the Task Force and the sub-committees and for others via
surveys, bulletin boards, and direct feedback to the Clinical Task Force and its sub-committees.
These discretionary coalitions used emails, teleconferences, and occasionally videoconferences for
their deliberations. However, when they had a particularly difficult decision to make they usually
made these in F2F meetings. As a member of one of the working groups stated: “One of the things
I demanded in my job on the toolbox has been face-to-face meetings of my team at regular
intervals. We’ve been doing everything by email and phone conferences, but there is nothing like a
face-to-face meeting.” It is also interesting to note that even though there were several
information and communication technology (ICT)-mediated vehicles for providing input, the ADC
Directors often used the public forums in their semi-annual meetings to raise issues or share their
concerns.
The communication involved in the ongoing deliberations to maintain the UDS are predominantly
technologically mediated: that is, the data is submitted electronically to the relational data base;
questions for clarification are posed and answered on an online bulletin board; and discussions
between the ADCs and the NACC regarding submitted data are conducted almost exclusively by
email and only in exceptional cases by telephone.
Even the day-to-day operations of the ADCs rely heavily on information and communication
technology. Here is how an interviewee described communications within their ADC (Personal
interview, 2011): Communications are partly face-to-face. They’re partly by email. They’re partly
by a system that my data management group created, which is online. It’s partly by electronic
reports that are also up on the Web. They’re generated each evening. And (in our center) they’re
partly through an online data management request system that we built, where you couple
communications with data management.
In sum, the routine deliberations characteristic of the D3 and D4 stages in the R&D continuum
were mostly virtual and technologically mediated. In comparison, the deliberation topics that
were more uncertain and complex (at the D1 and D2 stages in the R&D continuum), especially
those that involved important and sensitive trade-off decisions, tended to be addressed in F2F
interactions.
Culture of Collaboration
The NIA has been very explicit in its intention to foster an environment of collaboration across
multiple disciplines and the network of ADCs. Tony Phelps, NIA Program Director, described the
NIA’s vision for an ethos of collaboration on the 20th anniversary of the ADCs as follows:

21. When the ADC program was created, NIA’s leaders hoped that an environment of
cooperation would stimulate AD researchers to seek new pathways to scientific discovery
and to share their findings. The program has done just that. In the collaborative
atmosphere of the centers, specialists in biomedical, behavioral, pathological, and clinical
science are studying the causes, and possible prevention of AD, and developing new lines
of multidisciplinary research. As scientists at the centers uncover the complexities of
dementia, they spark a certain friendly competition with other research scientists
throughout the world. Their brilliance and enthusiasm creates an excellent training
ground for up and coming investigators, and inspires others to devote their energies to
AD research (National Institute of Aging website, 2005)
The following comment from our interviews provide anecdotal evidence that the NIA’s vision of
cooperation and ‘friendly competition’ is being realized (Personal interview, 2011):
I think we’re beginning to see young investigators and more established investigators
getting grants to do secondary data analysis, using the information, planning GWAS
studies, hooking up to other investigators within the group who have large data sets
that they’ve contributed, looking to see if they have other data that’s not necessarily
part of the UDS, but would help to augment analyses that they’re doing. I think we’re
seeing a lot of that now, and that will definitely be the metric for success going forward.
It’s becoming more and more important.
This researcher’s comment also captures the balance of competition and cooperation that now
characterizes the ADC network:
I would say cooperation is really the key behind what we’re doing right now in terms of
the UDS. And I think because we’re all part of the same project, it doesn’t make any
sense to be competing with one another because we all have the same common goal
whether we like it or not. We’ve got to collect the data for this protocol. Now, we may
compete in other ways in terms of other aspects of scientific research, but for this
protocol and for this project, we’re all on the same page. So we all have to work
together for a common good. I think it’s that spirit that we’re all one family for this
project, so we have got to figure out how we’re going to work together.
Building and maintaining the UDS and this collaborative network of researchers has been a
social task as well as one of technical or professional alignment. Admittedly there have been
pressures that would normally work against cooperative effort such as competition for funding
and recognition such as the Nobel Prize for Medicine, ego, professional jealousy and such. It
appears that these effects have been mitigated in large part by appealing to the compelling
mission and genuinely shared goal of eradicating AD, as well incentives such as continued
funding for one’s center.
The UDS has played a significant role in fostering and extending the culture of collaboration in
the AD research network. It has also played a role in other major collaborative research

22. initiatives both in the United States and now globally, including the Dominantly Inherited
Alzheimer Network, the Alzheimer’s Disease Genetics Consortium Genome Wide Association
Study, the Alzheimer’s Disease Neuro-Imaging Initiative (ADNI), and now, World Wide ADNI.
The UDS is a brilliant example of the benefits to be gained from virtual collaboration across
research sites and disciplines. Together, all of these contributors have created a powerful
instrument to help accelerate the identification of new treatments, preventions and ultimately,
a cure for this insidious disease.

23. References
Alzheimer's Disease International. World Alzheimer Report 2009.
www.alz.co.uk/research/world-report
Asaro, F. 2011. Universal Co-opetition: Nature's Fusion of Competition and Cooperation. Betty
Youngs Books.
Brandenburger, A. and Nalebuff, B., 1997. Co-Opetition: A Revolution Mindset That Combines
Competition and Cooperation, New York: Doubleday Currency
Dominantly Inherited Alzheimer Network, http://www.dian-info.org/
Evans, D.A., Funkenstein, H.H., Albert, M.S., Scherr, P.A., Cook, N.R., Chown, M.J., Hebert, L.E.,
Hennekens, C.H., and Taylor, J.O. 1989. Prevalence of Alzheimer's disease in a community
population of older persons. Higher than previously reported. JAMA. Nov 10; 262(18): 2551–
2556.
Friend, J., Power, J., and Yewlett, C. 1974. Public Planning: The Intercorporate Dimension,
London, Tavistock.
Gilbert, D.T., Fiske, S.T., Lindzey, Gardner, eds. (1998). The Handbook of Social Psychology.
Oxford University Press.
Kolata, G., 2010, Sharing of Data Leads to Progress on Alzheimer’s, August 12, 2010, New York
Times.
Mashey, J. (2008, December 3). What can we learn from Bell Labs about managing energy R&D?
Retrieved from http://bravenewclimate.com/2008/12/02/hansen-to-obama-pt-iv-where-to-from-
here/#comment-3023.
National Institute of Aging, 2013, Alzheimer’s Disease Research Centers. Retrieved from
http://www.nia.nih.gov/alzheimers/alzheimers-disease-research-centers
National Institute of Aging, 2013. Retrieved from http://www.nia.nih.gov/alzheimers/about-adear-
center
Purser, R.E, Pasmore, W.A., Tenkasi, R.V., 1992. The Influence of Deliberations on learning in
new product development teams, Journal of Engineering and Technology Management, 9, pp.
1-28.
Revkin, A. (2007, December 12), ‘R2-D2’ and other lessons from Bell Labs. New York Times.
Retrieved from http://dotearth.blogs.nytimes.com/2008/12/12/r2-d2-and-other-lessons-from-bell-
labs/.
2013 www.alz.washington.edu/WEB/NACCsum.pdf

24. Alzheimer’s Association re WW Adni http://www.alz.org/research/funding/partnerships/ww-adni_
overview.asp
2010 University of Pennsylvania School of Medicine http://alois.med.upenn.edu/adgc/
Williams, P. 2010. Special agents: The nature and role of boundary spanners. Paper presented
at the ESRC Research Seminar Series: Collaborative Futures: New Insights from Intra and Inter-
Sectoral Collaborations, University of Birmingham.

25. Appendix A - Methodology
We incorporated the diagnostic steps of open sociotechnical systems design to gather data and
analyze the impact of virtuality on the AD research network: conduct an initial scan of the
system, map the system, analyze the technical subsystem, and analyze the social subsystem
(Pava, 1983). We did this in two stages. First, we conducted an initial scan of the system, the
AD research network, and used this preliminary information to map the system and choose a
specific project to investigate in greater depth. We then conducted a series of interviews to
analyze the technical and social subsystems.
Conducting an Initial Scan and Mapping the System
The purpose of an initial scan is to discern the mission or goals of the system and the
governance processes and coordination mechanisms that enable collaboration in pursuit of the
mission. The mission and governance system provide the impetus for a self-regulating system
of players who define and iteratively evolve the technical subsystem in terms of the key
deliberations or issues they need to address in order to achieve the mission.
We conducted the initial scan of the AD research system/network by reviewing available
documentation on the major research initiatives and key stakeholders, by attending and
observing the Fall 2010 ADC Directors meeting in San Francisco, and by interviewing the
Principal Investigator of the National Alzheimer’s Coordinating Center (NACC) and the NIA’s
Program Director for the Alzheimer’s Disease Centers.
This afforded us a deeper understanding of the mission of the AD research network and the
emergence of the ADCs, the NIA-funded centers, and other major research programs, initiatives
and projects such as the UDS, ADNI, GWAS, NCRAD, and DIAN.
Analyzing the Technical Subsystem
Pava (1983) described deliberations as “equivocality reducing events”, i.e. choice points that
are critical to work systems involving knowledge generation and knowledge utilization. From
this general description, Purser (1990) defined deliberations in product development as “social
interactions in which knowledge is exchanged to define or solve a problem, make a decision, or
implement a solution”. However, deliberations are not simply the equivalent of decisions or
meetings; rather, they are sense-making exchanges (Weick, 1995), communications and
reflections in which people engage to reduce the equivocality of a problematic issue.
A deliberation is identified by the existence of an equivocal topic, which is explored in different
types of forums, involving a particular group of participants who contribute important
information or take-away important information. Deliberation analysis assesses the values and
perspectives of participants within forums, and “the interpretative dynamics among
interdependent parties who must forge a discretionary coalition” (Pava, 1983) to make
intelligent trade-offs from their respective values, priorities, and cognitive orientations (Tenkasi,
1994; 2000).

26. Deliberations connect people together to meaningful issues much like linear work processes
connect people together to a common outcome. For people who work virtually and may never
see one another as they work together, this connection to an issue may be vital to their
performance.
Deliberations in knowledge work such as R&D can be viewed in terms of intellectual bandwidth
(Nunamaker et al. 2001, 2002, Qureshi et al. 2002) and the ability to mobilize intellectual assets
in deliberations to create value. This model provides a framework for measuring the extent to
which an organization can create value from its intellectual assets by looking at two key elements
in deliberations. The first is the process of understanding the data and available information and
translating it into knowledge. The second addresses the interdependence of efforts and whether
it is primarily an individual work mode, a collected work mode and the sum of individual work, a
coordinated work mode in which there is sequential interdependence or a concerted work mode
in which all work in concert to produce joint deliverables.
Analyzing the Social Subsystem
The social system is defined in terms of discretionary coalitions that are needed to conduct the
deliberations effectively. These coalitions make the important trade-offs in creative work that
are made necessary by the presence of useful but inherently divergent values and perspectives.
For example, in traditional research environments, scientists typically compete against one
another for limited grant money and to publish articles in top journals, neither of which enable
the effective functioning of coalitions in a virtual project. The social system design does not try
to eliminate differences, but to create a mutual understanding and a common orientation such
that trade-offs can be settled on an intelligent and ongoing basis. Coalitions are to nonlinear
work what work groups or teams are to more routine work. Roles and responsibilities can be
defined for the parties involved in the coalitions as well as other changes in the coordinating
mechanisms in a way that supports and rewards the sort of integrative perspective necessary to
successful coalition functioning.
In classic sociotechnical systems analysis, the focus is on addressing variances in work processes
and performance. However, in the knowledge work environment of R&D projects (co-located
and virtual), variances manifest as knowledge barriers. Purser et al. (1992) identified four main
categories of “barriers” obstructing and delaying collaborative knowledge development: The
first was the failure to share knowledge due to lack of cooperation, missing parties, or
unrealistic timeframes; the second was the lack of a common frame of reference including
cognitive frame of reference barriers associated with differences in language, values, and
functional expertise; third was the lack of knowledge about the work, procedures and
processes, or the capabilities of virtual participants that can slow or derail progress; and the
fourth was failure to utilize knowledge.
In order to develop an interview frame, we performed an extensive literature review on
deliberations in virtual R&D organizations. We incorporated interview questions that would be
used in a sociotechnical scan of the technical subsystem in non-routine knowledge work. This
included listing the major deliberations, identifying the different forums in which these

27. deliberations were conducted (in person, videoconferences, teleconferences, email, etc.),
determining the structures of the forums (structured, semi-structured, and unstructured),
identifying the participants in these key deliberations, and surfacing the variances or knowledge
barriers in the deliberations.
To analyze the social subsystem within which the major deliberations occurred, we developed
questions that addressed issues such as the role of discretionary coalitions, key influencers or
participants, means of building support for decisions, and how they managed conflicting
objectives or perspectives.
Since the structure of the UDS had already been established and many of the early
implementation challenges had been addressed, we conducted retrospective interviews and
secondary data searches to understand the development process, the scope of involvement of
various personnel, and the kinds of decisions and actions taken to reach consensus on the
UDS.V1. However, at the time we conducted our interviews, the NACC and the Clinical Task
Force were in the process of revamping the batteries of instruments and measures for the
UDS.V3 and we were able to gather contemporaneous insights on how they were conducting
this task in a virtual environment. We also interviewed ADC researchers about current use,
decisions, and applications of the UDS to gain a perspective on how deliberations related to it
have changed over time.
We followed a protocol established by the VOSS research team, but customized the questions
for the ADC interviews in light of our discussions with the NACC PI and NIA’s Program Director,
the insights gathered at the ADC Directors semi-annual meeting, and our preliminary mapping
of the ADRN system. We also asked for their input on selecting a representative sample of
individuals who have been involved in the key deliberations related to the UDS. A total of 12
telephone interviews were conducted during the winter and spring of 2010-2011 with a cross-section
of ADC center directors, principal investigators, clinical core directors, neurologists,
psychiatrists, biostatisticians and other members of the research teams.
The interviews yielded considerable data about the technical and social subsystems. The
interviews were transcribed and members of the research team analyzed the content to
identify common themes regarding the major deliberations , the various forums in which they
occurred, the role and relative efficacy of virtual or technology-mediated forums and face-to-face
meetings, the sources of variances or knowledge barriers, and the coordinating
mechanisms used to address them.