1. Research | Mini-Monograph
Scientific and Legal Perspectives on Science Generated for Regulatory Activities
Carol J. Henry and James W. Conrad Jr.
American Chemistry Council, Arlington, Virginia, USA
Information Quality Act. The Information
Quality Act (IQA 2000) governs the quality of
This article originated from a conference that asked “Should scientific work conducted for purposes
information that federal agencies disseminate.
of advocacy before regulatory agencies or courts be judged by the same standards as science con-
The IQA required the Office of Management
ducted for other purposes?” In the article, which focuses on the regulatory advocacy context, we
and Budget (OMB) to issue an initial set of
argue that it can be and should be. First, we describe a set of standards and practices currently being
implementing guidelines (OMB 2002). Each
used to judge the quality of scientific research and testing and explain how these standards and prac-
federal agency then issued its guidelines apply-
tices assist in judging the quality of research and testing regardless of why the work was conducted.
ing the OMB guidelines to its particular cir-
These standards and practices include the federal Information Quality Act, federal Good Laboratory
c u m s t a n c e s [e.g., U.S. Environmental
Practice standards, peer review, disclosure of funding sources, and transparency in research policies.
Protection Agency (U.S. EPA) 2002a].
The more that scientific information meets these standards and practices, the more likely it is to be
I t is important to note that the IQA
of high quality, reliable, reproducible, and credible. We then explore legal issues that may be impli-
applies not only to information that agencies
cated in any effort to create special rules for science conducted specifically for a regulatory proceed-
generate themselves but also to information
i n g . Federal administrative law does not provide a basis for treating information in a given
developed by nongovernment parties, to the
proceeding differently depending on its source or the reason for which it was generated. To the con-
e x t e n t the government “disseminates” it,
trary, this law positively assures that interested persons have the right to offer their technical exper-
either by adopting or endorsing it as its own
tise toward the solution of regulatory problems. Any proposal to subject scientific information
view or by relying on it to make a decision.
generated for the purpose of a regulatory proceeding to more demanding standards than other scien-
[ I n f o r m a t i o n prepared for administrative
tific information considered in that proceeding would clash with this law and would face significant
adjudications is exempt from IQA guidelines,
administrative complexities. In a closely related example, the U.S. Environmental Protection Agency
but agencies construe this exemption narrowly
considered but abandoned a program to implement standards aimed at “external” information. Key
(U.S. EPA 2002a).] Thus, to the extent that
words: Administrative Procedure Act, agency proceedings, conflict of interest, financial disclosure,
businesses, universities, or other private enti-
industry science, Information Quality Act, interested persons, peer review, regulatory science, right
ties conduct research or testing and the results
to publish, scientific quality. Environ Health Perspect 116:136–141 (2008). doi:10.1289/ehp.9978
come into the possession of an agency such as
available via http://dx.doi.org/ [Online 7 November 2007]
the U.S. EPA, the results cannot form a basis
of the agency’s decision without becoming
Standards and Practices agencies, that specify the content and charac- s u b j e c t to IQA requirements. And those
teristics of studies to be conducted to meet requirements are most precise and demanding
for Judging the Quality
r e g u l a t o r y requirements, e.g., a standard in the area of scientific information.
of Scientific Work
bioassay for carcinogenesis) and “research” OMB’s guidelines (OMB 2002) prescribe
T h e project on Scientific Knowledge and (studies that are hypothesis driven, addressing fairly detailed standards for “objectivity.” As a
Public Policy (SKAPP) examines the nature b r o a d methodologic or mechanistic ques- general matter, information must be accurate,
of science and the ways in which it is used tions, e.g., the biologic activity of chemicals reliable, and unbiased. Scientific information
and misused in government decision making on the environment). m u s t be generated using sound research
a n d legal proceedings. Last year, SKAPP Both government and the scientific com- m e t h o d s . The sources of the information
commissioned papers to address the question munity outside of government have played must be disclosed and data should be docu-
“Should scientific work conducted for pur- independent but reinforcing roles in develop- m e n t e d . Scientific information must be
poses of advocacy before regulatory agencies ing and propagating these standards and prac- accompanied by supporting data and models.
or courts be judged by the same standards as tices. The private sector is actually driving “Influential” scientific information must be
s c i e n c e conducted for other purposes?” implementation of several relatively newer sufficiently transparent to be reproduced sub-
(SKAPP 2006). This article is adapted from practices, in part to address concerns about ject to several caveats. (“Influential” informa-
one of those papers. the credibility of industry-funded science. In tion is that which an agency “reasonably can
Science is a social enterprise, and scientific general, the regulatory implementations of determine will have or does have a clear and
work tends to be accepted by the community t h e s e concepts impose additional require- substantial impact on important public poli-
w h e n it has been confirmed. Crucially, m e n t s well beyond those conventionally cies or private sector decisions.”) Influential
experimental and theoretical results must be imposed outside of the regulatory context. information regarding risks to health, safety,
reproduced by others within the science com- These government requirements promote a or the environment must also be based on
munity and the validity of the work estab- high degree of reliability. As explained below,
l i s h e d by replication (Wikipedia 2007a). the cumulative result of their applicability is This article is part of the mini-monograph “Science for
However, such replication can take years, and that science conducted for regulatory purposes Regulation and Litigation.”
what constitutes replication in a given case in many cases is actually likely to be more reli- Address correspondence to J.W. Conrad Jr., Conrad
may also be disputable. Consequently, a vari- able than science conducted outside the regu- Law & Policy Counsel, 1615 L St. NW, Suite 1350,
Washington, DC 20036 USA. Telephone: (202) 822-
e t y of standards and practices have been l a t o r y arena. This is not to argue that the
1 9 7 0 . Fax: (202) 822-1971. E-mail: jamie@
established over the years for assessing the latter should be required to comply with these
conradcounsel.com
q u a l i t y of scientific work (Barrow and g o v e r n m e n t requirements to any greater At the time this article was accepted, C.J.H. and
Conrad 2006). These standards and practices extent than it already is, but only to emphasize J . W . C . were both employed by the American
apply to both “testing” (activities conducted that science conducted pursuant to them is Chemistry Council.
pursuant to protocols, prescribed by regulatory relatively more reliable as a result. Received 12 December 2006; accepted 5 July 2007.
136 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives
VOLUME

2. Science generated for regulatory activities
“the best available, peer-reviewed science and of the study in a clearly written protocol that The crucial role of reviewer independence
supporting studies conducted in accordance must be approved by the agency. These pro- a n d expertise. T h e hallmark of any peer
with sound and objective scientific practices; tocols have been validated and chosen, after review is the independence and expertise of
and . . . data collected by accepted methods or extensive and careful review, to provide an the peer reviewer. Reviewer knowledge, expe-
best available methods,” and must disclose acceptable degree of scientific certainty, in the rience, and expertise are central requirements
significant uncertainties and relevant peer- agency’s view, regarding the reliability and for instructive peer review and ensure “techni-
reviewed studies. relevance of test results, which in turn pro- c a l credibility” for the review. The peer
A c c o r d i n g l y , if a business submitted a v i d e s the confidence necessary for making reviewer should have expertise at least equiva-
paper to the U.S. EPA regarding research it safety determinations and other regulatory lent to that needed for the “original work” but
had conducted on the risks posed by one of its determinations. m u s t be independent of the work being
products, and the U.S. EPA was going to con- Conformance to GLPs does not, in itself, reviewed. These dual requirements for exper-
sider this paper as part of making a decision ensure that scientific work is reproducible, or tise and independence offer the best chance of
involving that family of products—a decision that the resulting data will be interpreted cor- obtaining objective, expert evaluation while
that would have a substantial impact on pro- rectly. However, when research studies adhere maintaining scientific integrity (NRC 2003).
ducers and customers of those products—that to GLPs, reviewers and those acting upon the C o n f l i c t of interest and bias in peer
paper would be subject to the most demanding science may have a high degree of confidence review. Identifying and managing potential
level of IQA objectivity standards. The under- t h a t the experimenters a) a d h e r e d to the conflicts of interest (COIs)—financial and non-
lying data and methods would have to be pro- experimental protocol employed, b) took all financial—is critical to meeting the require-
vided to the agency. Also, the agency (and the steps and measurements claimed to be ment for independent peer review. In turn, this
hence the submitter) would have to be able to taken during conduct of the study itself, and requires distinguishing between COI and bias.
document that the research reported in the c) accurately reported the test results. The National Academies (2003) defines
paper was conducted in accordance with sound GLPs are neither required nor common in COI as any financial or other interest that
and objective scientific practices. Finally, the research laboratories but have been imple- c o n f l i c t s with the service of an individual
paper would need to be based on the best avail- mented at all U.S. EPA contract laboratories because it could impair the individual’s objec-
able science, including any peer-reviewed work and at major universities that perform signifi- tivity or create an unfair competitive advan-
in the literature. [Notably, by virtue of being c a n t medical and toxicology testing and tage for any individual or organization.
provided to a federal agency, this information research for regulatory purposes. Sponsoring Conventionally speaking, some COIs are
w o u l d become subject to the Freedom of organizations can require that research and “actual,” with an unambiguous potential for
Information Act (FOIA 1966). Exemptions to testing studies comply with GLPs and can financial gain (e.g., the reviewer holds stock
FOIA exist for privacy and commercial consid- incorporate that request into contractual vehi- in an entity likely to be affected by the rele-
erations, although health effects data generally cles. As a result, GLPs have grown to be used vant regulatory action). Other actual COIs
cannot be claimed as confidential business f a i r l y extensively outside the specific U.S. might be employment conflicts or close pro-
information (Conrad 2006).] EPA and FDA regulatory contexts in which f e s s i o n a l financial relationships. Potential
G o o d Laboratory Practice regulations. they are required. f i n a n c i a l COIs may exist on the part of
B o t h t h e U.S. EPA and Food and Drug Peer review. While peer review has been an e x p e r t s remunerated by any individual or
Administration (FDA) have adopted compara- integral part of medicine for centuries, it has o r g a n i z a t i o n (International Life Sciences
ble sets of requirements specifying laboratory become a mainstay of the scientific process Institute 2005). Similarly, any scientist can
practices and procedures that must be followed o n l y since the mid-twentieth century have a COI, depending on the topic. As the
to ensure the quality and integrity of studies (Wikipedia 2007b). The National Research G o v e r n m e n t Accountability Office (GAO
submitted to support agency decisions (FDA Council (NRC) has defined peer review as pro- 2001) has noted, association with industry
2006; U.S. EPA 2006a, 2006b). These Good viding an “in-depth critique of assumptions, does not by itself indicate a COI. Conversely,
Laboratory Practice (GLP) standards prescribe calculations, extrapolations, alternate interpre- association with an environmental group does
essential, routine features of sound laboratory tations, methodology, and acceptance criteria not inoculate a scientist against COIs. There
science. All studies submitted to these agencies employed and conclusions drawn in the origi- well may be COIs among academics, because
in connection with the relevant statutory pro- nal work” (NRC 1998). At a minimum, peer of intense competition for research funds and
grams must be conducted in accordance with r e v i e w must exhibit the following features publications, that are difficult to identify.
these standards. The GLP rules are more rigor- (International Life Sciences Institute 2005): An individual with a COI generally may
ous than standards followed at university labo- • It must include multiple assessments. not participate in a peer review. Organizations
ratories (Anderson et al. 2001). • It must be conducted by scientists with no that use peer-review systems require candi-
GLPs have three basic elements: quality d i r e c t connection to the research, or its dates to disclose their organizational affilia-
assurance, standard operating procedures, and sponsors. t i o n s , financial interests, personal and
study protocols. • It must be conducted by scientists who have professional involvement, and other informa-
Quality assurance. GLPs mandate docu- experience with or expertise in the research tion that may be pertinent to the topic, erring
mentation of study conduct and results and in question. on the side of full disclosure, and to certify
ensure that a full record of the study is pre- A rigorous peer review is a key part of the that the information is true and accurate to
served for subsequent review, if necessary. foundation on which scientific excellence is the best of their knowledge.
S t a n d a r d operating procedures. T h e r e achieved in all research programs. Science is “Bias” is a partiality or loss of objectivity
must be written procedures for accurate and not “self-evident” and requires scientific judg- because of personal views or positions and may
full data collection under the nonclinical lab- ment, and because judgments vary, a well- be perceived as arising from the close identifi-
oratory study methods to be used, methods b a l a n c e d representation of intellectual cation or association of an individual with a
determined by management to be adequate to perspectives is needed. Where that is obtained, particular point of view or with a particular
ensure the quality and integrity of the data. the peer review is more likely to rebut any sci- group that may be affected by the research
Study protocols. Each study must state entifically untoward or untenable hypothesis being reviewed. For example, questions about
the objectives and all methods for the conduct and arrive at a successful review. t h e neutrality of a reviewer could arise if
137
Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008

3. Henry and Conrad
that person had represented an interest group OMB’s peer review bulletin. Many fed- Although journals have instituted disclo-
at a hearing. eral agencies have written policies for when sure policies (sometimes termed “competing
I t is generally recognized that bias is and how to conduct agency peer reviews (U.S. financial interests policies”) to increase trans-
pervasive and not inherently undesirable. A EPA Science Policy Council 2000). To bring parency for their readers, granting agencies
s u b c o m m i t t e e of the U.S. EPA Science greater consistency to and establish minimum and sponsors employ a variety of approaches
A d v i s o r y Board (2000) has opined that, requirements for such reviews, the OMB and and requirements, and it is too early to con-
“[a]lthough it is possible to avoid conflict of the Office of Science and Technology Policy clude that a broad consensus practice has been
i n t e r e s t , avoidance of bias is probably not in early 2005 issued their “Final Information established. The Long-Range Initiative (LRI)
p o s s i b l e . All scientists carry bias due, for Q u a l i t y Bulletin for Peer Review” (OMB of the American Chemistry Council (ACC)
example, to discipline, affiliation and experi- 2005). The bulletin requires all federal agen- requires its contractors to acknowledge ACC
ence.” Potential sources of bias in peer review- cies to conduct peer reviews of all “influential” as a sponsor of the research in all articles or
ers are managed through disclosure. scientific information—defined as under the publications pertaining to the research con-
Federal regulations generally implement IQA—that the agency intends to disseminate. ducted under agreement with the ACC.
these concepts. Although these rules address [As with the IQA, information disseminated Transparent research policies. Research
only federal employees, they include “special as part of an adjudication (e.g., a permit deci- p r o g r a m s are commonly regarded as more
government employees,” such as participants sion) is exempt from rules outlined in the bul- credible, and their results less suspect, when
i n panels organized under the Federal letin, unless it is novel or precedential and peer they employ transparent processes and proce-
Advisory Committee Act (1972), and there- review is practical.] dures regarding ownership of data, release of
f o r e they apply in the case of federal peer The bulletin sets especially high standards results, and publication of results. This prac-
review panels such as the U.S. EPA Science for “highly influential” scientific assessments. tice is growing for assessing scientific quality,
Advisory Board. These rules prohibit a federal For these documents, a) agency scientists gen- although it has not yet achieved broad accep-
employee from participating directly and sub- erally may not participate; b) reviewers must be tance within the research community.
stantially in a particular matter (as opposed to provided with background information suffi- When investigators own the data and the
“broad policy options”) that will have a direct cient to understand the key findings or conclu- s c i e n t i f i c information that they generate
and predictable effect on a) a financial interest sions of the draft assessment; c) where feasible, through their research efforts, they are in con-
of the employee (generally, employment or public comment should be sought and pro- trol of how those data will be evaluated, used,
stock ownership); b) the employee’s employer; v i d e d to the reviewers; and d) t h e agency and communicated. Sponsors who choose to
or c) organizations for which the person has should respond to the reviewers’ report. employ this approach do so to lend strength,
served in the last year in a paid capacity or as Under the guidelines of the OMB bul- objectivity, and credibility to the outcome of
an active participant, where a reasonable per- letin, peer review should now be applied sys- the research. The ACC LRI program includes
son would question the person’s impartiality t e m a t i c a l l y to all influential scientific this element of data ownership (ACC 2006).
in the matter, unless covered by an exclusion information published by federal agencies or With ownership of the data—the right to
o r issued a waiver (Office of Government used by them to make decisions—regardless of r e l e a s e data independently and to publish
Ethics 1997). why the work was conducted—in a manner without prior sponsor approval—inappropri-
P e e r review at federal agencies. that is more rigorous and revealing than that a t e sponsor interference can be avoided.
H i s t o r i c a l l y , the primary venues for peer occurring outside the federal government. Interactions with sponsors can be beneficial in
review have been in journal publication and Disclosure and acknowledgment of funding providing specialized knowledge and insight at
in grant evaluation. Increasingly, federal agen- sources. In the mid-1990s, concern arose about a level of detail other scientific resources might
cies have been conducting more demanding the integrity of scientific research because of not provide. However, the final decision on
peer reviews of studies that will form the basis increasing commercial links and consequent whether to accept the sponsor’s information or
of important agency decisions. For example, influences. In response, the Department of advice remains with the investigator.
the U.S. EPA has several standing advisory Health and Human Services (DHHS) and the Congress has required the federal govern-
bodies to conduct peer reviews, including its National Science Foundation (NSF) issued m e n t to take a slightly different tack in
Science Advisory Board, its Science Advisory their “Investigator Financial Disclosure Policy” the area of research transparency. As a result
Panel (for pesticide program decisions), and (DHHS/NSF 1995). This policy required dis- o f the “Shelby Amendment” (Omnibus
its Clean Air Science Advisory Committee. closure of an investigator’s significant financial Appropriations Act 1998), the OMB revised its
Some agencies request the National Research interest if it “would reasonably appear to be Circular A-110 governing federal grants, and
Council of the National Academies to con- affected” by activities funded or proposed for now all federal agencies must make available to
vene peer reviews for very prominent issues funding by NSF and DHHS. the public, pursuant to FOIA, final research
and assessments. Although the principles for As the scientific community became more d a t a generated by agency grantees that an
journal and agency peer review are the same, aware of the potential COIs for university agency cites in support of a rule or order (OMB
they involve very different practices, proce- investigators with commercial ties receiving 1999). Thus, when the federal government
dures, and decision pathways from the results federal grant support, questions also emerged funds scientific work and relies on it to support
and conclusions of the peer review. about the potential implications for interpreta- agency action with the force and effect of law,
Agency peer reviews are complex, time tion of research results and conclusions. By the data produced by that work will be placed
consuming, and most often involve forming a 2001 some of the major scientific journals had in the public domain if someone in the public
p a n e l of expert peer reviewers, where the established policies to encourage authors to asks for it. This way, federally funded research
panel will meet in public for their delibera- declare any competing financial interests in data can be available so that the scientific com-
tions. In some cases, peer reviews are con- relation to research papers. Financial disclo- munity can validate the work through replica-
ducted by scientists within the agency, and in sure forms are now routinely required to be tion. Although Circular A-110 does not apply
other cases by external scientists. Prior criti- submitted with manuscripts for use by the edi- to privately funded research submitted to fed-
c i s m s of agency peer reviews (GAO 2001) tors, and journal articles generally acknowl- eral agencies, the same prospect of disclosure is
have only heightened the scrutiny applied to edge research support, although the practice is a n inevitable effect of the IQA, discussed
them both within and outside agencies. not universal (Nature 2006; Science 2006). above, if the agency relies on that information.
138 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives
VOLUME

4. Science generated for regulatory activities
Federal Standards processes. In effect, Congress and the courts
science is premised on the idea that the self-
have determined that in an open democratic
interest of regulated parties that conduct or
for Considering
s o c i e t y administered by a bureaucracy
sponsor research creates a conflict with the
“Regulatory” Science
r e q u i r e d to act fairly and rationally, it is
interest of the truth, whether conscious or
The foregoing discussion outlines the standards important that agencies allow interested or
unconscious, that renders the research invalid
that one can use to judge the quality of scien- affected persons to provide information to
or at least suspect (Krimsky 2005).
tific research without regard to the purpose for them, and fairly consider that information.
At the outset, we acknowledge the grow-
which it was conducted. The following discus- The backbone of federal administrative
ing literature purporting to find that industry-
sion explains why regulatory agencies should law is the Administrative Procedure Act (APA
funded research produces results that favor its
not—and arguably cannot—treat science cre- 1 9 4 6 ) , which requires agencies to provide
sponsors more often than other research on
ated for purposes of an agency proceeding dif- notice and “give interested persons an oppor-
the same topic (Krimsky 2005; vom Saal and
ferently in that proceeding than science not t u n i t y to participate in [a] rulemaking
Hughes 2005). As the authors of one of those
created for purposes of that proceeding. t h r o u g h the submission of written data,
s t u d i e s has fairly observed in comparing
T h e breadth of research and testing views, or arguments. . . .” Courts interpreting
industry and government-funded studies, one
obligations. The data proffered by regulated Section 553 of the APA have made clear that
o r both of two things could be going on:
parties in agency proceedings often are not an agency must consider and respond to—
either “industry-funded scientists [are] under
solely the results of self-interested efforts to rather than discount—all significant matters
real or perceived pressure to find and publish
influence those proceedings. Rather, in many put before it. In particular, courts have specif-
only data suggesting negative outcomes” or
cases the parties have been obligated by regu- ically rejected the notion that the APA should
“government-funded scientists [are] under
lation or order to conduct particular studies, provide some basis for insulating agency offi-
real or perceived pressure to publish only data
according to particular protocols, and to pro- cials from the input of regulated parties. In
suggesting adverse outcomes” (vom Saal and
vide the results of that work to the agency t h e words of former D.C. Circuit Judge
Hughes 2005). Indeed, the source of a scien-
(Conrad 2006). Patricia Wald (Sierra Club v. Costle 1981):
tist’s funding may be less a cause of bias than
For example, under Section 4 of the Toxic a n effect of it. As an editor of The Lancet Under our system of government, the very legiti-
Substances Control Act (TSCA 1976), the (Horton 1997) has argued, financial conflicts m a c y of general policymaking performed by
unelected administrators depends in no small part
U.S. EPA has broad power to issue rules order- “may not be [more] influential” than underly-
upon the openness, accessibility, and amenability
i n g persons manufacturing, processing, or ing biases, because “interpretations of scien-
of these officials to the needs of the public from
importing a chemical to conduct further tests tific data will always be refracted through the which their ultimate authority derives, and upon
regarding the chemical’s health or environmen- e x p e r i e n c e s and biases of the authors.” whom their commands must fall. . . . Furthermore,
tal effects. TSCA Section 8(d) authorizes the S i m i l a r l y , a student of the science/policy the importance to effective regulation of continu-
U.S. EPA to compel, by rule, manufacturers interface (Sarewitz 2004) has argued that i n g contact with a regulated industry, other
affected groups, and the public cannot be underes-
and importers of a given chemical to submit stripping out conflicts of interest and ideological
timated. Informal contacts can . . . spur the provi-
lists and copies of existing, unpublished health commitments to look at ‘what the science is really
sion of information which the agency needs.
a n d safety studies for the chemical. TSCA telling us’ can be a meaningless exercise [because]
even the most apparently apolitical, disinterested In addition to setting a single set of quality
mandates have resulted in “more than 50,000
scientist may, by virtue of disciplinary orientation, standards for federally disseminated informa-
studies covering a broad range of health and
view the world in a way that is more amenable to
tion regardless of provenance, the IQA also
ecological endpoints” being filed with the U.S. some value systems than others.
authorizes “affected persons to seek and obtain
EPA since 1976 (U.S. EPA 2003a).
As vom Saal and Hughes argued in their arti- correction of information . . . disseminated by
S i m i l a r l y , the Federal Insecticide,
cle (2005), the appropriate technical response [a federal] agency that does not comply with
F u n g i c i d e , and Rodenticide Act (1972)
when confronted with science conducted by the guidelines issued by [OMB].” Far from
requires any potential pesticide chemical to
interested parties is to use that fact as an alert having their own information judged weighted
u n d e r g o more than 100 s c i e n t i f i c tests
to look, perhaps more deeply than one other- l e s s under the IQA, affected persons are
addressing chemistry, health effects, environ-
wise might have, for “what specific factors, empowered to use their own information to
mental effects, and residue chemistry to deter-
other than source of funding,” may be associ- obtain correction of government information
mine whether it can be used safely (U.S. EPA
ated with the results of that science. t h a t does not meet the common set of
2 0 0 6 c ) . Only after the information has
S u c h an approach is clearly the only standards that should apply to any information
undergone a thorough and rigorous review by
proper one in the case of interested research disseminated by the government.
the U.S. EPA can the product be “registered”
submitted to federal regulatory agencies, sim- Several other administrative law statutes
by the U.S. EPA for use to protect crops or
ply because the concept of “conflict of inter- embody the same orientation toward inter-
public health.
est” is not employed in federal laws governing ested persons and their rights to submit infor-
Most federal processes for evaluating the
the regulatory process (aside from the govern- m a t i o n to federal agencies and have it
safety of chemicals, including the FDA review
ment ethics rules noted above). Indeed, no c o n s i d e r e d . These include the Regulatory
of drug applications (Federal Food, Drug, and
federal laws, rules, or policies express a pre- F l e x i b i l i t y Act (1980), the Paperwork
Cosmetic Act 1938), depend heavily on pri-
sumption that agencies in a given proceeding R e d u c t i o n Act (1980), and the Federal
vately generated data. Indeed, historically and
should ignore or give less weight to scientific Advisory Committee Act (1972). The latter
for the foreseeable future, the vast majority of
w o r k on the basis of who conducted or also requires advisory committees to be “bal-
chemical and product testing has been and will
funded it or, more to the point, whether it anced,” which should equally prohibit exclu-
be borne by industry, not the public sector.
was prepared specifically for the relevant pro- sion of, as well as domination by, any interest.
Policies regarding the treatment of the result-
ceeding. To the contrary, federal administra- The upshot of this authority is not that
ing data need to bear this reality in mind.
t i v e law, as interpreted by the courts, r e g u l a t e d agencies are bound to accept
T h e rights of interested persons under
generally evinces a congressional mandate that unquestioningly any information that is gen-
federal administrative law. The notion that
a g e n c i e s give interested or affected parties e r a t e d specifically for that proceeding.
s c i e n c e generated for regulatory purposes
a c c e s s to and input into administrative A g e n c i e s can and, indeed, must assess the
s h o u l d be evaluated differently than other
139
Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008

5. Henry and Conrad
validity of information upon which they rely. challenges in “establishing quality assurance
also no question that many of the academics
But they cannot adopt blanket approaches practices to improve the reliability, accuracy,
working on these issues are highly invested
that judge science generated for a regulatory and scientific basis of environmental data”
both in their hypotheses and in the regulatory
proceeding differently than other science con- ( U . S . EPA Inspector General 2002). The
uses of their findings. Although these scien-
sidered in that proceeding. Inspector General’s memo expressed similar
tists may not have a financial or other tangible
W h a t sorts of proceedings and entities concerns with respect to the accuracy and reli-
stake in any particular regulatory proceeding,
would be covered. The next hurdle in imagin- ability of information generated by U.S. EPA
it seems artificial and formalistic to say that
ing a system that imposed different standards c o n t r a c t o r s . Critics argued that there was
their research is not being conducted, at least
on science created specifically for a given reg- ample justification for the agency to apply its
i n part, so that its results can be used in
ulatory proceeding is to consider what sorts of proposed assessment factors to that informa-
regulatory proceedings.
agency “proceedings,” and what sorts of enti- tion, as well as to information submitted by
Thus, any effort to establish special rules
ties conducting or sponsoring science, would third parties.
f o r consideration of science generated for
be covered. These turn out not to be simple Critics of the U.S. EPA draft assessment
regulatory proceedings will face difficult defin-
determinations in all cases. factors also argued that assessment factors for
itional challenges regarding what is a “pro-
W h e r e testing is being conducted by a external information created the undesirable
c e e d i n g , ” even more difficult evidentiary
m a n u f a c t u r e r of a chemical or product appearance of a double standard and opened
challenges determining whether and the extent
(e.g., an exposure study conducted in support t h e door to arbitrary agency decisions to
to which scientific work was being conducted
of a pesticide’s reregistration), there is little exclude otherwise appropriate information
for such proceedings, and politically loaded
question that the research is being conducted received from external sources on the basis of
challenges over when “unaffiliated” or “acade-
f o r those proceedings. But manufacturers the selective application of assessment factors
mic” work was in fact being conducted, at
frequently conduct or sponsor research and to information products. Most important,
least in part, for regulatory purposes.
t e s t i n g for product stewardship and other they contended that the standards the agency
Case study: the U.S. EPA assessment fac-
business reasons. In some cases, that work may offered for judging the quality and reliability
t o r s for external information. T h e issues
also be useful in some other agency “proceed- o f third-party data were no different than
raised above ultimately led the U.S. EPA to
ing” (e.g., establishing a “reference concentra- those that should be applied to evaluate infor-
abandon a related effort: to establish guide-
tion” value in the U.S. EPA Integrated Risk mation generated by the agency itself, U.S.
lines that treated “external” information dif-
Information System database (http://www. EPA contractors, or U.S. EPA permittees, and
ferently than information whose generation
epa.gov/iris). Such a proceeding may be ongo- hence a single set of assessment factors should
the U.S. EPA controlled.
ing, or the company may know the agency is apply to all.
Early on, the U.S. EPA realized that the
contemplating it, or the company may plan to When the U.S. EPA finalized the assess-
IQA would apply to information generated
p r o p o s e that the agency initiate it. What ment factors document, it clarified that “the
by third parties that the agency relied upon or
would the rules be in such “mixed-motive” document does not constitute a new standard
otherwise disseminated. The U.S. EPA draft
cases? Conversely, such an agency proceeding for information quality, nor does it describe a
“ A s s e s s m e n t Factors for Evaluating the
may arise later and be truly unanticipated by new process for evaluating third party infor-
Q u a l i t y of Information from External
the regulated entity. What sort of evidentiary m a t i o n . ” Second, and more important, it
Sources” (U.S. EPA 2002b) noted that
process would have to be established to deter- added that, “in general, we agree that consis-
the Agency . . . receives information that is volun-
mine what the research proponent intended or tent standards of quality should apply to both
tarily submitted to EPA by external sources (‘third
knew at the time research was initiated? All internally and externally generated informa-
parties’) in hopes of influencing Agency actions. . . .
The purpose of this document is to describe sets of
t h e s e circumstances would need to be tion, when used for the same purposes” (U.S.
‘assessment factors’ that illustrate the types of con-
addressed in a system that tried to treat “regu- EPA 2003b).
siderations that EPA takes into account when evalu-
latory proceeding” science differently. ating the quality and relevance of information that Conclusion
Moreover, many regulatory settings are is voluntarily submitted or that we obtain from
O n l y one set of standards and practices
more than bilateral; that is, entities other than e x t e r n a l sources in support of various Agency
should be used to judge the quality of scien-
the agency and a single regulated party may actions.
tific work in a given regulatory proceeding,
be able to submit scientific information. Even
The balance of the document consisted of an regardless of why the work was conducted. It
if one accepted the premise that science pre-
e l a b o r a t i o n on five “categories of general may be that, over time, more of these prac-
pared for purposes of a proceeding should be
assessment factors”: soundness, applicability tices and standards will apply to all scientific
treated differently than other science, there is
and utility, clarity and completeness, uncer- information. Many of these hallmarks of sci-
no inherent reason that science prepared for a
t a i n t y and variability, and evaluation and entific quality are incorporated into federal
p r o c e e d i n g by opponents of the permit
review. l a w , rules, and policy. These same federal
should be treated differently than science pre-
Critics argued that there is no basis, under authorities impose additional standards that
pared by its proponents. Indeed, many regu-
t h e IQA or any other legal authority, or further ensure the quality of scientific work
l a t o r y proceedings have multiple parties
i n d e e d , on any technical grounds, for the generated or submitted for regulatory pur-
aligned with and against the agency and other
U.S. EPA to assess “external” or “third-party” poses. Federal laws also ensure that interested
parties on different issues in complex ways,
information by different standards than first- parties have a right to submit information for
making it difficult in many cases to determine
or second-party information. On technical regulatory proceedings and to have that infor-
who is on anyone’s “side.”
grounds, critics noted that information gener- mation considered fairly and on its merits.
Finally, it will often be difficult to demar-
ated by the U.S. EPA or its contractors is not Any system of differential treatment for regu-
cate “nonregulatory” science, given the extent
immune from the same types of errors associ- latory science would face severe scrutiny in
to which academic scientists are participants
ated with information from external sources. light of that authority and would be difficult
or are at least partisans in regulatory disputes.
For example, the U.S. EPA Inspector General to administer. Most important, it would not
Although highly controversial issues generally
had just issued a memorandum noting that n e c e s s a r i l y lead to an improved scientific
raise important intellectual questions (e.g.,
t h e agency faced a number of unresolved foundation for regulations.
the effects of pollutants at low doses), there is
140 116 | NUMBER 1 | January 2008 • Environmental Health Perspectives
VOLUME

6. Science generated for regulatory activities
REFERENCES U . S . EPA (U.S. Environmental Protection Agency). 2002b.
N a t i o n a l Research Council. 1998. Peer Review in the
Developing assessment factors for evaluating the quality of
E n v i r o n m e n t a l Technology Development Programs.
information from external sources. Fed Reg 67:57225–57226.
Washington, DC:National Academies.
ACC (American Chemistry Council). 2006. Long-Range Research
U . S . EPA (U.S. Environmental Protection Agency). 2003a.
National Research Council. 2003. Peer Review in the Department
Initiative—Funding Opportunities. Available: http://www.
O v e r v i e w : Office of Pollution Prevention and Toxics
of Energy—Office of Science and Technology—Interim
uslri.com/home.cfm?id = funding [accessed 16 April 2007].
Programs. Washington, DC: U.S. Environmental Protection
Report. Washington, DC:National Academies.
APA (Administrative Procedure Act). 1946. Public Law 79-404.
A g e n c y . Available: http://www.epa.gov/oppt/pubs/
Nature. 2006. Competing Financial Interests. Available: http://
Anderson W, Parsons B, Rennie D. 2001. Daubert’s backwash:
oppt101c2.pdf [accessed 16 April 2007].
www.nature.com/authors/editorial_policies/competing.html
l i t i g a t i o n - g e n e r a t e d science. Univ Mich J Law Reform
U . S . EPA (U.S. Environmental Protection Agency). 2003b.
[accessed 16 April 2007].
34:619–682.
Clarification of Issues Identified Through Public Input on
O f f i c e of Government Ethics. 1997. Standards of Ethical
Barrow CB, Conrad JW. 2006. Assessing the reliability and credi-
t h e Environmental Protection Agency (EPA) Draft
Conduct for Employees of the Executive Branch. 5 CFR
bility of industry science and scientists. Environ Health
Assessment Factors Document. Copy on file with author.
2635. Washington, DC:U.S. Office of Government Ethics.
Perspect 114:153–155. Available: http://www.ehponline.
U.S. EPA (U.S. Environmental Protection Agency). 2006a. Good
OMB (Office of Management and Budget). 1999. Circular A-110.
org/mem/2005/8417/8417.pdf [accessed 16 April 2007].
Laboratory Practice Standards. 40 CFR 160. Washington,
Uniform administrative requirements for grants and agree-
Conrad, JW. 2006. Open secrets: the widespread availability of
DC:U.S. Environmental Protection Agency.
ments with institutions of higher education, hospitals, and
information about the health and environmental effects of
U.S. EPA. 2006b. Good Laboratory Practice Standards. 40 CFR
other non-profit organizations. Fed Reg 64:54926.
chemicals. Law Contemp Probs 69:141–165.
7 9 2 . Washington, DC:U.S. Environmental Protection
OMB (Office of Management and Budget). 2002. Guidelines for
DHHS/NSF (Department of Health and Human Services/National
Agency.
ensuring and maximizing the quality, objectivity, utility and
Science Foundation). 1995. Investigator financial disclosure
U.S. EPA. 2006c. Pesticides: Regulating Pesticides. Washington,
integrity of information disseminated by federal agencies.
policy. Fed Reg 60:35820.
D C : U . S . Environmental Protection Agency. Available:
Fed Reg 67:8452–8460.
FDA (Food and Drug Administration). 2006. Good Laboratory
http://www.epa.gov/pesticides/regulating/index.htm
OMB (Office of Management and Budget). 2005. Final informa-
Practice Standards. 21 CFR 58. Washington, DC:U.S. Food
[accessed 16 April 2007].
tion quality bulletin for peer review. Fed Reg 70:2664.
and Drug Administration.
U . S . EPA Inspector General. 2002. EPA’s Key Management
Omnibus Appropriations Act. 1998. Public Law 105-277.
Federal Advisory Committee Act. 1972. Public Law 74-220.
Challenges. Washington, DC:U.S. Environmental Protection
Paperwork Reduction Act. 1980. Public Law 96-511.
Federal Food, Drug, and Cosmetic Act. 1938. Ch. 675.
Agency. Available: http://www.epa.gov/oig/reports/2002/
Regulatory Flexibility Act. 1980. Public Law 96-354.
F e d e r a l Insecticide, Fungicide, and Rodenticide Act. 1972.
topchallenges2002.pdf [accessed 4 December 2007].
Sarewitz D. 2004. How science makes environmental contro-
Public Law 92-516.
U . S . EPA Science Advisory Board Environmental Health
versies worse. Environ Science Policy 7:385–403.
FOIA (Freedom of Information Act). 1966. Public Law 89-554.
Committee. 2000. Review of the Draft Report to Congress
S c i e n c e . 2006. Statement on Real or Perceived Conflicts of
GAO (Government Accountability Office). 2001. EPA’s Science
‘Characterization of Date Uncertainty and Variability in IRIS
Interest for Authors. Available: http://www.sciencemag.
Advisory Board Panels: Improved Policies and Procedures
Assessments, Pre-Pilot vs Post-Pilot.’ Washington, DC: U.S.
org/about/authors/prep/coi.pdf [accessed 16 April 2007].
Needed to Ensure Independence and Balance. Washington,
Sierra Club v. Costle. 1981. Case 79-1565, U.S. Court of Appeals Environmental Protection Agency Available: http://www.
DC:Government Accountability Office.
for the D.C. Circuit. epa.gov/sab/pdf/ehcl007.pdf [accessed 16 April 2007].
Horton R. 1997. Conflicts of interest in clinical research: oppro-
SKAPP (Project on Scientific Knowledge and Public Policy). U.S. EPA Science Policy Council. 2000. Peer Review Handbook.
brium or obsession? Lancet 349:1112–1113.
2 0 0 6 . Coronado Conference III. Available: http://www. Washington, DC:U.S. Environmental Protection Agency.
Information Quality Act. 2000. Public Law 106-554.
defendingscience.org/coronado_conference_papers/Coro Vom Saal FS, Hughes C. 2005. An extensive new literature con-
I n t e r n a t i o n a l Life Sciences Institute. 2005. Scientific Peer
nado-Conference-Papers.cfm [accessed 16 April 2007]. cerning low-dose effects of bisphenol A shows the need for
Review to Inform Regulatory Decisions. Washington, DC:
TSCA (Toxic Substances Control Act). 1976. Public Law 94-469. a new risk assessment. Environ Health Perspect 113:
International Life Sciences Institute.
U . S . EPA (U.S. Environmental Protection Agency). 2002a. 926–933.
Krimsky, S. 2005. The funding effect in science and its implica-
G u i d e l i n e s for Ensuring and Maximizing the Quality, Wikipedia. 2007. Scientific Method; Evaluation and Iteration;
tions for the judiciary. J Law Policy 13:43–68.
Objectivity, Utility and Integrity of Information Disseminated C o n f i r m a t i o n . Available: http://en.wikipedia.org/wiki/
National Academies. 2003. Policy on Committee Composition
b y the Environmental Protection Agency. Available: Scientific_method [accessed 16 April 2007].
and Balance and Conflicts of Interest for Committees Used
http://www.epa.gov/quality/informationguidelines/ Wikipedia. 2007b. Peer review; Confirmation. Available: http://
in the Development of Reports. Washington, DC:National
d o c u m e n t s / E P A _ I n f o Q u a l i t y G u i d e l i n e s . p d f [accessed en.wikipedia.org/wiki/Peer_review (accessed 4 December
A c a d e m i e s . Available: http://www.nationalacademies.
16 April 2007]. 2007).
org/|coi/bi-coi_form-0.pdf [accessed 16 April 2007].
141
Environmental Health Perspectives • VOLUME 116 | NUMBER 1 | January 2008