1. Determining Requirements
Compliance During the Design
Phase for a System of Systems
February 10, 2011
Alan Dickey, Wyle Integrated Science and Engineering
Stephen Chan, NASA JSC
Shana McElroy, Booz Allen Hamilton

2. Outline
 Introduction and Background
 Purpose and Objectives
 Methodology Details
 Influencing Design
 Connecting to Verification
 Making it Work for a System of Systems Architecture
- -
Requirements Design Compliance Page 2

3. Introduction – Project Life Cycle Phases
Requirements Development/
Achievability
Focus on establishing
program requirements that Design
demonstrate capability to
meet Agency goals and
objectives Design Qualification
Period of time used to and Verification
define the detailed design
of the system
Focus on developing confidence
that the integrated system will be
able to satisfy technical
requirements
SRR SDR PDR CDR SAR
Key
SRR – System Requirements Review
SDR – System Definition Review
PDR – Preliminary Design Review
CDR – Critical Design Review
SAR – System Acceptance Review
Requirements Design Compliance Page 3

4. Introduction - How Requirements Design Compliance Fits
 The design phase must adeptly
balance the interplay between
hardware/software design and
the requirements while managing
an acceptable level of safety/risk,
all within the context of the
defined operations concepts
Requirements Design Compliance Page 4

5. Introduction - Why Requirements Design Compliance?
 A system of systems is complex and has many variables – making an
objective decision to proceed from one design phase to the next is hard
and Requirements Design Compliance is a systematic method to bring
more objectivity into that decision making process
 Risk can be found early by measuring the requirement achievability of
design concepts with the same measuring stick that will be used in the
final product
 Requirement verification is the finish-line by which we declare success
for product delivery, therefore requirement compliance assessments can
be used to provide periodic dry-runs for verification
NPR 7123 Success Criteria (as it relates to requirements design compliance):
 At PDR: The preliminary design is expected to meet the requirements at an
acceptable level of risk
 At CDR: The detailed design is expected to meet the requirements with
adequate margins at an acceptable level of risk
Requirements Design Compliance Page 5

6. Introduction – Constellation Program Structure
Constellation (Cx)
Program Office
Requirements Design Compliance Page 6

7. Background - Requirement Flow Down and Decomposition
 Requirement decomposition, flow down and traceability of top level
Architecture requirements set the stage for Requirements Design
Compliance
● For Cx, Architecture requirements are implemented through Program and allocated
to Projects (systems), and flowed down to the design implementation level
● All Cx requirements are assigned owners and mapped to stakeholders
Ares 1 Cx
Ex-0010-01 ISS Crew Capacity
CA0426-PO Orion Habitable Volume Orion
Orion
Element
CA0447-PO Orion Low Earth Orbit Crew Capability
CV0085 Orion Automated De-Orbit from Low Earth Orbit
SS-SC-2256 Low Earth Orbit Capability
CV0874 Orion Un-Crewed Flight Configuration
CA0447-PO Orion Automated Rendezvous, Prox Ops, Dock/Undock EVA
Mission Systems
CA5129-PO Mission Systems Crew Training
EVA1067 EVA System Crew Size
CA1000-PO Ares 1 Lift Mass for ISS Missions
Requirements Design Compliance Page 7

8. Requirements Design Compliance Purpose & Objectives
 Purpose of Requirements Design Compliance is to identify and establish
resolution paths for integrated architectural performance/design issues
as early in the design cycle as possible
Proactively manage ‘acceptable level of risk’
Early issue resolution/prevention reduces cost and schedule impacts
End-to-End mission architecture perspective
 Key objectives of Requirements Design Compliance include:
Early identification of design aspects that are at high risk to meet the established
requirements and perform end-to-end
Utilize objective design evidence to determine success of design cycle, from
architecture perspective of the design reference mission (DRM) and operations
concepts (ops con)
Facilitate vertical integration of design compliance data with Projects and horizontal
integration across Program (i.e. provide bigger picture for lower level requirements)
Resolve, report and track significant architecture compliance issues (includes impact
analysis based on lover level non-compliances)
Engage architecture requirement owners in how the design will be verified and
create a relationship between the tools/process for compliance with that used for
verification (‘get their hands dirty’ early)
Connects the beginning (requirements) to the end (verification) and
provides a means to track the health of the architecture and minimize risk
Requirements Design Compliance Page 8

9. Requirements Design Compliance Methodology
Page 9

10. Requirements Design Compliance Methodology -
Issue Handling Processes
Project
Project
Project
Project
Project
Data
DCMs
DCMs
DCMs
DCMs
Requirement Requirement
Stakeholders Reqmt Owners Design
Owners Compliance
Assessments
Architecture/System
Processes
Status used as assessment input
Risk Plans
TPMs
Technology
New issue handling measures created to mitigate new issues
Analyses
Change Request
Requirement Owners use objective data provided by projects, stakeholders,
architecture analyses and issue handling processes to conduct requirement design
compliance assessments as an integrated team/process activity. Issues fed back to
architecture or system issue resolution processes as appropriate.
Requirements Design Compliance Page 10

11. Requirements Design Compliance Methodology, continued
 Requirements Design Compliance relies on objective evidence to identify
issues that could impact verifying the architecture requirements
• Objective evidence helps limit subjectivity when assessing realistic level of risk;
however, objective evidence must be combined with sound judgment – design phase
data may be incomplete and sound judgment is needed to help bridge the gap
• As the Project moves forward through its life cycle, more objective data is necessary to
lead to verification compliance
• Lack of any objective evidence indicative of either non-compliant design or design
behind schedule
 Acceptable objective evidence can be of many types, such as:
• Analysis and Independent Assessment Results – approved and peer reviewed
• Technical Performance Metric (TPM) Reports – e.g., monthly mass status
• Inspection and Demonstration Reports
• Test Reports
• Design Data Deliverables
• Engineering judgment based on historical similarity
 Traceability permits objective evidence to be readily linked to requirements
Objective evidence is key to Requirements Design Compliance but
thinking must stay in the equation
Requirements Design Compliance Page 11

12. Requirements Design Compliance Methodology - Definitions
 Architecture Compliance Definitions
Compliant (Green) – The overall technical design is expected to meet the
requirement at an acceptable level of risk.
Watch (Yellow) –
 The overall technical design does not currently meet the requirement,
but an acceptable mitigation plan has been identified and documented.
 There exists a specific significant threat(s) that requires additional
mitigation plans to be developed – actions assigned and accepted.
Non-Compliant (Red) – The overall technical design is not expected to meet
the requirement and an acceptable mitigation plan has not been identified. In
addition, the requirement has systemic threats that permeates the overall
design compliance.
Acceptable Level of Risk - No known technical issues exist which will impact
meeting the requirement; or, appropriate issue management processes (risk
mitigation plans, Technical Performance Measures, etc.) are in place and
indicate issue(s) will be mitigated to meet program baseline schedule. See
graphic on next chart.
Requirements Design Compliance Page 12

13. Requirements Design Compliance
Methodology - Acceptable Level of Risk
Current The Approved Mitigation Plan – Day 0
kg (1000)* Design Task 1
3.5 Task 2
Level of risk is acceptable if:
• A technically feasible/funded path forward exists to
3.0 provide a compliant design prior to a required date
High • Plan is executed and remains on schedule
2.5 Task 3 Task 4
2.0 Moderate Risk
Task 5
Required Level
1.5 Task 6 Date
Requirement*
1.0
Predicted path forward should be described
Low via a Risk Plan, TPM, or other project plan
0.5
and statused in the appropriate forum.
0.0
Time *Hypothetical requirement
Requirements Design Compliance Page 13

14. Requirements Design Compliance Methodology -
Tools Used
 Requirements
Design Compliance
embedded in
Architecture
requirement
database (Cradle)
 Full linkages
between
requirements and
assessments
 Assessment reports
and requirement
traceability reports
exported to Excel
 Any database/excel
can be used as tool
+
An intuitive, uncomplicated tool which links directly to requirements
database will yield the best results
Requirements Design Compliance Page 14

15. Using Requirements Design Compliance to Influence Design - Example
 Requirement Design Compliance at Ares PDR proved effective at
identifying significant integrated requirement non-compliances
Example 1 – analysis revealed the liftoff
clearance between the integrated stack vehicle
and the launch facility was insufficient and
caused pad re-contact & plume damage
Requirement validation and model refinement (e.g.
cantilever vehicle, thrust misalignment update) did
not resolve issue
Implemented thrust vector control (TVC) on liftoff to
preclude re-contact (e.g. command to counter
deterministic TVC bias, command to bias to the
South)
Example 2 – GN&C team analysis indicated a
roll attitude error build up during First Stage flight
that exceeded the 27 deg requirement
Defined forward work such as evaluating OML
changes to improve aero roll torques, re-working
CFD, and re-examining vehicle performance using
Monte Carlo Flight simulations to gain compliance
Combined hardware (aerodynamic strake) and
software (load relief algorithms) options to optimize
resolution of roll attitude error issue Example from Ares I-X Roll Control System
CFD Analysis
Requirements Design Compliance Page 15

16. Connecting Requirements Design Compliance to Verification
 During PDR phase, verification planning is in early development
● The Requirements Design Compliance effort is separate from verification
planning due to disparate maturity levels between the two efforts
– Verification and requirements compliance have a lot of overlaps
– Requirements compliance helps clarify verification planning aspects
 During CDR phase, verification is in final planning stage
● Requirements design compliance combined with verification pre-work
– Capture objective evidence in same database
– Look for and capture requirements compliance data that could be used as part of
verification closure data
 For example, integrated verification starts at CDR – integrated analyses to be
used for verification are often complete; you validate that the systems are
performing as you expected through qualification
 Requirements design compliance can help identify constraints for
verification work and/or potential operational constraints due to
verification limitations
Assessments supporting Requirements Design Compliance can provide the
integrated analysis for verification, leaving only validation of the system to
be done through qual
Requirements Design Compliance Page 16

17. Making it Work for a System of Systems Architecture
 The integration effort is easily underestimated, with specific
integration needed to
● Agree on roles and responsibilities between architecture and systems
– System data is needed at specific points in time to determine architecture compliance at
major milestones
– Many of the pieces from this effort came from collaboration with system implementers
– Agree on both the what and the how
– Create an overall community of practice focused on overall architecture requirements
design compliance
● Clarify that it isn’t just filling in a database with compliance data – it is a
technical assessment
– Time must be allocated for the technical team members to perform the assessment
● Learn from the different system engineering perspectives in order to
improve the success of the effort
– Ares was first system to be assessed - provided significant insight into what to do and
not do
● Communicate definitions and objectives across all teams
Requirements Design Compliance Page 17

18. Making it Work for a System of Systems Architecture, cont.
 Showing design is compliant to requirements is needed by all systems
to move from one phase to the next – communicate value/purpose at
all levels
 It always takes more time than you think it should – maximize time for
stakeholders and requirement owners assessment
 The earlier in the project lifecycle you start, the easier it is to
implement
● Starting during requirements development/achievability phase using available data
enables full PDR phase implementation and facilitates understanding of
requirements achievability
Requirements Design Compliance Page 18

19. Making it Work for a System of Systems Architecture, cont.
 Simplicity should be a high priority – people/teams get frustrated by
complicated processes and tools
● The objective is not to have a process/tool that does the engineer’s thinking, but
to have a simple enough process/tool that it enables the engineers to think
 Requirement traceability is a factor in the accuracy of the effort
 Acceptable level of risk has different meanings to different people
 It is not a panacea of all integration – it is more like a periodic health
report to catch anything integration missed
● While such a process can be intuitive for a small project, for a large Project you
don’t want to leave it to chance
Requirements Design Compliance Page 19

20. Questions
Requirements Design Compliance Page 20

21. If you would like further information…
Alan Dickey: john.a.dickey@nasa.gov, 281-244-5680
Stephen Chan: stephen.t.chan@nasa.gov, 281-483-1083
Shana McElroy: shana.mcelroy-1@nasa.gov, 281-483-9580
Requirements Design Compliance Page 21

22. Backup
Requirements Design Compliance Page 22

23. Why Integration Is Needed
As proposed by the As specified by the Design as interpreted
customer. Program. by Project A.
Design as interpreted Design as interpreted What was really
by Project B. by Project C. needed.
- -
Requirements Design Compliance Page 23

24. CxP PDR Architecture Metrics - Example
Cx Architecture Requirement Owner Green Yellow Red TOTAL
DIO (Architecture Integration) 2 1 3
Environments & Constraints 7 1 8
Flight Performance 10 2 12
Ground and Mission Operations 17 3 1 21
Human Systems 1 1
Integrated Systems Performance 2 4 6
Integrated Loads, Structures and Mechanisms 4 2 1 7
Integrated Power 1 1
Integrated Thermal/ECLSS 1 2 3
Software and Avionics 16 8 2 26
Safety, Reliability & Quality Assurance 2 2 4
Supportability, Operability, and Affordability 3 3
TOTAL CORE REQUIREMENTS 58 32 5 95
Color Definition
Green Compliant
Yellow Watch items
Red Non-compliant
Requirements Design Compliance Page 24

25. Requirements Design Compliance (RDC) Lifecycle
Customer
Constellation
E&C SIG Program,
NASA
Headquarters
FP SIG
PDR Products
HSIG
Projects ILSM
RIDs SIG Official Path of Projects
GS Issue Resolution
Power GS
Orion SIG
SIG/Project Community Orion
EVA of Practice Efforts Level II
SOA SIG Actions
EVA
Altair
T/ECLSS SIG/Project Community Altair
Ares SIG of Practice Efforts
Level III
PDR Ares
MS RIDs
Analysis Cycles GMO Board
SIG Actions MS
SR&QA
Analyses
Risks
SAVIO
DIO
HTI
Weekly RDC
ASET
Planning Meeting GOAL
To objectively determine if the preliminary design
can be expected to meet requirements to an
acceptable level of risk.
Requirements Design Compliance Page 25

26. NASA Center and Project Locations
Requirements Design Compliance Page 26