This presentation talks about the importance and techniques associated to validating and verifying models, more specifically simulation models.

1. Validation and Verification
Adapted from Jerry
Banks

2. Verification
 Concerned with building the model
right
 Comparison of conceptual model
and computer representation
 Is the model implemented correctly
in the computer?
 Are the inputs and logical
parameters represented properly?

3. Validation
 Concerned with building the right
model
 Accurate representation of the real
system
 This is achieved through the
calibration of the model
 Iterative process until accuracy is
acceptable

4. Model
Building,
Verification,
and
Validation
REAL SYSTEM
Conceptual Model
1Assumptions on system components
2Structural Assumptions (defines the interactions
between the system components)
3Input parameters and data assumptions
Operational Model
(Computer
Representation)
Conceptual
Validation
Model
Verification
Calibration and
Validation

5. Common sense suggestions
for verification
 Have someone check the
computerized model
 Make a flow diagram (with logical
actions for each possible event)
 Examine model output for
reasonableness
 Print the input parameters at the
end of the simulation

6. Common sense suggestions
for verification
 Make the computerized
representation as self documenting
as possible
 If animated, verify what is seen
 Use IRC or debuggers
 Use graphical interface

7. Three Classes of Techniques
for Verification
 Common sense techniques
 Thorough documentation
 Traces

8. Calibration and Validation
 Validation is the overall process of
comparing the model and its
behavior to the real system and its
behavior
 Calibration is the iterative process of
comparing the model to the real
system and making adjustments to
the model, and so on.

9. Iterative Process of
Calibration
REAL SYSTEM
Initial Model
Second
Revision of
Model
First Revision of
Model
Compare Model to
Reality
Compare Revised
Model to Reality
Compare second
Revised Model to
Reality

10. 3 Step Approach by Naylor
and Finger (1967)
 Build a model with high face validity
 Validate model assumptions
 Compare the model input-output
transformations to corresponding
input-output transformations of the
real system

11. Possible validation techniques in
order of increasing cost-value
ratio by Van Horn (1971)
 High face validity. Use previous research/
studies/observation/experience
 Conduct statistical test for data
homogeneity, randomness, and goodness
of fit test
 Conduct Turing test. Have a group of
experts compare model output versus
system output and detect the difference
 Compare model output to system output
using statistical tests

12. Possible validation techniques in
order of increasing cost-value
ratio by Van Horn (1971)
 After model development, collect
new data and apply previous 3 tests
 Build a new system or redesign the
old one based on simulation results
and use this data to validate the
model
 Do little or no validation. Implement
results without validating