The document discusses various types of validation metrics that can be used to measure the difference between results obtained from a simulation and experimental measurement. It describes several common validation metrics, including classical hypothesis testing, Bayes factor, frequentist's metric, area metric, and various error/correlation metrics such as vector norms, average residual, coefficient of correlation, Sprague and Geers metric, and normalized integral square error. It also provides examples of how validation and correlation metrics can be applied to compare experimental and simulation data.
7. SeoulĀ NationalĀ University ā 7 ā
ErrorĀ metric(orĀ correlationĀ metricĀ )ģ ģ¢ ė„
1. VectorĀ norms
2. AverageĀ residualĀ andĀ ItsĀ StandardĀ Deviation
3. CoefficientĀ ofĀ correlationĀ andĀ crossĀ relation
Limitations:Ā NotĀ ableĀ toĀ distinguishĀ errorĀ dueĀ toĀ phaseĀ fromĀ errorĀ dueĀ toĀ magnitude
Limitations:Ā PositiveĀ andĀ negativeĀ differencesĀ atĀ variousĀ pointĀ mayĀ cancelĀ out
2
1
1
( )
1
N
i
N
R R
S
N
ļ
ļ
ļ½
ļ
ļ„ ( )i iRi a bļ½ ļ
Limitations:Ā SensitiveĀ toĀ phaseĀ difference
NotĀ ableĀ toĀ distinguishĀ errorĀ dueĀ toĀ phaseĀ fromĀ errorĀ dueĀ toĀ magnitude
1 1 1
2 2 2 2
1 1 1 1
( )
( )
( ) ( ) ( ) ( )
N n N n N n
i i n i i n
i i i
N n N n N n N n
i i i n i n
i i i i
N n a b a b
n
N n a a N n b b
ļ²
ļ ļ ļ
ļ« ļ«
ļ½ ļ½ ļ½
ļ ļ ļ ļ
ļ« ļ«
ļ½ ļ½ ļ½ ļ½
ļ ļ
ļ½
ļ ļ ļ ļ
ļ„ ļ„ ļ„
ļ„ ļ„ ļ„ ļ„
*ComparingĀ TimeĀ HistoriesĀ forĀ ValidationĀ ofĀ SimulationĀ Models:Ā ErrorĀ MeasuresĀ andĀ Metrics,Ā H.Sarin,Ā M.Kokkolaras,Ā G.Hulbert,Ā P.Papalambro,Ā S.Barbat,Ā R.āJ.Yang,Ā
JournalĀ ofĀ DynamicĀ Systems,Ā Measurement,Ā andĀ Control(2010)
8. SeoulĀ NationalĀ University ā 8 ā
ErrorĀ metric(orĀ correlationĀ metricĀ )ģ ģ¢ ė„
4. SpragueĀ andĀ Geers metric
5. Russelās errorĀ measure
1
&G
1
cos ( ),AB
S
AA BB
P
ļ¹
ļ° ļ¹ ļ¹
ļ
ļ½
& 1,
2 2
& & &S G S G S GC M Pļ½ ļ«
2 2
1 1 1
, , ,
N N N
i i i i
i i i
AA BB AB
a b a b
N N N
ļ¹ ļ¹ ļ¹ļ½ ļ½ ļ½
ļ½ ļ½ ļ½
ļ„ ļ„ ļ„
Characteristics:Ā PhaseĀ errorĀ portionĀ considered
Limitations:Ā lumped theĀ entireĀ informationĀ intoĀ , ,
MagnitudeĀ :
PhaseĀ :
TotalĀ :Ā
10
( )log (1 )AA BB
R AA BB
AA BB
M sign
ļ¹ ļ¹
ļ¹ ļ¹
ļ¹ ļ¹
ļ
ļ½ ļ ļ«
Characteristics:Ā PhaseĀ errorĀ portionĀ considered
Limitations:Ā lumped theĀ entireĀ informationĀ intoĀ , ,
NoĀ magnitudeĀ error
*ComparingĀ TimeĀ HistoriesĀ forĀ ValidationĀ ofĀ SimulationĀ Models:Ā ErrorĀ MeasuresĀ andĀ Metrics,Ā H.Sarin,Ā M.Kokkolaras,Ā G.Hulbert,Ā P.Papalambro,Ā S.Barbat,Ā R.āJ.Yang,Ā
JournalĀ ofĀ DynamicĀ Systems,Ā Measurement,Ā andĀ Control(2010)