Expert judgment-based fragility functions to better characterize physical vulnerability worldwide

1. Expert judgment-based fragility functions to better characterize physical vulnerability worldwide
Kishor Jaiswal, Ph.D., P.E.
USGS/SynergeticsInc. 2014 EERI Housner Fellow
Golden CO

2. GEM Vulnerability Estimation Methods (GEMVEM)
Empirical
Analytical
Expert- Judgment
Empirical- National
USGS Role:

Contributing Empirical-National Vulnerability Models: Fatality & Economic loss models

Developing protocols and guidelines on Expert Judgment-based Vulnerability Estimation Methods

Collaborating with GEMGVC Team in developing Seismic Vulnerability Database & Mapping Schemes

3. Background
To understand and quantify the physical vulnerability of built infrastructure, we need a suite of vulnerability/fragility models
‣
“There is a rich literature of seismic vulnerability information, only exceeded by the vast need for more” (Porter et al., 2010).
‣
Expert judgments are essential for assessing vulnerability of building types for which the empirical data are lacking and the physical models have limited predictive capabilities.
‣
Expert judgments have been used in the past for assessing seismic vulnerability (ATC 13, 1985 survey used the modified Delphi process).
‣
We plan to use structured the elicitation procedure proposed by Cooke (1981) for eliciting judgments on seismic collapse fragility of global building types.

4. Cooke’s Approach (1991)
‣
It is astructured elicitation process
‣
Experts are treated as a statistical hypothesis
‣
The approach is subject to:
‒
Scrutiny/accountability [data/process open]
‒
Empirical control [quantitative, subject to empirical quality control]
‒
Neutrality[encourage experts to provide their true beliefs, and help avoid bias]
‒
Fairness[expertise is not pre- judged]
Credit: Aspinall2008
Cooke’s approach uses ‘performance-based’ scoring system instead of the traditional ‘self-rating’ and ‘consensus building’ approaches used in the Delphi procedure.

5. Calibration Score:
C(e) = 1 -χB2 [ Σ 2 * nb* I(sb , pb) ]
Average Response Entropy: (Inverse of entropy is called Information Score)
He(n) = [Σ nb* H(pb) ] / N where H(p)=-Σp *ln(p)
UnNormalized Weight:
w’eα= Ια[C(e)] * C(e) / He(n)
Let us assume that we have expert ‘e’who assesses the probability of each uncertain event by assigning a corresponding indicator function Ito one of Bprobability bins. Here each probability bin is associated with a distribution, and the term Pbindicates the occurrence probability Pof bin b.
Let nbdenote the number of variables assigned to bin b, and sbdenote the
sample distribution of variables in bin b, where b=1,…Bbins.
The term I(s,p)provides relative information of s with respect to p is given as
I(s, p) = Σ s * ln(s / p)
Performance-based Scoring Metric…

6. Key Aspects:
Seed Questions
‣
These are the questions for which the answers are already known to the facilitator
‣
Experts’ answers to seed questions are used to estimate the calibration and information scores
‣
Performance measured using both score is then combined to estimate the weights
Target Questions
‣
The answers to target questions are unknown, and thus the participating experts help to derive them collectively
‣
Each expert’s answers are weighted according to the
weights he/she received
‣
Multiple judgments are combined to provide estimate for an unknown quantity and to quantify associated uncertainties

7. Scoring approach:
Calibration Score
‣
Impetus is given on measuring the ‘statistical accuracy’ associated with elicited quantiles
‣
Penalizes heavily for larger deviations/mismatch
Entropy Score
‣
Measures the degree with which the elicited distribution is concentrated with respect to some background distributions
‣
Penalizes for presenting estimates with relatively poorer information

8. Summary: Deriving Generic Building Collapse Fragility Models
•
Supported by GEM Foundation
•
Focus on generic global construction typesthat are common in earthquake prone countries
•
Thirteenexperts from eight countries participated in the exercise
•
Construction types included:

Unreinforced masonry –6 Types

Concrete frame–6 Types
•
Supported by USGS, GEM and EERI
•
Focus on selected U.S. construction types that are deemed to be vulnerable
•
Twelveexperts from U.S. and onefrom Canada participated in the exercise
•
Construction types included:

Weak first story wood frame –2 Types

Unreinforced masonry (retrofitted and unretrofitted)–3 Types

Concrete types–20 Types
Lisbon Workshop [Sept 23, 2012]
Oakland Workshop [May 8, 2013]

9. Sample Responses…
Typical Response to
One of the Seed Question:
Typical Response to
One of the Target Question:

10. Generation of a Collapse Fragility Curve…
Collapse fragility estimate of seismically retrofitted unreinforced masonry
bearing wall construction

11. Summary
‣
Expert elicitation requires careful deliberation (e.g., recruiting experts, planning, preparation notes, survey instruments)
‣
Cooke’s method rewarded heavily to the experts who performed well in calibration exercise, but then it penalized significantly to the experts who failed to do so.
‣
The exercise resulted in generating dozens of generic seismic collapse fragility functionsto be used within the Global Earthquake Model (GEM)’s OpenQuakeengine as well as within the PAGER’s semi-empirical model
‣
Further research is necessary in harmonizing the influence that ultimately yielded in estimating the final score, i.e., scoring based on being statistically accurate vs. being a real expert who is not so good in quantifying answers in probabilistic sense.
‣
Collapse fragility results will be made available through the GEM Nexus and the USGS PAGER website.

12. Acknowledgements:
‣
Participating Experts:
Lisbon Workshop: Daniel Abrams, Edmund Booth, Michele Calvi, Dina D’Ayala, Ken Elwood, PolatGulkan, Michael Griffith, Jason Ingham, Sergio Lagomarsinho, Paulo Lourenco, Guido Magenes, David Mar, and Ahmed Yakut
Oakland Workshop: David Bonowitz, Kelly Cobeen, Craig Comartin, Greg Deirlein, Jim Harris, John Hooper, Charlie Kircher, Greg Kingsley, Bret Lizundia, Jay Love, Joe Maffei, Brian McDonald, John Sherstobitoff
‣
GEM Foundation:
‒
RuiPinhoand Helen Crowley
‣
Earthquake Engineering Research Institute:
‒
Jay Berger, Marjorie Greene, Maggie Ortiz, Jonathon Tai, Christopher Lee, Patrick Bassal
‣
University of Bristol, UK:
‒
Willy P. Aspinall
‣
U.S. Geological Survey:
‒
David Perkins, David Wald, and NicoLuco
‣
Designing of the Survey Questions:
Willy Aspinall, DimitriosVamvatsikos, Keith Porter, Bill Holmes, NicoLuco, AbbieLiel, David Perkins, Helen Crowley, and SiamakSattar

13. Except where otherwise noted, thiswork is licensed under:
creativecommons.org/licenses/by-nc-nd/4.0/
Please attribute to the GEM Foundation with a link to - www.globalearthquakemodel.org