The document summarizes the Comprehensive Safety Analysis (CSA) 2010 initiative from the Federal Motor Carrier Safety Administration. CSA 2010 introduces a new Safety Measurement System that uses all roadside inspection violations to evaluate carriers and drivers on seven behavior categories linked to crash risk. It also implements a new intervention process using a range of tools to address unsafe behaviors early. A proposed change would determine safety fitness based on current on-road performance measured monthly rather than after compliance reviews. The new approach aims to improve commercial motor vehicle safety by identifying problems earlier and demanding accountability.
1. Comprehensive Safety Analysis (CSA) 2010 A New Way To Measure and Address Commercial Motor Vehicle Safety OOIDA Briefing October 14, 2009 U.S. Department of Transportation Federal Motor Carrier Safety Administration
20. SafeStat vs SMS Today’s Measurement System: SafeStat CSA 2010 SMS Organized by four broad categories - Safety Evaluation Areas (SEAs): Accident, Driver, Vehicle, and Safety Management Organized by seven specific Behavior Analysis Safety Improvement Categories (BASICs) Identifies carrier for a compliance review (CR) Identifies safety problems to determine who to investigate and where to focus the investigation From roadside inspections, uses only out-of-service (OOS) and moving violations Emphasizes on-road safety performance, using all safety-based road-side inspection violations No impact on safety rating Used to propose adverse safety fitness determination based on carriers’ current on-road safety performance (future) Violations are not weighted based on relationship to crash risk Violations are weighted based on relationship to crash risk Assesses carriers only Two distinct safety measurement systems-one for individual carriers and one for individual commercial motor vehicle (CMV) drivers
21.
22.
23. Example of SafeStat vs SMS The following slides provide examples of key differences between SafeStat and the new SMS |
32. Current vs CSA 2010 Intervention Process Current CR Process CSA 2010 Intervention Process Broad one-size fits-all investigation regardless of extent or scope of safety deficiencies Array of interventions can be tailored to address extent and scope of specific safety deficiencies Resource intensive for agency and time consuming for carrier/fewer carriers contacted Less resource intensive for agency and less time consuming for carrier/more carriers contacted Focuses on broad compliance based on rigid set of acute/critical violations Focuses on improving behaviors that are linked to crash risk Discover what violations exist Discover what safety problem(s) are and why they exist, to facilitate corrective action Major safety problems result in fines (Notice of Claim (NOC)) When problems found, major focus on carrier proving corrective action; significant problems continue to result in fines Focuses on carrier Expands focus to include investigating individual drivers
Why is the U.S. DOT/FMCSA introducing CSA 2010? This graph shows the rate of fatalities, over time, per 100 million vehicle miles travelled. The chart illustrates that the introduction of the agency’s new safety programs over the years have had a positive impact on safety. Because the rate has flattened over the past several years, FMCSA, along with its state partners, is introducing CSA 2010 as a pro-active measure to further reduce the fatality rate, building upon its past success with introducing innovative safety programs
The leveling fatality rate motivated FMCSA to look carefully at its operational model. The agency found that: While the CR is an effective tool, it is labor intensive. In addition, safety ratings are tied to the labor intensive CR. With more vehicles on the road while enforcement resources remain stable, the need for more efficient tools became clear. FMCSA recognizes that many carriers have good records and effective safety programs and is introducing CSA 2010 as a way to better identify and address unsafe carriers.
What is CSA 2010 and what are the benefits? The program leverages the use of technology by using safety performance data collected at roadside and through crash reports. It uses that data to identify unsafe carriers and drivers and to pinpoint their safety performance problems. The wider array of tools enables FMCSA and state partners to employ the right tool to address the identified safety problems, resulting in more efficiency for both the agency and the carrier It is better for carriers as less time will be taken from their business and better for FMCSA and state partners as they will be able to reach more carriers. The CR is effective but is not always necessary when the safety problems identified are limited in nature. CSA 2010 takes the best of the CR process, enhances it, and supplements it with other tools to accomplish what’s needed in a less intrusive, more focused fashion (a comprehensive tool , similar to the CR, is still available for use to address the worst performers/highest risk carriers) By alerting carriers early that they are on FMCSA’s radar, the new model provides an opportunity to carriers to fix safety problems before they grow and to get off of the radar quickly without the expense of a CR. Because the data is updated monthly, the new model requires sustained accountability for safe performance for both carriers and drivers and gives carriers a tool to identify and address unsafe drivers.
CSA 2010 is a pro-active initiative to improve the efficiency and effectiveness of FMCSA’s enforcement and compliance program. Uses ALL roadside inspection results and crash reports to identify safety deficiencies Employs a wider array of interventions tailored to problems instead of solely the time-intensive Compliance Review process Enables more carriers to be contacted earlier Requires sustained accountability of carriers AND increases accountability of drivers
Is it working as designed? The new model is being tested in 6 states and early results are promising. These will be discussed in more detail later in the presentation. Carrier reaction has been generally positive according to many investigators involved with the new program, through feedback collected through the website, as well as through interviews conducted with some test state carriers.
This is a diagram of the new operational model once it is fully deployed (post Safety Fitness Determination Rulemaking.) Beginning on the left, the model shows how safety violations from roadside inspections and crash data from crash reports feed into MCMIS (FMCSA’s collection system of data) The raw data is then measured (green Measurement box) and carrier performance is measured in 7 BASIC categories – Behavior Analysis and Safety Improvement Categories (BASICs) as listed in the box. In turn, the measures are used for Safety Evaluation in two ways (green Safety Evaluation box). First, from a policy perspective an evaluation is made to determine intervention selection (see arrows and yellow interventions box). Second, to the right of the traffic lights in the box, the measurements may be used in the future to determine safety fitness (in rulemaking) .
The three main components of the new approach to CMV safety address HOW we are going to extend our reach beyond the population of carriers we currently address The Safety Measurement System is a new methodology – for OpModel Test States, results are available to industry in the Comprehensive Safety Information (CSI) system. For national roll out in the summer of 2010, results will be available to industry in a more powerful, nation-wide system. The proposed approach to Safety Fitness Determination will be in rulemaking in the fall of 2009. The proposed rule ties fitness to current on-road safety performance with a rating not limited to acute/critical regulation violations from a CR. The new intervention process provides an array of tools that can be used depending on the specific safety problems of the carrier . This includes an “early warning’ in the form of a warning letter.
In CSA 2010, the focus is on identifying and then changing unsafe behavior with SMS serving as the foundation. SMS measures on-road safety performance of carriers in Behavior Analysis and Safety Improvement Categories SMS enables FMCSA and its state partners to identify high risk carriers and drivers and determine their safety deficiencies more specifically. All safety-based violations found at roadside are taken into account and each violation is assigned a weight based on its relationship to crash risk. SMS organizes violations by unsafe behaviors (BASICs) that can cause crashes FAQs Who will have access to this data? During the OpModel test, only carriers have access to their information. It is expected that for national roll out, level of availability will be similar to that of SafeStat today. Does crash preventability factor into anything? Will the new system improve crash data or will industry be assessed on same data? FMCSA has this on its radar and a team working on the issue. As results come in and decisions are made, information will be updated for industry.
This slide lists the seven Behavior Analysis Safety Improvement Categories The methodology is designed to weight on-road safety data based on its relationship to crash risk and focuses on behaviors that lead to crash risk. The data is also time-weighted over a 24 month time period so that it is reflective of current on-road safety performance. If a carrier’s performance improves over the time, the safety performance score improves. HM regulation violations (171, 172, 173, 177, 178, 180) may also be found /included in other BASICs such as driver fitness, but the most concentrated BASIC for these is Cargo Related so they are listed there.
Key Message : Overview of SMS methodology Timing : 3 min Talking Points / Interactivity : We take all the data we have in our data system – MCMIS – and distribute into the “right buckets” We take on-road data and determine crash risk. A severity risk is assigned. After we get the data, we weigh the data and give the carrier a percentile based on its peers. For example, how does ABC trucking stand in with other carriers that have a fatigue BASIC? BASIC measures is “how much does this bucket weighs now?” The percentile tells us how the “bucket” compares to the carrier’s peers. Background Information : Materials :
Key Message : Safety event data is different for Carriers vs. Drivers Timing : 1 min Talking Points / Interactivity : Here we describe Safety Events. On the carrier side – we look at a 2 year window. We also look at interstate carriers On the driver side – we look at 3 years of history Background Information : Materials :
Key Message : Safety event data is sorted by BASIC Timing : 2 min (start approx 15 mins after webinar start time) Talking Points / Interactivity : You’ve seen this information – but here again you can see the data gets placed in different categories or “buckets” BASIC Data is our second step. Briefly reviews what each BASIC means… Background Information : Materials :
Key Message : BASIC data is converted in quantifiable measures Timing : 3 min Talking Points / Interactivity : After we get the data in the appropriate bucket – we weight the data. Several things are considered: Time weighting; Severity weighting; Normalizing – violations vs. power units; Single inspection cap – limits weight of a single violation - especially important to small carriers; Violation cap – helps establish uniformity and consistency Now we can see “how bad” a particular carrier is Some considerations we make: Time weighting – we focus on more recent events Severity – this is a “risk” weight Normalizing – this is a denominator for “exposure” – usually this is the number of inspections. Two different caps: Single Inspection and Violation Background Information : Materials :
Key Message : How a BASIC measure is calculated for Unsafe Driving Timing : 1 min Talking Points / Interactivity : Now for each BASIC – we will go through how each is measured. This is a similar construct for other BASICS The considerations are time and severity weight – and the number of power units. Background Information : Materials :
This slide highlights the differences between SafeStat and SMS SMS Applies risk-based weightings to violations in order to identify high crash-risk carriers demonstrating PATTERNS of unsafe behavior roadside SMS Matches poor safety performance with appropriate level of intervention SMS Uses all SAFETY BASED roadside data, allowing more carriers with unsafe behavior to be identified for intervention SMS Includes two new Safety Measurement Systems (SMS) – one for carriers (CSMS) and one for individual CMV drivers (DSMS).
This slide provides information about the Driver Safety Measurement System. The bullets on the slide answer most of the FAQs from industry about the driver part of the new model.
The following FAQ addresses the question more completely. What information is FMCSA making available to carriers to assess drivers’ safety performance? FMCSA will be making available “Driver Profiles” that contain roadside inspection and crash histories for an individual driver. Coming in summer 2009, the Commercial Drivers Pre-Employment Screening Program (PSP) will provide individual drivers' safety performance histories with their previous employers. Under the program, a driver would authorize release of this information, allowing FMCSA to provide, through a third-party contractor, this important driver data. Drivers will also be able to obtain their own crash and inspection file through the third party or through a FOIA request with FMCSA. NOTE: The PSP is not a component of the CSA 2010 program
The results seen in the following slides will likely be displayed differently upon national roll out but substantively , these screens show the comparison of SafeStat to SMS scores as well as information available in DSMS.
The next four slides show how the Carrier SMS is different from SafeStat The example is based on REAL DATA although the carrier information is blacked out. The Carrier in SafeStat is flying under the radar with no SEA values above 75. The safety rating here as of 8/2005 is satisfactory. The carrier has held this rating (the agency’s highest rating) and it’s been out in public, for several years. However, moving to the next slide, you will see that the carrier has very different results when measured using current on-road safety performance data provided by SMS.
This slide shows the carrier who was under the radar is SAFESTAT, is shown under SMS to have a serious safety deficiency related to Driver Fitness; the early identification of the problem is good news for both the carrier and the public The carrier’s rating is worse than 99% of the carriers evaluated in this BASIC in this peer group. The next screen shows the detailed information resulting in that percentile. From a carrier’s perspective, as well as the public’s perspective, identifying this safety problem early and putting actions in place to fix it before a crash occurs, increases safety for everyone, and in the long run saves the carrier money, the government time, and most important, saves lives. Such specific safety deficiencies may not warrant a full CR –but may indicate the need for a focused review which in turn can save carriers time while allowing the agency to effectively correct the specific safety problem
So why is this carrier’s Driver Fitness score so high? This shows that it has multiple occurrences of no and expired medical certificates violations Of the 300 inspections related to drivers, 66 inspections have resulted in a violation of the Driver Fitness BASIC SMS is identifying a pattern of behavior across multiple inspections
SMS Facilitates Problem Identification The new SMS website allows us to drill down to see further details related to this high driver fitness BASIC. It is clear that various drivers are having the same medical certificate issues, and these reports are coming in on multiple drivers from multiple states… the problem is not limited to one problem driver and indicates a process breakdown that the carrier is now in a position to fix.
Currently, only Carrier SMS data will be available in summer 2010 – driver SMS results (meaning percentiles) will not be available to industry (however, as mentioned earlier through the PSP program, FMCSA will make inspection/crash data available to carriers with driver release.)
New Interventions Process provides more tools to reach more carriers and influence safety compliance before crashes occur. SMS alerts FMCSA agents when an intervention is needed and recommends the appropriate type based on the safety problems. The focus of the new process is on changing behavior – For example: If a carrier has a particular problem, SIs will now look at WHY that carrier has that problem providing carriers insight and guidance to take corrective action. In the new model, FMCSA works with carriers to take real corrective action, while NOCs continue to be an important part of the process. SIs will use the Safety Management Cycle to walk carriers through their operations and to identify process breakdowns likely to result in safety problems. Safety Improvement Resources (SIRs) are a part of the new model and used to guide carriers in improving their operations.
This graphic illustrates the Safety Management Cycle that is used to help carriers understand WHY a carrier is having a problem and HOW to fix it. SIs use the Safety Management Cycle to walk carriers through their operations and to identify process breakdowns likely to result in safety problems. Safety Improvement Resources (SIRs) are a part of the new model and used to guide carriers in improving their operations.
This slide lists the interventions in increasing order of severity. While a progressive process is in place a carrier will enter the process anywhere from WL to on-site comprehensive depending on the nature and severity of the problems. Warning letters are a new intervention, though some states do send PRISM warning letters. The CSA 2010 warning letters are sent to a larger number of carriers, nationwide. This is likely to be some carriers’ first contact with FMCSA enforcement. It is a strong warning that the carrier is now on FMCSA’s radar and must improve its safety practices and results to get off of the radar. If a carrier continues with poor performance, it will be identified for an investigation. Off site investigations are a NEW intervention. They are used when a carrier passes the threshold of certain BASICs but an on-site intervention is not yet dictated. This intervention allows carriers and SIs to work together over the phone and computer to conduct the investigation and provide results. If a carrier doesn’t improve with this intervention, it will be identified for an focused or comprehensive on-site. On-site focused investigations are a NEW intervention. occur at a carrier’s place of business and are focused on specific problems. On-site comprehensive investigations are most similar to CRs. Cooperative Safety Plans are a new intervention. They are voluntary and may be used in conjunction with an NOV. They are NEVER used in lieu of an NOC. If an NOC is required, it is given. NOVs are not a new intervention but have a new focus in CSA 2010. This is the only follow up action that compels a carrier to do a corrective plan. NOCs and OOSs are the same as today’s NOCs and OOSs.
This slide compares the current CR process with the new CSA 2010 Intervention Process. CSA 2010 provides a set of tools is chosen based on a carrier’s specific safety problems with t he severity of intervention based on safety performance CSA 2010 Intervention includes finding out why a carrier has a specific safety problem and then focuses on facilitating corrective action. The CSA 2010 Intervention Process will continue to use comprehensive on-site investigations for the highest crash-risk carriers FMCSA agents work with the carriers in a systematic way to identify problems and offer solutions, while still serving as a strong enforcement presence.
Below is a history of the Operational Model Field Test Began as 30-month field test in 4 States: Colorado, Georgia, Missouri, New Jersey Randomly divided into control (34,493) and test (34,820) groups ~ 50%/50% Phased implementation of field test: Phase I Startup: Feb-08 to Sept-08 3 BASICS; emphasized off-site investigations; no A/B (high risk) carriers Phase II: Oct-08 to Jun-10 Fully operational – all BASICs, all interventions, and issuing safety ratings Additional states added at 100% (as opposed to 50/50%) May 2009: MT, MN Fall 2009: Two to four new states 100% of the State participates in CSA 2010 Offers a more accurate picture of efficiencies, capabilities and benefits Tests integration with national program goals and Congressional mandates Provides more data to evaluate test including workload and workforce analyses
Additional states selected based on Good CR programs (state and fed) Willing to accept the challenge Existing strong relationships (HQ and field, fed and state) 2 border states Throughout test, participant states were seeking 100% involvement: additional states were also requesting participation, but we needed to retain our original test design for evaluation purposes. Therefore, 100% in new states 100% offers are clearer picture of what implementation / steady state will be – with respect to efficiency, workforce and workload and integration with national program/ congressional mandates ****************** More accurate picture of operational model’s capabilities and benefits - efficiency Fully test integration of the OpModel with National program goals and Congressional Mandates More data to evaluate test including workload and workforce analyses EFFCICIENCY Losing efficiency: incur travel costs and hours to send a “test” SI to conduct an investigation on a “test” carrier when a control group SI is in the vicinity; IM loses efficiency because they are managing two separate workforces and workloads (test vs. control); NATIONAL GOALS Allows FMCSA to more fully test the efficiency and effectiveness capabilities that can be gained by a State when applying the model to the full array of carriers that must be addressed to meet program goals and mandates; allows FMCSA to test policies and procedures for carrier’s that are currently excluded from the test MORE DATA Increase in data collection with more investigators and carriers Provides FMCSA with a better picture of management and oversight requirements at the Division level (increased workload); better picture of potential increased case workload at the Service Center level
Data from Phase I & II: 10/01/08 – 6/30/09 (Colorado, Georgia, Missouri, New Jersey)
SFD increases the impact and effectiveness of the CSA 2010 program.
In summary… Roadside inspections discover and document violations and provide the basis for intervention and enforcement (as well as capturing good data that can raise a safety score) Interventions attempt to change the unsafe behavior indicated by the violations; data from interventions are also captured on a carrier Proposed SFD linked to continually collected and assessed data collected at roadside inspections, as well as crash reports, and not limited to rating via CR results Less resource intensive options allow for greater number of interventions In conclusion, the greater number of interventions (and eventual SFD) results in the agency reaching more carriers and will lead to increased safety, while decreasing the amount of disruption to carriers who are investigated. This program is good for good carriers; these components help to establish an equal playing field for industry.
![Presentation Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],|](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)