2. About Baltimore
• Form of Government: Strong Mayor
• Land Area: 81 square miles
• Population: 622,793 residents
• FY16 Operating Budget: $2.7 billion
• FY16 Capital Budget: $672.9 million
• FY16 Full Time Positions: 14,486
2
5. Baltimore CitiStat
“A leadership strategy that a mayor
can employ to mobilize city agencies
to produce specific results”
– Robert Behn, Harvard’s Kennedy School of Government
Tenets:
• Accurate and timely intelligence shared by all
• Rapid deployment of resources
• Effective tactics and strategies
• Relentless follow-up and assessment
5
6. •Regular Meetings with Mayor, senior
staff, and agency leadership.
•Data “What gets measured gets done.”
•Follow up on decisions made in the
room.
6
Getting Started : Structure
7. • Room for all key participants
• Lean start-up costs (low-tech)
• Dedicated analysts
Getting Started : The Room
7
8. Getting Started : Measurement
What did we already collect?
• Personnel and Financial data
• Reports to State, Federal, and private entities
• Internal management measures
Identify core service goals
• What is “doing a good job” in your field?
8
9. Performance Measures
9
1. How much did we do?
Who are our customers and what services do we
provide to them?
2. How well did we do it?
How well do we provide those services?
3. Is anyone better off?
What is the desired impact of those services for/on
our customers (a.k.a. “customer results”)?
10. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Regular Sources of Data
• Motorola 311 CitiTrack
Report and Citizen Survey
• Agency Templates
• Field Investigations
10
11. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
311 System
11
12. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
311 Online & Mobile App
12
13. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
311 & CitiTrack Report
WW-Surface Repair (Asphalt)
WW-Surface Repair (Stripings)
WW-Surface Repair/Stripings
Pending
WW Water Meter Cover
Missing/Damaged
WW Water Meter Leak
WW-Locate Meter
WW-Meter Reader Skipped
WW-Water Meter Installation
WW-Water Turn Off/On (Bill)
WW Storm Damaged Inlet
WW Storm Inlet Choke
WW-Storm Connection Choke
WW-Storm Damaged Inlet
Connection
WW-Storm Mainline Choke
WW-Storm Mainline Dig
WW-Storm Manhole/Grate
Replacement
WW-Storm Manholes Clean
WW-Storm Manholes Repair
WW-Storm Oil Spill
WW-Storm Routine Inlet Cleaning -
Proactive
WW-Surface Repair (Masonry)
WW Water Billing Audit
WW Water Senior Citizen Discount
WW-Delinquent Water Service Turn-
Off
13
14. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
CitiTrack Report
Department of Recreation and Parks - Statistic Summary Report
Created on Jul 22, 2013 7:59:50 AM
Date Range : From Jul 6, 2013 To Jul 19, 2013
Group SR Type Total Created Open Open % OpenTim
e
Overdue Overdue
%
Closed Closed
%
ClosedTi
me
Closedon
Time
Closedon
Time%
Internal
Duration
Days
Department of
Recreation and
Parks
RP-Citizen Complaint of
Employees
12 4 8 66.7% 87.6 5 62.5% 4 33.3% 31.2 1 25.0% 15
RP-Compliment of Employee 20 0 20 100.0% 2,292.9 10 50.0% 0 0 0 0 0 15
Department of Recreation and Parks 32 4 28 87.5% 1,190.3 15 53.6% 4 12.5% 31.2 1 25.0%
Forestry Forestry Blocked Traffic
Sign / Light
24 2 23 95.8% 41.9 21 91.3% 1 4.2% 17.9 0 0 14
Forestry Broken Branch in
Tree
26 13 21 80.8% 15.2 19 90.5% 5 19.2% 10.4 1 20.0% 1
Forestry Down Tree 111 90 75 67.6% 9.3 64 85.3% 36 32.4% 4.8 9 25.0% 1
Forestry Fallen Limb 160 126 93 58.1% 8.4 71 76.3% 67 41.9% 4.5 14 20.9% 1
Forestry Maintenance
Pruning
7,710 88 7,692 99.8% 3,672.4 6,252 81.3% 18 0.2% 1,720.9 12 66.7% 731
Forestry Priority Pruning 103 28 102 99.0% 66.7 6 5.9% 1 1.0% 27.2 1 100.0% 300
Forestry Stump Removal 2,592 97 2,590 99.9% 477.6 1,728 66.7% 2 0.1% 31.1 2 100.0% 180
Forestry Tree Inspection 1,256 304 1,098 87.4% 38.6 633 57.7% 158 12.6% 26.6 99 62.7% 30
Forestry Tree Removal 750 47 615 82.0% 220.2 268 43.6% 135 18.0% 322.6 20 14.8% 180
Forestry 12,732 795 12,309 96.7% 505.6 9,062 73.6% 423 3.3% 240.7 158 37.4%
14
15. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Dashboards from CitiTrack Report
15
16. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
311 Survey Results
16
17. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
CitiStat Template – Refuse Collection
17
18. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Field Investigations
18
19. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Former Special Events Permitting Process
19
20. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Current Special Events Permitting Process
20
21. Ac
14. Purchasing
reviews BOE approval
letter and puts it on
agenda
1. At the end of
production year Fleet
works with agencies to
determine needs
5. BBMR
approves
3. DGS
Accountant
enters request
into CitiBuy
2. Specs for each
component of bid created.
Volume needed estimated
7. Draft RFB is
reviewed by
purchasing and
Fleet
8. RFB released
and formally
Advertised
10. Comptroller
accepts bids
Wed @ 11AM
11.
Purchasing
picks bids up
12. Bids
brought to
Fleet
13. Fleet and Purchasing
review lowest bidder to
ensure conforms to specs
and BOE approval letter
created
19. Law signs off
and lets purchasing
know to create the
purchase order
18. Vendors
signs and
submits
certificate of
insurance
17. Contract Admin. sends
a letter and an agreement
to winner to legally bind
them to terms
15. BOE Approves
(wed mornings @
9:00AM)
21A. Agency
submits city buy
release
22. Release should
pass through
Specifications
Supervisor for approval
23. Approved by
Supervisor
24. Assigned to
Buyer to submit
purchase order
25. Vendor
places order
26. Vehicles
made to order
Brian takes all of the info off
the vendor delivery
information and puts it on
the vehicle sheet
Deliver
30. Sheet sent to
Specifications
Supervisor for
Fleet # and class
code entered
manually
33.Pre-Delivery
Inspection (PDI)
35. Specs. Supervisor
Creates rental
agreement and
emails agency
32. Office Assistant II enters
all of this information into
FASTER and then goes to
MVA to obtain Tag and Title
36. Quality Control
Issues Vehicle to
agency
6. Buyer Creates
the Request for
Bid (RFB)
4. Internal CitiBuy
approval (6 sigs.)
27. Delivered
28. Lead Mechanic
takes all of the info
off the vendor
delivery information
and puts it on the
vehicle sheet
31. Program Analyst
confirms that FASTER
codes and class codes are
assigned properly
21. Fleet Specifications
Supervisor submits city
buy release
9. Vendors
Submit Bids
DGS
Finance
Comptroller/
B.O.E
Law
Vendor
Agency
Car
Code?
Code 1
Code 2
Vehicle Purchasing Process – Blanket Purchase Order
(~90% of vehicle purchases)
34. Vehicle
Upfitting
29. Acct.
Supervisor
adds/reviews
billing
information
July – September
16.
Purchasing
picks up
approvals
From Box # 6
to 19 , target is
90 Days
60 – 120 Days
~7 Days1-2 Days
7 Days
1-2 Days7 - 10 Days 7 - 14 Days
2 - 28 Days
14 Days
10 – 14 Days
???1-2 Days
1-2 Days
14 - 56 Days
1 – 2 Days
270 Days
to
Purchase
Vehicle
CitiStat Tenet 3: Effective Tactics and Strategies
Fixing Internal Processes
21
22. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
CitiStat Memos
List of highlighted topics
Dashboard showing recent
core service performance
22
23. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Charts: Many ways of visualizing an issue
Crew comparisons, year-over-year growth, or progress towards a goal.
23
24. CitiStat Tenet 1: Accurate and Timely Intelligence Shared by All
Maps identifying problem locations
24
33. OutcomeStat Background
OutcomeStat is based on the principles of
Results-Based Accountability as laid out in
Mark Friedman’s book Trying Hard Is Not
Good Enough.
33
34. OucomeStat Goals
Create a strategic plan for the City of Baltimore structured
around the measurable indicators for each of the Mayor’s
seven Priority Outcomes.
Better align Outcome Budgeting and CitiStat in a way that
builds on the innovative frameworks that already exist, and
keeps Baltimore at the forefront of innovation in
Government.
Identify multi-year targets for each of the Priority Outcome
Indicators, and an action plan for turning-the curve.
Increase opportunities for Citizen Participation in the City’s
strategic planning and decision making processes.
34
35. Mayoral Priority Outcomes & Indicators
• A Safe & Healthy Start
• Kindergarten Readiness
• Academic Achievement
• College & Career Ready
• Shootings
• Property Crime
• Perception of Safety
• Blight Elimination
• Neighborhood Investment
• Sustainable Transportation
• Recreation Visits
• Employment Rate
• Number of Jobs
• Visitors
• Procurement
• Prompt Payment
• Special Events
• Cleanliness of Waterways
• Energy Usage
• Recycling Rate
• Perception of Cleanliness
• Heroin-Related Deaths
• Mental Health
• Childhood Asthma
35
Better Schools
Safer Streets
Stronger Neighborhoods
Growing Economy
Innovative Government
Cleaner City
Healthier City
50. OutcomeStat Conference Results
Buy-in on Story Behind the Curve for all indicators
Understanding of what other organizations view as challenges for
Baltimore and how they’re willing to help
Prioritized strategies for turning-the-curve on all indicators
Group of people engaged in OutcomeStat and awaiting next steps
50
51. OutcomeStat Conference Results
Results from Participant Survey:
95% are supportive of the OutcomeStat process after
attending the Conference
88% believed team accomplished the objectives for
each indicator
Listed opportunity to collaborate and develop
strategies with variety of stakeholders as most
valuable
Wanting to know next steps was common outstanding
question 51
52. OutcomeStat Going Forward
Align with Budget Process
Budget Proposals from Agencies will demonstrate alignment
with indicator goals & the action plan items
Regular OutcomeStat Sessions
Meet regularly bi-annually to measure progress on indicators
& action plan items
Grow OutcomeStat Website
Link indicators to various service level Performance Measures,
Maps, & Real-Time Data Visualizations
52
53. Conference, Lessons Learned
Location Matters
Space couldn’t accommodate everyone who needed/wanted to
be at the table
Good Discussion v. Deliverables
Some teams used this time to have healthy conversations but
did not necessarily agree on a set of strategies
Demonstrating need for good follow-up sessions
53
54. CitiStat
CitiStat has given rise to many other
State and City models
Bridgeport,
Connecticut
Buffalo,
New York
Philadelphia,
Pennsylvania
Somerville,
Massachusetts
Atlanta,
Georgia
New Orleans,
Louisiana
State of
Maryland
State of
Washington
54
55. To learn more about the CitiStat or OutcomeStat,
visit the City of Baltimore’s website:
www.baltimorecity.gov
55
For further reading on how CitiStat started visit
http://www.businessofgovernment.org/pdfs/BehnReportCiti.pdf
CitiStat history and description
Photo of CitiStat room, Innovations Award, copycats
Accurate And Timely Intelligence Shared By All
There are three main sources of information on which the CitiStat process relies. The first is the 311 service request system. Residents of Baltimore City can call 311 to report all non-emergency problems from potholes in the road to traffic hazards. Detailed information about the call, the service request and the agency response are recorded and available for review at CitiStat sessions.
A second source of information is statistical reports that the agencies are responsible for preparing themselves. These reports, called templates, measure areas not covered by 311 services requests. Examples include overtime hours, traffic citations issued and new employees hired.
The third and final source comes from field work. Staff analysts are expected to follow up on leads provided by community liaisons and statistical trends to locate, photograph and present evidence of inefficient policies and procedures.
Rapid Deployment Of Resources
Solid evidence and good ideas can only be implemented with an efficient response system. The CitiStat model provides a forum for frank discussions and direct communication with the Mayor and her cabinet. If service managers are having difficulty responding to citizens, every two weeks they will have the opportunity to address issues concerning response capabilities. In addition, a key measurement associated with the CitiStat model is the setting of response goals and measurement of actual response times.
Effective Tactics And Strategies
Analyzing data and reporting statistics is only the beginning of the CitiStat process. The CitiStat session is one that promotes frank discussions and novel solutions. The keys to promoting the development of effective tactics and strategies include: requiring all relevant personnel to attend the session, possessing the technological capabilities to immediately provide all relevant data, dedicated analysts responsible for investigating agencies’ procedures and performances and preparing the participants for every session, and focused measurement and follow-up on the performance of all tactics and strategies in place.
Relentless Follow-Up And Assessment
Perhaps the most important tenet of the CitiStat model is to relentlessly follow-up and assess the measures developed or identified at CitiStat. Agencies are required to adhere to a rigid reporting schedule. Sessions are held monthly or bimonthly and are rarely cancelled or postponed. During times of emergency or for initiatives of heightened importance, sessions can be held as frequently as every week. In addition, staff analysts are responsible for carefully recording promises made and issues discussed at previous meetings, so they can be addressed again at future meetings. Finally, within hours of every CitiStat session, a detailed memorandum is send directly to the agency head listing all the agreements made in the previous session and requiring that a progress report be provided before the next session
10
11
12
13
14
15
16
17
18
19
20
22
23
24
CitiStat strengths
Cool charts, maps
Results
**This chart shows where citizens requested rat treatments (left) and where DPW was actually baiting (right) – demonstrating visually the problem that needs to be addressed.
26
Enter OutcomeStat
Logo
Trying Hard Is Not Good Enough
New Priority Outcomes and Indicators
New Priority Outcomes and Indicators
Turn the Curve infographic
Turn the Curve infographic
Turn the Curve infographic
Fiscal 2016 budget process (Program Level)
Scorecard screen shot
Example of good budget proposal
Fiscal 2016 budget process (Program Level)
Scorecard screen shot
Example of good budget proposal
Getting to Population Level Turn the Curve Plans
COLTS
Conference