1.
Best practice from Portugal
OCDE WORKSHOP
THE PERFORMANCE OF UTILITIES
FOR WASTEWATER COLLECTION AND
TREATMENT
Susana Rodrigues, Quality Department
31 january, 2023

2.
2
AGENDA
1. Wastewater Regulation in Portugal
2. Qualility of Service Evaluation System
3. Wastewater KPIs
4. How benchmarking impacts
5. Challenges and final remarks

3.
WASTEWATER REGULATION
IN PORTUGAL

4.
4
Wastewater regulation in Portugal
ERSAR Regulation model
Structural regulation Regulation of utility
behaviour Monitoring
Other additional activities
Drinking water quality
regulation
Quality of service
regulation
Economic regulation
Legal and contractual
regulation
User interface regulation
Contribution to the
clarification of sector’s
rules
Contribution to the sector’s
legislation
Information gathering
and processing
Capacity building
Sanctioning and infractions
Investigation, innovation and studies
Inspection and fiscalization
actions
Auditing and monitoring
actions
Contribution to the
sector’s organisation
Cooperation
Cooperation with national
authorities
Cooperation with
international authorities
Holistic and
integrated approach

5.
5
Wastewater regulation in Portugal
ERSAR Regulation universe
o National level (mainland)
o All the utilities: vast size and diversity of operators with different governance models (State-owned,
municipal-owned, private companies) and activity scopes (water, wastewater, urban waste; bulk, retail)
ERSAR
Universe of
354
operators
… State owned and
municipalities … in
direct management
model, by delegation
or concession …
operating bulk or
retails systems or both
239 water supply entities
237 wastewater management
services entities
260 urban waste management
entities
Ten million consumers

6.
QUALITY OF SERVICE EVALUATION SYSTEM
HOW?

7.
7
Quality of service regulation.
HOW to promote quality of service of
monopolistic utilities?
CARROT
STICK
Sunshine regulation: KPI assessment system,
simple and transparent Performance Benchmarking

8.
8
Clear procedures and definitions
Technical guide
Universal, applicable to every operator, regardless of the
activity scope, nature, management model or size
Comparable and transparent, establishes all the definitions
of data and indicators required: formulas, reference ranges,
data base sources, and reliability
Uptaded, new KPI and targets are defined to address new
legal and strategic challanges
Quality of service regulation. How?

9.
9
Adequacy of service to users
Accessibility of service
Quality of service provided
Service management sustainability
Economic sustainability
Infrastructural sustainability
Physical productivity of human
resources
Environmental sustainability
Efficient use of natural
resources
Circularity and valorisation
Efficiency on pollution
prevention
3 Objectives, and 8 Criteria
Quality of service regulation. How?
Objectives and criteria
Performance assessment, to compare operators and promote efficiency

10.
10
Quality of service regulation. How?
KPI assessment system
1. Profile data (operator and system)
2. Base data
3. Key Performance Indicators
4. Reference values
5. Operators' individual performance file
(after results validation)
quantitative assessment of efficiency
and effectiveness for certain aspects
of the service
allows benchmarking,
using clusters
code, definition, and rules for
reliability
semaphore codes for easier
perception
System Components:

11.
11
Frequency: Annual Cycle
Beginning of the
cycle
January
Data treatment
by ERSAR
June to August
Results publication,
publicising and awards
for excellence
November
Data validation
by ERSAR
(audits)
May to June
Data report
by the utilities
March and April
Right-of-reply by
the utilities
September
Technical guide and files
Individual file
Portal ERSAR
Annual report
Website
App ERSAR
Audits
Awards and
certificates
Quality of service regulation. How?

12.
12
Quality of service regulation. How?
Reliability and Data Validation: Audits
Audits for Data Validation
100% of the reported data are
validated
Reliability assessment
to validate sources
and measurement procedures
Right of Reply
Final Validation
REGULATOR value
OPERATOR value
AMOUNT
Data
More
data
Even
More data
………...
All the data
available
QUALITY
Uncertain
information
Better
information
Even better
information
………..
Best available
information
COMPARISON
Benchmarking
Better
benchmarking
Even better
benchmarking
………….
Best possible
benchmarking
STEP 1
STEP 2
STEP 3
……….
STEP N
Audits Treatment
Audits Treatment
Audits
Audits
Treatment
Treatment

13.
13
Transparency: Are the data being collected in a timely and efficient manner?
Portal ERSAR
IT system, through a single platform:
Centralises information of all operatos
Standardises data collection and analysis
Otimises reporting procedures
Improves comunication with operators
Reduces information assymetries
Quality of service regulation. How?

14.
WASTEWATER KPIs

15.
15
Wastewater services
Adequacy
of
service
to
users
Accessibility of service for users
B - Physical accessibility (service coverage) (%)
R - Service coverage through fixed networks and septic tanks (%)
R – Affordability (%)
Quality of service provided to users
Flooding occurrences [n.º/(100 km collector.year)]
Reply to suggestions and complaints and information requests (%)
Service
management
sustainability
Economic sustainability
Cost recovery ratio (%)
B - Connection to the service (%)
R - Connection to the service through networks (%)
Infrastructural sustainability
Sewer rehabilitation (%)
Sewer collapses [n.º/(100km.year)]
Monitoring of the structural condition of sewers (%)
Treatment infrastructures’ use (%)
Physical productivity of human resources
B - Adequacy of human resources on transport & treatment [n.º/106m3.year)]
R - Adequacy of human resources on treatment [n.º/106m3.year)]
R - Adequacy of human resources on collection and drainage [n.º/(100km.year)]
Environmental
sustainability
Efficient use of natural resources
Energy efficiency of pumping stations [kwh/(m3.100 m)]
Sludge production (kg/m3)
Circularity and valorisation
Production of water for reuse (%)
Own energy production (%)
Efficiency on pollution prevention
Control of emergency discharges (%)
Compliance with the discharges permit (%)
21
Fed by
108
variables
Quality of service regulation
Wastewater KPI

16.
16
Wastewater KPI
Quality of service regulation.
OCDE ERSAR IWA
Compliance with the UWWTD
AR21 - Compliance with the discharges
permit (%/year)
wEn1 - Compliance with
legislation on wastewater
treatment plant discharges
(%/year)
Greenhouse gas emissions
PAR04 - Indirect greenhouse gas emissions
(kgCO2e/m3)
-
Energy use PAR06 - Energy consumption in treatment
(kWh/m3 /year)
wOp18 – Energy consumption
in treatment facilities
(kWh/p.e./year)
Production
AR19 - Own energy production (%) -
Quality of service AR02 - Service coverage through network
and septic tanks (%/year)
AR05 - Response to complaints,
suggestions and information requests
(%/year)
-
wQS27 - Response to
complaints (n.º/1000
inhabitants/year)

17.
17
Wastewater KPI
Quality of service regulation
Number of structural collapses per 100 km of collector.
AR10ab – Sewer collapses [No./(100 km ⋅ year)]
AR10ab =
Sewer structural collapses (No./year)
Total length of sewers (km)
x 100
dAR54ab
dAR31ab
Reference values for bulk systems:
Good service quality 0,0
Average service quality ]0,0; 1,0]
Unsatisfactory service quality ]1,0; + ∞]
Reference values for retail systems:
Good service quality 0,0
Average service quality ]0,0; 2,0]
Unsatisfactory service quality ]2,0; + ∞]
Clusters: reference values may be
different for retail/bulk and for
predominantly urban, medium
urban or predominantly rural
intervention areas

18.
HOW BENCHAMRKING IMPACTS

19.
19
Quality of service regulation
How Benchmarking impacts: public information
Evolution assessment for each
indicator of quality of service
Annual assessment of the
quality of service for each
operator
ANNUAL
BENCHMARKING
between operators
regarding the quality of
service
2017 2018 2019 2020 2021
AR08b - Média do indicador
(n.º/(100 km.ano))
1,6 2,8 3,9 1,6 1,1
N.º de EG com resposta 241 245 245 210 208
N.º total de EG 257 257 257 227 225
Fiabilidade dos dados *** (%) 76 80 82 83 86
Fiabilidade dos dados ** (%) 7 6 8 8 6
Fiabilidade dos dados * (%) 17 14 10 9 8
0,0
1,0
2,0
3,0
4,0

20.
20
Quality of service regulation
How Benchmarking impacts. Sunshine regulation
o All information is publicly available on our website
to be used by researchers and by the public
o Awards and certificates excellence, in 5 categories:
• Drinking Water Quality
• Water Supply Service
• Wastewater Service
• Waste Management Service
• Efficient Use of Water
o Rewards best practices and good behavior towards ERSAR and
the consumer (and improves consumer awareness)
o Results are used in operators' access to EU funds, a major
incentive for improvement: artificial competitive environment

21.
21
Quality of service regulation
How Benchmarking impacts
60%
86%
28%
0%
20%
40%
60%
80%
100%
1993 1996 1999 2002 2005 2008 2011 2014 2017 2020
Access to wastewater services
(1993-2020)
Access to drainage Access to treatment
∆= +58
∆= +26
p.p.
✓ Better performance in
critical KPIs
✓ Increase in the number
of responses
✓ Improvement in data
quality

22.
22
Quality of service regulation
Linking quality of service and willingness-to-pay
5%
2%
2%
5%
17%
31%
32%
6%
4%
2%
2%
4%
17%
30%
33%
9%
2%
1%
1%
4%
14%
31%
34%
12%
0% 5% 10% 15% 20% 25% 30% 35% 40%
NS/ NR
1
2
3
4
5
6
7
% de respostas
Escala
de
avaliação
Avaliação positiva Avaliação razoável Avaliação negativa
Abastecimento de água Saneamento de águas residuais Resíduos indiferenciados
Qualidade muito
baixa dado o preço
Qualidade muito
alta dado o preço
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AR01b AR02b AR03b AR04b AR05b AR06b AR07b AR08b AR09b AR10b AR11b AR12b AR13b AR14b
Retail System
Good quality Average quality Unsatisfactory quality No reply
End user perception
versus reality
High service quality
evaluation
considering the cost
Low service quality
evaluation
considering the cost
Doesn’t know/ answer

23.
CHALENGES AND FINAL REMARKS

24.
24
Quality of service regulation
Challenges and Final remarks
Performance benchmarking is:
o A key tool for regulation (monitoring national strategies)
o Provides reliable information for structural (optimal operation scale)
and operational efficiency (cost reduction and productivity increase),
economic regulation (invest or subsidise), for compliance control (EU
targets; performance based contracts on concessions)
o Promotes capacity building (e.g. climate change resilience).
The role of Regulators
Command and control
instrument: setting targets
The Challenge is on the definition and practical implementation of
KPIs, to compare only what is comparable. Who? How?

25.
25
Quality of service regulation
The role of Regulators
Compare what is comparable:
opportunities of digital
transformation
o Accuracy, Transparency and Reliability: clear, accurate and
detailed information on clusters definition and reference
values, input data and KPIs definitions, normalisation
factors, and validation process for member states
o Communication: information to the public versus
performance benchmarking results (smart phone apps, websites)
o Operators: reliability of information has progressed slowly,
as it requires an adaptation on the operator's IT systems
o Regulators: developments needed to simplify and
automatize validation process (at least annually)
Challenges and Final remarks

26.
susana.rodrigues@ersar.pt
THANK YOU