Slides from the webinar co-presented by Andrew Arch and Neil Jarvis in September 2023 on the value in measuring your accessibility maturity.
The webinar explored the importance of measuring your maturity, and how doing so can help you create an action plan for improving accessibility within your organisation. A recording of the webinar is available on Intopia's YouTube channel.
Development Lifecycle.pptx for the secure development of apps
The value of measuring your accessibility maturity
1. The value of measuring your
accessibility maturity
Andrew Arch
Principal Consultant
Neil Jarvis
Senior Consultant
September 2023
2. What is Digital Accessibility Maturity
– Why would we measure it
3. What is Digital Accessibility Maturity?
• A measure of how well an
organisation is doing with respect
to ensuring all their digital products
and services are
• Various models can be used to
measure this
Diagram: https://www.icon-uk.net/Fraud_Masterclass/Fraud_Risk_Digital_Maturity_Assessment.html
4. Why do we want to know our maturity level?
• To measure the current effectiveness
• To assess gaps in capabilities
• To enable improvement
It should encourage and enable improving overall accessibility performance
over time. This will also help mitigate risk.
5. How to measure maturity
• Depends on the model we’re adopting
• Lots of commonality across them
7. W3C Maturity Model dimensions
1. Communications
2. Knowledge and skills
3. Support
4. ICT development life cycle
5. Personnel
6. Procurement
7. Culture
8. • Only measure proof points for dimensions being evaluated
• Proof points indicate the stage of maturity
How do we measure these dimensions
Proof points for evidence
9. W3C model maturity stages
Stages Criteria
Inactive No awareness and recognition of need.
Launch
Recognized need organization-wide.
Planning initiated, but activities not well organized.
Integrate
Roadmap in place.
Overall organizational approach defined and well organized.
Optimize
Incorporated into the whole organization, consistently evaluated,
and actions taken on assessment outcomes.
11. Communications dimension
• Launch stage
• Recognized need for accessibility of internal and external communications
• Integrate stage
• Communications roadmap in place
• Integration of accessibility criteria into most communications underway
• Optimise stage
• Full accessibility processes and tools implemented
• Fully accessible communications
12. Knowledge and skill dimension
• Launch stage
• Recognized need for disability inclusion and accessibility expertise
• Integrate stage
• ICT accessibility skills and training roadmap in place
• Integration of process to assess and increase accessibility knowledge
• Optimise stage
• Organization-wide, role-based expertise in accessibility and disability inclusion
• Accessibility knowledge and skills are consistently implemented
19. Recommendations
• Strategic and tactical
• Leadership messages
• Support groups
• Implementers skills uplift
• Portal for accessibility support
• Recruitment
• Outsourcing /procurement
• Priorities
• Timelines
• Depends on the organisation
20. Other support
• Suggestions for policy and strategy improvements
• Suggestions for improving specific documents we’ve reviewed
• Practical suggestions around software and platforms in use
• List of different disability days to celebrate and increase awareness
21. Next Steps
Remember it’s much more than just ICT
– it’s all things that touch a customer or employee
22. “If you can’t measure something, you can’t
understand it. If you can’t understand it,
you can’t control it. If you can’t control it,
you can’t improve it.”
“Measurement is the first step that leads to
control and eventually to improvement.”
H. James Harrington
23. Let’s chat later
Andrew Arch
Principal Consultant
andrew@intopia.digital
https://intopia.digital/
Neil Jarvis
Senior Consultant
neil.jarvis@intopia.digital
Hinweis der Redaktion
Neil
Andrew
Define what we’re taking about
Andrew
A measure of how well an organisation is doing with respect to ensuring all their digital products and services are
Various models can be used to measure this
Diagram is from Icon in the UK - It has 5 levels:
Initial – process unpredictable, poorly controlled and reactive
Repeated – processes are defined and documented
Defined – processes are standardised
Managed – Processes are monitored and controlled
Optimised – processes are continuously improved
Sources:
Diagram: https://www.icon-uk.net/Fraud_Masterclass/Fraud_Risk_Digital_Maturity_Assessment.html
Definitions: https://www.techtarget.com/searchsoftwarequality/definition/Capability-Maturity-Model
Neil
Why might we want to know what our maturity level is?
To get a measurement of the current effectiveness of a group or an organisation’s accessibility practices
To assess gaps between the current capabilities and the next level of accessibility maturity
Knowing where you are on a given scale and what the gaps in culture, knowledge, process, etc should encourage and enable improving overall accessibility performance over time
Mitigate risk – legal, reputational, financial
A couple of organisations we’re working with at present hope it will give them ‘permission’ to work on accessibility as BAU.
Andrew
There are many good models out there which can be used to measure accessibility maturity. Lots of commonality and all of them have their pros and cons.
For the purposes of demonstrating the value of maturity assessments, we’re going to take a look at the draft model currently being designed by the World Wide Web Consortium (W3C). You should find a model which is right for you.
The W3C model is composed of dimensions and maturity stages. Each dimension has associated proof-points and deliverables by which progress is measured. You can set goals and outcomes for each stage which are relevant to the dimension being assessed.
We will first provide an overview of the dimensions, and then we will pick a couple of dimensions and take a closer look at examples of goals, outcomes and proof points relevant to those specific dimensions.
Neil
The W3C Accessibility Maturity Model is a tool that assist you to:
assess the current effectiveness and capabilities of an entire organization or just teams within the organization
identify gaps between the current capabilities and the next level of accessibility maturity
Plan for improvements to your organisation's accessibility performance over time
Neil
7 Dimensions to measure in W3C’s model:
Communications: Information as it relates to an organization’s accessibility, as well as accessibility of all internal/external communications.
Knowledge and Skills: Ongoing education, and outsourcing practices to fill gaps for accessibility operations.
Support: Accessibility assistance provided to internal employees and external customers with disabilities.
Information and Communication Technology Development Life Cycle: Incorporation of web, software and hardware accessibility considerations in development processes - from idea conception, to design, development, testing, ACR production, user research, maintenance and obsolescence.
Personnel: Job descriptions, recruiting, disability-related employee resource groups necessary to provide lived-experience to accessibility efforts.
Procurement: A strategic process that concentrates on finding and acquiring accessible products required by an organization. Activities may include: sourcing, negotiation, and selecting goods and services.
Culture: The attitudes, sensitivity, and behaviors around accessibility, including internal interaction, perception, and decision-making.
Each dimension can have a goal and outcome associated with each stage, though strictly speaking, the first stage (Inactive) has neither since it is inactive).
Andrew
How is each dimension measured?
To assess the maturity stage for each dimension, a series of Proof Points have been suggested which can be used as evidence. These are:
evidence-based, organizational deliverables which are used to measure the maturity stage.
specific to the dimension being focused on. For example, if only procurement maturity is being measured, only procurement proof points should be evaluated.
likely to span several stages. They can be partially completed at the Launch and Integrate stages, but must be fully completed for the optimize stage.
The proof points demonstrate the completed outcomes, and should be relevant to the dimension and to the organisation.
If an outcome is not relevant, it should be marked as Not Applicable.
for instance, it would be pointless requiring an organisation to meet outcomes relating to native apps if they don't have them.
Neil
Using the evidence or proof points, you can measure an organisation’s progress on each dimension which leads to a determination of the maturity level of the organisation or department within it.
The four Stages of maturity in the W3C models are:
Inactive - No awareness and recognition of need.
Launch - Recognized need organization-wide. Planning initiated, but activities not well organized.
Integrate - Roadmap in place, overall organizational approach defined and well organized.
Optimize- Incorporated into the whole organization, consistently evaluated, and actions taken on assessment outcomes.
The W3C is producing an assessment template which can be used against this model, with worksheets corresponding to the dimensions, each organised to reflect the four stages above, with a list of relevant proof points. Each can then be scored with a final overall score helping to determine the level of maturity.
Time does not permit us to take you through each of the seven dimensions in detail but we will use two as an example. This will enable us to demonstrate likely dimension-related goals and outcomes for each stage of maturity, and the proof points which might be used as evidence. It is assumed that there are no goals and outcomes at the Inactive stage of maturity.
Andrew
Time does not permit us to take you through each of the seven dimensions in detail but we will use two as an example. This will enable us to demonstrate likely dimension-related goals and outcomes for each stage of maturity, and the proof points which might be used as evidence. It is assumed that there are no goals and outcomes at the Inactive stage of maturity.
We’ll consider
Communications
Knowledge and skills
Andrew
Goals and outcomes relevant to communications dimension might include:
Launch Stage
Goal: Recognized need for accessibility of internal and external communications. Plans begun or in place to identify communications where accessibility criteria should be integrated.
Outcomes: Plans in place for making all internal / external communications accessible (and compliant with accessibility regulations, where applicable).
Integrate Stage
Goal: Communications roadmap in place, integration of accessibility criteria into most internal and external communications is being implemented. Non-accessible tools replaced, or procurement of accessible tools in process.
Outcomes: Internal and external delivery of accessible communications across most media and platforms
An accessibility policy that includes requirements for a feedback mechanism for users, including a formalized process for handling accessibility complaints.
Optimise Stage
Goal: Full accessibility processes and tools implemented across all internal and external media resulting in fully accessible communications.
Outcomes: All processes, procedures, and tools are in place, used consistently, and are regularly reviewed and refined to ensure that all internal and external communications are fully accessible.
Proof points used to measure progress might include but by no means be limited to:
Accessible corporate document templates (word, ppt, etc)
Processes and procedures for creating communications are documented, and infrastructure is in place to ensure accessibility
Websites, e-mail marketing and social media are accessible per regulatory requirements (e.g. conforms to WCAG)
Websites may have an accessibility statement or contain a statement of commitment to accessibility
There is a clear feedback mechanism for handling questions and accessibility complaints
Neil
Goals and outcomes relevant to this dimension might include:
Launch Stage
Goal:
Recognized need for organization-wide accessibility and disability inclusion, as well as accessibility expertise.
Planning initiated, but activities not well organized
Outcomes:
Skill areas identified, plans for organization wide surveys to identify gaps initiated, but not implemented
Mostly Ad hoc training (professional development is not required or monitored.)
Some training courses established, e.g. for onboarding or annual refresher
Integrate Stage
Goal:
Workforce ICT accessibility skills and training roadmap in place.
Integration of process to assess and increase accessibility knowledge and skills is in progress.
Outcomes:
Role-based training available for people’s role plus current information on new technologies, platforms, and tools
Training metrics are established
Optimise Stage
Goal:
Organization-wide, role-based expertise in accessibility and disability inclusion is well-defined, evaluated, remediated, and continuously enhanced.
Accessibility knowledge and skills are consistently implemented across the organization
Outcomes:
Required and preferred knowledge and skills are consistently communicated to all personnel
Workforce is periodically evaluated to ensure knowledge and skills are current with the most up to date standards and accessibility practices
Periodic analysis used to identify gaps in knowledge as well as training materials
Annual training (conferences, events, online, etc.) is provided to maintain skills current with ICT accessibility requirements and industry best practices.
Workforce inclusion training incorporates accessibility for persons with disabilities.
Tracking systems in place and consistently used to maintain training inventory, measure skills, and track completion
Proof points used to measure progress might include but by no means be limited to:
Assessing Current Skills to Identify and Address Gaps such as:
Organizational surveys that identify current skill levels and gaps
Internal database(s) to track employee training for ICT accessibility skills
Keeping skills up-to-date with current requirements
Accessibility criteria integrated into individual performance measurements
Building and Maintaining Organizational Capacity such as
Implementation of role-based training plans and curriculums
Accessibility training when onboarding all new employees
Accessibility requirements included in position descriptions
Subject matter experts (SMEs) appointed within the organization to provide training and support
Andrew
Undertaking a gap analysis approach to assessing maturity – four aspects:
Interviews
Documentation reviews
Process reviews
Output reviews
Neil
Interviews, conversations really, with:
Product owners
Designers
Developers
Content creators
Testers / QA
Leadership
Staff with disabilities
Marketing and Comms
People and Culture (incl D&I)
Call centre support
Looking for different things from different teams and individuals
understanding across the organisation
Commitment
Leadership
Internal support
Client support
Recruitment
communication
What is happening at the coal face
Understanding of policies
Level of knowledge
Role responsibility
Stage considered
Tools used
Andrew
Processes considered - as appropriate
Design and development
Publishing
Procurement
Events
Marketing
Recruitment
Support, including feedback and complaints
Neil
Documents reviewed - depends what’s documented
Looking for commitment
Looking for measurement
Looking for plans
Looking for process
Documentation might include:
Policies and strategies
Annual reports, including disability action plans
Risk register
Procurement documentation
Accessibility support materials
Job descriptions
Professional development
Andrew
Sample output to look at – might include:
Client / customer communications
Website content
Any audits if conducted
Especially looking for critical and high impact failures
also common failures
Neil
Bring it all together
What is their maturity by dimension considered
What do we recommend, next slide, but:
We want people to know what’s working well and to keep doing it
What did we find that is holding them back
Neil
Recommendations
Strategic and tactical
Leadership messages
Support groups
Implementers skills uplift
Portal for accessibility support
Recruitment
Outsourcing /procurement
Priorities
Timelines
Depends on the organisation
Andrew
Suggestions for policy and strategy improvements
Eg in D&I policy
Suggestions for improving specific documents we’ve reviewed
Eg branding guidelines
Practical suggestions around software and platforms in use
Basis for an a11y support portal
Eg MS Office or AEM etc
List of different disability days to celebrate and increase awareness, eg
Basis for an a11y support portal
Eg MS Office or AEM etc
Neil
Remember it’s much more than just ICT – it’s all things that touch a customer or employee
Rinse and repeat
Andrew
So, why do we care about measuring digital accessibility maturity?
“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”
― H. James Harrington