1. A macroscopic Web accessibility
evaluation at different
processing phases
Nádia Fernandes, Luís Carriço
9th International Cross-Disciplinary Conference on Web Accessibility –
16/17th April 2012 – Lyon, France.

2. Motivation
9th International Cross-Disciplinary Conference on Web Accessibility –
2
16/17th April 2012 – Lyon, France.

3. Introduction
• The Web is being used by all kinds of people;
• Web sites must be accessible;
• Modern Web development transcends static HTML;
• A more real evaluation is necessary.
9th International Cross-Disciplinary Conference on Web Accessibility –
3
16/17th April 2012 – Lyon, France.

4. Previous work
9th International Cross-Disciplinary Conference on Web Accessibility –
4
16/17th April 2012 – Lyon, France.

5. QualWeb evaluator
• Accessibility evaluation of Web
pages:
– using WCAG 2.0 ,
• 18 HTML techniques;
– at different processing
Before
phases
• BEFORE and AFTER
Browser processing;
After
9th International Cross-Disciplinary Conference on Web Accessibility –
5
16/17th April 2012 – Lyon, France.

6. QualWeb evaluator: execution
9th International Cross-Disciplinary Conference on Web Accessibility –
6
16/17th April 2012 – Lyon, France.

7. Some improvements...
9th International Cross-Disciplinary Conference on Web Accessibility –
7
16/17th April 2012 – Lyon, France.

8. PhantomJs
• is a command-line tool that uses WebKit ,
• works like a WebKit-based Web browser (simulation);
• can be controlled using Javascript.
9th International Cross-Disciplinary Conference on Web Accessibility –
8
16/17th April 2012 – Lyon, France.

9. Problems Solved!
1. It avoids data injection at the browser level;
2. Evaluated Web page before and after browser processing is
exactly the same;
3. Integration with a crawler.
9th International Cross-Disciplinary Conference on Web Accessibility –
9
16/17th April 2012 – Lyon, France.

10. Research Question
How do the macroscopic properties emerging from Web
accessibility change in respect to the processing phase of
delivery?
9th International Cross-Disciplinary Conference on Web Accessibility –
10
16/17th April 2012 – Lyon, France.

11. Experimental Study
• We evaluated a set of Web pages from a list provided by the
Portuguese Web Archive (version of 2008);
• We used the QualWeb evaluator.
9th International Cross-Disciplinary Conference on Web Accessibility –
11
16/17th April 2012 – Lyon, France.

12. Metrics
• Results of the evaluation are presented in terms of:
– PASS, WARN, and FAIL (Applicable = PASS + WARN + FAIL)
• Metrics used:
– rate conservative =
– rate optimistic =
– rate strict =
• The results are between accessible (100%) and not accessible (0%).
9th International Cross-Disciplinary Conference on Web Accessibility –
12
16/17th April 2012 – Lyon, France.

13. Results
• 24, 462 Web pages evaluated.
• Total Number of HTML elements:
– before processing - 24,918,720
– after processing - 41,967,072
– (Ratio ≈ 1.7).
• Average Number of HTML elements per page:
– before processing - 1010
– after processing – 1710
– (Ratio ≈1.7).
9th International Cross-Disciplinary Conference on Web Accessibility –
13
16/17th April 2012 – Lyon, France.

14. The differences of an HTML
document between both
Processing Phases were observed.
9th International Cross-Disciplinary Conference on Web Accessibility –
14
16/17th April 2012 – Lyon, France.

15. Results: Average Outcomes
• Successes
– before processing - 9 elements
– after processing - 87 elements
– (Ratio ≈ 9.7).
• Failures
– before processing - 46 elements
– after processing – 176 elements
– (Ratio ≈ 3.8).
• Warnings
– before processing - 262 elements
– after processing – 451 elements
– (Ratio ≈ 1.7).
9th International Cross-Disciplinary Conference on Web Accessibility –
15
16/17th April 2012 – Lyon, France.

16. Results: rates Before and After processing
• Conservative rate:
– The average quality increases after
processing.
– Accessibility quality between 60% and 90%
disappeared after processing.
• Optimistic rate:
– The average quality decreases after
processing.
– Some results lower than 20% disappeared
after processing.
– Decrease of higher accessibility quality,
leading to a lower accessibility average.
9th International Cross-Disciplinary Conference on Web Accessibility –
16
16/17th April 2012 – Lyon, France.

17. Results: rates Before and After processing
• Strict rate:
– The average increased after processing.
– Results higher than 85% disappeared
after processing.
– Worse pages before processing get higher
scores after processing.
– Better pages before processing are
ranked lower after processing.
9th International Cross-Disciplinary Conference on Web Accessibility –
17
16/17th April 2012 – Lyon, France.

18. Limitations
1. Techniques coverage
2. Dynamic content
3. Automatic evaluation
9th International Cross-Disciplinary Conference on Web Accessibility –
18
16/17th April 2012 – Lyon, France.

19. Discussion
• Evaluations before processing clearly is not the best option!
• We used/percept/interact with the after processed version.
• Considering the rates…
– Web pages possess higher uniformity
– That can be explained with reusable code
9th International Cross-Disciplinary Conference on Web Accessibility –
19
16/17th April 2012 – Lyon, France.

20. Discussion
• Impact on Designing Accessible Web Pages
– Importance of sharing reusable code;
– High quality reusable code produce better quality pages.
• Impact on the Perception of Accessibility
– Each metric identify different perspectives;
– It is important that evaluations assess what user perceive.
9th International Cross-Disciplinary Conference on Web Accessibility –
20
16/17th April 2012 – Lyon, France.

21. Conclusion
• We presented a large-scale study of accessibility on the Web.
• We were able to characterize some accessibility properties of the
Web, pointing some differences between processing phases.
• The results obtained on the evaluation of pages after browser
processing tend to be more homogeneous than before.
• Considering that the end-user interacts with the after processed
pages then most studies about Web quality should be redone.
9th International Cross-Disciplinary Conference on Web Accessibility –
21
16/17th April 2012 – Lyon, France.

22. Future Work
1. Enlarge the coverage of WCAG 2.0 implemented tests
2. Evaluate Rich Internet applications
3. Perform a comparative set of studies
9th International Cross-Disciplinary Conference on Web Accessibility –
22
16/17th April 2012 – Lyon, France.

23. nadiaf@di.fc.ul.pt
9th International Cross-Disciplinary Conference on Web Accessibility –
23
16/17th April 2012 – Lyon, France.