Automating Google Workspace (GWS) & more with Apps Script
Using Acceptance Tests to Validate Accessibility Requirements in RIA
1. Using Acceptance Tests to
Validate Accessibility
Requirements in RIA
Willian Massami Watanabe
Renata Pontin de Mattos Fortes
Ana Luiza Dias
9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
2. Agenda
• Research context
• State of art
• Hypothesis
• Acceptance Tests
• Preliminary results
• Advantages and limitations
• Next steps
2/35
3. Agenda
• Research context
• State of art
• Hypothesis
• Acceptance Tests
• Preliminary results
• Advantages and limitations
• Next steps
2/35
29. Agenda
• Research context
• State of art
• Hipothesis
• Acceptance Tests
• Preliminary results
• Advantages and limitations
• Next steps
15/35
30. Acceptance Test
Definition
“A formal test that determines if a system satisfies its
acceptance criteria and allow the user to identify
when a system will be accepted or not.”
16/35
40. Agenda
• Research context
• State of art
• Hipothesis
• Acceptance Tests
• Preliminary results
• Advantages and limitations
• Next steps
21/35
41. Preliminary results
[W3C,
Focus management 2011]
HTML Acceptance
validators tests
flyout menu flyout menu
accordion menu accordion menu
overlay menu overlay menu
tabbed menu tabbed menu
[Welie,
2008] 22/35
42. Preliminary results
[W3C,
Focus management 2011]
All web tools that test navigation
HTML
validators
accessibility requirements
http://www.w3.org/WAI/ER/tools/complete
Cynthia Accessibility check
DaSilva Achecker
EvalAccess FAE report
WAVE Hera
23/35
43. Preliminary results
[W3C,
Focus management 2011]
All web tools that test navigation
HTML
validators
accessibility requirements
http://www.w3.org/WAI/ER/tools/complete
Cynthia Accessibility check
DaSilva Achecker
EvalAccess FAE report
WAVE Hera
23/35
47. Preliminary results
Projects that present defects that
[W3C,
Focus management not2011]
were identified by the Tool
Acceptance
HTML Validators
tests
26/35
49. Preliminary results
Projects that might be put
Integração Contínua
in production event though
they contain accessibility tests
• Unit
errors • Integraiton tests
• Compile
• Acceptance tests
Source code
repository
Developers
Report
27/35
51. Preliminary results
[W3C,
Focus management 2011]
Projects that present no
Acceptance
accessibility errors however are
HTML Validators
tests
not put into production
28/35
55. Preliminary results
[W3C,
Focus management 2011]
Acceptance
Warnings only
HTML Validators HTML
tests
30/35
56. Preliminary results
[W3C,
Focus management
Do not make distinction between interface
2011]
components that were accessible and the ones
that were not Acceptance
Validadores HTML
tests
30/35
57. Agenda
• Research context
• State of art
• Hipothesis
• Acceptance Tests
• Preliminary results
• Advantages and
limitations
• Next steps 31/35
58. Advantages
• In the experiment, the use of acceptance tests
presented a higher number of correct
assertions
• Higher effort required in order to execute the
tests
• Assertions made agains the DOM structure
inside of an use context makes it possible to
evaluate ARIA requirements
32/35
59. Limitation
• The limitation of this hyphotesis is the higher
cost to describe all contexts of the users
33/35
60. Agenda
• Research context
• State of art
• Hipothesis
• Acceptance Tests
• Preliminary results
• Advantages and limitations
• Next steps
34/35
61. Next steps
• Implement new use scenarios for the
acceptance tests
• Evaluate ARIA requirements
35/35
62. REFERENCES
• Seojin Kim, Sungjin Park, Jeonghyun Yun, and Younghoo Lee. 2008.
Automated Continuous Integration of Component-Based Software: An
Industrial Experience. In Proceedings of the 2008 23rd IEEE/ACM
International Conference on Automated Software Engineering (ASE '08).
IEEE Computer Society, Washington, DC, USA, 423-426.
DOI=10.1109/ASE.2008.64 http://dx.doi.org/10.1109/ASE.2008.64
• ROMEN, D.; SVANAES, D. Evaluating web site accessibility: validating the
wai guidelines through usability testing with disabled users. In: NordiCHI
’08: Proceedings of the 5th Nordic conference on Human-computer
interaction, New York, NY, USA: ACM, 2008, p. 535–538.
• W3C Conformance evaluation of web sites for accessibility. 2008a.
Available at: http://www.w3.org/WAI/eval/conformance.html
63. REFERENCES
• BRAJNIK, G. Web accessibility testing: When the method is the culprit. In:
Computers Helping People with Special Needs, Springer Berlin / Heidelberg,
2006, p. 156–163 (Lecture Notes in Computer Science, v.4061).
Available at: http://www.springerlink.com/content/ h2573t61gu3n8533/
• C. A. Velasco, D. Denev, D. Stegemann, and Y. Mohamad. A web compliance
engineering framework to support the development of accessible rich
internet applications. In Proceedings of the 2008 international cross-
disciplinary conference on Web accessibility (W4A), W4A ’08, pages 45–49,
New York, NY, USA, 2008. ACM.
• W3C Accessible Rich Internet Applications (WAI-ARIA) 1.0. 2011.
Available at: http://www.w3.org/TR/wai-aria/.
• Welie. 2008.
Available at: http://www.welie.com/patterns/.
64. Thank you very much!
Contacts:
watanabe willian@yahoo.com
renata@icmc.usp.br
anadias@icmc.usp.br
URLs:
http://www.pyccuracy.org
https://github.com/watinha/Pyccuracy-Accessibility-Actions
http://watinha.com/pyccuracy_test_1/templates/
9th International Cross-Disciplinary Conference on Web Accessibility – 16/17th April 2012 –
Hinweis der Redaktion
Good morning! My name is Ana Luiza and I will talk about “ Using Acceptance Tests to Validate Accessibility Requirements in RIA ”
I will talk about: Research context, State of art, Hypothesis, Acceptance Tests, Preliminary results, Advantages and limitations and Next steps.
I will start discussing about Research context.
Nowadays, there are many web applications available, such as: amazon, google, twitter, youtube and facebook.
These Web applications have been developing considering the Web 2.0, that have many characteristics.
One of them is Perpetual Beta, because these Web applications are always under development.
It happens because it is necessary to consider users, and user`s needs can change.
Than, it`s necessary to implement new features and to improve Web applications.
But, there is a challenge. It`s necessary to do these, spending less than an hour.
To implement new features and to improve Web applications, it`s common to follow all these steps: Requirements, Design, Implementation, Tests and Deployment.
In this context, it`s important to consider Continuous Integration…
… because in each steps, it`s necessary to check the quality.
Then in this case, when developers insert any code in repository it`s necessary to make tests.
The result of theses tests is a report…
… and this report, can show if the quality is OK…
… or not OK. This part of Continuous Integration is related to tests.
The idea is to follow these steps spending less than an hour, because user’s needs change very fast. So, Web application need to change as well; and automatic tools can validate if the changes are OK or NOT faster.
But, we have to consider how these automatic tools evaluate accessibility.
Nowadays, there are researches that check accessibility features like user testing, but if use them, you will spend more than one hour.
That`s why it`s useful to use automatic accessibility tests.
But, there is a problem because when we develop a Web application, we use many technologies, mainly if we will develop rich interface applications.
This kind of automatic tools usually checks just HTML…
… and it`s not enough to evaluate the accessibility.
So, our hypothesis…
… is to consider user`s context, rich applications features….
… and also accessibility in rich applications.
… . and this report will consider the accessibility too, because these acceptance tests consider the whole technologies.
Acceptance tests is “a formal test that determines if a system satisfies its acceptance criteria and allow the users to identity when a system will be accepted or not”.
This test doesn’t consider external interface because usually people with disabilities don’t use it, it’s more common to use assistive technologies. This test reflects User Stories because it considers how users can interact with Web applications.
And these Users Stories are represented by DOM structure witch there are the possibilities that people can navigate on Web applications and, we consider these possibilities to create User Scenario.
For example, we created a scenario considering this Web application. In this case, the user needs to find tennis news. What the user needs to do?
Firsty, user needs to go to Yahoo page.
In order to observe how our hypothesis works, we implemented the same Web application twice. One had keyboard accessible and another one we inserted some accessibility errors. To do the test, we considered only navigation and keyboard accessibility issues.
Theses Web applications was implemented using some interactions design patterns from Welie like flyout menu, accordion menu, overlay menu and tabbed menu.
So to do this test, we used our hypothesis and automatic tools recommended by W3C, as DaSilva, EvalAccess and others.
But some of them were not available to use in that moment.
Therefore, we used DaSilva, EvalAccess, WAVE and FAE report in contrast with our hypothesis.
Firstly, we checked if the tools could identify if the Web application is accessible or not. It was possible to notice that acceptance tests identify just one incorrect assertion.
In incorrect assertions there are false positives and false negatives.
False positives represent menus that were not accessible. And, these tools didn`t identify this.
So, the results could be OK, but on the other hand, it`s not OK, because the tools didn`t identify that the menus were not accessible.
In this case, it`s possible to put this Web application in production and it is not good because there are errors.
In contrast to false positives, false negatives represent menus that were accessible.
But these tools didn`t identify this.
So, the results couldn`t be OK.
In this case, it isn`t possible to put this Web application in production.
It`s necessary to say that many results considered…
… only warnings, because the tools didn`t identify errors of accessibility.
As well as, these tools didn`t identify any distinction between Web application which presented accessible design solutions and the one that did not.
