360-Degree Web Testing: Post 3 of 4
Automated accessibility testing is an integral part of your 360-degree approach to accessibility testing, but there are some issues that require manual testing. To get a truly comprehensive testing process, you have got to bite the bullet and experience the page with:
- A screen reader
- Speakers and microphone off
- No mouse
You will need to run three test cycles: one with “no sight,” one with “no sound,” and one with “no mouse.” You see, if you don’t conduct this type of usability testing during the web development process, people with disabilities in the real world become the usability testers. And that is not okay.
A Practical Accessibility Testing Plan
In my former life, I was the lead accessibility expert at a very large university. I developed the following practical accessibility testing plan as a way to monitor over a million pages maintained by over a thousand web developers and content contributors:
- Comprehensive Testing
- High Tech
- Enterprise Accessibility Scans
- High Tech
- Representative Pages
- High Touch + High Tech
- Page by Page Testing Tools
- High Touch
- Expert Functionality Testing
- Screen Reader
- No Sound
- Keyboard Only
- Expert Functionality Testing
- High Touch + High Tech
- Key User Paths
- High Touch
- Usability testing with people who have disabilities
- High Touch
Where to Focus Your Energy
None of us have the resources to conduct comprehensive manual accessibility testing on every page of our site(s). Even if we did have the resources to do this, the moment we finished the test, we’d have to start all over again, because our sites are constantly changing. So, the smartest way to leverage your manual testing is to focus on the following types of pages:
- Site-wide Templates
- Representative Pages that include
- Informational Images
- Active Images
- Forms
- Data Tables
- Multimedia
- Dynamic Pages
- Modal Windows
- Color
- Timers
- Different Languages
- Critical Pages
- Key Entry Points
- Key User Paths
- Highest Traffic Pages
- The Obvious
- Feedback Forms
- Accessibility Policy Pages
As you develop your list of representative pages for manual accessibility testing, keep these additional factors in mind:
- Audience for this Page
- Impact of Not Having Access to the Information / Functionality on this Page
My final consideration when reviewing the list of representative pages is to imagine I’m talking to a disabled user. Picture yourself explaining how you used automated testing to proactively look for problems and used focus manual testing to make sure that key user paths were truly accessible to people with disabilities. Will they see that you really cared about equal access and did everything within your power to make progress and maintain accessibility?
Focused Hands-on Accessibility Testing and WCAG 2.0
Once you have selected the representative pages you will test manually, you need to make sure you have a comprehensive, consistent testing methodology that is effective and efficient. Here at Deque, we have analyzed the 38 individual WCAG 2.0 AA success criteria and determined that the following checkpoints must be manually verified as follows:
| WCAG 2.0 AA Success Criteria | Keyboard-Only Testing Required | No-Sound Testing Required | Screen Reader Testing Required |
|---|---|---|---|
| 1.2.1 Transcript |
X | ||
| 1.2.2 Captions (prerecorded) |
X | ||
| 1.2.3 Captions (live) |
X | ||
| 1.3.3 Sensory |
X | ||
| 1.4.2 Audio Control |
X |
X | |
| 2.1.1 Keyboard |
X |
X | |
| 2.1.2 No Keyboard Trap |
X |
X | |
| 2.2.1 Timing Adjustable |
X |
X | |
| 2.2.2 Pause, Stop, Hide |
X |
X | |
| 2.4.3 Focus Order |
X |
X | |
| 2.4.7 Focus Visible |
X | ||
| 3.2.1 On Focus |
X |
X | |
| 3.2.2 On Input |
X |
X | |
| 3.3.1 Error Identification |
X | ||
| 3.3.1 Error Identification |
X | ||
| 3.3.2 Labels or Instruction |
X |
X | |
| 3.3.4 Error Prevention |
X |
X |
Note the overlap between the twelve success criteria that must be verified by keyboard alone and the eleven success criteria that must be verified by screen reader testing. The only difference between the two is for keyboard only testing you need to test for 2.4.7 Focus Visible, while this test is not critical for screen reader users. It is important to keep in mind that screen reader testing is not identical to testing with keyboard alone. Remember, that sighted people with manual dexterity disabilities that cannot use a mouse are not likely to be using a screen reader. Powerful features in modern screen readers certainly help people with visual disabilities, but we can’t lose sight of accessibility issues that cause barriers for keyboard only users.
Remembering Why
As you are considering your strategy for manual and automated accessibility testing, remember to keep the core purpose in mind. The goal is to remove barriers on the web so that all people, regardless of disabilities, can obtain the same information and perform the same functions. So, every time you can honestly report “no sight, no sound, no mouse: no problem!” know that you have played a part in building a web for all.
Posts in this series include:
- Part 1: Why a Consistent Methodology is Critical in Accessibility Testing
- Part 2: The strengths of automated testing.
- Part 4: Bringing it all together
If you are looking into web accessibility software for your business, download our checklist. It will be a big help when evaluating the different software options out there.
