360-Degree Web Testing: Post 4 of 4
So far, we’ve talked about the two main methodologies that Deque uses to perform accessibility testing. We presented a post on automated testing using Deque’s Axe platform and a post on manual testing to provide depth and focus of expertise where it’s needed.
While both of the methodologies are important, it’s how they are used together that makes the difference. Let’s walk through a fictionalized case study of a client I’ll call Fiction Bank. While our illustration uses a financial institution as the organization doing the testing, it could be any company, government agency, or educational institution that could use and benefit from this approach.
Fiction Bank has a problem. They have received a complaint about the accessibility of their site via a state regulator. The single problem has raised the issue of how accessible their site is overall and the need to comply with the ADA to ensure all web users, and particularly people with disabilities, can use the site without barriers.
Before engaging in automated or manual testing, some important organizational work needs to be accomplished up front:
- A list of all the “transaction paths” (user tasks and the clicks required to accomplish them) needs to be created. In the case of Fiction Bank, example transaction paths would include logging into their account, checking their account balance, paying a bill, and contacting customer support. It’s not uncommon for the list of transaction paths to be quite long, depending on the size and sophistication of the web application that’s being evaluated.
- The transaction paths are then prioritized, based on their popularity as reflected in usage logs, and based on the level of importance as core functionality as defined by the business. The ranking of the transaction paths helps to ensure that the most important and heavily used features have the urgency and the resources assigned to them that’s appropriate. For Fiction Bank, contacting customer service by email or via an online form was prioritized higher than contacting customer support by fax, a feature that the product team indicated was rarely used.
Accomplishing items #1 and #2 create the start of an action plan. The team knows what is the most important areas of the site to review first, and what has less urgency.
At this point, the portfolio of transaction paths can be used to design a testing plan that allows the team to take a complete 360-degree approach and draw on the strengths that automated and manual testing offers.
For Fiction Bank, the combined high priority transaction paths totaled several hundred thousand pages. Clearly, to conduct manual review of those pages would have taken a great deal of time and an unrealistic budget. So, here automated testing can help ensure that the breadth of pages receives that attention it needs.
Additionally, a number of the transaction paths are data-driven–the pages render differently based on the selections and input the customer makes. Different states, for example, have different regulations regarding credit cards, so which state the user selects impacts the content that they see. Testing 50 state permutations would be time-consuming and costly using manual testing, but WorldSpace Sync allows customer data entry to be scripted and played back during testing, achieving efficiency and scale.
There were some areas where manual testing was important. For example, Fiction Bank had several pages with complex tables that compared the financial products they offered. Automated testing could accomplish a high-level sense of the level of accessibility, but manually testing these pages to ensure a screen reader user could navigate the information successfully provided a more conclusive and thorough outcome.
Taken together, manual and automated testing–combined within a testing plan that identifies and prioritizes transaction paths–provides a complete, 360-degree view of the level of accessibility of the site. Because it draws on the strengths of both testing approaches, Deque has found it provides the most cost-effective and thorough results.
Posts in this series include:
- Part 1: Why a Consistent Methodology is Critical in Accessibility Testing
- Part 2: The strengths of automated testing.
- Part 3: Where manual testing is indispensable.
If you are looking into web accessibility software for your business, download our checklist. It will be a big help when evaluating the different software options out there.
