Automated Testing as Part of Your 360˚ Approach to Web Testing

Share on FacebookShare on LinkedInShare on Twitter

360˚ Web Testing: Post 2 of 4

How do you test for web accessibility?  Whether you are testing for WCAG 2.0, 508, or any of the other accessibility standards used internationally, you will benefit from using a mix of manual and automated testing techniques, applied regularly and consistently.

While manual testing can yield detailed and accurate results, the cost of this method is high, and the shelf life of the testing results can be extraordinarily short.

On average, experienced web accessibility experts conducting manual tests spend anywhere from 1 – 3 hours doing a thorough examination of a page for violations of WCAG 2.0.   Factors that affect the length of the review include complexity of the page, number of violations on that page, requests to identify every single violation on every page, and misleading browser and assistive technology bugs.  In addition to discovering each issue, the expert is often asked to clearly explain, in writing, exactly where the problem is located and how to resolve it.

Now imagine a project where the objective is to manually review 100 pages on a specific site.  The client has a modern, dynamic and interactive site that is updated regularly with new content.  The release cycle for infrastructure changes to the site is based on an agile timeline, so changes to the underlying template and code happen as frequently as every 4 weeks.

  • Approximate hours for manual accessibility review for 100 pages = 100-300 hours
  • Potential shelf life of that review – 4 weeks

Clearly this approach is not feasible for most organizations.

So, how do you balance practicality with accuracy?   The smart solution is to use automated accessibility scans to monitor the breadth and depth of your site(s), coupled with focused manual testing on key representative pages.  In other words, use the power of automation as a proactive warning system and use manual testing to confirm that key users paths are truly accessible.

What is the right mix of manual testing (high touch) and automated testing (high tech)?  Based on my experience, the most effective and efficient use of your resources is to employ the 80/20 rule.  80% of your testing effort should be automated and 20% should be manual.  I recommend monthly, automated scans for unambiguous or explicit accessibility issues.  These smart scans will alert you if accessibility testing was skipped during development, or if  obviously inaccessible third party content was added.

 

What are the top 5 explicit accessibility issues that can be identified with an automated scan?

 

Accessibility Issue

WCAG 2.0 Success Criteria

508 Paragraph

Images missing alt attributes

1.1.1

(a)

Active images with a null alt attribute

1.1.1

(a)

Form fields missing explicit labels and/or titles

1.1.1

(n)

No title on a web page

2.4.2

(k)

No primary language specified for the web page

3.1.1

Not applicable

 

Running a monthly scan across your site to identify explicit accessibility issues is one of the very best methods for detecting accessibility problems when you have limited resources.  These monthly scans will help you get your head turned in the right direction so you can focus your detailed manual testing efforts in the smartest way possible.  An approach that combines robust automated testing with select manual testing is an approach that is practical and realistic for organizations to implement.

Remember, what gets measured gets done.  If you don’t monitor the accessibility health of a web site on an ongoing basis, it will decay.

 

This post is the second in a series on what we call “360 Degree” web testing: the approach that Deque advocates for obtaining the most detailed and accurate sense of a site’s accessibility.

Future posts in this series will include:

  • A piece on where manual testing is indispensable.
  • A miniature case study that puts the entire 360 degree approach together in a story of how a customer used this approach to strengthen the level of accessibility their organization achieved.

 

Download our Accessibility Checklist Now!

 

 

About 

Glenda Sims is the Team A11Y Lead at Deque, where she shares her expertise and passion for the open web with government organizations, educational institutions, and companies ranging in size from small business enterprise. Glenda is an adviser and co-founder of AIR-University (Accessibility Internet Rally) and AccessU. She serves as an accessibility consultant, judge, and trainer for Knowbility, an organization whose mission is to support the independence of people with disabilities by promoting the availability of barrier free IT. In 2010 Glenda co-authored the book InterACT with Web Standards: A holistic approach to Web Design.

1 comment

Leave a Reply

You can use these HTML tags:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>