Setting up a screen reader test environment

Back in the day when web pages were web pages and not web applications you could just about get away with not testing with a screen reader. Using a combination of text based browsers, accessibility testing tools such as the Web Accessibility Tools consortium Accessibility Toolbar and and browser tools such as Preferences settings in Opera (see Opera > Quick preferences or Preferences), you could more or less muddle your way through.

This is not the case now though as browsers, screen readers, guidelines and technology has moved on and in many ways is catching up with one another. What follows are some tips to get you started when setting up a screen reader testing environment.

Why test with screen readers

  • WCAG 2.0 and JavaScript: Where WCAG 1.0 had a blanket ban on JavaScript WCAG 2.0 now allows it providing it’s accessible to assistive technologies, or in WCAG 2.0 talk accessibility supported.
  • WAI-ARIA: Screen readers can in fact handle most uses of JavaScript. It’s when they encounter areas of a page that update without full page refreshes or encounter functionality that is mouse orientated such as sliders, drag and drop or collapsible menus that they start to stumble. WAI-ARIA is a solution to help fix that by hooking into HTML to making it speak to screen readers. Gez Lemon’s Introduction to WAI ARIA, also available in Spanish, French and German, gives a good overview of what ARIA is.
  • Browser support for WAI-ARIA: Varying levels of support can be found in Opera 9.5, Firefox, Safari and IE8.
  • Screen reader support for WAI-ARIA: Screen readers are also catching up with varying levels of support in Jaws, WindowEyes, NVDA and Orca.
  • JavaScript library support for WAI-ARIA: The following libraries all have varying levels of support: Dojo from IBM, YUI! and JQuery.

Set up a sandbox

Whether on Mac or PC you want to set up a screen reader testing sandbox. This means you can isolate glitches with installs as well as crashes and reboots. If you’re on a Mac there is more need for a sandbox as screen readers tend to be built for Windows. There’s plenty of virtualization software out there to help you do this including Parallels, VMWare, Virtual PC and so on.

Test with a variety of screen readers

Next you need to find screen readers that support WAI-ARIA. It’s not practical to buy every screen reader on the market although it is advisable to get what you can. If budgets are low there are plenty of free options available to you:

  • Jaws from Freedom Scientific: designed for Windows the free demo allows for you to test for 40 minutes at a time before rebooting.
  • WindowEyes, GW Micro: a fully functional demo is available for Windows but times out after 30 minutes.
  • NVDA, NV Access: again just for Windows and only works with Firefox however it is available in full for free.
  • Firevox: available in full for free on Windows and Mac but only works with Firevox.
  • Orca: available on Linux, free and open source.
  • Fangs: this screen reader emulator  is a Mozilla Firefox extension that displays a text representation of a web page similar to how a screen reader would read it.
  • Thunder: available for Vista or XP.

Configuring screen readers

Screen readers are designed for the non-sighted user so you may want to set preferences to disable or change what you don’t find useful. For example I always switch off automatic page scroll in Jaws as I don’t want the page to scroll in line with the content being read out. You may also want to disable features that have the screen reader start up automatically as you boot your system up.

You may also find it useful to configure your screen reader keyboard shortcuts so that they are all the same in each one. That way you don’t have to remember one thing for Jaws and another for NVDA.This can be especially useful with regards to shortcuts for switching the virtual buffer on and off.

One final tip is to take a snapshot of everything once it is installed, this means you have it at hand to revert back to should anything go wrong. This can be done at various different stages throughout the set up on the sanbox.

Install browsers

Ensure that you have all the browsers that support ARIA installed on your machine (see above).You really should be testing across browsers just as you would in any development process.Note that while Opera is building in WAI-ARIA support that it works best with a screen reader on Mac with Windows to follow.

Testing advice

  • Get comfortable with a screen reader first: finding your way round using a screen reader can be a tricky process. I’d recommend never diving in and testing unless you’ve spent some time just browsing the web in general and getting familiar with the environment. One really good trick is to spend an afternoon, day or a week browsing the web with just a screen reader with the monitor and mouse switched off. This is the ultimate crash course and once you’ve done it you’ll have a much better understanding of the types of challenges people face when browsing.
  • Use real users: ultimately real users will reveal much more than just you testing or even just one genuine screen reader user testing. Use any screen reader users in-house you may have but also make sure you test with regular users. Often screen reader users within a development team are power users so mixing it up a bit with regular users is important. Never assume you know what the best solution is.
  • Switch off the monitor: so often it’s not the technical accessibility that is a problem but how easy it is to follow and discover information. Simply put is what the user hears making sense? Have you described roles, states, properties and live regions appropriately? Do you need to teak the text on the page to make it clearer what the page is doing?
  • Disable your mouse: check that all widgets and functionality can be accessed and triggered using only the keyboard just as you would if accessing desktop widgets. Check that you can use either shift or arrow keys to get focus onto elements and manipulate them. For example tab onto a slider then use the arrow keys to move the slider back and forth.
  • ARIA test cases: Before you start testing make sure you know what support there is for the ARIA being tested. The best plac to look for this is the ARIA test cases page on Codetalks.

References and resources

  • Developing accessible widgets, video from Todd Kloots. An excellent summary covering much of what is above as well as the specifics of ARIA implementations.
  • Introduction to WAI ARIA, also available in Spanish, French and German
  • Accessibility of Rich Internet Applications, WebAim. A good overview of ARIA and what it is.
  • Designing for screen reader computability, WebAim.
  • Survey of preferences of screen reader users, WebAim. Touches upon ARIA but is a good overview of what versions of screen readers and what browsers people use and preferences when browsing.
  • Codetalks community, wiki and information on ARIA.
  • Free ARIA excellent mailing list that covers all things ARIA contributed to by some of the best people in the business.
  • Establishing a screen reader test plan, an article of mine over at Spotless Interactive
  • 13 thoughts on “Setting up a screen reader test environment

    1. Brilliant post Henny,

      I would like to add a very important point here: get the RIA tested by the end users – screen reader users.
      There are a few who’ll understand the functioning of these dynamic applications easily but a large portion of them, who will revert to the traditional method of accessing the web page information.

      Its this group who will help us all develop more user friendly applications as they will give us inputs of their expected behavior of a web app/page.

      Always BPositive!

    2. Hi Priti, you are absolutely right and although I do say test with end users I probably should have flagged this up more obviously at the start of the post.

      When I wrote this post I was really targeting it at the developer who is using ARIA who must always test even if they will be having it user tested by screen reader users after they have built the app.

      It would be great to hear from anyone who has been involved in user testing either as a tester or a developer. I’m sure the findings would be using to everyone and not just those involved in the project being tested.

      Lovely to catch up with you at CSUN by the way 🙂

    3. Well pointed out Mike, thanks. I’ve now added it to the list and also put down Thunder as well. Not sure why I didn’t have them originally…may be because I don’t really use them so much so they were off the radar.

    4. Pingback: stefsull (Stephanie Sullivan)

    5. sirs
      Automatic page updates causing problems with your screen reader
      I have the above problem on my computer screen while started.
      pls show solution

    6. Great post. As an SEO, I think heavily about accessibility from a search engines point of view. However, you just can’t ignore the fact that at least 10% of the users out there may not even be able to read your text.

      Great post, i’m now looking into developing my own with my own points, I will notify you if you like to see it.

    7. Thank you for an interesting post. However, I would like to point out that the license for the free JAWS “40 minute version” is for product evaluation purposes only. The license explicitly states that development and testing is not permitted. You will have to buy a full license if you want use JAWS to test and develop.

    8. I want to also note that testing with a screen reader is an _interoperability_test_ and is fine to determine if the specific assistive technology (AT) you are using works for people with disabilities, but since it is assistive technology, sometimes it ‘cheats’ and provides more accessibility to the end user than is properly coded into the application or content under review. Testing with AT should not be the primary mode of inspecting for accessibility! Inspecting the platform APIs, e.g., via Inspect32 on Windows, or JavaFerret for Java applications and the document markup via Web Accessibility Toolbar and other API/Document object model inspection-based approaches should be used. Not only does this eliminate confusion in determining if the technologies under inspection are coded properly, it allows clarification if AT has defects or simply doesn’t yet support the standard APIs or interfaces available, so that can be taken back to the AT vendor for a fix or feature request. It also really helps developers who may not know AT, but who can determine if it is coded properly.

    9. hi,
      Is Flash accessible to JAWS-12 in below browsers?
      ie. IE (All version)
      Firefox (Versions 3.0 onwards)

      iheni Reply:

      Hi harsit – while not about Jaws 12 itself perhaps this article on Flash and keyboard focus across browsers will be useful.

      In general (aside from just Flash support) Jaws is for Windows wont work with Safari. Chrome has pretty poor support for screen readers and Opera only supports VoiceOver on Mac.

      Marco Zehe from Mozilla has the latest on accessibility support for Firefox 4 and will be able to answer questions about Jaws 12.

    Comments are closed.