Tag Archives: Testing

Quick tip: testing web content for screen readers without a screen reader

We can’t all get access to a screen reader better yet find the time to tame them so here is a quick tip on how to test how well your web content supports screen reader users. All you need is a browser, a plugin and 5 minutes.Continue Reading Quick tip: testing web content for screen readers without a screen reader

Establishing a screen reader test plan

The lovely folk over at Spotless Interactive have invited me to be a guest writer. An honour indeed given the good stuff they do and publish.

In my first post I take a walk through what you need to consider when establishing a screen reader test plan. It’s an overview look at what you need to consider when incorporating screen reader testing in an overall website test plan and is intended to go with my previous post here: setting up a screen reader test environment.

There’s some good articles over at Spotless and I’m looking forward to contributing more.

“Where’s my Googlebox?!” – adventures in search for silver surfers

I forget that at its core the web is all about  ”search” so it was humbling and eye opening to spend two days in the company of 8 silver surfers aged 60 to 80 testing  Opera desktop and observing, amongst other things, how they went about carrying out searches.

It’s more or less the first skill you learn when you’re new to the web (our testers had between a month and 18 months experience each) and by far the most essential. It took me right back to how I felt when I first used the web and it was fascinating to watch how people tried to differentiate between web content, a browser and a search engine, often getting it wrong but for entirely for logical reasons. Our testers all came from the analogue world with little or no experience using computers.

So here are a few rough findings around the subject of search for older users new, or relatively new, to the Internet.

First let me describe the set up.

We had a vanilla install of Opera 10.10  with www.bbc.com/news set as the home page. We left the side panel open not because we were testing it as such but because we were curious to see how people used it when carrying out tasks. Finally we removed all additional toolbars that a user would not typically have.

The BBC search field centered at the top of the page below the browser address box.

The BBC search field centered at the top of the page below the browser address box.

Website search versus the browser address field

All participants had a hard time distinguishing between the search field in the web page (positioned top-centre just below the browser address box), the browser address box and the browser search box. When asked to look up www.tesco.com most would write the URL in the BBC search field and hit search.

When this didn’t work people would eventually venture up to the browser address box and start typing there  often typing text in the middle of the BBC URL.

Text typed into existing text in the browser address box

Others would click in the browser address box, highlight the existing URL then not know they could over-ride it by either writing  or using the ‘delete’ key. Only one tester knew to use the delete key. Not using the keyboard for anything other than typing text was a common theme as this group seemed to rely totally on the mouse to get about making me wonder if using a keyboard was only relied on when it had to be. I also had a sense that having a URL address box populated with text put people off using it.

The main, and obvious issue here though was people not being able to differentiate, or understand what the browser was and what web content was. The focus was very much on content with the browser menus and features ventured into as a last resort. This is something that we’ve already come a cross before in tests and is not an issue restricted to just this group.

Browser search versus website search

Very few of our testers ventured to the browser search box opting instead to use the search field of the site. When they did there was a degree of confusion around what the field did. Most looked for a ‘Go’ button and in lieu of that accessed the drop down menu (showing various search engine options).

The browser search field and drop down menu of search engine options

It was clear that typical user behavior was to take the hands away from the keyboard and use the mouse to hit ‘Go’. In other words hitting ‘Enter’ was not commonly known linking back to this groups preference to do everything (bar typing text) using the mouse.

Using the Home browser button

When testers got lost default behaviour was to go for the browser ‘Home’ button or, in a couple of instances close the browser and start again. I’m really glad I saw this as I’d all but written off the ‘Home’ button as a bit of browser UI clutter (based on personal and peer preference admittedly).

Given the combined preference to set Google search as the home page and the almost universal avoidance of the browser search field this made a lot of sense.

“Where’s my Googlebox?”

As we worked with more testers it became evident that the preferred home page of choice was Google search. This may well account for people confusing the BBC website search field for the browser address box.

My Mum in law first brought this to my attention when, after we’d just set her up with browsing. I heard her shout in absolute frustration from the other end of the flat:

WHERE’S MY F@^&ING GOOGLEBOX?!

That’s when I realised that familiarity is key and having a ‘safe’ place to start from and return to makes all the difference when starting out with using the web. It all links into the confusion between the browser, web page and definition of what a search engine is. Being able to search the web from the browser is a hard concept to grasp and understanding that the browser is not the web page, or vice versa, problematic.

This is of course the tip of the iceberg (and only part of what we looked at during our testing) but I remain convinced that we have a lot to learn from this group. Some of the issues and barriers they hit I’ve seen seasoned users stumble upon and I think if we are going to make truly usable websites and browsers we need to go back to the source and learn from new and older users.

A big learning point for me, with a developer hat on, is to consider how your content works within the context of the browser – something that is rarely considered, if at all. This was evidenced by the placement of the BBC search field in the top centre of the page under the browser address box. While I don’t think BBC are wrong it is something that is worth considering especially given they are such as well known website (globally) and is prone itself to being confused with a search engine.

A second learning point was to not fall into the trap of making assumptions. Not everyone knows what a browser is, not everyone uses the keyboard for simple shortcuts (including ‘Enter” and ‘Delete”) and what we may think as logical as a result of doing something repetitively may not be to others.

A big thank you

We couldn’t have done these tests without the wonderful Digital Access Media Group at Dundee University, especially David Sloan. David provided the space, facilities, and hospitality for us and the wonderfully helpful participants who were great company as well as fantastic testers.

Thank you also to Lawrence Eng from Opera who flew in from San Diego especially to lend his extensive knowledge of Opera and user behaviour to the project.

We hope to do more testing and are already looking at how our findings can influence decisions on improving browser features and accessibility. Watch this space!

Setting up a screen reader test environment

Back in the day when web pages were web pages and not web applications you could just about get away with not testing with a screen reader. Using a combination of text based browsers, accessibility testing tools such as the Web Accessibility Tools consortium Accessibility Toolbar and and browser tools such as Preferences settings in Opera (see Opera > Quick preferences or Preferences), you could more or less muddle your way through.

This is not the case now though as browsers, screen readers, guidelines and technology has moved on and in many ways is catching up with one another. What follows are some tips to get you started when setting up a screen reader testing environment.

Why test with screen readers

  • WCAG 2.0 and JavaScript: Where WCAG 1.0 had a blanket ban on JavaScript WCAG 2.0 now allows it providing it’s accessible to assistive technologies, or in WCAG 2.0 talk accessibility supported.
  • WAI-ARIA: Screen readers can in fact handle most uses of JavaScript. It’s when they encounter areas of a page that update without full page refreshes or encounter functionality that is mouse orientated such as sliders, drag and drop or collapsible menus that they start to stumble. WAI-ARIA is a solution to help fix that by hooking into HTML to making it speak to screen readers. Gez Lemon’s Introduction to WAI ARIA, also available in Spanish, French and German, gives a good overview of what ARIA is.
  • Browser support for WAI-ARIA: Varying levels of support can be found in Opera 9.5, Firefox, Safari and IE8.
  • Screen reader support for WAI-ARIA: Screen readers are also catching up with varying levels of support in Jaws, WindowEyes, NVDA and Orca.
  • JavaScript library support for WAI-ARIA: The following libraries all have varying levels of support: Dojo from IBM, YUI! and JQuery.

Set up a sandbox

Whether on Mac or PC you want to set up a screen reader testing sandbox. This means you can isolate glitches with installs as well as crashes and reboots. If you’re on a Mac there is more need for a sandbox as screen readers tend to be built for Windows. There’s plenty of virtualization software out there to help you do this including Parallels, VMWare, Virtual PC and so on.

Test with a variety of screen readers

Next you need to find screen readers that support WAI-ARIA. It’s not practical to buy every screen reader on the market although it is advisable to get what you can. If budgets are low there are plenty of free options available to you:

  • Jaws from Freedom Scientific: designed for Windows the free demo allows for you to test for 40 minutes at a time before rebooting.
  • WindowEyes, GW Micro: a fully functional demo is available for Windows but times out after 30 minutes.
  • NVDA, NV Access: again just for Windows and only works with Firefox however it is available in full for free.
  • Firevox: available in full for free on Windows and Mac but only works with Firevox.
  • Orca: available on Linux, free and open source.
  • Fangs: this screen reader emulator  is a Mozilla Firefox extension that displays a text representation of a web page similar to how a screen reader would read it.
  • Thunder: available for Vista or XP.

Configuring screen readers

Screen readers are designed for the non-sighted user so you may want to set preferences to disable or change what you don’t find useful. For example I always switch off automatic page scroll in Jaws as I don’t want the page to scroll in line with the content being read out. You may also want to disable features that have the screen reader start up automatically as you boot your system up.

You may also find it useful to configure your screen reader keyboard shortcuts so that they are all the same in each one. That way you don’t have to remember one thing for Jaws and another for NVDA.This can be especially useful with regards to shortcuts for switching the virtual buffer on and off.

One final tip is to take a snapshot of everything once it is installed, this means you have it at hand to revert back to should anything go wrong. This can be done at various different stages throughout the set up on the sanbox.

Install browsers

Ensure that you have all the browsers that support ARIA installed on your machine (see above).You really should be testing across browsers just as you would in any development process.Note that while Opera is building in WAI-ARIA support that it works best with a screen reader on Mac with Windows to follow.

Testing advice

  • Get comfortable with a screen reader first: finding your way round using a screen reader can be a tricky process. I’d recommend never diving in and testing unless you’ve spent some time just browsing the web in general and getting familiar with the environment. One really good trick is to spend an afternoon, day or a week browsing the web with just a screen reader with the monitor and mouse switched off. This is the ultimate crash course and once you’ve done it you’ll have a much better understanding of the types of challenges people face when browsing.
  • Use real users: ultimately real users will reveal much more than just you testing or even just one genuine screen reader user testing. Use any screen reader users in-house you may have but also make sure you test with regular users. Often screen reader users within a development team are power users so mixing it up a bit with regular users is important. Never assume you know what the best solution is.
  • Switch off the monitor: so often it’s not the technical accessibility that is a problem but how easy it is to follow and discover information. Simply put is what the user hears making sense? Have you described roles, states, properties and live regions appropriately? Do you need to teak the text on the page to make it clearer what the page is doing?
  • Disable your mouse: check that all widgets and functionality can be accessed and triggered using only the keyboard just as you would if accessing desktop widgets. Check that you can use either shift or arrow keys to get focus onto elements and manipulate them. For example tab onto a slider then use the arrow keys to move the slider back and forth.
  • ARIA test cases: Before you start testing make sure you know what support there is for the ARIA being tested. The best plac to look for this is the ARIA test cases page on Codetalks.

References and resources

  • Developing accessible widgets, video from Todd Kloots. An excellent summary covering much of what is above as well as the specifics of ARIA implementations.
  • Introduction to WAI ARIA, also available in Spanish, French and German
  • Accessibility of Rich Internet Applications, WebAim. A good overview of ARIA and what it is.
  • Designing for screen reader computability, WebAim.
  • Survey of preferences of screen reader users, WebAim. Touches upon ARIA but is a good overview of what versions of screen readers and what browsers people use and preferences when browsing.
  • Codetalks community, wiki and information on ARIA.
  • Free ARIA excellent mailing list that covers all things ARIA contributed to by some of the best people in the business.
  • Establishing a screen reader test plan, an article of mine over at Spotless Interactive
  • Ask MAMA what the web is made of

    Last year Opera released data from MAMA (Metadata Analysis and Mining Application), a search engine that trawls web pages and returns results detailing page structures, what HTML, CSS, and script is used.

    MAMA examined 3,509,180 URLs in 3,011,668 domains  and returned results on how many pages validate (only 4.13%), how many use Flash (33.5%), how FRAMES were used, images, CSS and so on.

    This has given us such useful insights into how web developers are using code that Brian Wilson, who runs MAMA, is planning a second run and is interested in knowing what more we can check for. Some of the things I’d like to see assessed are support for ARIA, HTML5 attributes and how headings are structured.

    If you have ideas leave a comment and we’ll look at getting it included. I can’t promise it will be possible to add everything but we’ll do what we can. Don’t forget to check MAMA and see what we already cover first.

    Beta testers needed for a shiny new accessible Elearning system

    It has to be said that I’ve never had more pleasure than working with Barrier Break Technologies (the guys behind India Techshare) when it come to accessible Flash. Working closely with Adobe, RNIB in the UK, Vision Australia and disability organisations and groups all over the world I’d say they have a pretty good handle on things.

    So I was very happy to find in my inbox this morning a request for beta testers for the BarrierBreak Course Authoring Tool (BCAT). Simply put BCAT is designed and developed to assist teachers/authors in creating accessible Flash based Elearning courses. Spot on, I thought, Flash is such a great technology for Elearning but it’s fairly unrealistic to expect teachers to know how to create Flash let alone accessible Flash.

    So Barrier Break are inviting users with disabilities to have a look and share their thoughts with them. So without further ado here are the details about the tool and how to get involved:

    Right to education is universal. Education has evolved from blackboard to keyboard. Elearning has broken the geographical barriers and made it possible for people to access knowledge with ease and convenience.

    Despite the benefits offered by Elearning, it is not possible for students with special needs to access Elearning courses. Students using assistive technologies can with little ease access HTML based elearning but are deprived of the rich learning experience provided by technologies such as Flash. The common myth that prevails is that ‘Flash cannot be made accessible’ Its time that we over come this myth and change the mind set to ‘Flash can be made Accessible’.

    Net Systems Informatics & BarrierBreak Technologies have taken the initiative to break this myth and have launched the Beta version of “BarrierBreak Course Authoring Tool (BCAT)”. BCAT is designed and developed to assist teachers/authors in creating accessible Flash based Elearning courses. BCAT is an easy to use tool with in-built keyboard and screen reader support enabling students with disabilities to experience the power of Elearning!

    BCAT comprises of two main components, Course Authoring Tool based on Moodle Learning management System and Flash Architecture to access the course contents. In addition, the teacher/author can add various accessibility options to the course such as: Alternate Text, Captions, and Transcripts.

    Once the course is published, the contents are generated in to XML and users can access the course using a Flash player. The Flash course thus created provides users with an “Accessibility Panel” to meet their needs:

    • Show/Hide Captions
    • Audio Transcript
    • Change Themes
    • Small/Large Icons
    • Show/Hide Labels
    • Change Text Size

    We invite users with disabilities, teachers/authors, accessibility community at large to access BCAT and provide their valuable feedback and suggestions to make BCAT a tool for inclusive education.

    Access and sign up to the Beta version of BCAT

    Once you sign up, you will receive an email including further instructions. Along with the email, you will receive a ReadMe file, documentation for creating a course, documentation for using the course and bug report file.

    The Beta Test Run is open until 20th January 2009. All suggestions are highly appreciated and will help us to make BCAT more user-friendly and accessible for all.

    It’s good to see that accessibility in education is getting a little love from projects such as this one Project:Possibility and the Opera Web Standards Curriculum. More exciting things are to come too with the Web Standards Project Curriculum Framework, expect some news about that soon.