It’s more or less the first skill you learn when you’re new to the web (our testers had between a month and 18 months experience each) and by far the most essential. It took me right back to how I felt when I first used the web and it was fascinating to watch how people tried to differentiate between web content, a browser and a search engine, often getting it wrong but for entirely for logical reasons. Our testers all came from the analogue world with little or no experience using computers.

So here are a few rough findings around the subject of search for older users new, or relatively new, to the Internet.

First let me describe the set up.

We had a vanilla install of Opera 10.10  with www.bbc.com/news set as the home page. We left the side panel open not because we were testing it as such but because we were curious to see how people used it when carrying out tasks. Finally we removed all additional toolbars that a user would not typically have.

The BBC search field centered at the top of the page below the browser address box.

Website search versus the browser address field

All participants had a hard time distinguishing between the search field in the web page (positioned top-centre just below the browser address box), the browser address box and the browser search box. When asked to look up www.tesco.com most would write the URL in the BBC search field and hit search.

When this didn’t work people would eventually venture up to the browser address box and start typing there  often typing text in the middle of the BBC URL.

Text typed into existing text in the browser address box

Others would click in the browser address box, highlight the existing URL then not know they could over-ride it by either writing  or using the ‘delete’ key. Only one tester knew to use the delete key. Not using the keyboard for anything other than typing text was a common theme as this group seemed to rely totally on the mouse to get about making me wonder if using a keyboard was only relied on when it had to be. I also had a sense that having a URL address box populated with text put people off using it.

The main, and obvious issue here though was people not being able to differentiate, or understand what the browser was and what web content was. The focus was very much on content with the browser menus and features ventured into as a last resort. This is something that we’ve already come a cross before in tests and is not an issue restricted to just this group.

Browser search versus website search

Very few of our testers ventured to the browser search box opting instead to use the search field of the site. When they did there was a degree of confusion around what the field did. Most looked for a ‘Go’ button and in lieu of that accessed the drop down menu (showing various search engine options).

The browser search field and drop down menu of search engine options

It was clear that typical user behavior was to take the hands away from the keyboard and use the mouse to hit ‘Go’. In other words hitting ‘Enter’ was not commonly known linking back to this groups preference to do everything (bar typing text) using the mouse.

Using the Home browser button

When testers got lost default behaviour was to go for the browser ‘Home’ button or, in a couple of instances close the browser and start again. I’m really glad I saw this as I’d all but written off the ‘Home’ button as a bit of browser UI clutter (based on personal and peer preference admittedly).

Given the combined preference to set Google search as the home page and the almost universal avoidance of the browser search field this made a lot of sense.

“Where’s my Googlebox?”

As we worked with more testers it became evident that the preferred home page of choice was Google search. This may well account for people confusing the BBC website search field for the browser address box.

My Mum in law first brought this to my attention when, after we’d just set her up with browsing. I heard her shout in absolute frustration from the other end of the flat:

WHERE’S MY F@^&ING GOOGLEBOX?!

That’s when I realised that familiarity is key and having a ‘safe’ place to start from and return to makes all the difference when starting out with using the web. It all links into the confusion between the browser, web page and definition of what a search engine is. Being able to search the web from the browser is a hard concept to grasp and understanding that the browser is not the web page, or vice versa, problematic.

This is of course the tip of the iceberg (and only part of what we looked at during our testing) but I remain convinced that we have a lot to learn from this group. Some of the issues and barriers they hit I’ve seen seasoned users stumble upon and I think if we are going to make truly usable websites and browsers we need to go back to the source and learn from new and older users.

A big learning point for me, with a developer hat on, is to consider how your content works within the context of the browser – something that is rarely considered, if at all. This was evidenced by the placement of the BBC search field in the top centre of the page under the browser address box. While I don’t think BBC are wrong it is something that is worth considering especially given they are such as well known website (globally) and is prone itself to being confused with a search engine.

A second learning point was to not fall into the trap of making assumptions. Not everyone knows what a browser is, not everyone uses the keyboard for simple shortcuts (including ‘Enter” and ‘Delete”) and what we may think as logical as a result of doing something repetitively may not be to others.

A big thank you

We couldn’t have done these tests without the wonderful Digital Access Media Group at Dundee University, especially David Sloan. David provided the space, facilities, and hospitality for us and the wonderfully helpful participants who were great company as well as fantastic testers.

Thank you also to Lawrence Eng from Opera who flew in from San Diego especially to lend his extensive knowledge of Opera and user behaviour to the project.

We hope to do more testing and are already looking at how our findings can influence decisions on improving browser features and accessibility. Watch this space!

The World Health Organization estimates 278 million people worldwide have some form of hearing impairment.

A Nielsen study suggests that there has been over a 300 percent increase in online video watching since 2003. Further, most watching is done during work hours. Workplace computers are often muted or have no speakers.

Several billions of videos are watched monthly worldwide, with many of them in different
languages.

See 22frames.com for more about this.

I had my first foray into captioning this week for a short video that a colleague Daniel Davis (@ourmaninjapan) did on remote debugging with Opera Dragonfly and Opera Mobile 10 to mark the release of Opera Mobile 10 Beta (go try it, it’s free).

I’m slightly pink faced to say I’ve not done any captioning before having always opted to transcribe video and audio so I had to start from scratch sourcing the right tool and figuring out how to go about editing and setting up a process. Whilst I set out to caption a video my purpose was also to see how easy or difficult it was as captioning is the poor cousin of accessibility considered to be expensive, time-consuming and only relevant to hand-full of people.

Before I launch into my findings below is the final product captioned in English, Japanese and Russian using  Overstream and hosted on YouTube. Big thank you to Daniel for the translation and original video and Vadim for the Russian. You can also watch the video on Easy YouTube.

There’s also a hidden Easter Egg in there, see if you can spot it.

Captioning benefits

Accessibility – this is the obvious benefit as you’ll be opening up your content to deaf and hard of hearing users as well as people find it easier to read rather than listen (or do both together). If you don’t have translated captions some non-native speakers may also find content easier to consume when reading captions.

Localisation – adding translations to your captions widens your potential audience massively. There are plenty of tools out there such as dotSUB that enable you to crowdsource translations and many hosts such as YouTube which support multiple caption tracks.

Mobile – users with mobile phones who may not have earphones or are in a noisy place also benefit. I do wonder how much can be visible on some small screens but certainly some people will find it useful.

Search – site indexing may also get a boost. For example YouTube supports video searching of caption data which also filters through into Google search.

Getting the right tool

There are more captioning tools out there than I’ve had hot dinners so I thought I’d narrow it down scientifically and just ask over Twitter what people recommended. My only stipulations were that it had to be quick, easy and free (what else!).

CaptionTube

I gave Google’s web based tool CaptionTube a go first. It’s super easy to get started as you just use a Gmail login and from there you upload video from your YouTube collection. So far so simple.

What I didn’t find so intuitive was the captioning interface itself. When dropping text into the timeline I wasn’t able to clearly see when text started and ended as the end time was measured in how long the segment was rather than when it stopped in the overall timeline. This just didn’t work for me.

The CaptionTube interface fails to show captions overlaid the video

In addition to that I had to flick between the Timeline and Preview screens to see the captions I’d just created overlaid on the video. With pages taking a time to download, not to mention breaking the rhythm of what I was doing, this really held me back. Too much buffering for my liking.

Overstream

Being a newbie to all this I wasn’t sure if I was expecting too much or missing the point but after a chat with Antonia Hyde – who knows a thing or two about accessible multimedia – I decided to switch to Overstream which had originally been recommended by AbilityNet.

This was altogether a lot better plus Overstream support a number of video providers: YouTube, Google Video, MySpace Video, Dailymotion, Veoh Blip.tv and Archive.org. It was pretty easy to upload a YouTube video but equally easy to miss a crucial instruction that you need to have the video in question playing in YouTube when you hit the upload button.

Overstream shows the edit box and video with captions overlaid on the same page.

The interface gave me much more of an integrated toolbox and by now I had an idea of what I wanted which helped. One huge bonus was being able to add text to the timeline, complete with start and end times, adjust time lengths and see in real time the text overlaid on the video on the same page.

I had a few problems trying to play the video once done in a new window with a URL warning popping up but it was easy enough to download the .srt file (with all the captions and timeline in) and upload that in turn to YouTube.

MAGpie

Next on my list to try is the downloadable tool MAGpie, from the National Centre for Accessible Media. I didn’t try it this time as Overstream got the job done plus MAGpie supposedly doesn’t play nicely with Intel based Mac’s. I did have a quick look at it however and while very clunky and old looking it does give you an the option to style captions which looks pretty good. I’ll be looking at this in more depth when I next caption something.

Stanford Captioning Service

John Folliot pointed me to Stanford Captioning Service which looks like an excellent service. All you need to do is upload a video file which then is put in multiple formats – FLV, MP4, MP3. These are then transcribed by Stanford contractors for a small fee. When the transcription is done Stanford do automatic timestamp generation to turn transcript into various formats – this part is free.

For my short video I was happy to transcribe and caption the audio myself but if I had longer videos to get caption I’d almost certainly use these guys. Victor Tsaran, head of accessibility at Yahoo!, used the Stanford Captioning service to caption a video about himself recently.

dotSUB

Lastly I dug out my login to dotSUB, who’s main selling point is enabling subtitling of videos on the web into, and from, any language. It’s also a collaborative tool so you can crowdsource community input and/or work collaboratively with your team to get the captions done. Of the tools tested this was by far simplest and easiest to use. 

Captioning tips

As soon as I got started I realised that I needed to have a process as to how I approached doing the actual work. Here are a couple of things that worked for me – let me know if you have any more worth adding to the list:

• Transcribe text before you start captioning – you can do this yourself, pay a professional to do it or use voice recognition. Even though the last two options are less labour intensive you will need to edit and double check text – especially with voice recognition.
• Break it down – once you have your transcript you’ll have a clear idea of the volume of words and quality. You can then break text into short sentences that fit on screen without obscuring too much of the screen real estate. All I did was use a text file and hit return after short sentences or natural breaks in a sentence. Once I started adding text to the timeline this had to be reworked as I went along but having it already drafted was a big help.
• Editing text - if you have a text that works verbatim then great, but this is unlikely and there’s nothing wrong with removing repetitions or false starts to sentences. The key is to keep it succinct while maintaining the original meaning and flavour of the language as well as the character of the speaker.
• Punctuation – I found that less is more. Obviously you want full stop at the end of sentences but Andrew Kirkpatrick, head of accessibility at Adobe, recommends removing commas at the end of lines. We don’t ‘see” punctuation when we hear people so visually breaking text down like this makes sense to me.
• Timing – you can create a bit of drama, suspense and humour by remaining faithful to how people speak and using timing to replace tone. For example, someone getting excited may talk in short sentences so break the transcript down so that it is given in short segments rather than having longer segments.

Check out captioning tips from the WGHB Media Access Group, captioning tips and tools from NCDAE and W3C Multimedia FAQ for more.

How long did the whole process take?

Captioning the 4.27 minute video took be the best part of 10 hours BUT this included researching tools, false starts as well as a bit of reading around the subject. If it’s a long video you definitely want it to be transcribed for you but if a short one like this you could estimate 1 to 2 hours depending on your typing speed and how audible the sound is.

After that, once you have the hang of adding text to a timeline you should be ok. I added text and allocated times as I went along but you can add text then allocate time second if breaking the two tasks work better for you. This probably took me about 1.5 hours.

All in all I’d average out a 4 minute video at 3 hours – but this will no doubt get better as it becomes more familiar.

It’s a bit fiddly to start with but smooth running once you get the hang of it and seeing the end result is completely worthwhile. It’s satisfying to know that the captions will help not just deaf users but also non-native English speakers as well a people looking at video on their mobile phone.

Update 20 November 2009

Google have just announced automated captioning of YouTube video which will include automatic time stamping as well as transcripts. This should be available soon and will have a huge impact for many users as well as influence in promoting captioning overall.

Cognition is probably one of the most under researched areas of accessibility and the least understood. As a result design of usable and accessible websites is incredibly speculative and hard to get right. Unlike accommodating users with screen readers (which is more of a precise art) making colours, text, fonts, layouts, images and icons speak to everyone ranging from those of us with learning difficulties, problems with perception, memory, cognition and comprehension is a hard task to fill. As autistic web pro Jamie Knight said:

Autism is a big spectrum, and in a very real way everyone is autistic!

We were lucky to get four fantastic speakers as well as a over 50 people who had a lot to contribute themselves in questions, discussions and comments over Twitter. Thank you to everyone who gave their time to make it such a valuable and informative event especially the speakers Antonia Hyde, Jamie Knight, David Owens and Ian Pouncy.

Huge thanks also to Kath Moonan who helped us find a room and the Opera Developer Network who sponsored the event.

Antonia Hyde – Accessibility Beyond Code

With years of experience working with people with cognitive and learning disabilities Antonia had a truly captive audience with her talk accessibility beyond code which included commentary on videos she had taken of Martin using websites and what he found tricky when using them

What struck me was how much her testers hit upon issues that I myself find problematic. Antonia explained just how important it is to be literal on the web and not assume that people understand even the most common of icons such as “i” for information: using icons and text is always a good way to go.

The image she showed below is a good case in point when it comes to being literal. The sign “Use escalators safely” is unhelpfully illustrated with a man running down the escalator looking to all intents as if that’s just the right thing to do. Perhaps a red cross through him would have illustrated the point better.

Antonia finished up highlighting that it’s not just the designer’s responsibility to make content accessible but also the developer and the content editor’s responsibility. I would add to this that it’s also the browser’s responsibility to help render and provide access to content that really helps the user. Something that we are keenly aware of at Opera.

Huge thanks to Martin for contributing his time and being videod.

Jamie Knight and Lion – Autism, the Internet and Antelopes

Jamie and Lion wowed us with his unique take on Autism, the Internet  and Antelopes (blog post and slides from the man himself). Being a “generalist” as he describes himself he took us through some of the projects he has worked on and what he does as a web professional.

He told us about his personal screen reader he’d built to help him read web pages when he was tired and illustrated how screen readers should not just be seen as access tech for the blind but for just about anyone who might find it useful listening to web pages. He made a good point as I occasionally (er, not enough) listen to my blog post content through VoiceOver on Mac in Opera to check my grammar which I really struggle with.

Jamie also talked in depth about how video, as great as it is, can be a problem when played automatically on download (especially when his screen reader is running) or the spoken word is too fast. He talks a little bit about this in his interview as well. He also explained that when he gets sensory overload he wished there was a way to convert text into sign to help give meaning to words. I couldn’t help thinking this would make sense for many of us who are more visual.

David Owens –  lessons learned doing usability testing

We got a first hand walk through from David on lessons learnt from user testing and I have to admit I was impressed to hear of an organisation out in the real world who were taking this group seriously. When his company initially set out to do user testing they didn’t specifically recruit people with cognitive impairments but along the way realised that they had some testers who were able to highlight key issues with their site.

David discussed style switchers on websites (A, AA and AAA for small, medium and large text etc) and how important these were for users who would never be comfortable changing their browser settings. Representing a browser maker I’m acutely aware of how important discoverability of features and preferences to help you browse are. It’s important to educate people and make them aware of how to access these however David’s point was that there are often people who will just never be comfortable changing things in the browser.

David also highlighted that you can’t assume a user remembers how to do something on your site if they’ve done it before. This was picked up by Patricia, my soon-to-be-mum-in-law, who has Fibromyalgia and often can’t remember how to do certain things. She summed it up perfectly:

Thank god other people find the same things difficult, I thought it was only me!

If we achieved nothing else on Saturday I’m happy that David helped Patricia realise that she wasn’t the one who is broken!

Ian Pouncy – content and cognition

Rounding off the day Ian gave excellent bitesize pieces of advice on content and cognition learned from his time working at Yahoo!. What I loved about is talk was that he wrapped in cross-overs with internationalsation such as clear, well worded text. He also did a walk through of a mock-up site showing us how he’d improve it.

His talk triggered some debate over lightboxes (those pesky ‘popups’, often of images that overlay a page) and how confusing they are to users who don’t understand where they have come from or how to get rid of them. I’d add to this quirks of keyboard access where they seem to be keyboard focusable but as part of the rest of the content of the underlying page making it a huge effort to tab to ‘close’ buttons (if they exist).

I was happy to see a Tweet from Alastair Campbell however suggesting he might look into the problem:

After #standardsnext (and considering an upcoming project), I think accessible lightboxes will have to be on the to-do list.

Go Alastair!

Final thoughts for me

Since joining Opera I’ve increasingly used our accessibility features within the browser while looking more closely at how people discover features within the browser that help access content in a way that is meaningful to them. As with any browser this is a huge challenge and there is an important balance in exposing access to preferences without further confusing the user.

It was great to gather feedback on what people wanted from the browser, or indeed expected, and will tie in nicely with user testing that David Sloan of the Digital Media Access Group at Dundee university and I will be doing this autumn with older users to see what they can teach us about improving our browser. So watch this space – we hope to have something to share with you early next year.

Links and things

Blog posts and Twitter coverage

• Jamie Knight
• Jeff Van Campen
• Léonie Watson
• Ian Pouncy
• Bruce Lawson – leave a comment for Bruce if you have any thoughts about stylable HTML5 forms
• Stanards.Next
• Tweets with the hash tag #standardsnext

Tools and resources

Here’s a list of bits and pieces mentioned on the day plus a few more that may help:

• WebAim research on cognition – Stepping Stones project
• WebAim: cognitive accessibility part 1 and part 2
• BBC HTML5 media player with captions
• Techdis tools
• RNIB Surf Right toolbar
• Flashblock for Firefox
• Stop autoplay for Firefox
• Accessing web content with Opera
• Using Opera 10 with VoiceOver on Mac
• WAI-AGE – Web Accessibility Initiative Ageing Education and Harmonisation
• Videos from Antonia Hyde of people with cognitive problems using the web
• Photos from Jeff Van Campen on Flickr

1. Go to Wii-ware.
2. Click Start.
3. Click Shopping.
4. Choose Wii Channel (third choice after Virtual Console and Wii-ware).
5. Scroll to bottom of list and click Internet Channel.
6. Download and select whether to save to Wii itself or to SD card.
7. Info page – “This software supports the Wiimote and selected keyboards”. Click OK.
8. Do you want to update/install? Click OK.
9. Go back to the Wii menu click “Internet channel” and away you go.

Thank you to @ourmaninjapan for the pointers.

As an extra bonus Nintendo are now offering the Wii Internet Channel for free. If you want to check it out but don’t have a Wii there’s plenty of information about it in the My Opera Wii community.

The issue

Plugin support typically needs an API that acts as a doorway connecting the plugin, browser and user. IE gets round this by using ActiveX – itself a closed propitiatory format – so users can tab into and out of the Flash. Of course keyboard access within the Flash content itself is handled by Adobe and is now considered to be keyboard accessible. So it’s really support for entering and leaving the plug-in with keyboard that is the issue.

The browser fix

Ideally there needs to be a standardized API that can be used across industry to enable plugin support across browsers. The most common  API is Netscape Plugin Application Programming Interface (NPAPA). First developed for Netscape it has subsequently been implemented in other browsers including Opera, Safari, Firefox, Konqueror, Google Chrome, and some versions of Microsoft Internet Explorer.

The kind of access NPAPI supports includes scripting, printing, full screen plugins, windowless plugins and content streaming but is not as powerful as ActiveX, and is still evolving – in particular tabbing into and out of the Flash movie.

Help is at hand however.

There is currently a proposal to solve the issue of Flash support being lead by Mozilla, Adobe, Opera, Apple, IBM and Sun which has now been accepted. Implementation will depend on collaboration between all stakeholders including plugin vendors and of course Adobe.

Aside from the the plugin API solution is there an alternative quick fix for Flash support in Opera?

There have been some discussions internally but it seems there is no quick fix that will completely, and satisfactorily, address the issue. There are two ways that NPAPI plugins can work. The first (default) is called “windowed”. This is essentially an OS window rendered on top of the browser. Keyboard input is therefore direct and not via the browser.

There are a couple of drawbacks with “windowed” however. Firstly it can pose security issues. Secondly it’s not a complete keyboard access solution because while getting focus into the plugin is possible getting focus out is not. This is key and really negates the point of being able to tab into the Flash Player because as a keyboard only user you’ll only get stuck there.

The second mode is called “windowless”, where the browser controls more of the plugin rendering. Here keyboard input goes via the browser (possibly depending on OS) and in turn is intercepted. The drawback with this solution is that real world support is limited as most plugins do not support this mode, and for those who do it’s not that widely used due to performance issues.

By far the best and most secure solution is standardising the NPAPI API so that it works across browsers with all plugins. Better not just for Opera but the web in general.

In terms of a solution for Opera it seems the fixes available now fall far short of what we would want to give our users. The good news however is that to implement support once the plugin API is ready should be fairly straight forward.

The developer fix

So where does this leave you as a developer, and more importantly your users? There is a hack you can use to give Flash keyboard access using a method in your Flash movie to focus a chosen element. You can then create a text link that calls this method to “skip into Flash”. This isn’t something I’ve tried and tested and I’d be interested to hear comments from anyone who has.

Update –  As Andrew Kirkpatrick points out there is another way to give Flash focus using the SWFFocus class. While the technique showcased discusses this in the context of Firefox I did a quick test in Opera 10 Beta and Safari 4 but had no luck accessing the content.

But I suppose the real question is why hack what you can already do using existing technologies supported across all browsers? Don’t get me wrong, I like Flash and have two thumbs up at Adobe for the work they have done to make it accessible, but if I’m building a site using Flash and knowingly locking out all non-IE users then I can’t use it.

As Christian Heilmann points out much of what Flash does can be done with existing technologies supported in the browser:

Using the DOM and JavaScript I can create HTML elements that work with all kind of assistive technology. Instead of hoping that keyboard users can access my Flash content I use what browsers already have – links, buttons and form fields – to interact with the it.

WAI-ARIA is also a core way to build screen reader accessible and tabbable web apps and widgets. Added to this the HTML5 <video> element will soon give us native support for video across browsers; something that Flash is used for extensively today.

So there are ways and means now to avoid the keyboard trap that Flash content poses for keyboard only users plus there is work to provide a universal solution in the form of the proposed plugin API. But for now I’d personally always opt for the standards based cross browser solution so as to ensure happy users and avoid additional work and hacks.