TalkBack: An Open Source Screenreader For Android
Posted:
Tuesday, October 20, 2009
Earlier this year, we blogged about project Eyes-Free — a collection of Android applications that enable efficient eyes-free interaction with your mobile phone. Since then, one of the questions we have received most often is about a complete access API to enable general purpose adaptive technologies such as screenreaders.
We are happy to announce the first version of such an API as part of the latest Android release (Donut). This new API is now available within the Android 1.6 SDK , and we welcome developer feedback. The Android Access framework generates android.view.accessibility.AccessibilityEvent in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.
You can see this API in use within our Open Source Android screenreader TalkBack. With TalkBack installed, standard Android user interface elements such as ListView produce spoken feedback during user interaction. Applications SoundBack (for producing non-spoken auditory feedback) and KickBack (for producing haptic feedback) generate additional augmentative output and demonstrate how multiple access applications can be active simultaneously.
in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.
What This Means For Developers
If you are interested in developing innovative access solutions on Android and have been eagerly waiting for our access APIs, the Donut SDK contains what you have been waiting for — including a set of free voices for English (US and UK), French, Italian, German and Spanish. You can use TalkBack, SoundBack and KickBack as a starting point for designing your own access innovations.
If you are an Android developer interested in making your applications more widely usable, you can use TalkBack and friends to quickly verify whether your applications remain usable when not looking at the screen. In this context, here are a few coding tips to ensure that your applications work out of the box with these tools:
End-users of Android 1.6 (Donut) can enable TalkBack, SoundBack and KickBack via the Accessibility section of the Settings menu. You need to do this only once i.e., once enabled, these access applications remain active across restarts. Note that depending on your Android device, you may need to install these applications from the Android Market; we will post videos that demonstrate step-by-step instructions for specific Android devices in the Eyes-Free channel on YouTube.
Providing Feedback
We (T. V. Raman, Charles L Chen, and Svetoslav Ganov) will be continuously improving the underlying APIs and access tools, and we look forward to your questions and feedback on the Android Developers Group.
We are happy to announce the first version of such an API as part of the latest Android release (Donut). This new API is now available within the Android 1.6 SDK , and we welcome developer feedback. The Android Access framework generates android.view.accessibility.AccessibilityEvent in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.
You can see this API in use within our Open Source Android screenreader TalkBack. With TalkBack installed, standard Android user interface elements such as ListView produce spoken feedback during user interaction. Applications SoundBack (for producing non-spoken auditory feedback) and KickBack (for producing haptic feedback) generate additional augmentative output and demonstrate how multiple access applications can be active simultaneously.
in response to user interaction events; the event payload contains additional details about the event, e.g., the user interface control that received focus. This access framework enables the creation of general purpose screenreading applications that make all of Android's user interface, as well as native Android applications built with standard Android widgets usable without looking at the screen.
What This Means For Developers
If you are interested in developing innovative access solutions on Android and have been eagerly waiting for our access APIs, the Donut SDK contains what you have been waiting for — including a set of free voices for English (US and UK), French, Italian, German and Spanish. You can use TalkBack, SoundBack and KickBack as a starting point for designing your own access innovations.
If you are an Android developer interested in making your applications more widely usable, you can use TalkBack and friends to quickly verify whether your applications remain usable when not looking at the screen. In this context, here are a few coding tips to ensure that your applications work out of the box with these tools:
- Ensure that all visually drawn UI controls have meaningful textual labels.
- Ensure that users can navigate to all controls in your application using the trackball.
- Ensure that navigating controls in your application with the trackball results in a meaningful traversal order.
End-users of Android 1.6 (Donut) can enable TalkBack, SoundBack and KickBack via the Accessibility section of the Settings menu. You need to do this only once i.e., once enabled, these access applications remain active across restarts. Note that depending on your Android device, you may need to install these applications from the Android Market; we will post videos that demonstrate step-by-step instructions for specific Android devices in the Eyes-Free channel on YouTube.
Providing Feedback
We (T. V. Raman, Charles L Chen, and Svetoslav Ganov) will be continuously improving the underlying APIs and access tools, and we look forward to your questions and feedback on the Android Developers Group.
Really cool!
ReplyDeleteIs there any list available which features all eyes-free apps?
I think what you guys ar contributing to the world of accessibility is a revolutionary step to making sure blind people have access to phones. For the longest time, we've had to purchase screen readers such as mobile speak just to gain access to the device and it's features. Now, this will change with the talkback development. Thanks.
ReplyDeleteIt is refreshing to have this functionality out of the box but I think it is critical to include internet and email support of at least the included browser and email clients. With email I only have speech access to the sender and subject in the email app and no access at all in the gmail app. There is also no support at all in the browser. I think this is in the right direction but I have not heard any info as to when these two issues are to be addressed. I think all the other potential issues with this application such as not being able to stop speech or not having caller ID speech without buying an app are all very minor compared to these two gaping holes in speech support. I am supporting you guys buy spedning the money and buying an android device and buying into this screen reader in its infincy and I want to have it succeed but these are really important applications and until there is some accessibility for these internet applcations I can't see any wide acceptance of this platform as an alternative to something like mobile speak or the iPhone (neither of which I care for). I hope someone can provide some updated info as to when talkback can support email and internet. I will be waiting anxiously. Good first steps.
ReplyDeleteI had a discussion with a friend of mine about the fact that the accessibility api is stil not as developed as it is on the iPhone. I was wondering if the talkback screen reader will be more streamlined with the main api to provide better access to mainstream apps.
ReplyDeleteAs a blind end user, I want to strongly agree with the poster who is asking for support of email and web browsing. Any user who is willing to spend money on a high end phone will expect to be able to access those features, and both the iphone and MobileSpeak offer users these options. You guys have a really interesting product here. I currently use an iPhone, but have spent a lot of time playing with my husband's Motorola Droid, and like it a lot. We are looking to get both of us on the same carrier,so if you were to make those apps accessible, I'd switch in a heart-beat. Thanks for all your work so far, you're off to a great start!
ReplyDeleteOne thing, there is no clear documentation o n google part on what accessibility feature is supported and hat is not. TV Raman and Chen are doing absolutely wonderful service, but google should get into the game and make a corporate attempt at accessibility, put several more engineers at work.
ReplyDeleteemail and browser support is very critical and i am surprised android, even froyo does not have that. It reads only subject and I cannot get talkback to read contents of email. Please have this in gingerbread. Having nexus one for 3 days and i love it, but we need full accessibility support and easy to configure.
is there any app that reads a webpage aloud? or lets you pick an email to read aloud, rather than only reading new ones that come in? talkback only reads actions, and the webpage address..
ReplyDeleteThis may be "necro" raising, but I had been searching everywhere and had not found any info.
ReplyDeleteSo I'm asking this:
Do Android Honeycomb 3.0 (a long way from 1.6 discussed here) have screen magnifier?
I DO NOT! want audio assistive technology for a simple reason: I'm Deaf and find them useless. I have low vision and use screen magnifier.
At present, iPad 2 already have full screen zoom app pre-installed in it and is looking to be one I would buy. However, I'm fascinated by Motorola Xoom that have Android Honeycomb.
So if Android 3.0 doesn't have what I need, then iPad is it for me.
Thanks for your help.
It's really great!!!
ReplyDeleteI installed the screenreader with a OCR-Software on the Galaxy-Tab for my grand-father. But the screenreader stops speaking after a few lines of the scanned text. Is there any possibility to fix this?
Thanks a lot!
Lucas Kolb.
Hello,
ReplyDeletewhat languages are supported by TalkBack?
Awesome idea. Just one thing I have an application installed on three different devices. Handheld device from Motorola, Samsung and HTC. When I open up my application on Motorola while accessibility is turned on (talkback app installed) it provides me better options. for e.g: 4 edittexts in an application. if I click one it will say "1" and so on(on Motorola). But in Samsung and HTC it says click instead of numbering. How can I achieve the same counting in other devices?
ReplyDelete