TalkBack: An Open Source Android Screenreader

Video: Introducing TalkBack, An Open Source Screenreader

1 Video: Introducing TalkBack, An Open Source Screenreader

Device Used: Motorola Droid On Verizon

We briefly introduced TalkBack in the previous video while enabling Accessibility from the settings menu.Here, we show off some of this screenreader's features.

TalkBack is designed to be a simple, non-obtrusivescreenreader. What this means in practice is that you interactdirectly with your applications, and not withTalkBack. TalkBack's job is to remain in the background andprovide the spoken feedback that you need.

TalkBack works with all of Android's native user interfacecontrols. This means you can configure all aspects of the Androiduser interface with TalkBack providing appropriate spokenfeedback. What is more, you can use most native Androidapplications --- including those downloaded from the AndroidMarket with TalkBack providing spoken feedback.

Here are some examples of Android applications (both from Google as well as third-party applications available onmarket) that work with TalkBack:

  • Google Maps: Perform searches, and listen to directions.
  • YouTube: Search, browse categories and play.
  • Simple Weather: Listen to local weather forecasts.
  • Facebook: Moving around on the social Web.

But in this video, we'll demonstrate the use of a very simple butuseful Android application --- the Android Alarm clock.

  • Launch: I launch the alarm clock from Marvin's eyes-free application launcher.
  • TalkBack: TalkBack takes over and starts speaking.
  • Navigate: Navigating with the trackball speaks the alarmunder focus.
  • Activate: Activating with the trackball produces appropriate feedback.
  • Navigate: Selected alarm displays its settings in a list-view which speaks as we navigate.

Author: T.V Raman<raman@google.com>

Introducing The Android Access Framework

Video: Introducing The Android Accessibility Framework

1 Video: Introducing The Android Accessibility Framework

Device Used: MotoRola Droid on Verizon

Starting with Android 1.6 --- fondly known as Donut --- theplatform includes an Accessibility API that makes it easy toimplement adaptive technology such as screenreaders. Android 1.6comes with a built-in screenreader called TalkBack that providesspoken feedback when using Android applications written in Java.

The next few videos will progressively introduce TalkBack,SoundBack and KickBack, a suite of programs that augment theAndroid user interface with alternative output.

All of these special utilities are available through optionAccessibility in the Android Settings menu. Once activated, theaccessibility settings are persistent across reboots, i.e., youneed enable these tools only once.

Notice that because I have accessibility enabled on my phone,all user actions produce relevant auditory feedback. Thus, eachitem is spoken as Imove through the various options in the settings menu. The spokenfeedback also indicates the state of an item as appropriate.

Activating SoundBack produces non-spoken auditory feedback;KickBack produces haptic feedback.

Author: T.V Raman<raman@google.com>

Connecting The Dots: Marvin And Android Access

Video: Connecting The Dots: Marvin And Android Access

1 Video: Connecting The Dots: Marvin And Android Access

When we first launched project eyes-free in early spring 2009, wepromised to post frequent video updates to the eyes-freechannel. Well, sadly, we have been remiss in keeping that promise --- but all in a good cause --- we were busy building out theneeded accessibility APIs in the core Android framework.

We're now returning with a fresh set of video updates thatdemonstrate the new accessibility framework in Android, and howthese access related tools mesh with the Eyes-Free shell shownearlier.

To summarize:

  1. All of the eyes-free utilities from project Marvin continueto be developed in order to provide fluent eyes-free interaction.
  2. The Marvin shell that we demonstrated last time continues tobe my default home screen.
  3. We have added an application launcher on the Marvin screenthat can be launched by stroking 8.
  4. This launcher uses stroke dialing to quickly navigate andlaunch applications.
  5. With the launch of the Accessibility API in Android1.6, and the accompanying Open Source TalkBack screenreader,I can now launch any Android application, e.g., GoogleMaps or YouTube.
  6. TalkBack provides spoken feedback for native Androidapplications, including the settings menu.
  7. You can use Android Market to install third-partyapplications, many of these work outof the box with TalkBack.

We'll demonstrate these, and a variety of other new coolenhancements in these forthcoming videos, stay tuned!

Author: T.V Raman<raman@google.com>

Date: 2009-03-30 Mon

HTML generated by org-mode 6.08c in emacs 23

Eyes-Free Home: The Marvin Shell

Video: Eyes-Free Home: The Marvin Shell

1 Video: Eyes-Free Home: The Marvin Shell

Device Used: T-Mobile G1 from HTC

The Marvin shell pulls together available eyes-free applicationsto provide an integrated user experience. Note that talkingapplications can come from many sources, with project Eyes-Freebeing but one such source. For other exciting talkingapplications that use our open Text To Speech (TTS) APIs, seethe Android Marketplace, where you will find many useful toolsthat integrate seamlessly with Marvin.

when you install the Eyes-Free Shell, you can choose to makeMarvin your default home screen --- this means that pressingthe home button always brings up the Marvin shell. To return tothe default Android home screen, hold down the back button for3 seconds or more.Here is a brief description of the Marvin user interface.

1.1 Single Touch Access To Useful Tools

The Marvin shell uses the Stroke Dialerto provide single touch access to useful tools right from thehome screen. You can explore this interface by moving your finger around the screen --- as you move over the buttons,Marvin speaks the associated action. Lifting up the fingerexecutes the current action. As an example,the top row of the keypad, i.e., 1, 2, and 3provide status information.Stroking to 4 brings up your favorite short-cuts, and 6speaks your current location using geo-location informationobtained from Google Maps. Pressing 7 connects to your voice-mailbox, and pressing9 invokes Voice Search to obtain quick spokenanswers from Google e.g., current weather for your location.Finally, the applications that appear on the shortcuts screencan be customized by editing XML file


/sdcard/eyesfree/shortcuts.xml
on your SD-Card --- as is apparent, this is a power-user feature:-)!

2 Talking Mini-Applications For Single Touch Access

Here, we demonstrate some of the talking mini-applications thatcan be accessed from the Marvin screen.All of these mini-applications speak useful information withoutthe need for the user to do some form of context switch.

2.1 Device State

Available from 1 on the Marvin screen, this mini-applicationannounces useful information such as signal strength, andavailability of WiFi networks.

2.2 Date And Time

Available on 2 on the Marvin screen, this mini-application provides single-touch access to currentdate and time.

2.3 Battery State And Power

Pressing 3 on the Marvin screen speaks the current batterylevel and announces if the phone is presently being charged.

2.4 Knowing Your Location

Available as 6 from the Marvin home screen,this mini-application announces your present location based oninformation acquired via GPS and the cell network. It speaks yourcurrent heading using the built-in magnetic compass, looks up thecurrent location on Google Maps, and announces the location interms of a nearby address and street intersection.

Author: T.V Raman<raman@google.com>

Date: 2009-03-30 Mon

HTML generated by org-mode 6.08c in emacs 23

Talking PhoneBook: Eyes-Free Communication Device

Video: Talking Phonebook: Eyes-Free Communication Device

1 Video: Talking Phonebook: Eyes-Free Communication Device

Device Used: T-Mobile G1 from HTC

Pressing the menu button while in the Talking Dialer togglesbetween dialing mode and phonebook.When in phonebook, you get eyes-free access to your contactswith the ability to quickly move to the contact that you wishto call.

When in the phonebook, you can scroll through your contacts and press the call button to callthe current contact. In addition, you can use stroke dialingas explained below to quickly move to a specific contact.

1.1 Entering Letters Using Stroke dialing

We covered eyes-free input with the touch screen in the earlier video on stroke dialing--- in that video, we illustrated the concept via a traditionalphone keypad. Here, we extend that techniqueto enable textual input. In the explanation below, we will use compass directions to helpwith orientation.As before, we will use relative positioning i.e., for the rest ofthis explanation, you can start anywhere on the touch-screen--- though we recommend (for reasons that will become evident)that you start somewhere close to the middle of the screen.

1.2 The Eight Compass Directions

Defining the center as where you first touch down on thescreen, notice that you can stroke in any one of the 8 compassdirections, and that opposite pairs of compass directions e.g.,North and South, can be thought of as opposites.So we get 4 pairs. We enumerate these below,associate them with the 4 Google colors, and equate themto their equivalent strokes from the stroke dialer:

  • Red: North-West and South-east 1 and 9.
  • Blue: North and South --- 2 and 8.
  • Green: North-East and South-West --- 3 and 7.
  • Yellow: East and West --- 4 and 6.

Now, let's place the letters of the alphabet on these 4 circlesas follows:

  • Red: A ... H
  • Blue: I ... P
  • Green: Q ... X
  • Yellow: Y ... Z.

To input a given letter, we stroke to the circle containing thedesired letter, trace along the circle till we hear the letter wewant, and lift up the finger to make the selection. Letters arespoken in a female voice while moving along the selected circle;lifting up the finger speaks the selected letter in a male voice.

Notice that conceptually, we have defined a fairly simple mappingfrom strokes to letters of the alphabet!

1.3 Skimming The Contact List

So to cut a long story short, you dont need to scroll throughthe contact list. To quickly jump to a contact, use the techniquedescribed above to input the first letter from the contact's name--- the aplication jumps to contacts starting with thatletter. At that point, you can either scroll, or enter additionalletters to further filter the contact list.

1.4 Examples Of Using Strokes For Letters

Notice from the mapping shown earlier that we can enter eachcircle either at the top or bottom. Thus, entering the redcircle at the top gets to A, while entering it at thebottom gets us to E. This means that the 8 letters on anygiven circle are no more than 3 steps away --- for example, toenter C, one needs to trace clockwise from A, orcounter-clockwise from E. As an example, H is only 1 stepfrom A on the red circle. similarly, P is only 1 step fromI on the blue circle.

Author: T.V Raman<raman@google.com>

Date: 2009-03-30 Mon

HTML generated by org-mode 6.08c in emacs 23

Talking Dialer: Eyes-Free Communication Device

Video: Talking Dialer: Eyes-Free Communication Device

1 Video: Talking Dialer: Eyes-Free Communication Device

Device Used: T-Mobile G1 from HTC

The MotoRola Droid does not have a call button. Instead, Use the search capacitivebutton, i.e. the button on the extreme right, in place of the call button.

So now, let's use the stroke dialer for something practical ---let's make phone calls with our smart phone!Well, we know Marvin would disapprove if we just made phonecalls, so rest assured, we'll do a lot more later!

Pressing the call button on Android phones launches thebuilt-in dialing application. When using the Marvin shell,pressing this button launches the Talking Dialer application--- if you are not using Marvin as your home screen, you canlaunch this dialer as you would launch any Android application.

The Talking Dialer announces dialing mode upon start up.You can start dialing using the technique described in theprevious video on stroke dialer --- if you make a mistake, simplyshake the phone to erase. Once you have finished dialing, pressthe call button to initiate the call. The application speaksthe number you're about to dial, and makes the call once youpress the call button to confirm.But you say


Dialing phone numbers is so passe'!

--- well, there is still hope for the Talking Dialer. Inaddition to dialing mode, the Talking Dialer provides an easyto use Talking Phonebook that provides eyes-free access to yourcontact list --- we will cover this in our video on the talking phonebook.

Author: T.V Raman<raman@google.com>

Date: 2009-03-30 Mon

HTML generated by org-mode 6.08c in emacs 23

Stroke Dialler For Android

Video: Stroke Dialer For Android

1 Video: Stroke Dialer For Eyes-Free Keypad Input

Device Used: T-Mobile G1 from HTC

The stroke dialer enables one-handed keypad input using thetouch-screen --- and that without having to even look at thescreen. Here is how it works --- we start with a briefdescription of the problem that asks the rightquestion. The answer becomes self-evident as you follow thisvideo.

1.1 The Problem

On-screen keyboards typically show some buttons on the screenthat you activate by touching the screen. To activate suchbuttons, one needs to look at the screen, because the buttons areplaced at specific points on the screen, i.e., they areabsolutely positioned. So what if you want to activate suchbuttons without looking at the screen?From the foregoing description, it's clear that the only reasonone is forced to look at an on-screen keyboard is because thebuttons are absolutely positioned. So let's relax thatconstraint, let's use relative positioning to place thebuttons.

We'll start with a keyboard we're all familiar with, thetelephone keypad. Since we're using relative positioning,let's place the center of the keypad wherever you firsttouch the screen.So, to dial a 5, you just touch the screen.

Now, you know where 5 is --- it's where you first touchdown. But look, since you know the layout of a phone keypad, youcan now find all the other digits relative to the 5. So forexample, 2 is directly above 5 --- so to press 2, youtouch down on the screen, and stroke up before lifting yourfinger.similarly, you stroke down for an 8, or diagonally up for a1.

In real life, we both hear and feel as we press physicalbuttons. This form of synchronized auditory and tactile feedbackis essential for creating user interfaces that feelrealistic. The stroke dialer produces a slight vibration as thefinger moves over the various buttons that is synchronized withan auditory tick to achieve this effect. It also producesspoken feedback to indicate the button that was pressed.

To conclude this video, let's dial a few numbers.

Author: T.V Raman<raman@google.com>

Powered by Blogger