Crowdsourcing iPhone App Lets Sighted People Lend Their Eyes to the Blind

By Veronique Greenwood | May 12, 2011 1:23 pm

vizwizWith VizWiz, the blind can take a picture, ask a question, and get an answer back from a real person in seconds.

What’s the News: With the web as their eyes, the blind will able to read menus, identify canned foods, and tell whether that park has any free benches without having to walk over. That’s the vision of a team of computer scientists behind an iPhone app called VizWiz, which lets people take a photo of whatever’s perplexing them, record a question like “What denomination is this bill?” and send it off to real people online who’ll respond in a matter of seconds with “That’s a 20.”

How the Heck:

  • Blind people have workarounds for the kinds of tasks the sighted use their eyes for—folding dollar bills in certain shapes, keeping the cans of tomatoes separate from the cans of beans, and so on—but these measures often require the input of a sighted person at some point, and they’re not very efficient. An app like this would give the blind more independence.
  • Many simple tasks, like reading an address off a letter, are phenomenally difficult for computer intelligences. So the scientists are working with Amazon’s Mechanical Turk, a system that employs people around the world to do odd jobs computers can’t, like choosing the best picture for a product website or reading signs in photographs.
  • Then, to solve the problem of speed, the team wrote a program called quikTurkit that works to recruit people even before the question is sent, so there’s always someone on hand to answer. In the latest version of VizWiz, the average turnaround time on a question was 27 seconds. Not bad.

The Future Holds: VizWiz, which is being tested by teams of blind volunteers, hasn’t left the lab yet. But the volunteers are fans: it would be “very useful,” one said (via New Scientist), “because I get so frustrated when I need sighted help and no one is there.” Though an in-depth study [pdf] on VizWiz was released last year, there’s no word yet from the scientists on when this will hit the market. Soon, one hopes.

(via New Scientist)

Image credit: Rochester Human Computer Interaction Group

CATEGORIZED UNDER: Technology, Top Posts
  • Elissa

    Wow! What a great example of using the power of technology and the goodwill of people to do something productive and positive in the world. I hope this gets developed further and goes to good use!

  • Gwen H

    I have a question that I know a lot of people want to ask: How does a blind person know what is in front of the lens? The pictures probably do not look as good as the cans pictured here. But good idea nonetheless.

  • Cathy

    Blind people actually have a very good mind’s eye visual-spacial recognition that’s tied into their sense of touch and orientation. It’s like how you can find the light-switch in the dark – you can tell roughly where it is supposed to be, and you can figure out the exact position with no more than 2-3 touches on the wall. If they know there are two cans in front of them, they can touch the cans once or twice and then point an iPhone camera at them pretty accurately.

  • http://tongal.com James D.

    Wow.

    Seems like a great application for Mechanical Turk–I’m sure Turks will enjoy and be rewarded more helping the blind identify objects than pointless (but maybe fun) things like translating Moby Dick into emoji.

  • Anon

    You can use touch and memory, or even use echolocation[1] to detect cans or other stuff.

    Might be able to shake the can to figure out which is corn and which contains baked beans, but figuring out the brand of baked beans can be harder, or other stuff which provide even fewer clues (bottled drinks), and that’s where this app might be helpful.

    [1]
    http://www.youtube.com/watch?v=uobuBc2GO0o
    http://www.youtube.com/watch?v=qLziFMF4DHA

  • http://www.parkerpensnz.com parker pen

    That song is nice。

    Great Blog. I add this Blog to my bookmarks.

    Nice post. Thanks for sharing these tips
    .
    Thanks for sharing information.Looking forward to more stuff.

    That was so cool.

    Your site theme looks fantastic.

    уж… комментарий так комментарий

    i hope you have a good day with your Friends……

    Write well, support you to move on and look forward to your next article.

    This article is really great, strong support

    I really liked your article and I shared with my friends in my facebook account ..

    This is a nice site and I wanted to post a note to let you know, good job!

  • Brian Too

    Is there any kind of vetting or quality control system?

    I can imagine bad advice being given, either by carelessness, unclear inputs, a misplaced sense of fun, or even outright cruelty.

  • http://thatguylam.posterous.com ThatGuyLam

    I agree with above, but with looking behind the story, it looks like they are recruiting people to work this job specifically… possibly incorporating a feedback system where the blind customers can know who gave them answers and Vote Up/Down (or something similar) would be nice. These smart people have probably already thought of that…

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

80beats

80beats is DISCOVER's news aggregator, weaving together the choicest tidbits from the best articles covering the day's most compelling topics.
ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »