I'm the developer of the Speak app for the blind

You're doing a great service! How do you manage QA testing? Do you have a core group of users who helps you out?

The coding I do myself usually late at night. I am a members of a google group of visually impaired people named ANTAD. I spin my ideas there, ask for suggestions and ask for the new features to be tested and get feedback.

This will probably be the most obvious question but,

Why did you make that app? What drove you to do it?

I have been working for a while for giant tech companies. Today's cutting edge technology is led by these companies. I started having ideas how state of the art tech can benefit people with disabilities. The problem is that giant companies do not address niche groups for reason of focus. They something do it for reasons of PR but usually don't bring such ideas to full production.

On the other hand if small companies raise money in order to address the needs for niche groups such as the visually impaired community the products will be very expensive due to the small market.

So I got the idea I might start working on such solutions volunterally. What I try to do is the best I can distributed for free, meaning I cannot use any cloud services for example.

I started working with the visually impaired community knowing very little about all the different kinds of problems people face and even less on how people perform day to day tasks.

Since then I found the positive feedback extremely rewarding.

Have you heard of WebAIM (https://webaim.org/)? When I used to do freelance web development, I subscribed to their mailing list, and it was really enlightening. Accessibility technology/standards have existed for a while now, but very few developers bother with them unless they work in a sector where accessibility compliance is legally required, resulting in the web being a lot less usable than it could be. It's a real shame.

I didn't hear about it. I took a quick look, looks very serious. Accessibility it a very complex issue one of those things no one gets to. I am trying to contact Google now. They released an amazing app called Google Lens. It can give blind people great value but the app is not compatible with the accessibility settings. It's very annoying. Hope I will be able to reach someone there and have it fixed.

What popular toys do you think could be adapted for children with visual impairments or blindness? Maybe by simply adding Braille to them?

(Been working on writing about toys for children with visual impairments/blindness, and was saddened by how little is on offer)

Oh wow, what an interesting question. It never crossed my mind... I suppose it differs very much by the age. I think for very small children it should be things with a lot of touch and sound feedback. For the older ones I just don't know. I think I saw once a video game made especially for the blind, it was an adventure game that was only audio but I never encountered another.

It is quite sad, never thought about it.

How does your app recognize text captured by the camera?

I use a google text recognition library. If you mean how does a computer recognize text that's another question.

how does a computer recognize text

Yup, that's what I meant.

I'll try to answer it's a complex question. Till recently that way it was done was to explain to the computer how a letter A looks like. Then how a letter B looks like. That didn't work well at all. When people invented the first computers they defined the most difficult problems to solve as math problems and playing chess. They saw that as the highest of human abilities. They got that done really quickly but then realized that teaching a computer to understand an image or speach is way more difficult, we humans just do it seamlessly because our brains are built that way.

About 7 years ago a new technology started showing a lot of promise. The technology is called machine learning. It is what enables self driving cars, supermarkets without human cashiers, electronic assistants and also text detection. The idea in machine learning is basically take a lot of annotated data, in our case a lot if images with text together with the expected text on it. Show it to the computer again and again till it "learns" how text looks like and it can do it by itself. In order to get these things to work properly you need a lot of data, so the companies that train these systems are those that have access to a lot of data. Google, Apple, Microsoft, etc. I used Google text recognition software. There is a free and paid version, I use the free version since I want my app to be distributed for free. Similar apps to mine use paid cloud services which forces then to take subscription fees from their users.

Machine learning has been around for decades. But yeah, the big companies make it popular more recently.

That's true machine learning has been around for a while. But the technology didn't work, not beyond basic digit recognition. The problem was, although no one knew it, was much more data and much more computation. The combination of running machine learning training on GPUs and feeding it with much more data did the trick. Now it seems obvious but at the time no one expected that to be the case.

If this interests you, there is a very interesting TED talk by one of the pioneers of this method.

https://www.ted.com/talks/fei_fei_li_how_we_re_teaching_computers_to_understand_pictures?language=en

I have an idea for an app for a different disability but have no coding, software, etc experience or knowledge. How/where can I start?

It's such a general question I don't know how to answer. You might want to colaborate with some coder there is no shortage in those. You can also learn app development online but not everyone is good with those things.

How does the app handle the variety of input fonts? What about input issues from users who might not be able to see exaxtly where they need to point the camera?

The app recognizes different fonts since its core technology is a machine learning algorithm. That means that everything it saw during its training it will recognize. The software was created by Google and they have huge amounts of data with different fonts.

For blind users the app has a scanner feature. The user can point at things and the app will identify text, color, objects and barcodes. The text scanner works in the following manner, when text is in the frame the phone will vibrate. When the text is completely inside the frame it is read.

Do.you know the app taptapsee?

Yes of course, it is a single feature app. It performs image description. It's slow but very impressive. Such technology can be great for auto tagging large databases of images. Not sure how a blind person would use this app in his day to day life.

How privacy friendly is the app, do you guys sell user data to third party data brokers?

Speak app works offline, does not require internet and saves and sends nothing to the cloud.

This is a serious matter. I heard from blind people that they don't like using apps that connect them to volunteers because they are afraid about what they will see.

What is your take on the lawsuit filed,I think, in L.A. against Fenty’s website and their lack of accessibility to apps like yours?

I never heard about it, I'll check it out...

Hey! Thanks a lot for the job you are doing!

I wanted to ask a more personal question. What drove you into creating this app? Was there someone from your inner circle that had visual problems?

Bonus question: When was the first time that you saw a person using the app by both the person in need and the helping party ? How did you feel?

I have been working for a while for giant tech companies. Today's cutting edge technology is led by these companies. I started having ideas how state of the art tech can benefit people with disabilities. The problem is that giant companies do not address niche groups for reason of focus. They something do it for reasons of PR but usually don't bring such ideas to full production.

On the other hand if small companies raise money in order to address the needs for niche groups such as the visually impaired community the products will be very expensive due to the small market.

So I got the idea I might start working on such solutions volunterally. What I try to do is the best I can distributed for free, meaning I cannot use any cloud services for example.

I started working with the visually impaired community knowing very little about all the different kinds of problems people face and even less on how people perform day to day tasks.

Since then I found the positive feedback extremely rewarding.

Most of the apps I have seen appear to be for visually impaired people - how easy would it be for somebody who has no vision at all to use?

I'm not sure if you ask about my app or generally. Blind people don't use the phone in a regular way. They use it in a mode called talkback. With that mode the controls are completely different. My app works differently when talkback mode is activated. Sone features just make no sense fir blind people like changing font size. Regarding features I gave special features for the blind like color detection, that could be used to identify your clothes. Barcode detection that could be used to identify products when shopping.

I was talking about your app specifically - I suppose I am dubious about some of the usability of some of these features - how do they find the barcode on an item to scan it for example?

If the barcode is anywhere inside the frame it will be read properly ever if it is upside down. So the blind person, let say at the supermarket that an item and flips it till the barcode is detected.

I made a short video demonstrating this.

https://m.facebook.com/story.php?story_fbid=10157240250232351&id=670102350

I m legally blind. Will this app be able to read labels if I point the phone camera at it?

What do you mean by labels?

How much information do you gather on your volunteers? And how do you use it?

I don't really understand the question. I don't use any volunteers. I am a member of a google group named ANTAD they are visually impaired and help out test new features.

I don’t see any proof that you’re the developer, rather than a rando who is pretending you created it. It’s customary to link to a professional page, or a page with your name on it, and have that page say you’re doing an AMA.

I put a link to google play page, you can contact the developer by mail from there, I'll answer you that it's me.

Will that work?