Originally posted in The Washington Post
April 5 at 11:46 AM

Three women test out the new automatic alt text feature on their Facebook mobile app. (Facebook)

Matt King sat down one Saturday morning six years ago with what seemed like a simple mission: He wanted to add a few friends on Facebook.

“I eventually accomplished my goal,” he said. “But I have to say, it took me an entire morning — at least 4 hours — to do what somebody else probably could have done in a few minutes.”

The reason it took so long is that King, in addition to being an engineer with a past as a world-class cycling athlete, is blind. He was born with a degenerative disease that slowly cost him almost all of his vision by the time he was in college.

Now, he’s one of the people trying to make Facebook usable for everyone — including the new feature that Facebook is rolling out Tuesday that will let blind Facebook users “see” pictures that their friends post like never before. This feature, called Automatic Alternative Text, is just one way tech companies are changing the way that blind people experience the world, on and offline.

How blind people “see” the Internet

There are an estimated 39 million blind people around the globe, according to the World Health Organization. And just because they can’t see, it doesn’t mean they can’t use the Internet.

But the way they experience it is very different from how sighted people do.

“You can see a whole screen full of information and because of that you’re able to make decisions about how to get to the information you need,” explained Chris Danielsen, director of public relations for the National Federation of the Blind, or NFB. “For a blind person, it’s a bit more cumbersome.”

The blind rely on tools called screen readers to narrate websites and apps or translate them into braille — one line of text or link at a time. Instead of using a mouse, they also generally jump around a page using keyboard shortcuts.

And because screen readers work by looking at a website’s code, not just what is visible on a display, sites can be almost incomprehensible if they weren’t designed to be read by the tech.

But there are simple things, often invisible to a sighted viewer of a site, that a developer can do to help. One is making sure images on a site have “alt texts,” descriptions of what is in the pictures so they can be described by a screen reader.

“There are all kinds of little tricks like that,” said Danielsen. “If the website is properly coded, you can get around it pretty easily — otherwise, it can be very frustrating to navigate.”

The early days of the Internet were actually easier for blind users because almost everything was text, according to King. “But overtime not just the content, but also the design has become much more visual in nature,” he said.

How Facebook is helping them “see” more

Social media has been a big part of that shift. Facebook says that people share more than 2 billion photos per day across its various products and apps alone. But those pictures can cause an accessibility headache because they often don’t come with text that describes what they show.

Facebook is trying to fix that with Automatic Alternative Text. It works when a user has Apple’s built-in screen reader turned on and selects a picture in the app. Then Facebook uses artificial intelligence algorithms to detect basic features in the image and create a new alt text that the screen reader will share with the user — basically describing what’s in the photo.

If a picture shows a couple overlooking the ocean while wearing sunglasses, for instance, the new alt text will look something like this: “This image may contain: two people, smiling, sunglasses, sky, outdoor, water.”

How Facebook’s new feature helps the blind ‘see’ pictures

Facebook’s new automatic alt text feature allows visually impaired users to interact with the Facebook iOS mobile app in a new way. (Facebook)

It’s not exactly poetry — and eventually, King said, the hope is to make the alt texts more like a narrative. But it still gives blind users a new way to imagine what’s in images they otherwise couldn’t really picture.

The feature will only be available for the English-language version of Facebook’s main iOS app at launch — and it will only recognize around 100 basic concepts because Facebook only wants to suggest that an image contains things its artificial intelligence tech has a good track record of identifying, according Jeffrey Wieland, who leads the company’s accessibility team.

An industry-wide effort

Facebook is far from alone in trying to make the Internet more usable for people who are blind or suffer from impaired vision. Twitter recently announced a new feature that will finally allow users to attach an alt text to the images they upload.

And most major tech companies have dedicated accessibility teams. In fact last summer, Facebook poached King from IBM — where he had worked on accessibility for a quarter of a century.

Many tech companies, including Facebook, along with several universities are also part of a working group focused on better preparing students to create inclusive technology.

The World Wide Web Consortium, an online standards body, has also long had international best practices for how to design sites that everyone can use. But not all sites adhere to them.

And while there have been many strides in making the Internet more accessible for the blind, getting to this point has been tough. The NFB, for example, has brought legal action against tech companies in the hopes of making their services more accessible: It sued AOL in 1999, and in 2008 Apple agreed to a settlement with the NFB and the Massachusetts attorney general aimed at making iTunes accessible to the blind.

Those legal actions are generally brought under the Americans With Disability Act. But the 1990 law was put in place before the Internet became the driving force it is today, and the Justice Department has delayed guidelines that will cover what private companies need to do to make their online services accessible to blind people under the Act until 2018.

Yet as more and more of our lives go online, it’s even more important that the blind are able to stay connected, according to Danielsen.

“This is not just a matter of making life easier for blind people — it’s a matter of how they’re actually going to get along in the 21st century,” he said.

What’s next?

There are other technologies on the horizon that could take the same artificial intelligence approach to pictures that Facebook is pushing online now into the physical world.

Microsoft showed off an app dubbed “SeeingAI” at its Build 2016 developers conference last week. In a video presentation, Saqib Shaikh — a blind developer who has been with Microsoft for a decade — used the app with a pair of smart glasses that let him take pictures in real time and have the world around him narrated.

The company said it doesn’t have a timeline for when the app actually will be released. But its development hints at how wearable computing could have the same sort of game-changing effect on how blind people experience the real world as screen readers have had for their digital world experiences.

King says there’s still more to be done on the online front, however. He easily can imagine Facebook using its artificial intelligence tech and facial recognition data to tell him not just how many people are in a photo, but also who they are, for instance.

There may be privacy concerns about individual users, but it could at least be used to help identify public figures, he said.

“If everyone else can see that’s a picture of Donald Trump, I should be able to know that too,” King argued.

Given Facebook’s move-fast-and-break-things approach to development, the accessibility team will no doubt have more features in the pipeline. And Wieland says the work King and the rest of the team does is essential to Facebook’s larger goal.

“Our company’s mission is to connect the world,” he said. “Accessibility is actually a requirement to help us achieve that mission.”

Andrea Peterson covers technology policy for The Washington Post, with an emphasis on cybersecurity, consumer privacy, transparency, surveillance and open government.