New AI from Microsoft will auto-caption images for the visually impaired

Microsoft has created a new artificial intelligence (AI) that will automatically caption images in emails and documents for the visually impaired.

Using VIsual VOcabulary (VIVO), the machine learns a visual vocabulary by controlling large amounts of paired image-tag data. The AI software will then read out the captioned images.

To help teach the AI how to best describe the images, a second dataset of correctly captioned pictures is used. Saqid Shaikh, a software engineering manager with Microsoft’s AI platform group, states:

 

Ideally, everyone would include alt text for all images in documents, on the web, in social media – as this enables people who are blind to access the content and participate in the conversation. But, alas, people don’t

 

Ultimately, the AI is expected to be twice as efficient as Microsoft’s existing captioning system. This new auto-captioning system is already available in Azure Cognitive Services’ Computer Vision package, so developers who want to use this when building apps can now do so.

The improved auto-captioning feature is also expected to be available in Outlook, Word, and PowerPoint later this year.