Finger-mounted reading device for visually impaired

A finger-mounted device with a built-in camera which has the ability to convert written text into audio has been built by the researchers at the MIT Media Laboratory for visually impaired users. The device guide the users, when their finger is along the line of text and provides feedback in either tactile or audible way as the system generates the corresponding audio in real time.

It works as a translation device for visually impaired person, it is like the finger is ‘seeing’ the audio. The only problem with this device is the maintenance of connection because if it is broken, then it breaks the illusion. Therefore people really need a fast, real-time feedback to maintain this connection. The project also describes the report of results of a usability study conducted with vision-impaired volunteers. The researchers tested several variations of their device while testing it on volunteers. Two haptic motors, one on top of the finger and the other beneath it were including to understand the vibration of the motors. It indicates whether the subject should raise or lower the tracking finger.

They have another version with audio feedback and it does not include motors. The audio has a musical tone which the volume is increase when the user’s finger began to drift away from the line of text. When the researchers tested the motors and musical tone in conjunction, there was no consensus among the subjects. Therefore, they are concentrating on audio feedback, since it allows for a smaller, lighter-weight sensor.

An algorithm used for processing the camera’s video feed is the key to the system’s real-time performance. As the user positions the finger at the start of a new line, every time the algorithm makes a host of guesses about the baseline of the letters. Most of the alphabets and words tend to cluster together but the algorithm selects the median value of the densest cluster. That particular value constrains the guesses and the system makes with each new frame of video, which reduces the algorithm’s computational burden.

The algorithm also tracks each individual word as it slides past the camera, when estimate of the baseline of the text is given. The device have broader applications apart from visually impaired person, for instance, children with dyslexia can also make use of this device. They are also developing a finger-mounted device that gives visually impaired users information about distant objects.