Eye controlled robotic arm

A research team at Indian Institute of Science, Bengaluru has designed a robotic arm that can be manipulated by eye movement using a computer interface, to help people with Severe Speech and Motor Impairment (SSMI).

This research, unlike other eye tracking systems, uses only webcams and computers instead of head implantation devices. Until now, it was very difficult for people with SSMI to physically operate devices such as joystick, mouse or trackball, or use speech recognition systems.

Using a robotic arm designed by the team of researchers led by Prof. Pradeepta Biswas at the Center for Product Design and Manufacturing (CPDM), disabled people can use their eyesight to perform mechanical tasks. It helps them to work on fabric printing and other handicrafts on par with the Normal peers.

The project has been commissioned at Vidya Sagar (Spastics Society of India) in Chennai and is in use. Many of the disabled people (who have cerebral palsy) are unable to focus precisely at a single point in their visual field, due to uncontrolled gaze movement. They are also not comfortable to look at all portions of the visual field equally. Prof. Biswas and his team used computer vision and machine learning algorithms to analyse live feeds of facial video from the users, and were able to estimate where the user was looking. Later they coupled this with an Augmented Reality application. This way the team has shown that the user can perform tasks like picking up and dropping objects using robotic arms, and placing them where they desire, as long as it was within reach of the robotic arm.

The research team is using video games as a medium to teach this new technology and may open the gateway to e-learning future for disable people.