Animated face helps deaf with phone chat

From: New Scientist, UK - Aug 4, 2004

15:08 02 August 04

NewScientist.com news service

Software that creates an animated face to match someone talking on the other end of a phone line can help people with hearing difficulties converse, suggests a new study.

The animated face provides a realistic impersonation of a person speaking, enabling lip-readers to follow the conversation visually as well as audibly.

The prototype system, called Synface, helped 84 percent of participants to recognise words and chat normally over the telephone in recently completed trials by the UK's Royal National Institute for the Deaf (RNID).

The RNID trials involved hard-of-hearing volunteers trying to decipher preset sentences and also taking part in real conversations.

Synface takes around 200 milliseconds - one fifth of a second - to generate the animated annunciations. But the system incorporates a fractional delay, so that the face is perfectly synchronised with the voice on the end of the line.

Regional dialects

Synface runs on an ordinary laptop and can be connected to any type of phone, including a cellphone. It uses a neural network to match voice to mouth movements. This mimics the way neurons operate inside the brain and can be trained to recognise patterns.

The neural network used by Synface identifies particular sounds, or "phonemes", rather than entire words. This has been shown to be a particularly fast way of matching words to animation. By concentrating on sounds the system can also represent words that it has not encountered previously.

The technology is not meant to assist people who are profoundly deaf, but rather those who have some hearing difficulties. Around one in seven people in western countries fall into this category. So far, Synface has been trained to work in English, Swedish and Dutch. It could also be fine-tuned to recognise different regional dialects.

"The accuracy still needs to be improved," admits Neil Thomas, head of product development at the RNID. But he says it could eventually make life easier for many people who have trouble hearing.

"There are a lot of people who struggle with using the telephone," Thomas told New Scientist. "It really gives them an added level of confidence."

The system was developed by researchers at Royal Institute of Technology, in Stockholm, Sweden, University College London, UK as well as Dutch software company Viataal and Belgian voice analysis firm Babletech.

Will Knight

© Copyright Reed Business Information Ltd.