Hot Stuff »

January 27, 2015 – 12:25 am | Edit Post

 by John R. Quain, Senior Writer
 What is it? – You’ve heard of butt dialing. Get ready for wrist dialing. The Samsung Gear S is a smart watch that can make calls on its own, no …

Read the full story »
Big Picture

trend articles

An App A Day

the latest mobile apps that work for Boomers

Hot Stuff

short, timely product reviews

Guest Stars

contributions from noted tech experts and journalists

Tricks of the Trade

finding accessibility options in hardware and software

Home » Guest Stars

Helping Hawking

Submitted by on September 12, 2012 – 8:09 am 5 Comments

Intel’s Effort to Help the World’s Most Famous Physicist Communicate

By Justin Rattner, VP and CTO, Intel Corporation

As Chief Technology Officer at Intel, it’s my job to invent new technologies and to judge their market potential. We have been working for some time at Intel on something called facial feature recognition or FFR.  Imagine how much time you could save if instead of having to enter a password, your computer could look at your face and verify your identity. We’ve also been looking at using FFR to enable something we call avatar chat. Instead of sending a video stream of you sitting in front of a camera, we first send the other party your avatar, a 3D representation of you. It can be anatomically correct or it can take certain liberties with your image making you look younger or older or having more hair.

It was in October of last year when I realized there was another application for FFR that nothing to do with identity or chat. I was visiting with the eminent astrophysicist and popular author, Dr. Stephen Hawking, at his office in Cambridge, England. Hawking had invited me in the hope of finding some new technology that would allow him to improve his ability to communicate. Hawking suffers from what the Brits call motor neurone disease or, as it is known in the U.S., Lou Gehrig’s disease.

This degenerative disease causes the patient to lose muscle function over time. Most people with MND don’t live past their 40s, but thanks to the extraordinary care he receives, remarkably Stephen just celebrated his 70th birthday. Over time he’s tried a number of different communication schemes, but none has worked as well as the simple system he began using more than a decade ago. It consists of a light sensor attached to his eyeglasses that can detect when he squeezes his right cheek muscles. The light changes on the sensor and is detected by a bit of electronics which in turn tells his PC to stop moving its cursor.

Stopping the cursor over a particular character on the screen of the PC selects it and appends it to the end of the text string he’s typing. The cursor then resumes its scan of the characters on the screen. As Professor Hawking has aged, the system has become more error prone and his word output has declined. His hope was that I might have an insight on how best to improve on the simple light sensor.

While visiting Hawking and his staff It didn’t take me too long to realize that FFR might be just such an improvement. I asked the staff if Stephen had more than one facial expression. They hesitantly acknowledged that he did indeed. Well, then, I said, “if you can tell them apart, we can teach a computer to do the same.” They seem intrigued, but concerned that like many earlier efforts he might find such a system too invasive or too cumbersome. While searching the lunch buffet for something sweet, Professor Hawking’s administrative assistant came up to me and quietly said, “Stephen has five or six expressions that I can recognize.” With that I knew we were on to something. I explained the idea to Stephen who seemed genuinely excited. Returning home, I quickly engaged the Human Interface Technology team at Intel Labs in an effort to test my belief that FFR could be the next step in helping Stephen communicate with greater speed and accuracy.

To make sure we didn’t unduly burden Professor Hawking, the team scheduled a visit to see him in Cambridge.  They spent the better part of a day capturing his various expressions on high definition video. Stephen was the perfect subject and never tired of the exercise despite many hours of recording.

Once they’d captured the video they returned to the U.S. and began the analysis of Professor Hawking’s facial expressions. To everyone’s delight, the facial feature recognition technique worked quite well. They also went to work on improving the software that Stephen had used for many years. They built an improved version that Stephen would feel quite at home with as well as a new version that was designed to increase his writing speed. They were quite excited about testing it on him. Despite his condition Professor Hawking comes to the U.S. every spring to visit colleagues at Caltech and around the country. This year’s visit created an opportunity to test the FFR technology at Intel rather than at Cambridge. The tests were very successful, and Stephen encouraged the team to move ahead with plans to bring the system to Cambridge once it was fully implemented and debugged.

We are planning to install the system using the personal computer in Stephen’s wheel chair. An unobtrusive video camera will be mounted on the bracket that holds the computer screen. The light sensor on his glasses will be removed, and the system totally controlled by his facial expressions just as I first imagined it would work less than ten months ago. The whole team at Intel Labs is excited to see how well it all works in the field, and we expect we’ll all find out in the blink of an eye.  I’ll be back with more when I have the field trial data to report.

Justin Rattner is Intel’s Chief Technology Officer, Vice President and an Intel Senior Fellow.   He is responsible for leading Intel’s microprocessor, communications and systems technology labs and Intel Research at universities and research labs around the world. .

  • Amedar Consulting

    Wow! This can be one particular of the most useful blogs We have ever arrive across on this subject. Basically Wonderful. I’m also a specialist in this topic so I can understand your hard work.

  • Computer Repair

    hi!,I love your writing very much! proportion we keep up a correspondence extra approximately
    your article on AOL? I require an expert on this area to resolve my problem.

    Maybe that’s you! Taking a look ahead to peer you.

  • iscsi

    Very soon this website will be famous amid all blogging users,
    due to it’s good articles

  • Wesley Culbertson

    great points altogether, you just gained a new reader. What would you suggest in regards to your post that you made some days ago? Any positive?

  • Dorcas

    I do believe all the concepts you have presented in your post.
    They are really convincing and will definitely work. Nonetheless, the posts are very quick for beginners.
    Could you please extend them a bit from subsequent time?
    Thanks for the post.