Memory screening using eye-tracking on mobile devices

Investigators at Emory Brain Health Center have developed a platform for evaluating visual memory, while someone views photos for a few minutes on an iPad.

Emory researchers, led by Goizuieta Alzheimer’s Disease Research Center director Allan Levey and biomedical informatics chair Gari Clifford, are working with the company Linus Health to develop the VisMET (Visuospatial Memory Eye-Tracking Test) technology further. Results from the most recent version were published in the journal IEEE Transaction on Biomedical Engineering, and the Emory/Linus team continues to refine the technology.

The goal is to screen people for memory issues, identifying those with mild cognitive impairment (MCI) or Alzheimer’s disease. The task — difficult to call it a test — was designed to be more efficient, easier to administer, and more enjoyable than tests currently used.

“We think this could be a sensitive and specific method for detecting visual memory impairment, and it’s convenient enough for use on a wider scale,” Levey says.

The VisMET technology is based on this observation. When someone with MCI or Alzheimer’s views a photo twice, and the photo has been changed the second time (example: an object in the scene has been removed), their eyes spend less time checking the new or missing element in the photo, compared with healthy people. This is because the regions of the brain that drive visual memory formation, such as the entorhinal cortex and hippocampus, are some of the earliest to deteriorate in MCI or Alzheimer’s.

Currently, when someone is evaluated for memory loss, they get a battery of “paper and pencil” tests to assess verbal memory. Researchers say the alternative of viewing photos on a tablet could be less intimidating for those taking the test, as well as easier to administer and score. The only instruction given to study participants was to enjoy the images.

“The current way memory tests are implemented can be stressful,” says software engineer Alvince Pongos, who is co-first author of the IEEE TBME paper, now at MIT’s McGovern Institute for Brain Research. “The difficulty of standard memory tests can lead to test-givers repeating task instructions many times, and to test-takers being confused and frustrated. If we design simpler tasks and make our tools available in the comfort of one’s home, then we remove barriers allowing more people to engage with their health information.”

Using cutting-edge eye-tracking technology, the mobile version of VisMET can identify people with Alzheimer’s disease or MCI with a test accuracy of 70 percent. For the study published in IEEE TBME, the 250 participants came from the Emory Healthy Aging Study and the Goizueta Alzheimer’s Disease Research Center — in all cases, people who had detailed clinical evaluations.

A previous version of the test, implemented on a desktop computer monitor and an external eye tracker, was published in Learning and Memory in 2019. The newer version uses the built-in camera from a tablet.

Reaching an accuracy of 80 percent is the goal to make VisMET mobile for broader clinical and at-home use, Pongos says. The researchers were able to improve the detection of AD and MCI to 76 percent if they restricted their analyses to only subjects whose eye-tracking calibration passed a minimum quality assurance threshold.

To make VisMET available on tablets, the researchers built an iOS app that collects pictures of users’ faces and eyes as they look at a calibration screen. The researchers then analyzed the pictures with iTracker, a neural network-based system for estimating gaze based on 1.5 million publicly available photos of faces. Once the system was calibrated, the Emory researchers used the model to estimate users’ gaze while they viewed VisMET on the tablet. Using an AI technique known as “transfer learning”, the system was adapted to work on the MCI subjects using a webcam.

Emory and Linus Health researchers are continuing to improve the technology by fine-tuning the images that are presented to study participants, and by making sure that it works robustly under different lighting and background conditions.

Linus Health was founded in 2019 with the mission of improving brain health across the spectrum of neurocognitive care by delivering scientifically validated tools and data-driven insights when and where they are needed. Linus is focused on developing remote digital health tools that harness artificial intelligence in order to improve brain health outcomes.


Posted on by Quinn Eastman in Neuro Leave a comment

About the author

Quinn Eastman

Science Writer, Research Communications qeastma@emory.edu 404-727-7829 Office

Add a Comment