Monitoring Parkinson’s: Putting your finger on it

An important aspect of developing new potentially ‘curative’ treatments for Parkinson’s is our ability to accurately test and evaluate them. Our methods of assessing Parkinson’s at the moment are basic at best (UPDRS and brain imaging), and if we do not improve our ability to measure Parkinson’s, many of those novel treatments will fail clinical testing and forever remain just  “potentially” curative.

Researchers from Madrid (Spain) and the Massachusetts Institute of Technology have developed new technology that could aid in better measuring and monitoring of Parkinson’s over time.

And it is as easy as typing on your keyboard or sending a text message.

In today’s post, we will look at how the monitoring of typing could provide a useful ‘real world’ method of assessing people with Parkinson’s over time.


Measuring stuff. Source: Medium

When we think about new technology for the monitoring of Parkinson’s, we all too often think of a device that is strapped on to the body in order to measure tremor or speed of movement (Click here to read a previous post on wearable tech).

Or perhaps a smart phone app that has simple tests on it that individuals can use to assess themselves over time (Click here to read a previous post on this topic).

One of the issues with these approaches, however, is ‘adherence‘ – these devices require effort from the individual being assessed (they have to strap on the motion sensing device or remember to complete the task on the smart phone). And after the first week or so of using the device or the app, the novelty wears off and recordings may be less frequent.

Many of these methods are also slightly ‘unnatural‘, and they may deviate the individual from their normal way of life. For example, wearable tech is amazing, but the individual may find it uncomfortable to wear all the time or may alter aspects of their behaviour to better suit the wearing of the device.

Source: Cloudtweaks

A better approach would be to have methods of monitoring that require no effort from the individual. Tools that silently and seamlessly slip into the background of their lives and monitor continuously – the individual completely forgets about them, which provides a more unbiased assessment.

We have previously discussed some examples of more ‘real world/natural’ approaches (such as smart pills – Click here to read that post – and also with regards to sleep monitoring – Click here to read that post), and today we will explore another example: keyboard stroke monitoring.

What is keyboard stroke monitoring?

Keyboard stroke monitoring is the action of recording (or logging) the keys struck on a computer keyboard or tablet device, and using the information gathered for further analysis. Given that this data collection is typically covertly – the person using the keyboard may be completely unaware that their actions are even being monitored – there is a sinister/”big brother” aura to this technology.

But recently researchers from the Madrid-MIT M+Visión Consortium (a partnership between Fundación para el Conocimiento madri+d and the Massachusetts Institute of Technology) have been exploring this technology for the purpose of diagnosing and monitoring Parkinson’s.

And the results of their research are really interesting.

In this video, the scientists behind the research explain what they are trying to do:

And this research has recently been bearing fruit.

In 2015, investigators involved in the project published this report:

Title: Psychomotor impairment detection via finger interactions with a computer keyboard during natural typing.
Authors: Giancardo L, Sánchez-Ferro A, Butterworth I, Mendoza CS, Hooker JM.
Journal: Sci Rep. 2015 Apr 16;5:9678.
PMID: 25882641                   (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers presented a novel algorithm based on key press latencies that is able to detect the motor deficit caused by sleep inertia in healthy subjects.

Hang on a second. What does any of that actually mean? What is ‘key press latencies’?

Ok, so when I am typing this sentence, my fingers are performing actions that can be broken down into specific components. This may sound ridiculous, but there are actually three phases for a single keystroke:

  • key mechanism compression
  • finger impact
  • fingertip pulp compression followed by release

The ‘key mechanism compression‘ phase starts when the finger first makes contact with the key and it ends when the key has been fully pressed. And yes – I kid you not – researchers have measured this very precisely: this first initial phase accounts for approximately 12% of the total ‘Hold Time’ (the period during which the key is depressed – see image below).

When a key reaches full compression, there is a period of finger deceleration and peak force occurs. This phase accounts for approximately 11% of the hold time. Next, the tip of the finger continues to move down for less than a millimeter due to the skin compression, and then it is finally lifts up from the depressed key and this phase takes up most of the remaining 77% of the hold time.

(Seriously, have you ever in your life stopped to consider how you actually press a key on your key board?!?)

Now, what the researchers found in their initial analysis was that the action of pressing a button is significantly less likely to be affected by things like the actual text being typed or the speed of the typing or even the typing skills of the individual. No, the action of pressing a button is affected by:

  • the flight time
  • release latency
  • press latency
  • number of keys typed

And of course, all of these things can be monitored, measured and assessed. So the researchers generated an algorithm that could do just that, and they found press latency to be a very useful measure.

Importantly, by focusing on these basic components of key pressing, the researchers could avoid the issue of variability between typists. We all type differently, which could affect the results of any analysis. For example, some typists are ‘hunt and peck’ while others have a more practised style.

Remarkably, the investigators had found that all of these styles of typing have similar hold times. So it doesn’t matter how you type, the way we all press a key is very similar.

The investigators decided to test their algorithm on a group of 14 healthy individuals (7 males, 7 females; aged between 20 & 39 years). They tested algorithm by inducing a subtle psychomotor impairment on the study participants by waking each of them up during the night and asking them to type.

This abrupt wake up caused a period of sleep inertia.

What is sleep inertia?

Sleep inertia is a physiological state that is characterised by impaired cognitive and sensory-motor performance. It occurs immediately after awakening and it persists during the transition of sleep to full wakefulness. The individual will experience feelings of drowsiness, disorientation and issues with motor dexterity.

Sleep inertia: Basically me at anytime before 4pm each day. Source: Purple

The participants in the keyboard study were tested once during the day, when they felt well rested and fully awake. They were asked to select any Wikipedia page of their choice and type the main body contents of that page onto another web page for 15 minutes. The subjects were free to select any type of text as long as they had never typed it previously.

That evening (after the day time test), the participants were asked to send a text message to the experimenters just before going to sleep. They were then woken up by a phone call 70–80 minutes later. This period of time between the text message and the phone call allowed approximately 20 minutes for the participant to fall asleep and an additional 50 minutes of stage 3/4 of a sleep cycle.

Stages of sleep. Source: Wikipedia

If participant reported that they could not fall asleep within the 20 minutes, that particular test was considered invalid and they would repeat another night.

Once awakened, the participants were again asked to select any Wikipedia page of their choice and type the main body contents of that page onto another web page for 15 minutes. And again, the subjects were free to select any type of text as long as they had never typed it previously.

These daytime/night time test sessions were repeated for each participant (one week separating the days of testing), and thus the researchers accumulated 14 hours of typing from all of the participants (4 x 15 minutes from each of the 14 participants).

And what did they find?

The researchers found that their algorithm could clearly distinguish between typing made during sleep inertia (in the middle of the night) and typing performed when fully rested (during the day).

Examples of Key Hold Time in rested subjects (left) vs sleep inertia (right) for 3 participants

And the investigators considered this a proof-of-principle result for using keystroke patterns to assess impaired motor function.

Next, they turned their attention to a condition that is characterised by impaired motor function: Parkinson’s.

And here is the research report of that study:

Title: Computer keyboard interaction as an indicator of early Parkinson’s disease.
Authors: Giancardo L, Sánchez-Ferro A, Arroyo-Gallego T, Butterworth I, Mendoza CS, Montero P, Matarazzo M, Obeso JA, Gray ML, Estépar RS.
Journal: Sci Rep. 2016 Oct 5;6:34468. doi: 10.1038/srep34468.
PMID: 27703257                    (This report is OPEN ACCESS if you would like to read it)

In this study, the investigators recruited 42 people with recently diagnosed Parkinson’s (average age: 59 years; 43% females) and 43 control subjects (60.1 years; 60% females). Within the Parkinson’s group, 24 were ‘de-novo’ cases (that is newly diagnosed and drug naïve), while the remaining 18 were less than 5-years since diagnosis.

Each subject was asked to visit the assessment clinic twice, where the motor tests and clinical evaluation were performed. There was at least a week between each visit. People with Parkinson’s who were taking levodopa were asked to refrain from taking it for 18 hours before the visit.

In addition to a full Unified Parkinson’s Rating Scale (UPDRS-III) assessment, the participants were asked to perform finger tapping tests and the the typing test. The finger tapping test involved pressing a single button for 60 seconds, as fast as possible. This was performed twice: first with their dominant hand, then with the non-dominant hand. And for the typing test, the participants were asked to type out a story for 14 minutes.

The results of the typing test demonstrated that the researcher’s algorithm could clearly differentiate between the control group and the Parkinson’s group (both the de novo and less than 5 years groups):

And, importantly, the performance of this test was comparable to the finger tapping test.

Next, the researchers wanted to assess if this approach to detecting Parkinson’s could be used on a technology that is gradually replacing the keyboard and computer: the mobile smart phone.

Here is that research report for that study:

Title: Detection of Motor Impairment in Parkinson’s Disease Via Mobile Touchscreen Typing.
Authors: Arroyo-Gallego T, Ledesma-Carbayo MJ, Sanchez-Ferro A, Butterworth I, Mendoza CS, Matarazzo M, Montero P, Lopez-Blanco R, Puertas-Martin V, Trincado R, Giancardo L.
Journal: IEEE Trans Biomed Eng. 2017 Sep;64(9):1994-2002.
PMID: 28237917

In this study, the researchers tested an algorithm to determine if it could detect Parkinson’s by analysing the typing activity on smartphones, independent of the content of the text being typed. They conducted their study on a sample of 21 people with Parkinson’s and 23 control subjects. After the clinical assessment, each participant was asked to create an account on the project website (neuroqwerty).

Participants were asked to type a randomly-selected piece of text for 5 minutes on an Android touchscreen smart phone, and in this study, all of the participants with Parkinson’s were tested during their ON state (when they have taken their medication).

And (again) the researchers found that their algorithm could clearly differentiate the typing patterns between the two groups, providing ‘a home-based, high-compliance, and high-frequency Parkinson’s motor test by analysis of routine typing on touchscreens’.

For more information regarding this study, please  see the study website: neuroqwerty.

But given that this typing-based technology approach had not been testing in a ‘home-based setting’, the researchers next turned their attention to performing just such as evaluation:

Title: Detecting Motor Impairment in Early Parkinson’s Disease via Natural Typing Interaction With Keyboards: Validation of the neuroQWERTY Approach in an Uncontrolled At-Home Setting.
Authors: Arroyo-Gallego T, Ledesma-Carbayo MJ, Butterworth I, Matarazzo M, Montero-Escribano P, Puertas-Martín V, Gray ML, Giancardo L, Sánchez-Ferro Á.
Journal: J Med Internet Res. 2018 Mar 26;20(3):e89.
PMID: 29581092                     (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers wanted to extend the in-clinic results that they had collected to more natural settings by validating the applicability of their neuroQWERTY approach in the uncontrolled setting of the participant’s home. The goal was to determine if using the typing data from subjects’ natural interaction with their laptop to enable remote and unobtrusive assessment of Parkinson’s.

The investigators collected data from 52 participants (25 people with Parkinson’s and 27 healthy controls). The typing of the participants was monitored for 7 days, and those who were taking levodopa treatment were allowed to keep taking it during the study period.

To assure that a comparable amount of typing activity was made, only subjects who aggregated at least 15 min of typing data during their at-home period were included in the analysis.

The results demonstrated that an algorithm trained on ‘in-clinic’ data has comparable performance when tested on data collected in the natural home environment. The researchers concluded that the results get them ‘one step closer to our ultimate goal of providing an objective ambulatory tool to monitor Parkinson’s progression‘.

The researchers are now planning a longitudinal study to further validate and evaluate the precision of their typing monitoring approach in its ability to monitor Parkinson’s progression over time.

Interesting, but this sounds a bit big brother-ish, no?

Yes indeed.

There is massive potential for idiotic abuse of this type of technology. A dodgy insurance company quietly keeping an eye on everyone, looking for any signs of problems and the opportunity to raise premiums. Yes sir. Lots of opportunity for silliness. Give the world an interesting piece of technology, and someone will find something stupid to do with it.

‘Only two things are infinite, the universe and human stupidity, and I’m not sure about the former

Albert Einstein

But this is all beside the point.

Here at the SoPD what we are looking for are methods of assessing Parkinson’s. Thus, we will be keeping an eye on this technology as it continues to be developed.

So what does it all mean?

We urgently need better methods of assessing and monitoring Parkinson’s – both for the general well being of individuals within the affected community, but also for better testing of novel therapies in clinical trials. Once every month, I like to put out a post on research regarding a novel method of assessing Parkinson’s.

To my mind, research efforts on new methods of assessing Parkinson’s are CRITICAL to us providing better therapies in the future. Our current methods of assessing folks with Parkinson’s are limited to standard clinical rating scales and brain imaging techniques, which in the setting of a clinical trial are conducted on a episodic basis and provided limited information.

The need for new and different approaches to monitoring Parkinson’s is so urgent that the testing of a novel methods of assessment should be viewed as a standard ‘added value’ aspect of any new clinical trial being proposed – in this manner, even if the experimental treatment fails the clinical test, new information can be gained from the study with regards to the assessment aspect. Useful information must be taken away from every clinical trial, beyond simply whether a drug/treatment worked or not.

And ideally, new methods of measuring Parkinson’s will be designed to monitor people continuously and in the home environment, providing a frequency of real life assessment vastly greater than the current standard of care. In addition, these methods of assessment should leave little or no footprint on the life of the individual being monitored. There should be no burden what-so-ever on the individual.

Typing on a keyboard or touch screen seems like an every day activity that could provide just such a source of assessment.


The banner for today’s post was sourced from funtechsummercamps.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.