We need to talk about the UPDRS

 

“The measure of who we are is what we do with what we have” – Vince Lombardi

The measuring of Parkinson’s is complicated. There is such enormous variability between individual cases that the task of assessing people is very difficult.

The primary method that is used in clinics around the world is the Unified Parkinson’s Disease Rating Scale (or UPDRS). It is by no means perfect, and recently (in the wake of several unsuccessful clinical trials) there has been heated debate as to whether it is really up to the task.

Does it accurately reflect the condition? Does it really capture the lived experience? Can it pick up subtle changes associated with potentially disease modifying therapies in clinical trials? Or is it simply a “we’ve always done it this way” kind of tool?

In today’s post, we will look at what the UPDRS is, discuss some of the criticisms associated with it, and consider what solutions to those issues could look like.

 


Source: Wired

This is Andy Grove and his story is rather remarkable.

Born in 1936 to a Jewish family in Budapest, he managed to survive the Nazis, and then fleed Hungary when Soviet tanks started rolling in. Arriving in the US with absolutely nothing, he taught himself English, before going to City College of New York and later the University of California (Berkeley) where he received a PhD in chemical engineering.

And that was just the start of his amazing tale.

After completing his PhD (and publishing a textbook on semiconductors), Grove joined the seminal Silicon Valley company – Fairchild Semiconductor – in 1963. He worked his way up from researcher to assistant director of development, before becoming the first person that Robert Noyce and Gordon Moore (of Moore’s Law fame) hired after they departed Fairchild to start their own little company in 1968.

The name of that company was Intel.

Source: Wikipedia

Grove also worked his way up the ladder at Intel – from director of engineering to CEO – and he is credited with transforming the company from a struggling memory chip maker into the processor powerhouse it is today. He was Time’s ‘Man of the Year’ in 1997 and he was a widely revered figure in Silicon valley.

Source: Time

But the path to success was not easy.

Having survived prostate cancer in 1995, Grove was diagnosed with Parkinson’s in 2000. Viewing the situation as a problem solving exercise, he poured tens of millions of his own money into researching Parkinson’s.

Andy & Michael J Fox. Source: MJFF

But coming from the world of ‘Moore’s Law’, Grove became frustrated by a.) the slow speed of progress in the world of biomedical research and b.) the tools used to assess it.

In particular, he disliked the UPDRS, which he referred to as a “piece of crap” (Source – you should read the linked article).

What is the UPDRS?

The Unified Parkinson’s Disease Rating Scale is a standardised clinical method that is widely used for assessing Parkinson’s.

It is considered “the gold standard“, and many (if not most) clinical trials focused on disease modification use it for measuring their primary end point (determining whether a treatment has worked or not).

Source: Assessment

It is available in multiple languages (Click here to read more about this), and if you have ever been involved with clinical research, the chances are you have had a UPDRS assessment (whether you are aware of it or not… and you probably should have been told!).

What does it involve?

The current version of the UPDRS has four parts:

  • I: Non-motor experiences of daily living
  • II: Motor experiences of daily living/Activities of daily living
  • III: Motor examination
  • IV: Motor complications.

The MDS-UPDRS rates 65 items in comparison to 55 on the original UPDRS. 48 of these items have a 0 to 4 option (0 = normal, 5 = severe) and 7 yes/no response items. And the items are divided across the 4 parts of the UPDRS:

Part I – 13 items
Part II – 13 items
Part III – 33 items (in effect there is only 18 items, but several address which side of the body, or which limb is affected)
Part IV – 6 items

Twenty items are completed by the patient/caregiver.

It is estimated that the UPDRS should only take 30 minutes in the clinic, with 10 min for the interview items of Part I, 15 min for the motor assessment of Part III, and 5 min for motor complications of Part IV (Several items in Part I and all of the questions on Part II are filled in by the patient at their leisure).

And this is not the first version of the rating scale.

It was originally developed in the 1980s, and consisted of 42 items (Click here to see an example of the original). It was widely adopted and used, but limitations with the scale were noted and in the early 2000s, this report was generated:

Title: The Unified Parkinson’s Disease Rating Scale (UPDRS): status and recommendations.
Authors: Movement Disorder Society UPDRS Revision Task Force.
Journal: Mov Disord. 2003 Jul;18(7):738-50.
PMID: 12815652

In this report, the Movement Disorder Society UPDRS Revision Task Force recommended that the Movement Disorder Society should sponsor the development of a new version of the UPDRS based on several issues that had been communicated following usage of the old UPDRS. These issues included:

  • ambiguities in the text
  • the need for better instructions for raters
  • several metric flaws
  • the need for screening questions on non-motor aspects of Parkinson’s

The last issue in particular was important in the revision.

The task force also encouraged efforts to establish its “clinimetric properties”, which would address the need to accurately define what the minimal clinically relevant difference is and a Minimal Clinically Relevant Incremental Difference, as well as testing its correlation with the current UPDRS.

Critically, the task force also suggested that the new scale should be culturally unbiased and evaluated in different racial, gender, and age-groups.

The result of this process was the publication of a revised UPDRS:

Title: Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results
Authors: Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, Poewe W, Sampaio C, Stern MB, Dodel R, Dubois B, Holloway R, Jankovic J, Kulisevsky J, Lang AE, Lees A, Leurgans S, LeWitt PA, Nyenhuis D, Olanow CW, Rascol O, Schrag A, Teresi JA, van Hilten JJ, LaPelle N; Movement Disorder Society UPDRS Revision Task Force.
Journal: Mov Disord. 2008 Nov 15;23(15):2129-70.
PMID: 19025984

In 2008, the Movement Disorder Society published a revision of the UPDRS, which was now known as the MDS-UPDRS.

The modified UPDRS retained the original four-scale structure, but there was significant reorganisation of the various subscales, which we outlined above. And it is this version of the UPDRS that is now used widely in clinical trials, particularly the UPDRS Part III (which is focused on motor symptoms as determined by the assessing clinician). Primary and secondary end points in Parkinson’s clinical trials will often involve participants being assessed on the UPDRS Part III during OFF state (no medication since the night before) and the ON state (standard medication).

These were the primary and secondary endpoints for the recent Bristol GDNF clinical trial – Click here to read more about this).

Is the UPDRS Part III accurate?

It is the best – most well characterised – tool that we currently have.

(There. That is a diplomatic, fair and unbiased answer)

There is evidence, however, that the activities of daily living (UPDRS Part II) is actually better at determining the progression of Parkinson’s:

Title: UPDRS activity of daily living score as a marker of Parkinson’s disease progression
Authors: Harrison MB, Wylie SA, Frysinger RC, Patrie JT, Huss DS, Currie LJ, Wooten GF.
Journal: Mov Disord. 2009 Jan 30;24(2):224-30
PMID: 18951537                   (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers analysed Parts I-III of the UPDRS scores from 888 people with Parkinson’s to assess which of the Parts of the UPDRS was associated more with disease progression. That is to say, which part of the UPDRS was a better measure of progression. Interestingly, independent of the medication status of the individuals and across longitudinal analyses, the Part II section of the UPDRS (which measures Motor experiences of daily living/Activities of daily living) showed a stronger and more stable association with disease duration and progression than any of the other UPDRS subsections.

So why didn’t Andy Grove like the UPDRS?

Well, it should be noted that Andy was not alone in his distaste of the UPDRS.

There are many criticisms of the UPDRS.

Such as?

Where does one begin? The UPDRS has it’s problems:

1.  Firstly, the UPDRS is episodic. It is made during a half-hour period on just one day out of the year in most cases, and the outcome depends solely on how the individuals being assessed (AND the clinician) may be feeling that particular day. Below is the a schematic demonstrating the 1 hour per year (out of the 8,765 hours) that Parkinson’s activitist (and all round misfit) Sara Riggare gets for an assessment to determine how her Parkinson’s is progressing.

selfcare-infographic-english

A schematic illustrating the episodic nature of assessments. Source: Riggare

2.  The UPDRS is variable. The scoring can vary dramatically depending on how recently patients have taken medication, how well they slept the night before, the time of day, etc. Plus, what happens if the individual being assessed was offered a coffee when they arrived at the clinic (which some clinics do), or smoked a cigarette to calm their nerves before entering the clinic? How does the UPDRS account for these things?

Source: PSU

3.  The UPDRS is also extremely subjective. Two neurologists will often give a two different scores for the same patient (Click here to read more about this). In addition, when patients are asked, for example, how are they sleeping, their answer is basical based on “Good”, “Ok” or “bad”. Not what percentage of your sleep time was in REM state. There is limited granularity in the UPDRS score.

Source: Thesocietyonline

4.  The UPDRS scale is non-linear. By this I mean that a score of 50 is not twice as bad as 25. And this has important implications in clinical trials when one participant may improve from 50 to 40, which would be very different to someone improving from a score of 30 to 20. And this is a crucial thing to consider when most clinical trial involve very heterogeneous (or mixed) groups of people with different baselines scores.

5.  Importantly, the UPDRS score has no meaning. Two people may have the same Part III motor score of 30, but given that one is tremor dominant and has lived with the condition for 10 years, while the other has more rigidity issues and only had the condition for 4 years, it is difficult to define what a score of 30 might actually means.

Source: tru-signal

For the record, I am not an expert on UPDRS – far from it (and I am happy to be corrected on anything written here). Nor am I seeking to pick a fight with anyone.

By writing this post, I am simply trying to start a conversation.

And I know it is the easiest thing in the world to state the problem with something.

It is a far greater matter to provide a solution.

Plus, to their credit, the Movement Disorder Society – which owns and maintains the UPDRS – does monitor, review and revise the test. But even they can admit that it is a slow process (the last revision was in 2008).

Overall, one can not help but think that there is an element of sunk cost effect at play here.

What is the sunk cost effect?

The sunk cost effect is the tendency for humans to continue investing in something even when it is apparent that it isn’t working (like me with my gym and beauty salon membership for example). Our basic nature is to avoid admitting failure, and people will often continue spending time, effort and/or money to try and fix a problem that isn’t working, instead of simply cutting our losses and moving on.

So what did Andy Grove do about the UPDRS?

Coming from the world of Intel, he threw technology at the problem to address some of the issues associated with the UPDRS that we mentioned above, such as the episodic and subjective nature of the rating scale.

And in 2009, he was amongst the authors of this published report:

Title: Testing objective measures of motor impairment in early Parkinson’s disease: Feasibility study of an at-home testing device.
Authors: Goetz CG, Stebbins GT, Wolff D, DeLeeuw W, Bronte-Stewart H, Elble R, Hallett M, Nutt J, Ramig L, Sanger T, Wu AD, Kraus PH, Blasucci LM, Shamim EA, Sethi KD, Spielman J, Kubota K, Grove AS, Dishman E, Taylor CB.
Journal: Mov Disord. 2009 Mar 15;24(4):551-6.
PMID: 19086085                      (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers developed a box that contained a series of motor tasks that could be performed on a weekly basis, which they called the At-Home Testing Device (or AHTD).

The AHTD. Source: NCBI

The data that was collected was stored on a USB memory stick and sent via the internet – as an encrypted data package – to a central database (pretty high tech huh!). 52 individuals recently diagnosed with Parkinson’s were enrolled in the study and 50 completed the six month-long trial. Each week the participants had to conduct a range of assessments, and remarkably, compliance with the weekly 30-minute assessments was 90.6%. The at-home AHTD assessments were compared to clinic-based UPDRS scores at 3 and 6 months.

Interestingly, the participants had no access to their prior scores when performing the battery of AHTD tests, which included tremor measurements, finger tapping on key strokes, hand tapping movements, “pegboard plugging”, and speech.

When the researchers analysed the results, they found that UPDRS motor scores worsened over the 6 months, and that the alternating finger taps, tremor, and speech tests were best associated with that change. Most of the measures assessed by the AHTD remained stable over the 6 months of the trial, which highlights the difficulty with measuring Parkinson’s.

There was also an unexpected progressive improvement in reaction time on the tests, which indicates the possibility of a ‘practise effect’ coming into play on some of the test being repeatedly conducted.

Has anyone ever continued characterising the AHTD?

Not that I am aware of (and I’d be happy to be corrected on this).

But advanced in new technology have left the AHTD behind, and there are now a wide range of wearable devices and gadgets that are being used to more accurately measure (quantify) aspects of Parkinson’s motor features.

In addition, video analysis technology is taking the subjective element out of the UPDRS and providing a more quantifiable assessment, as this video from Machine Medicine Technologies shows:

But does this solve all of the issues with the UPDRS you ask? It probably depends on who you ask – many folks in the parkinson’s affected community do not understand the point of the finger tapping test.

It is not relevant to their quality of daily living.

So what does it all mean?

An accurate assessment of disease progression in Parkinson’s is absolutely criticial for evaluating the impact of symptomatic and potentially disease modifying therapies that are currently being developed. But this task is made difficult as our understanding of the condition is incomplete, and there is tremendous variability between individual cases.

I recently sat with a clinical trials program director. He ran a massive program for a completely different set of medical conditions (not neurology). When I spoke of the difficulties with assessing Parkinson’s because “there is so much variability between cases“, he simply shook his head, smiled, and said “it’s the same with every medical condition. Everyone says this. Cancer, immunology, diabetes, multiple sclerosis – they are all plagued with the variability between cases issue. If the situation involves humans, there is variability. It’s not a problem, it’s just an excuse“.

And he was right.

This begs the question, however, whether a standardised ‘one-size-fits-all’ assessment approach for everyone is the right way forward. If there is variability between cases, surely a more personalised approach to testing would be more prudent. If someone has rigidity, but no tremor, what is the point in scoring change in tremor?

Either way, as I suggested above, this post is simply a call for more discussion on this topic.

It is not intended as a “the emperor has no clothes” protest – rather it is a “the emperor has poor fashion sense and needs everyone’s help” suggestion.

The UPDRS is still the best tool we have for the assessment of Parkinson’s, but it is by no means ideal and we need to consider alternatives rapidly as we push more resources at large and expensive clinical trials.

I leave the last word to Dr Grove:

“To be sure, there are additional fundamental differences between the two industries [the health science/health care industry and the microchip industry]. One industry deals with the well-defined world of silicon, the other with living human beings. Humans are incredibly complex biological systems, and working with them has to be subject to safety, legal, and ethical concerns. Nevertheless, it is helpful to mine this comparison for every measure of learning that can be found” – Andy Grove, 2005 (Source)

 

For Carol


The banner for today’s post was sourced from Medium

11 comments

  1. Lesley Gosden

    Excellent article. This should start the ball rolling in the right direction.

    UPDRS needs reviewing urgently and thoroughly. Thousands of pounds are going to waste on clinical trials that are doomed to failure if they continue to rely on this unreliable tool.

    Like

  2. jaynewithay66

    I read this article with huge interest and absolute relief that finally we are facing into the need for real change in the measurement of Parkinson’s

    Like

  3. gavinmogan

    PD clinical assessments are a joke. No, seriously. I present Parkinson’s humor at support groups and this is some of my prime content. My neurologists and I have had many laughs at this process over the years.

    Like

  4. Rick Bentley

    I worked for Andy at the Grove Foundation as we made objective measures of Parkinson’s disease a reality. The device shown (aka the QMAT) was an early effort with Intel’s now defunct Digital Health group, we had a mobility solution as well (originally off the shelf wired sensors and then wireless ones from APDM). A couple hundred were made and I got them deployed in trials from the NIH to the Netherlands. Once we were out of devices we pivoted to online and smartphone driven solutions for scale.

    We built a web-based keyboard test that measured manual dexterity and a smart phone test that measured mobility (timed up and go, as well as postural stability and sway; smartphone was attached to the patient’s waist). Andy took a turn for the worse as this was rolling out and Parkinson’s effort of the Grove Foundation came to an end. I made sure the code was open sourced and picked up the ball at Google Life Sciences (now Verily) for a bit as well.

    The work is here on Github: https://github.com/openOPDM. I’d be happy to help anyone who wants to get it started again.

    Like

    • wildflowerhouse

      Rick, I am interested in this. I have a teeny bit of experience with programming in python, but that’s all. I am willing to learn more or to contribute in another way. What can I do?

      Like

  5. Claire Ford-Hutchinson

    A very interesting article. My husband was diagnosed with PD in 2000 at the age of 48yrs. His strong will and stubborn nature has definitely helped him cope with this disease. However I feel that the treatment is too generalised. Everyone’s PD symptoms are different. At present my husband is under medicated and consequently experiences constant freezing, tremor, slowness and ongoing low mood and more. He has recently been prescribed a new drug but has not had this yet as he has a tendency to high blood pressure and one of the cautions related to this drug is high blood pressure. I feel that a one hour appointment to see the PD nurse every 6 months is insufficient to be able to monitor what treatment or help my husband requires. We are constantly being informed of clinical trials and research in this awful disease but I can’t see that much if any progress has been made in treating ,not to mention curing it in all the time , 20 yrs that my husband has had it.

    Like

    • John Turner

      Claire,

      Seeing a specialist nurse or doctor once every six months is not frequent enough to give good care for PD. You’re always playing catch-up: by the time you have titrated to the required dose you need the dose to be increased.

      Faced with the same situation myself, I took control of my dosing. If you don’t have dyskinesia, there is no reason to be under medicated.

      Rather than always take the same dose at the same time, I “dynamically” dose: I aim to read my symptoms and time it so that I take another dose at the right moment for it to become effective, just as the previous dose wears off.

      A metric for PD should be able to measure this situation. UPDRS does not have the resolution to be of use in this case.

      John

      Like

  6. Kevin McFarthing (@InnovationFixer)

    Hi Simon – absolutely bang on target. Having been measured on quite a few observational studies, two of them long-term longitudinal ones, I have had lots of UPDRS testing. I vary from day to day. The conduct of the test varies massively from rater to rater. Let’s take the pull test as an example. The strength applied is highly variable, resulting in (at least in my case) from 0 to 2 steps back. By the way, I’m not blaming the raters, each one believes they are doing it right; but we should not have to rely on their inherent subjectivity.

    The test is trainable (I’ve heard anecdotes of people practising for their next UPDRS). It is not only inaccurate, it is imprecise as there is no possibility to interpolate between numbers. My biggest symptom is slowness; UPDRS does not capture this. It relies on subjective assessment from patients, especially part IV on motor complications. I recently tried to keep a dyskinesia diary and found it really hard to be accurate and stop real life getting in the way. Guessing is the default.

    Clinical trial researchers are forced to ascribe much greater credence to UPDRS than it deserves. For example, the T0 assessment in any study becomes a fixed reference point against which any percentage decline or improvement is measured. That T0 number is no less variable than any other, potentially destabilising the whole study.

    So from my (subjective) n=1 experience, UPDRS is broken and needs urgent replacement with objective, device-based measurement.

    Like

  7. GK

    Hello Simon,

    The only test is materially confirmed, which allows you to say whether there is a disease or not is PET a test. It seems to me in this direction we have to move. Elaborating on that – in the direction of test simplification, cheapening, security and depth of analysis. And this is obvious. Otherwise, it is always a subjective assessment of the patient of his own brain functions. Perhaps connect the latest scanning technology of brain activities with patterns of reactions of brain in normal state and in PD condition.
    This is my thinking as amateur person hopefully useful.
    Best Regards,

    Like

  8. Jellywoman

    Can I add the Montreal Cognitive Assessment Test as something else in desperate need of a rethink? Being a serial volunteer for research projects, I actually know the test by heart (face, velvet, church, daisy, red). I have often mentioned to the researcher that in all honesty, the test can have no validity if you use it again and again (I only know that John is the one to come today) but the reply is always that it needs to be the same for comparison (they’re all measurement devices).
    Really?
    Exactly the same?
    Surely it would be fairly simple to have a few tests of equal weight – thinking of words starting with ‘g’ instead of ‘f’ perhaps. Setting the clock hands to twenty past four instead of ten to eleven. Naming daffodils and tulips rather than lions and camels.
    You’d be amazed how fast I can count back in sevens.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.