We need to talk about the UPDRS

“The measure of who we are is what we do with what we have” – Vince Lombardi The measuring of Parkinson’s is complicated. There is such enormous variability between individual cases that the task of assessing people is very difficult. The primary method that is used in clinics around the world is the Unified Parkinson’s Disease Rating Scale (or UPDRS). It is by no means perfect, and recently (in the wake of several unsuccessful clinical trials) there has been heated debate as to whether it is really up to the task. Does it accurately reflect the condition? Does it really capture the lived experience? Can it pick up subtle changes associated with potentially disease modifying therapies in clinical trials? Or is it simply a “we’ve always done it this way” kind of tool? In today’s post, we will look at what the UPDRS is, discuss some of the criticisms associated with it, and consider what solutions to those issues could look like.

---

Source: Wired

This is Andy Grove and his story is rather remarkable.

Born in 1936 to a Jewish family in Budapest, he managed to survive the Nazis, and then fleed Hungary when Soviet tanks started rolling in. Arriving in the US with absolutely nothing, he taught himself English, before going to City College of New York and later the University of California (Berkeley) where he received a PhD in chemical engineering.

And that was just the start of his amazing tale.

After completing his PhD (and publishing a textbook on semiconductors), Grove joined the seminal Silicon Valley company – Fairchild Semiconductor – in 1963. He worked his way up from researcher to assistant director of development, before becoming the first person that Robert Noyce and Gordon Moore (of Moore’s Law fame) hired after they departed Fairchild to start their own little company in 1968.

The name of that company was Intel.

Source: Wikipedia

Grove also worked his way up the ladder at Intel – from director of engineering to CEO – and he is credited with transforming the company from a struggling memory chip maker into the processor powerhouse it is today. He was Time’s ‘Man of the Year’ in 1997 and he was a widely revered figure in Silicon valley.

Source: Time

But the path to success was not easy.

Having survived prostate cancer in 1995, Grove was diagnosed with Parkinson’s in 2000. Viewing the situation as a problem solving exercise, he poured tens of millions of his own money into researching Parkinson’s.

Andy & Michael J Fox. Source: MJFF

But coming from the world of ‘Moore’s Law’, Grove became frustrated by a.) the slow speed of progress in the world of biomedical research and b.) the tools used to assess it.

In particular, he disliked the UPDRS, which he referred to as a “piece of crap” (Source – you should read the linked article).

What is the UPDRS?

The Unified Parkinson’s Disease Rating Scale is a standardised clinical method that is widely used for assessing Parkinson’s.

It is considered “the gold standard“, and many (if not most) clinical trials focused on disease modification use it for measuring their primary end point (determining whether a treatment has worked or not).

Source: Assessment

It is available in multiple languages (Click here to read more about this), and if you have ever been involved with clinical research, the chances are you have had a UPDRS assessment (whether you are aware of it or not… and you probably should have been told!).

What does it involve?

The current version of the UPDRS has four parts:

• I: Non-motor experiences of daily living
• II: Motor experiences of daily living/Activities of daily living
• III: Motor examination
• IV: Motor complications.

The MDS-UPDRS rates 65 items in comparison to 55 on the original UPDRS. 48 of these items have a 0 to 4 option (0 = normal, 5 = severe) and 7 yes/no response items. And the items are divided across the 4 parts of the UPDRS:

Part I – 13 items
Part II – 13 items
Part III – 33 items (in effect there is only 18 items, but several address which side of the body, or which limb is affected)
Part IV – 6 items

Twenty items are completed by the patient/caregiver.

It is estimated that the UPDRS should only take 30 minutes in the clinic, with 10 min for the interview items of Part I, 15 min for the motor assessment of Part III, and 5 min for motor complications of Part IV (Several items in Part I and all of the questions on Part II are filled in by the patient at their leisure).

And this is not the first version of the rating scale.

It was originally developed in the 1980s, and consisted of 42 items (Click here to see an example of the original). It was widely adopted and used, but limitations with the scale were noted and in the early 2000s, this report was generated:

Title: The Unified Parkinson’s Disease Rating Scale (UPDRS): status and recommendations.
Authors: Movement Disorder Society UPDRS Revision Task Force.
Journal: Mov Disord. 2003 Jul;18(7):738-50.
PMID: 12815652

In this report, the Movement Disorder Society UPDRS Revision Task Force recommended that the Movement Disorder Society should sponsor the development of a new version of the UPDRS based on several issues that had been communicated following usage of the old UPDRS. These issues included:

• ambiguities in the text
• the need for better instructions for raters
• several metric flaws
• the need for screening questions on non-motor aspects of Parkinson’s

The last issue in particular was important in the revision.

The task force also encouraged efforts to establish its “clinimetric properties”, which would address the need to accurately define what the minimal clinically relevant difference is and a Minimal Clinically Relevant Incremental Difference, as well as testing its correlation with the current UPDRS.

Critically, the task force also suggested that the new scale should be culturally unbiased and evaluated in different racial, gender, and age-groups.

The result of this process was the publication of a revised UPDRS:

Title: Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results
Authors: Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, Poewe W, Sampaio C, Stern MB, Dodel R, Dubois B, Holloway R, Jankovic J, Kulisevsky J, Lang AE, Lees A, Leurgans S, LeWitt PA, Nyenhuis D, Olanow CW, Rascol O, Schrag A, Teresi JA, van Hilten JJ, LaPelle N; Movement Disorder Society UPDRS Revision Task Force.
Journal: Mov Disord. 2008 Nov 15;23(15):2129-70.
PMID: 19025984

In 2008, the Movement Disorder Society published a revision of the UPDRS, which was now known as the MDS-UPDRS.

The modified UPDRS retained the original four-scale structure, but there was significant reorganisation of the various subscales, which we outlined above. And it is this version of the UPDRS that is now used widely in clinical trials, particularly the UPDRS Part III (which is focused on motor symptoms as determined by the assessing clinician). Primary and secondary end points in Parkinson’s clinical trials will often involve participants being assessed on the UPDRS Part III during OFF state (no medication since the night before) and the ON state (standard medication).

These were the primary and secondary endpoints for the recent Bristol GDNF clinical trial – Click here to read more about this).

Is the UPDRS Part III accurate?

It is the best – most well characterised – tool that we currently have.

(There. That is a diplomatic, fair and unbiased answer)

There is evidence, however, that the activities of daily living (UPDRS Part II) is actually better at determining the progression of Parkinson’s:

Title: UPDRS activity of daily living score as a marker of Parkinson’s disease progression
Authors: Harrison MB, Wylie SA, Frysinger RC, Patrie JT, Huss DS, Currie LJ, Wooten GF.
Journal: Mov Disord. 2009 Jan 30;24(2):224-30
PMID: 18951537                   (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers analysed Parts I-III of the UPDRS scores from 888 people with Parkinson’s to assess which of the Parts of the UPDRS was associated more with disease progression. That is to say, which part of the UPDRS was a better measure of progression. Interestingly, independent of the medication status of the individuals and across longitudinal analyses, the Part II section of the UPDRS (which measures Motor experiences of daily living/Activities of daily living) showed a stronger and more stable association with disease duration and progression than any of the other UPDRS subsections.

So why didn’t Andy Grove like the UPDRS?

Well, it should be noted that Andy was not alone in his distaste of the UPDRS.

There are many criticisms of the UPDRS.

Such as?

Where does one begin? The UPDRS has it’s problems:

1.  Firstly, the UPDRS is episodic. It is made during a half-hour period on just one day out of the year in most cases, and the outcome depends solely on how the individuals being assessed (AND the clinician) may be feeling that particular day. Below is the a schematic demonstrating the 1 hour per year (out of the 8,765 hours) that Parkinson’s activitist (and all round misfit) Sara Riggare gets for an assessment to determine how her Parkinson’s is progressing.

A schematic illustrating the episodic nature of assessments. Source: Riggare

2.  The UPDRS is variable. The scoring can vary dramatically depending on how recently patients have taken medication, how well they slept the night before, the time of day, etc. Plus, what happens if the individual being assessed was offered a coffee when they arrived at the clinic (which some clinics do), or smoked a cigarette to calm their nerves before entering the clinic? How does the UPDRS account for these things?

Source: PSU

3.  The UPDRS is also extremely subjective. Two neurologists will often give a two different scores for the same patient (Click here to read more about this). In addition, when patients are asked, for example, how are they sleeping, their answer is basical based on “Good”, “Ok” or “bad”. Not what percentage of your sleep time was in REM state. There is limited granularity in the UPDRS score.

Source: Thesocietyonline

4.  The UPDRS scale is non-linear. By this I mean that a score of 50 is not twice as bad as 25. And this has important implications in clinical trials when one participant may improve from 50 to 40, which would be very different to someone improving from a score of 30 to 20. And this is a crucial thing to consider when most clinical trial involve very heterogeneous (or mixed) groups of people with different baselines scores.

5.  Importantly, the UPDRS score has no meaning. Two people may have the same Part III motor score of 30, but given that one is tremor dominant and has lived with the condition for 10 years, while the other has more rigidity issues and only had the condition for 4 years, it is difficult to define what a score of 30 might actually means.

Source: tru-signal

For the record, I am not an expert on UPDRS – far from it (and I am happy to be corrected on anything written here). Nor am I seeking to pick a fight with anyone.

By writing this post, I am simply trying to start a conversation.

And I know it is the easiest thing in the world to state the problem with something.

It is a far greater matter to provide a solution.

Plus, to their credit, the Movement Disorder Society – which owns and maintains the UPDRS – does monitor, review and revise the test. But even they can admit that it is a slow process (the last revision was in 2008).

Overall, one can not help but think that there is an element of sunk cost effect at play here.

What is the sunk cost effect?

The sunk cost effect is the tendency for humans to continue investing in something even when it is apparent that it isn’t working (like me with my gym and beauty salon membership for example). Our basic nature is to avoid admitting failure, and people will often continue spending time, effort and/or money to try and fix a problem that isn’t working, instead of simply cutting our losses and moving on.

So what did Andy Grove do about the UPDRS?

Coming from the world of Intel, he threw technology at the problem to address some of the issues associated with the UPDRS that we mentioned above, such as the episodic and subjective nature of the rating scale.

And in 2009, he was amongst the authors of this published report:

Title: Testing objective measures of motor impairment in early Parkinson’s disease: Feasibility study of an at-home testing device.
Authors: Goetz CG, Stebbins GT, Wolff D, DeLeeuw W, Bronte-Stewart H, Elble R, Hallett M, Nutt J, Ramig L, Sanger T, Wu AD, Kraus PH, Blasucci LM, Shamim EA, Sethi KD, Spielman J, Kubota K, Grove AS, Dishman E, Taylor CB.
Journal: Mov Disord. 2009 Mar 15;24(4):551-6.
PMID: 19086085                      (This report is OPEN ACCESS if you would like to read it)

In this study, the researchers developed a box that contained a series of motor tasks that could be performed on a weekly basis, which they called the At-Home Testing Device (or AHTD).

The AHTD. Source: NCBI

The data that was collected was stored on a USB memory stick and sent via the internet – as an encrypted data package – to a central database (pretty high tech huh!). 52 individuals recently diagnosed with Parkinson’s were enrolled in the study and 50 completed the six month-long trial. Each week the participants had to conduct a range of assessments, and remarkably, compliance with the weekly 30-minute assessments was 90.6%. The at-home AHTD assessments were compared to clinic-based UPDRS scores at 3 and 6 months.

Interestingly, the participants had no access to their prior scores when performing the battery of AHTD tests, which included tremor measurements, finger tapping on key strokes, hand tapping movements, “pegboard plugging”, and speech.

When the researchers analysed the results, they found that UPDRS motor scores worsened over the 6 months, and that the alternating finger taps, tremor, and speech tests were best associated with that change. Most of the measures assessed by the AHTD remained stable over the 6 months of the trial, which highlights the difficulty with measuring Parkinson’s.

There was also an unexpected progressive improvement in reaction time on the tests, which indicates the possibility of a ‘practise effect’ coming into play on some of the test being repeatedly conducted.

Has anyone ever continued characterising the AHTD?

Not that I am aware of (and I’d be happy to be corrected on this).

But advanced in new technology have left the AHTD behind, and there are now a wide range of wearable devices and gadgets that are being used to more accurately measure (quantify) aspects of Parkinson’s motor features.

In addition, video analysis technology is taking the subjective element out of the UPDRS and providing a more quantifiable assessment, as this video from Machine Medicine Technologies shows:

But does this solve all of the issues with the UPDRS you ask? It probably depends on who you ask – many folks in the parkinson’s affected community do not understand the point of the finger tapping test.

It is not relevant to their quality of daily living.

So what does it all mean?

An accurate assessment of disease progression in Parkinson’s is absolutely criticial for evaluating the impact of symptomatic and potentially disease modifying therapies that are currently being developed. But this task is made difficult as our understanding of the condition is incomplete, and there is tremendous variability between individual cases.

I recently sat with a clinical trials program director. He ran a massive program for a completely different set of medical conditions (not neurology). When I spoke of the difficulties with assessing Parkinson’s because “there is so much variability between cases“, he simply shook his head, smiled, and said “it’s the same with every medical condition. Everyone says this. Cancer, immunology, diabetes, multiple sclerosis – they are all plagued with the variability between cases issue. If the situation involves humans, there is variability. It’s not a problem, it’s just an excuse“.

And he was right.

This begs the question, however, whether a standardised ‘one-size-fits-all’ assessment approach for everyone is the right way forward. If there is variability between cases, surely a more personalised approach to testing would be more prudent. If someone has rigidity, but no tremor, what is the point in scoring change in tremor?

Either way, as I suggested above, this post is simply a call for more discussion on this topic.

It is not intended as a “the emperor has no clothes” protest – rather it is a “the emperor has poor fashion sense and needs everyone’s help” suggestion.

The UPDRS is still the best tool we have for the assessment of Parkinson’s, but it is by no means ideal and we need to consider alternatives rapidly as we push more resources at large and expensive clinical trials.

I leave the last word to Dr Grove:

“To be sure, there are additional fundamental differences between the two industries [the health science/health care industry and the microchip industry]. One industry deals with the well-defined world of silicon, the other with living human beings. Humans are incredibly complex biological systems, and working with them has to be subject to safety, legal, and ethical concerns. Nevertheless, it is helpful to mine this comparison for every measure of learning that can be found” – Andy Grove, 2005 (Source)

 

For Carol

---

The banner for today’s post was sourced from Medium