Differences in clinician versus patient recording of comorbidities in PROMs

Methodology

A data request was made from the Health and Social Care Information Centre (HSCIC) regarding patients who had undergone a primary total knee replacement (TKR) at the Norfolk and Norwich Hospital in 2014. In total, 576 patients had received post-operative PROMs questionnaires in 2014. Complete information was available for 195 patients, which forms the basis of this analysis. The patient letters and the pre-operative assessment documentation on our electronic system (Bluespier) were then reviewed. The comorbidities that the clinician felt would apply to that patient were recorded from the list provided in the Oxford Knee Score (OKS) and were then compared with what the patients had recorded.

In total, there were 189 additional comorbidities identified from our notes review. Of these, 95 would alter the predicted OKS score in 77 patients. There was a significant change in average predicted OKS score from 33.7 ± 3.9 to 32.3 ± 4.0 (p = 0.02) in the 77 patients who had additional OKS-altering comorbidities. When looking at the case-mix adjustment, the original mean adjustment was -0.83 (± 1.1). After adjusting for clinician-reported comorbidities, there was a significant change in the mean to -1.40 (± 1.4) (p < 0.0001). After the relevant recalculations were carried out, the adjusted average health gain went from 15.254 to 15.907. This is an improvement of 0.653.

The small change of ensuring accurate comorbidity recording can have an impact on the adjusted average health gain for a hospital. This is important information: patients report comorbidities differently to clinicians, and often overrate their health. Despite the limitation of this comorbidity data, hospital performance data, which are publically available, are based on this case-mix and comorbidity adjustment. Care clearly needs to be taken in the interpretation of these case mix-adjusted scores.

Introduction

The PROMs (Patient-Reported Outcome Measures) programme, embedded within the NJR, is an evaluation of surgical outcomes based on questionnaires completed by patients before and after their surgery. Eligible patients are those treated by or on behalf of the English NHS for the following procedures: hip replacements, knee replacements, varicose vein surgery and groin hernia surgery.

The increasing use of PROMs acknowledges the patients’ perspective as a marker of quality and effectiveness by placing them at the centre of decision making. It enables comparison of health services, identifies strengths and weaknesses of health care delivery, drives quality improvement, informs commissioning, and promotes choice. However, it does then rely on patients filling out forms and returning them. Efforts have had to be made to make shorter, more reliable tools, in order to increase patient participation, particularly among underrepresented patient populations. Widespread use of PROMs may be limited by the costly and time consuming process of collection, analysis and data presentation. The internet opens up opportunities but is not currently in widespread use.¹ Clinician reported outcomes are based on a clinician’s observations or interpretations of the patient’s global level of functioning. A certain level of knowledge is required to perform this assessment but it can offer an educated insight to the patient’s condition. However, there can be a failure to comprehensively capture patient interpretation of their function and there is a risk of over-estimation of recovery status.²

PROMs data and analyses, including those from Hospital Episode Statistics (HES)-PROMs linked data, are published each month by the Health and Social Care Information Centre. Publications include monthly summary statistics, a more detailed quarterly set of statistics, including extensive reusable datasets, and either an analysis of a topic of interest from the datasets or a detailed annual report of the latest finalised annual data.

PROMs in use in arthroplasty surgery comprise both generic and disease-specific scores. The EQ‑5D Index collates responses given in five broad areas (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression), combining them into a single value. EQ VAS is a simple and easily understood ‘thermometer’‑style measure and is a patient’s self-scored assessment designed to measure their general health and quality of life on the day that they completed their questionnaire. This is obviously also an indication of their general health and is not necessarily determined by the condition for which they underwent surgery and which may have been influenced by factors other than health care. The OKS questionnaire is a disease-specific score that aims to combine a number of disease-specific questions into a single, validated score.

The PROMs outputs supplement mortality and complications data, and allow the DoH and NHS England to monitor progress towards their strategic objectives, such as those specified in the NHS Outcomes Framework (NHSOF). The aim is to allow local commissioners and service providers to assess, from the perspective of the patient, the results of treatment and care, with the goal of improving overall quality of care. It enables patients and clinicians to make an informed decision on the choice of treatment provider.

Adjusted average health gains are calculated using statistical models which account for the fact that each provider organisation deals with patients with different case mixes. The objective of the case-mix or risk adjustment process is to adjust the reported PROMs health status data, taking account of variables such as patient age, sex and comorbidities, amongst others, across the country. These variables are beyond the control of the provider and the adjustment can allow comparison between providers on a like-for-like basis.³ Random variation in patients means that small differences in averages, even when case-mix adjusted, may not be statistically significant. ‘Control limits’ are therefore defined and calculated, which represent boundaries. Providers falling outside of these limits may be stated with statistical validity to be significantly better (if above the upper limit) or significantly worse (if below the lower limit) than England as a whole.

In order to allow for this case-mix adjustment, patients are expected to recall their comorbidities and list them on the form. There is no formal guidance on how to do this and this may represent a problem. It is unclear how well patients can recall all of their medical problems and as a result they may fill in the form incorrectly. A common example might be in the case of hypertension – most patients feel that, whilst on medications, their blood pressure is controlled and hence they do not have “high blood pressure”.

The patient’s predicted post-operative functional score is calculated, based on several variables. The case-mix adjustment model sets out these different variables for the various scores that make up PROMs. Each variable has a coefficient that affects the predicted score, either in a positive or negative manner. This is to try and quantify the impact that different comorbidities may have on the outcome of the individual patient. The different variables have different impacts on the various scores. The OKS is the most relevant of these to this study. Table 1 shows the list of comorbidities collected in the PROMs booklet. Only some of these have been shown to affect OKS scores, and as such only these have had their coefficient listed.⁴