Improving health outcomes, through the prevention, diagnosis and treatment of illness, is the stated reason why most healthcare systems exist. It is therefore unfortunate that robust outcome measures are not routinely collected as part of the provision of care. If patients are even followed up, clinicians often record only subjective, unstandardised measures of response to treatment. At the provider and system level, crude uni-dimensional outcome measures, such as mortality, are sometimes recorded. More often, process measures are used as proxies for outcomes, but these often do not capture what was intended and are subject to gaming (Dunbar-Rees 2015).
Outcomes measurement is a key building block of a Learning Healthcare System (McGinnis 2015). Unless it includes standardised details of how outcomes vary under different circumstances, routinely collected data will be of limited value in improving healthcare delivery (Bates 2015).
There is recognition that measuring outcomes is central to improving the quality of healthcare (Dunbar-Rees 2015, Rubin 2015, Stowell 2015). Currently collected routine data can sometimes provide insights into outcomes (Morrow 2015) and many providers and national organisations have begun measurement programmes (Akerman 2015, ICHOM 2015). These efforts have often used process measures or a narrow range of outcomes, for example, just mortality, to make judgements about the quality of care. Even in 1863, Florence Nightingale had highlighted the reductionist nature of this approach:
“If the function of a hospital were to kill the sick, statistical comparisons of this nature would be admissible.”(Nightingale 1863)
Porter (Porter 2010) has proposed a three-tiered outcome measures hierarchy that captures the multiple dimensions of a patient’s health . This hierarchy highlights the importance of survival/mortality, but also takes account of other significant outcomes such as, degree of recovery, time to recovery, harm caused during treatment, recurrences and long-term consequences of care.
This hierarchy does not indicate which outcome measures should be collected in a particular situation. That will depend on the condition in question and on what is important to that particular patient group.
The International Consortium for Health Outcomes Measurement (ICHOM) has been set up to define global Standard Sets of outcome measures for all medical conditions and to drive adoption and reporting of these measures worldwide (Akerman 2015). Essentially, these standard sets populate Porter’s hierarchy (Porter 2010) for each condition. They are created through international collaborations among patients, clinicians and outcomes researchers. ICHOM also support a scalable implementation network to assist partners to implement the Standard Sets (Akerman 2015).
Currently, it is unusual for outcome measures to be collected in such a structured and robust way (Porter 2010, Dunbar-Rees 2015), but by 2017, ICHOM aim to have published 50 Standard Sets covering outcomes measurement for more than 50% of the global disease burden in developed countries (Akerman 2015). This work is open source and ICHOM is aware of 60 partners who are already implementing them in 18 countries (Akerman 2015). ICHOM have the goal of covering half of all medical care with transparent medical data in 10 years (Stowell 2015). This study has detected significant interest in adopting this approach within the English health system.
Participants believed that in the near future, there will be a completely different landscape for outcomes measurement. There will be increased focus on the patient, with reimbursement to providers based on outcomes (Akerman 2015, Dunbar-Rees 2015). An increasingly fragmented provider market may mean that the payer/commissioner or a national body, such as the HSCIC in England, must take a more proactive role in tracking outcomes (Stowell 2015). This is already the case with billing data in the US and HES data in England.
As the volume of outcomes data increases, it will become easier to make risk adjustments for case mix and other factors (Devlin and Appleby 2010).
Data Collection
Research trials collect detailed and standardised outcomes, but these are often too narrow or too burdensome for use in routine practice. There is a strong sense that clinicians are already overstretched and will not tolerate additional tasks that do not enhance the care of the patient who they are currently treating (Foley and Fairmichael 2015).
While detailed clinician completed outcome scales may be appropriate in certain circumstances, they will not be a sustainable universal solution. For many one off treatments, it does not make sense to bring patients back, just to record their outcomes (O’Hanlon 2015). Several other approaches have been proposed.
Some smaller suppliers of electronic records have already incorporated outcome measurement into the EHR and ICHOM have detected increasing interest in this approach from larger suppliers. For example, MD Anderson, in the US, has been exploring how the ICHOM Standard Sets could be integrated with Epic (Akerman 2015).
Natural Language Processing might be used to create structured outcome data from free text clinical notes (Platt 2015, but this could be subject to error and in many cases, the clinical notes simply do not contain sufficient detail.
There is now a range of vendors who offer services to automate the collection of patient-reported outcomes (Dunbar-Rees 2015). These systems can automatically call patients and elicit outcome measures or can use web portals and mobile apps to engage patients. The cost of implementation is falling quickly (Stowell 2015).
Organisations are beginning to explore passive recordings from mobile apps as an alternative to actively collecting Patient Reported Outcome Measurements (PROMs) (Rubin 2015). For example, mobility following treatment can be measured by asking the patient or it can be collected through the sensors on the patient’s smart phone. This is an example of “out of app behaviour”. Another gaming app estimates the severity of autism using “in app behaviour”, behaviour patterns within a game, and “out of app behaviour”, the way the iPad is held and the way that the screen is touched (Dunbar-Rees 2015).
Ginger.io use passive mobile data to examine how people interact with social networks via their smart phone. They also incorporate PROMs via condition specific surveys and combine the results using behavioural analytics that can produce a depression score (Dunbar-Rees 2015).
Using a range of different collection methods will allow triangulation of results across the outcomes measurement hierarchy, making the system more resistant to gaming and bias.
Leadership buy-in
Buy-in from both senior management and clinical leadership is critical. Clinical leaders are often quickly convinced of the value of measuring outcomes, but they may not be in the management of their organisation, therefore it is essential to gain strategic buy-in from the senior management as well (Stowell 2015). This can be achieved when they realise that what is currently measured is often administrative data and process measures that do not represent value for the patient (Akerman 2015).
Conclusion
Outcomes measurement will form an important building block of the Learning Healthcare System. There appears to be a solid trend in that direction. This is enabled by new technological solutions and by recognition that it is difficult to improve what is not measured. The most obvious use of outcomes data will be in driving improvement by benchmarking teams and organisations, but this data will also drive other aspects of the Learning Healthcare System, such as predictive modelling and comparative effectiveness research. These will be discussed in Use Cases.
Evidence