Advanced analytics to improve performance: can healthcare replicate the success of professional sports?
=======================================================================================================

* Aidan Mcparland
* Alun Ackery
* Allan S Detsky

*   continuous quality improvement
*   quality improvement
*   quality measurement

The Tour de France is revered as one of sports’ most challenging events. Team Sky, a cycling team from the UK, recently captured its sixth Tour victory in 7 years, a feat that makes it one of sport’s greatest dynasties. Dave Brailsford, the coach and architect behind Team Sky, became the Billy Beane of the cycling world by using advanced analytics to identify areas where his team could make marginal gains in their performance producing substantial improvements in outcomes. Beane’s success using this approach with the Oakland Athletics was made famous by a book and movie *Moneyball* and has been followed by many major success stories in the sporting world over the last decade revealing a similar trend. National Basketball Association players use heat maps and wearable devices to improve shooting accuracy. National Hockey League teams optimise line performance using entropy mapping. Moreover, professional soccer teams develop novel metrics to assess fatigue in an attempt to reduce injury. These examples have made the term ‘advanced analytics’ a household expression, a term essentially encompassing the use of automated collection and examination of big data using tools such as artificial intelligence, neural networks and machine learning to analyse otherwise hidden areas of potential performance improvement. Advanced analytics differs from basic analytics in that it involves more granular data, sometimes derived from machine learning, and more sophisticated statistical techniques. Table 1 shows some examples of differences between the two types of analytics in both sports and healthcare.

View this table:
[Table 1](http://qualitysafety.bmj.com/content/29/5/405/T1)

Table 1 
Evolution of basic analytics metrics compared with advanced analytics in sports and healthcare

Healthcare is similar to professional sports in many ways, and examples of how high-functioning sports teams, such as those of Formula 1 teams, may pave the way for healthcare improvement has sparked widespread discussion.1 Akin to athletes, clinicians make quick consequential decisions in physically and cognitively demanding environments. Yet, despite the fact that human factors are now regarded as a leading cause of medical error and death,2 3 advanced analytics measurement practices have not yet affected healthcare providers or system performance to nearly the same degree that it has in professional sports. Where success has been seen using analytics in healthcare, it has largely only begun to occur at institutional levels to implement macrolevel improvements.4–6 This viewpoint discusses reasons why healthcare is not ready to emulate athletics in the era of advanced analytics for provider performance analysis and focuses on the major differences between healthcare and sports in this context (table 2 summarises these differences)).

View this table:
[Table 2](http://qualitysafety.bmj.com/content/29/5/405/T2)

Table 2 
Differences between professional sports and healthcare that affect adoption of advanced analytics

The first difference is the consequences of performance variation. If professional athletes exceed or fail to meet athletic demands, their value and opportunity to continue playing can be enhanced or reduced. In healthcare, once physicians are granted a licence, they can almost always continue to work (barring egregious errors). Of course, at the core of medical practice is the intrinsic motivation to provide high-quality patient care; in reality, some combination of income, prestige and pressures like medical malpractice dominate individual and institutional improvement motivation. While unique pressures drive intense performance assessment in sports, it clearly will not work the same way in healthcare because providers will not be ‘taken out of the line-up’ if they do not improve.

Standardising ‘success’ is the second major difference. Sports uses wins as the main measure of success, reflecting easy to measure binary outcomes. More recently, sports has transitioned from using intermediate metrics like home runs, pass completions and goals to more sophisticated metrics like ‘wins above replacement’ and ‘power calculation’. What does success look like in healthcare? Relevant provider metrics have yet to be defined and vary across specialties and settings. For example, mortality adjusted for patient characteristics might seem like an obvious performance measurement, yet it is not clear what variations in adjusted mortality across hospitals or providers really means, because eventually all patients die. Surgical wound infections make more sense, yet they are only one measure in a specific area of healthcare. Recent experience with 30-day readmission rates also shows conflicting results.7 While Nike’s goal to crack the 2-hour marathon was something that had never been done, the outcome was clear, and achieving this milestone required identifying microscopic aspects of performance improvement for athletes that are largely already practising within an optimised system. Even if healthcare had clear ‘success targets’, major, not minuscule, barriers are often the required targets of improvement, and the current systems are hardly optimised already. Tackling emergency department wait times in the era of systemic hallway medicine is like trying to improve running times on a road filled with potholes. Before we can start to target marginal gains, we must first bring up the average level of performance within these metrics.

Differences in the culture of improvement between sports and healthcare is a third issue. Sports teams have invested heavily in analytics because the industry recognises the value of identifying areas for marginal gain. For example, Arsenal uses technology that assesses player fatigue by measuring the length of time an athlete’s foot strikes the ground when on-field to enhance injury prevention. Lebron James and Stephen Curry used analytics in the form of heat maps to modify their shooting ranges and to increase efficiency. Open source sharing of professional and amateur sports data has created a spectator culture of investing in strategy and overall improvement of play. Alternatively, healthcare practice does not promote exposure of error; in contrast, there may be a culture of excuses when feedback is given (eg, ‘my patients are sicker and older’). Physicians and hospitals fear of malpractice lawsuits and media coverage of errors induces behaviour that often games the system, changing the way performance is measured rather than truly improving. An example of this phenomenon is the Province of Ontario, Canada’s attempt to improve hand hygiene in hospitals through public reporting.8 One proposal to identify errors in healthcare during surgery in a non-punitive fashion is the use of ‘black box’ technology, similar to what has been used effectively in aviation for many years.9 By recording sound and video during operations, surgeons may identify causes of error when they occur and ultimately prevent them.

The question of who will use performance data to inform improvement is a fourth difference between professional sports and medicine. The concept of a coach is essential in sports. In some cases, the coach is also the person who decides whether a player is used and is therefore also the judge of his or her performance. However, in many cases, particularly in individual sports, the coach is hired by the athlete and plays no role in judging the performance. The greatest players of all time still operate under this feedback paradigm whereas one of the most commonly cited challenges in physician performance feedback is the source of that feedback. Throughout training, feedback is largely provided by attending physicians who also function as adjudicators without actual observation of performance, a problem that ‘Competency Based Evaluation’ attempts to address.10 Feedback in medicine is still often vague, involving advice like ‘be more thorough’ or ‘listen to the patient’. Informing cyclists that a climb during a certain phase of the race was ineffective is one thing, but showing them exactly what their VO2 maximum heart rate and nutritional state were is a lot more powerful and more directly conveyed.

The final difference between sports and medicine is the amount of time devoted to practice before performance. In sports, much more time is devoted to preparation than the actual competition, and athletes are paid to do both. Physicians and other healthcare professionals do not have the same proportion of their work lives available for practice, and in the absence of salaried reimbursement that includes time devoted to performance improvement, there is little chance physicians will embrace spending valuable time on this endeavour. Indeed, physicians use the word practice to mean delivery of care rather than honing their skills to improve. While a Formula 1 team may employ detailed assessments of engine function and team performance into hundreds of hours of training outside of a race, healthcare improvement focuses on analysis of actual ‘game time’. This is not likely to change.

Beane and Brailsford have shown us that in the correct environment, the power of advanced analytic performance assessment can yield tremendous gains. While many similarities in performance demands exist between healthcare and professional sports, glaring differences between the practice environments summarised in table 2 limit healthcare’s ability to adopt similar performance improvement practices. Table 2 offers some potential directions for healthcare to resolve those differences, but those steps will involve a tremendous amount of effort. While looking to other industries like sports that have exhibited widespread performance improvements may appear logical, recognising the barriers that will not allow healthcare to quite yet make this jump is important. We argue here that healthcare is not ready to directly emulate the analytics that have revolutionised next-level performance in professional sports due to gross limitations within our own healthcare systems. While other baseball teams may have looked to emulate Beane’s practices to become competitive, perhaps physicians should be studying other already elite practitioners and healthcare teams to first improve macro-level deficiencies.

## Acknowledgments

We thank Kaveh Shojania MD, Trevor Jamieson MD and Sacha Bhatia MD, all at the University of Toronto, for comments on an earlier draft of this essay.

## Footnotes

*   Twitter @Adetsky

*   Contributors All authors participated in the conceptulalisation, writing and revising the paper. AM is the the guarantor for the overall content.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests None declared.

*   Patient consent for publication Not required.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data availability statement There are no data in this work.

## References

1.   Atul G . Cowboys and pit crews, 2011. Available: [https://www.newyorker.com/news/news-desk/cowboys-and-pit-crews](https://www.newyorker.com/news/news-desk/cowboys-and-pit-crews) 
    
    

2.   Shojania KG , Dixon-Woods M . Estimating deaths due to medical error: the ongoing controversy and why it matters. BMJ Qual Saf 2017;26:423-428.[doi:10.1136/bmjqs-2016-006144](http://dx.doi.org/10.1136/bmjqs-2016-006144) 
    
    

3.   Makary MA , Daniel M . Medical error-the third leading cause of death in the US. BMJ 2016;353:i2139–5.[doi:10.1136/bmj.i2139](http://dx.doi.org/10.1136/bmj.i2139) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNTMvbWF5MDNfMTIvaTIxMzkiO3M6NDoiYXRvbSI7czoxODoiL3FoYy8yOS81LzQwNS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.   Komorowski M , Celi LA , Badawi O , et al . The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018;24:1716–20.[doi:10.1038/s41591-018-0213-5](http://dx.doi.org/10.1038/s41591-018-0213-5) 
    
    

5.   Chan T , Sebok-Syer S , Thoma B , et al . Learning analytics in medical education assessment: the past, the present, and the future. AEM Educ train, 2018.
    
    

6.   Phillips CJ , Greene JA , Podolsky SH . Moneyball and medicine. N Engl J Med Overseas Ed 2012;367:1581–3.[doi:10.1056/NEJMp1211131](http://dx.doi.org/10.1056/NEJMp1211131) 
    
    

7.   Ibrahim AM , Dimick JB , Sinha SS , et al . Association of coded severity with readmission reduction after the hospital readmissions reduction program. JAMA Intern Med 2018;178:290–2.[doi:10.1001/jamainternmed.2017.6148](http://dx.doi.org/10.1001/jamainternmed.2017.6148) 
    
    

8.   Muller MP , Detsky AS . Public reporting of hospital hand hygiene compliance--helpful or harmful? JAMA 2010;304:1116–7.[doi:10.1001/jama.2010.1301](http://dx.doi.org/10.1001/jama.2010.1301) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.2010.1301&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=20823438&link_type=MED&atom=%2Fqhc%2F29%2F5%2F405.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000281559800023&link_type=ISI) 

9.   Jung JJ , Jüni P , Lebovic G , et al . First-Year analysis of the operating room black box study. Ann Surg 2020;271:122–7.[doi:10.1097/SLA.0000000000002863](http://dx.doi.org/10.1097/SLA.0000000000002863) 
    
    

10.  Cavalcanti RB , Detsky AS . The education and training of future physicians: why coaches can't be judges. JAMA 2011;306:993–4.[doi:10.1001/jama.2011.1232](http://dx.doi.org/10.1001/jama.2011.1232) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.2011.1232&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=21900141&link_type=MED&atom=%2Fqhc%2F29%2F5%2F405.atom)