The statistician's page

Cumulative Sum Curves and Their Prediction Limits

Training is essential for the safe and uncomplicated placement of hemodialysis catheters. This study explores the learning curve of this procedure.

In this prospective study, 60 patients who needed emergency hemodialysis without vascular access were included. All catheters were placed under ultrasound guidance. One nephrologist was included in each two groups, one to be consisted of a junior, and one to be consisted of a senior. Learning curves were created using the cumulative total methodology and receiver operating characteristic curve analyses.

The patients’ mean age was 67.92 ± 14.23 years. The mean catheter insertion time of the senior nephrologist was significantly shorter than that of the junior. According to cumulative total analysis, the junior group's maximum learning point overlaps with patient 22. When the confidence intervals of the study durations of both groups were examined, they overlapped in the 95% confidence interval starting from the 19th patient. When the mean catheter insertion time of the senior and the mean of the last 12 patients of the junior were compared, there was no significant difference between them (F = 15.827, P = .092). The receiver operating characteristic curve analysis showed a cutoff value of 320 seconds for the junior group compared with the senior group, indicating an overlap in case 22 for the junior nephrologist.

This study suggests that 22 catheter insertions under the supervision of a senior nephrologist are needed to complete the learning curve for a junior nephrologist. If the number of nephrologists at the center is limited, safe catheter insertion may be allowed after 19 insertions.

Reporting of risk-adjusted surgical outcomes is commonly used to compare providers and track changes over time. Preferred graphical methods use the relationship of the observed to the expected values of outcome events, including their ratio (O/E), and the cumulative sum (CUSUM) of their differences over time, called Risk-Adjusted CUSUM (RA-CUSUM) or Variable Life Adjusted Display (VLAD). We demonstrate these methods using operative mortality data for 7255 isolated coronary artery bypass graft patients from January 2014 to June 2017. RA-CUSUM and VLAD are excellent techniques to display risk-adjusted outcomes and, unlike O/E, can provide continuous monitoring as performance varies over time.

To determine the number of cases required to achieve competency in phacoemulsification in terms of the posterior capsule rupture rate.

Singapore National Eye Centre, Singapore.

Retrospective cohort study.

The posterior capsule rupture rate of the first 300 phacoemulsification cases performed by each trainee in the Singapore National Eye Centre (2004 to 2012) was charted using cumulative sum graphs. Competency was primarily defined as a posterior capsule rupture rate of 2% or lower and secondarily as 5% or lower.

Twenty trainees fulfilled the inclusion criteria. Four trainees (20%) and 19 trainees (95%) achieved a posterior capsule rupture rate of 2% and 5%, respectively. At least 41 cases were required to achieve a 5% posterior capsule rupture rate (mean 106, median 83).

There was considerable variation in the trainees’ abilities. Only 20% of the trainees achieved a posterior capsule rupture rate of 2% or lower and required at least 212 cases.

EUS is an operator-dependent procedure and requires more technical and cognitive skills than a routine endoscopic procedure. The learning curve for the staging of gastric cancer, however, has not been evaluated.

To evaluate the threshold number of EUS examinations for gastric cancer T staging.

Retrospective study.

University-affiliated tertiary care hospital in the Republic of Korea.

Four trainees with no previous EUS experience.

Analyzing performance of EUS trainees in gastric cancer T staging by using cumulative sum (CUSUM) analysis.

CUSUM plot and a minimal number of procedures for reaching a plateau.

A total of 553 initial EUS examinations for treatment-naïve gastric cancers, performed by trainees, were enrolled in the study. The final T stage was determined by experts by using EUS in 332 gastric cancer cases, whereas the T stage of the other 221 lesions was determined by trainees by using EUS. The accuracy of EUS examinations performed by trainees and experts was 72.6% and 84.3%, respectively. The number of EUS examinations required to reach the first plateau in each trainee was 20, 41, 60, and 65.

Retrospective study with a relatively small number of trainees.

The CUSUM scores of all of 4 trainees in the study reached a plateau by the 65th examination.

Utilization of statistical graphical methods to detect deterioration and to compare performance of health providers (e.g., surgeons, surgical departments, centers, etc.) has been rapidly increasing. These methods rely heavily on assumptions that may not be applicable in all surgical scenarios. The results produced by those methods could have major potential impact on funding, court rulings, insurance rates, etc. Thus, if one wants to use these graphical methods, it is imperative that the methods produce highly reliable results, even when some of the assumptions that underlie such methods are violated. In this manuscript, we discuss some of the assumptions that underlie such methods. We examine the performance of these methods when some assumptions are violated by using simulations based on analyses of plausible data. Our results show that using current graphical methods to compare two or more health providers when the assumptions are not met could result in misleading conclusions. Hence, researchers should apply these types of graphical methods with appropriate care, and only after making sure that the underlying assumptions are valid or the methods are robust enough to those violations.

Public reports of organ transplant program outcomes by the US Scientific Registry of Transplant Recipients have been both groundbreaking and controversial. The reports are used by regulatory agencies, private insurance providers, transplant centers and patients. Failure to adequately adjust outcomes for risk may cause programs to avoid performing transplants involving suitable but high-risk candidates and donors. At a consensus conference of stakeholders held February 13–15, 2012, the participants recommended that program-specific reports be better designed to address the needs of all users. Additional comorbidity variables should be collected, but innovation should also be protected by excluding patients who are in approved protocols from statistical models that identify underperforming centers. The potential benefits of hierarchical and mixed-effects statistical methods should be studied. Transplant centers should be provided with tools to facilitate quality assessment and performance improvement. Additional statistical methods to assess outcomes at small-volume transplant programs should be developed. More data on waiting list risk and outcomes should be provided. Monitoring and reporting of short-term living donor outcomes should be enhanced. Overall, there was broad consensus that substantial improvement in reporting outcomes of transplant programs in the United States could and should be made in a cost-effective manner.