Article Text
PDF
Abstract
Objective To assess quality of care for children presenting with acute abdominal pain using validated indicators.
Design Audit of care quality for acute abdominal pain according to 21 care quality indicators developed and validated in four stages.
Setting and participants Medical records of children aged 1–15 years receiving care in 2012–2013 were sampled from 57 general practitioners, 34 emergency departments (ED) and 28 hospitals across three Australian states; 6689 medical records were screened for visits for acute abdominal pain and audited by trained paediatric nurses.
Outcome measures Adherence to 21 care quality indicators and three bundles of indicators: bundle A-History; bundle B-Examination; bundle C-Imaging.
Results Five hundred and fourteen children had 696 visits for acute abdominal pain and adherence was assessed for 9785 individual indicators. The overall adherence was 69.9% (95% CI 64.8% to 74.6%). Adherence to individual indicators ranged from 21.6% for assessment of dehydration to 91.4% for appropriate ordering of imaging. Adherence was low for bundle A-History (29.4%) and bundle B-Examination (10.2%), and high for bundle C-Imaging (91.4%). Adherence to the 21 indicators overall was significantly lower in general practice (62.7%, 95% CI 57.0% to 68.1%) compared with ED (86.0%, 95% CI 83.4% to 88.4%; p<0.0001) and hospital inpatient settings (87.9%, 95% CI 83.1% to 91.8%; p<0.0001).
Conclusions There was considerable variation in care quality for indicator bundles and care settings. Future work should explore how validated care quality indicator assessments can be embedded into clinical workflows to support continuous care quality improvement.
- clinical practice guidelines
- quality improvement
- general practice
- paediatrics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Evidence-based or consensus-based clinical practice guidelines (CPG) and recommendations are accepted as central in supporting care quality and safety.1 CPGs can only be effective if implemented into practice by front-line clinicians, and yet adherence is rarely measured.2 The CareTrack studies in adults1 and in children3 report that approximately 60% of care is delivered as recommended in CPGs for common conditions. Acute abdominal pain in children is a common reason for accessing medical care in all healthcare settings, including general practice (GP) and emergency departments (ED). It is associated with high hospital admission rates and significant morbidity.4 5 There are no reliable Australian prevalence estimates for childhood acute abdominal pain, but 10% of all visits with general practitioners (GPs) are for acute abdominal pain and approximately 13% of children aged <18 years who experience abdominal pain will seek medical attention.6 7
Acute abdominal pain is an important symptom of many clinical conditions, ranging from mild and self-limiting conditions (eg, gastroenteritis, constipation and urinary tract infections) to life-threatening conditions requiring urgent surgical intervention (eg, intestinal obstruction, incarcerated inguinal hernia, testicular torsion, intussusception, volvulus and appendicitis).4 5 8 In adolescent girls, ovarian torsion or cyst rupture and ectopic pregnancy should also be considered as differential diagnoses.4 9 Abdominal pain may also be a feature of systemic conditions like diabetic ketoacidosis.4 10 Appropriate and definitive management of acute abdominal pain therefore relies on an accurate and timely differential diagnosis of the aetiology. This, in turn, requires clinicians taking and documenting a detailed history, thorough physical examination and ordering appropriate investigations when indicated.
The CareTrack Kids (CTK) study developed and validated care quality indicators for 17 common childhood conditions including acute abdominal pain, according to available international CPGs.3 The overall findings of CTK have been previously reported.3 In this paper, we focus on the CTK care quality indicators for acute abdominal pain in Australian children presenting to the GP, ED and inpatient hospital settings. To our knowledge, the quality of care in children with acute abdominal pain has not been previously measured in a systematic manner and at scale.3 11–13
Methods
Study design
This study is part of the CTK programme of research that assessed the quality of documented care in Australian children aged 0–15 years during 2012 and 2013 for 17 childhood conditions.3 CTK is the second large-scale population-based study to assess the quality of care provided to children, and the first to include acute abdominal pain.3 The CTK methods have been described in detail elsewhere.3 14–16 Here, we describe those aspects of the methods that relate specifically to acute abdominal pain.
Development of indicators
We defined a clinical care quality indicator as a measurable component of a standard or guideline, with explicit criteria for inclusion, exclusion, time frame and practice setting. Thirty recommendations were extracted from eight available CPGs which covered recommendations for GP, ED and inpatient settings.15 Of the 30 recommendations three were excluded because they were imprecise, for example, used indefinite auxiliary verbs such as ‘may’ or ‘consider’.
Candidate recommendations underwent a three-round modified Delphi internal review by two paediatricians and one general practitioner involved in CTK. Four paediatricians and two GPs external to CTK further reviewed and modified the recommendations, again using a modified Delphi method.14 A modified RAND-UCLA method was used for both internal and external reviews.17–22 Additionally, reviewing clinicians recorded the level of clinical impact and whether it was feasible to extract information about the recommendations from patient records.14 The final seven recommendations were reformatted as 21 care quality indicator questions (see online supplementary appendix table 1). For example, a recommendation that children should not be inappropriately referred for imaging was used to create three separate indicators, ABDO13–ABDO15 (table 1).
Supplemental material
Description of care quality indicators and components of bundles
Sample size, sampling process and data collection
The general sampling methods have been published3; additional details specific to acute abdominal pain can be found in online supplementary appendix 2. CTK sampling targeted at least 400 medical records for acute abdominal pain and 6000 medical records for the 16 other conditions. Nine purpose-trained paediatric nurses screened medical records for visits for acute abdominal pain. They reviewed the selected records for adherence to quality indicators and collected relevant data.14 16 Participating sites (GP, EDs and hospitals) were selected from randomly chosen administrative units (‘health districts’) in Queensland (Hospital and Health Services), New South Wales (Local Health Districts) and South Australia (Local Health Networks). Of the invited sites, 92% of hospitals and 24% of GPs agreed to participate (see online supplementary appendix 2). Records of children aged 1–15 years receiving care in 2012 and 2013 were assessed in each site. Records of children aged less than 12 months were not assessed because of the difficulty in confirming whether the presenting problem was acute abdominal pain or not.
Supplemental material
Analysis
Adherence was measured on three levels: overall adherence to the 21 indicators, adherence with each individual indicator and adherence with indicator ‘care bundles’. At the individual indicator level, adherence was measured as the percentage of each indicator scoring a ‘yes-adherent’ for all eligible visits. Individual indicators describing similar or related aspects of care were grouped into three bundles: bundle A-History, bundle B-Examination and bundle C-Imaging (table 1). Indicators within a bundle were given equal importance as the CPGs from which the indicators originate did not rank their importance. Adherence with a bundle required all indicators in that bundle be scored ‘yes-adherent’ for a particular visit. Sampling weights were constructed as specified in online supplementary appendix 2 to adjust for oversampling of states and healthcare settings and for sampling within health districts.3 Similar methods were used to calculate estimates for each healthcare setting across all indicators, and across all indicators in each of the three indicator bundles.
The weighted data were analysed in SAS V.9.4 using the SurveyFreq procedure. Variance was estimated by Taylor series linearisation and the primary sampling unit (health district) was specified as the clustering unit. Stratification and, where appropriate, domain analyses were used (see online supplementary appendix 2, online supplementary appendix table 2). Exact 95% CIs were generated using the modified Clopper-Pearson method, except when adherence was 100%, where the unmodified method was used. In both individual indicator and indicator bundle reports, results were suppressed if there were <25 eligible assessments. Statistical significance, where calculated, was based on the F-test approximation of the Rao-Scott χ2 test, which adjusts for the design effect.
Ethical considerations
We received primary ethics approvals from all hospital networks and the Royal Australian College of General Practitioners (HREC/14/SCHN/113; HREC/14/QRCH/91; HREC/14/WCHN/68; NREEC 14-008), in addition to 34 site-specific approvals allowing for data collection from medical records without individual patient consent as the study entailed minimal risk to healthcare professionals and patients.16 To protect participants, statutory immunity from litigation was obtained in recognition of CTK as a quality assurance activity, from the Federal Minister for Health under Part VC of the Health Insurance Act 1973 (Commonwealth of Australia).
Results
Characteristics of audited medical records
Five-hundred and fourteen children had 696 visits for abdominal pain across three states of Australia (figure 1). Visits for acute abdominal pain were retrospectively reviewed in 57 GP, 34 ED and 28 inpatient settings. A median of 14 indicators were assessed per record with a total of 9785 indicator assessments undertaken. Most children (484, 94.2%) had one or two visits. Only 4.3% were 1–4 years old, 14.6% were 3–4 years old, 59.1% were 5–11 years old and 22.8% were 12–15 years old.
Geographical and clinical settings of the study participants. Total number of consultations (n=696): 228 in New South Wales, 254 in Queensland, 214 in South Australia. 327 emergency department, 81 hospital admissions and 288 general practice (GP) consultations (adapted from https://mapchart.net/, CC BY-SA 4.0).
Overall adherence
The overall adherence for all indicators and settings was 69.9% (95% CI 64.8% to 74.6%). Overall adherence was significantly lower in the GP setting (62.7%; 95% CI 57.0% to 68.1%) compared with ED setting (86.0%; 95% CI 83.4% to 88.4%; p<0.0001) and inpatient hospital setting (87.9; 95% CI 83.1 to 91.8; p<0.0001).
Adherence to individual indicators
Table 2 summarises adherence to individual indicators. The highest adherence was recorded for indicator ABDO13, appropriate use of imaging, at 91.4% (95% CI 77.1% to 98.1%). The lowest adherence was recorded for indicator ABDO08, assessment of severity of dehydration, at 21.6% (95% CI 13.0% to 32.5%). Other indicators with less than 50% adherence included: documentation of vital signs (ABDO07; 43.9%; 95% CI 30.2% to 58.3%) and examination of the inguinoscrotal area in boys (ABDO12; 31.6%; 95% CI 16.0% to 51.0%) (table 2).
Proportion of encounters where practice was assessed to be adherent to the care quality indicators
Adherence to indicator bundles
Across all settings, adherence to bundle A-History (29.4%) and bundle B-Examination (10.2%) was lower than to bundle C-Imaging (91.4%) (table 2). Adherence to bundle A-History was significantly lower in the GP setting at 12.6% (95% CI 4.8% to 25.1%) compared with 67.9% (95% CI 57.1% to 77.5%; p<0.0001) in the ED setting and 73.3% (95% CI 57.4% to 85.8%; p<0.0001) in the hospital inpatient setting (table 3). Adherence to bundle B-Examination was also significantly lower in the GP setting at 2.3% (95% CI 0.5% to 6.8%) compared with 27.9% (95% CI 20.3% to 36.5%; p<0.0001) in the ED setting and 35.7% (95% CI 17.5% to 57.6%; p<0.001) in the inpatient setting.
Adherence to bundles of related indicators by healthcare setting
Discussion
For the first time we have measured and demonstrated gaps in care quality for paediatric abdominal pain according to validated indicators. The overall adherence to care quality indicators for acute abdominal pain was 69.9%, which is higher than the overall average adherence for all 17 CTK conditions combined (59.8%).3 In the current study, health professionals in all healthcare settings demonstrated a high level of adherence to indicators related to avoidance of unnecessary imaging when there was no history of abdominal trauma, and no signs or history of obstruction or perforation, or ingested foreign objects. This aligns with the recommendations in the current Evolve guidelines of Royal Australasian College of Physicians.23 24 The WHO recently highlighted the potential risks of ionising radiation for children due to unnecessary medical imaging.25 In our study, inappropriate imaging may have occurred in approximately 10% of encounters in the GP setting, representing low-value care that could be avoided.
Our results suggest that education about the importance of thorough physical examination and assessment of vital signs is needed to prevent potential misdiagnosis of serious conditions associated with paediatric acute abdominal pain. For example, regularly assessing and documenting vital signs helps with the early detection of deteriorating patients,26 yet adherence with this indicator occurred in less than half of the visits in our study. Similarly, a comprehensive physical examination is needed to avoid missing serious diagnoses such as incarcerated inguinal hernia or testicular torsion, but examination of the inguinoscrotal area was done in only ~30% of visits for abdominal pain in boys. Approximately a quarter of children presenting to ED with severe acute abdominal pain did not receive appropriate pain relief. Pain relief in children is thought to make healthcare safer, alleviating anxiety, allowing for a more thorough examination, and it does not interfere with differential diagnosis or treatment.10 27
Adherence to indicator bundle A-History and bundle B-Examination was low across all settings which may indicate that clinicians adhered to some quality indicators within a bundle, but rarely adhered to all. Adherence to bundle A-History and bundle B-Examination was significantly lower in the GP setting. There are several potential reasons for this. The assessment of adherence to quality care indicators relied on appropriate documentation in patients’ medical records. It is possible that GPs asked all recommended history-taking questions and conducted all examinations and assessments but failed to document this sufficiently. We know that medical records are often incomplete and sometimes inaccurate.28 29 An average visit with an Australian GP is approximately 15 min.30 Undertaking and documenting a thorough assessment and physical examination in a sick child in this short time frame is challenging for GPs.
Clinicians tend to look first to their own professional college or association for guidance, however, there were no specific guidelines for paediatric acute abdominal pain endorsed by the Royal Australian College of General Practitioners at the time of data collection for our study. Although the indicators used in our study were derived from several guidelines, including those intended for use in ED and hospital settings, all indicators were deemed appropriate by independent paediatrician and general practitioner reviewers for the Australian clinical context, including the GP setting.14 16 Adherence to guidelines in the ED and hospital settings is often supported by sophisticated clinical governance structures, more reliance on teamwork, use of decision aids such as flow charts and prompts embedded in electronic medical records, but this is seldom the case in the GP context. GPs see over 160 different conditions in their daily practice,31 and clinical recommendations are ever proliferating, but few are embedded into GP workflows and medical software.32 Initiatives such as the development and dissemination of clinical pathways have the potential to support GPs in their clinical decision-making and referral practices.33
Strengths and limitations
The CTK study has several strengths including population-based sampling, robust methods for the development of quality care indicators, its size and scope, with data drawn from 113 healthcare delivery settings including GP, ED and inpatients, from three Australian states, and assessment of 9785 individual visits.3 14 Only 24% of invited GPs participated in the CTK study which might represent a selection bias towards those GPs more likely to be adherent to care quality indicators.3 It is estimated that approximately 10% of non-adherence to the indicators may be due to lack of documentation,12 and this is a limitation as the study assessed what was documented and not what was done. Furthermore, the clinical outcomes for the audited visits were not collected, limiting the ability to assess the consequences of adherence or non-adherence for patients.
Auditing of patients’ medical records in person, by trained nurse auditors, is a barrier to repeating the study to monitor changes over time, due to cost and logistical constraints, multiple ethics permissions, variability of medical record platforms and their quality. The current lack of unified electronic medical record platforms and the inability to harvest data across jurisdictions, sectors and facilities remains a significant barrier.
Practical implications and next steps
Our study confirms previous findings showing that embedding recommendations into clinical practice to effect changes in the care delivered to patients remains a long-standing challenge for healthcare systems.34 35 However, developing and validating a set of care quality indicators for acute abdominal pain that are applicable across healthcare settings affords opportunities to apply these indicators to monitor care quality improvement over time. To do this effectively without overburdening clinicians, automated solutions that interface with clinical software could support routine data collection to increase the completeness of documentation submitted by clinicians. Real-time data analysis and feedback to individual clinicians, teams and organisations shows promise to support benchmarking and to reduce unwarranted variation in care.36
Adoption of clinical recommendations could be further improved by encouraging codevelopment and codesign of dissemination, implementation and evaluation plans alongside the development of content of CPGs.37 Embedding recommendations into electronic decision support tools,32 using clinician champions,35 and developing companion recommendations for consumers are increasingly used to support adoption.38 39 Evaluation of the relationship between adherence to care quality indicators and patient outcomes and experiences may be a strong incentive for clinicians to change practice.40 Research in this area should be actively encouraged. There may be opportunities to embed the measurement of care quality indicators in pay-for-performance programmes such as the soon-to-be-implemented Australian Practice Incentive Program—Quality Improvement, which aims to reward GPs who collect and share data about quality performance indicators.41 42
Conclusion
Our study has, for the first time, enabled measurement of gaps between recommended care and documented care for acute abdominal pain in children in all healthcare settings in Australia. The significantly lower adherence in GP may reflect setting-specific challenges routinely faced by GPs, such as limited time, seeing children infrequently and limited decision support and clinical governance structures. Future work should explore how the validated indicators can feasibly be applied by individual clinicians and organisations in a structured and automated manner using clinical software, where possible, to help them to continuously assess and improve care quality.
References
Footnotes
Twitter @YvonneZurynski, @peter_hibbert
Contributors All authors read and contributed critical comments and suggestions that were incorporated into the manuscript. YZ interpreted the results, ensured additional analyses were undertaken and led the writing of the manuscript. KC and GA contributed substantial intellectual input into the interpretation of results and writing of the manuscript. PDH was the programme manager for the CTK study, oversaw data collection and analysis and significantly contributed to the interpretation of data, through his deep knowledge of the CTK study methodology. CM and LKW helped design the data collection instruments, collected data, carried out the initial analyses and interpreted the results. HPT and GA led the statistical analyses and contributed to the interpretation of data. JB was the chief investigator on the CTK study—he conceived the study and its design contributing intellectual guidance at all stages, and approved the final manuscript. LKW and CdW contributed to the draft and final manuscripts. All authors had full access to data and statistical analysis.
Funding This study was funded by the National Health and Medical Research Council (grant number: APP1065898) (http://dx.doi.org/10.13039/501100000925).
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Royal Australian College of General Practitioners (HREC/14/SCHN/113; HREC/14/QRCH/91; HREC/14/WCHN/68; NREEC 14-008).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.