Article Text
PDF
Abstract
Objective The authors examined the validity of documentation produced during paediatric emergency care to determine if a patient-driven health information technology called ParentLink produced higher-quality data than documentation completed by nurses and physicians.
Design The authors analysed the quality of information across elements of allergies to medications and the history of present illness (HPI) collected during a quasi-experimental intervention study where control periods with usual care alternated with intervention periods when ParentLink was operational. Documentation by emergency department (ED) providers was abstracted and compared with information generated through ParentLink. The criterion standard for the history of allergies to medications was a structured telephone interview with parents after the ED visit. A valid report for a medication allergy was one that was both accurate and complete. Completeness of the HPI for acute head trauma was evaluated across seven elements relevant to an evidence-based risk assessment.
Results Of 1410 enrolled parents, 1111/1410 (79%) completed the criterion standard interview. Parents' valid reports of allergies to medications were higher than those of nurses (parents 94%, nurses 88%, p<0.0001). Parents' valid reports of allergies to medications were greater than those of physicians (parent 94%, physicians 83%, p<0.0001). ParentLink produced more complete information on HPI for head trauma than the medical record for five of seven elements.
Conclusion ParentLink provided electronic information that met or exceeded the quality of data documented by ED nurses and physicians.
- Paediatrics
- emergency medicine
- parent
- medical history taking
- patient-centred care
- documentation
- medical informatics
- information science
- assessment
- communication
- emergency department
- information technology
- patient-centred
Statistics from Altmetric.com
- Paediatrics
- emergency medicine
- parent
- medical history taking
- patient-centred care
- documentation
- medical informatics
- information science
- assessment
- communication
- emergency department
- information technology
- patient-centred
Background
Delivery of high-quality care in complex systems requires high-quality information. Technologies that support clinical care through decision support and order entry depend on electronically available information that is accurate and complete. Acute care settings such as the emergency department (ED) are challenged in how to efficiently populate and update a centralised repository with relevant, correct and up-to-date information.1–4
The Institute of Medicine has called for a health information infrastructure that provides ‘immediate access to complete patient information and decision-support tools for clinicians and their patients.’5 Prior research suggests that patients can contribute valid data in electronic form to a health information infrastructure for risk screening and management of chronic disease, but examples of patient-inclusive information strategies that target latent errors in prescribing and testing during acute care are limited.6–16
We developed a patient-centred technology called ParentLink that captures clinically relevant, parent-reported information to support the delivery of emergency care to children. In this study, we evaluated the validity of data produced through the use of ParentLink during real-time emergency care. We sought to determine if ParentLink produced a more accurate allergy history than that documented by nurses or physicians, and if ParentLink produced a more comprehensive history of present illness for head trauma than that documented by ED providers.
Methods
Overview
We analysed the quality of information across elements of current symptoms and allergies to medications collected during a quasi-experimental intervention study that evaluated the effect of a patient-centred health information technology (HIT) during paediatric emergency care. During the study, 3-month control periods alternated with 3-month intervention periods when a parent-driven HIT called ParentLink was operational. ParentLink prompted parents to report on the child's current symptoms and allergy history, and provided tailored output to both parents and ED providers. Documentation by ED providers was abstracted and compared with information generated through parents' use of ParentLink. The study was conducted between June 2005 and June 2006 at the ED of an urban tertiary care children's hospital (site 1) and the ED of a suburban, general community hospital (site 2). The Institutional Review Boards at both sites approved the study.
ED sites
Site 1, the urban children's hospital ED, sees an average of 55 000 patient visits per year. At the time of the study, the environment included computer-based charting for physicians (EMstation, Cerner Corporation, Kansas City, MO, USA). During the study period, nurses documented on paper.
Site 2, a general community ED, sees an average of 18 000 paediatric visits a year with an overall patient volume of 77 000 visits per year. Physicians charted via phone dictation or computerised documentation. Nurses charted on paper and transitioned to electronic charting during the study period.
Subjects
Parent–child dyads presenting for ED care were eligible if:
parent reported English or Spanish as primary language, agreed to complete a follow-up telephone interview and reported being the primary caretaker for the child;
child's triage status was non-critical (level 2–5 for the emergency severity index);
physician's evaluation and treatment not yet initiated;
child met one of the following age/chief complaint combinations
child under age 12 years presenting with head trauma, ear pain, or dysuria;
child between 1 year and 12 years with respiratory symptoms and a history of asthma;
child between 3 months and 2 years presenting with fever.
Study protocol
The consent and enrolment process remained consistent during control and intervention periods. Parents were approached for participation after nursing triage.
ED-based research steps
During control periods, usual care proceeded without intervention. During intervention periods, parents used ParentLink on a mobile kiosk (Seepoint Technology) to enter data on their child. A research assistant (RA) coordinated the informed consent process and introduced parents enrolled during intervention periods to the kiosk; once the introductory screen was displayed for the parent, the RA did not assist the parent further. Output from the kiosk was handed to the provider caring for the child or placed on the bedside chart prior to evaluation. Providers reviewed the printed output generated from parents' interaction with the technology at their own discretion.
Details of ParentLink's functionality, implementation and clinical impact have been previously reported.17 ,18 Ninety-four per cent of parents (538/575) who started use of the kiosk entered enough information to produce output for review. The median time for parents to complete the computer interview was 11 min.
Criterion standard interview
Parents completed a structured telephone interview within 10 days of the incident ED visit. This interview, conducted by a trained research assistant blinded to parents' kiosk entries and to ED providers' documentation, was completed in English or Spanish according to the parent's preference. The telephone interview was conducted with the parent who enrolled as a study subject at the incident ED visit. The history of allergies to medications was assessed using a previously published set of questions intended to maximise sensitivity in the capture of the allergy history.19 ,20 These questions first probe whether the child has ever had a problem or reaction to a medication that they were given and then further elicit from the parent the details specific to the reported problem or reaction.
Abstraction of the medical record
A trained research assistant reviewed the nurse and physician records to abstract documentation specific to any history of allergies to medications. A notation of ‘NKDA’ or ‘no allergies’ was considered to represent a negative history of allergies to medications. A lack of documentation regarding the allergy history was coded as missing. For each recorded medication allergy, notation was made of any details concerning reaction type.
Head trauma is an exemplar condition where parent-reported symptoms map directly to a critical decision step in ED care—whether to order a head CT. For the subset patients presenting with head trauma, the research assistant abstracted the nurse and physician records across a series of data elements germane to risk assessment of intracranial injury, including date/time of trauma, mechanism, loss of consciousness, seizure, vomiting, mental status and presence of scalp haematoma.21–24 The nurse and physician records were considered together; in cases where the documentation was in conflict, the physician's assessment was considered final.
Outcomes
The primary outcomes were: (1) the percentage of parent–child dyads with a valid list of medication allergies as documented by the parent, nurse and physician, and (2) the percentage of responses for each element of the acute head trauma history that were complete as documented by the parent and the ED clinical record.
Analysis
We scored the validity of allergy lists as documented from parents using ParentLink and by physicians and nurses in the medical record. We limited our analysis to a prespecified list of 100 common oral medications as determined by an expert paediatric pharmacist. These medicines included common antibiotics, antipyretics, respiratory-specific drugs, seizure-specific drug, gastrointestinal-specific drugs, antihistamines and medicines used to treat pain.
A response (either positive or negative) for each allergy to medication was considered valid if it correlated with the answer from the criterion standard interview. Invalid responses were classified as either inaccurate (‘over-reporting,’ ie, reporting an allergy which was not reported in the criterion interview) or incomplete (‘under-reporting,’ ie, failing to report an allergy that was reported in the interview, or the absence of any data recorded for that element). For each data reporter (parent, nurse, physician), the overall response was considered valid if the allergy list documented matched the allergies reported on the criterion standard.
The accuracy of the allergy lists was summarised by the percentage of parent–child dyads with valid lists as reported by the parent (during intervention periods only), and by the nurse and physician (during both intervention and control periods). Pairwise comparisons of these percentages between parent, nurse and physician were made using the McNemar test. Comparisons with parent responses could only be performed during intervention periods since the parents' responses were made through ParentLink.
For HPI elements relevant to head trauma, we first compared the percentage of cases where the parent reported the information through the ParentLink kiosk with the percentage of cases where the information was documented in the medical record, using the McNemar test. For each element, we then considered the subset where both parent and physician reported information and assessed agreement on the information reported, using the κ statistic.
To better understand providers' usual practice of documenting the HPI for head trauma, and to investigate whether exposure to ParentLink might decrease providers' propensity to document this information, we compared the mean number of elements documented during periods when ParentLink was and was not in operation, using a t test.
All tests are two-sided, and p values less than 0.05 were considered significant.
Results
A total of 2002 parent–child dyads were screened. One hundred and ninety-two were ineligible (10%), and 399 (20%) declined. Reasons given by parents for declining included: too much time/effort required (6%), does not like research (3%) and child too ill (3%). Ultimately, 1411 (70%) were enrolled, of which one protocol violation was excluded, leaving 1410 subjects for analysis.
One thousand one hundred and eleven of 1410 (79%) completed the telephone interview that determined the criterion standard. Nursing records were available for 1106/1111 (99%) subjects for whom the criterion standard exists; physician records were available for 1086/1111 (94%) subjects. Of 575 parent–child dyads enrolled during the intervention period, 453 (79%) parents completed the telephone interview. Of these 453, parents' documentation, represented by the output from ParentLink produced by the parents' data entry, was available for 430 (95%) subjects.
Table 1 presents the characteristics across the entire cohort, the 1111 subjects who completed telephone follow-up, and provides a breakdown of those subjects completing follow-up in the control and intervention periods. Parents who completed follow-up were more likely to self-report as: more educated (p=0.02), Caucasian (p=0.006), non-Latino (p=0.005), not having a prior ED visit with their child (p=0.04) and having more exposure to technology (p=0.004). Parents of children evaluated for head trauma were more likely to complete the follow-up interview compared with parents of children evaluated for asthma, otitis media or urinary-tract infection (p=0.02). Parents who enrolled in control periods were more likely to have an older child, have self-report race as Caucasian and have made prior ED visits for the child.
Characteristics of study subjects
Valid report of allergies to medications
The criterion standard found a total of 84 instances of allergies to medications across the 1111 subjects who completed the telephone interview postdischarge. These allergies were clustered within 73/1111 (7%) subjects. Table 2 displays the distribution of valid, inaccurate, incomplete and inaccurate/incomplete documentation across the three reporters.
Parent, nurse and physician report of allergies to medications
The validity of parents' report of allergies to medications was compared with that of nurses and physicians across subjects where a criterion standard interview was complete, and a record was available from which to abstract for each reporter (N=430, parent and nurse; N=422, parent and physician). Parents' reports (94%) were significantly more valid than those of nurses (88%, p<0.0001) and physicians 83%), p<0.0001).
Nurses' reports of allergies to medications were compared with that of physicians across subjects where clinical records were available from both, and a criterion standard interview occurred (N=1086). Nurses' valid reports of allergies to medications were significantly higher than those of physicians (nurse 90%, physician 83%, p<0.0001).
Report of data elements specific to the history of acute head trauma
From the 538 subjects in whom ParentLink output was available, 193 cases of acute head trauma were identified where the physician's final diagnosis included a code specific to trauma, and the parent endorsed head injury as the chief complaint. Table 3 summarises results that compare parents' documentation with that of the treating clinicians. Across five of seven data elements, parents' documentation was significantly more complete than that of clinicians.
Comparison of documentation for history of acute head trauma (N=193)
We found a variation in the agreement between parents and treating clinicians across relevant historical data for acute head trauma. Of note, parents' answers often reflected a more serious concern than that recorded by treating clinicians. For the data element, loss of consciousness, parents in 18 cases reported ‘stunned’ or ‘lost consciousness’ when the treating clinician recorded ‘no loss of consciousness,’ whereas in only one case did a clinician record ‘stunned’ when the parent documented ‘no loss of consciousness.’ Similarly, for the data element mental status, parents in eight cases documented ‘still sleepy/not right’ when the treating clinician recorded a response of ‘normal,’ whereas in only one case did the treating clinician record ‘still sleepy/not right’ when the parent documented ‘normal.’
Providers' documentation of the HPI for trauma during control and intervention periods
During control periods, 297 subjects were enrolled who were evaluated for head trauma; only 36% of these subjects had at least 6/7 items relevant to the HPI for head trauma recorded by providers. Of the 193 subjects with output from ParentLink during intervention periods, 50% had at least 6 out of 7 items documented. The mean number of HPI items documented by providers was significantly lower during control periods than during intervention periods (5.1 (SD 1.1) vs 5.4 (SD 1.0), p=0.002).
Discussion
Accurate and complete data to support real-time decision-making is the cornerstone for prevention of errors specific to medications and diagnostic testing.25 ,26 Parents' report of allergies to medications via ParentLink produced a significantly higher rate of valid information than the usual documentation practices of nurses and physicians. Parents' use of ParentLink also provided more complete data needed for an evidence-based decision for testing around acute head trauma compared with providers' documentation. Together, our findings suggest that a structured process of data-gathering yields higher-quality data than existing unstructured styles of documentation and present one potential solution—a patient-driven and technology-mediated approach. A patient-inclusive redesign of the information environment that supports emergency care may eliminate latent errors that spring from inaccurate or missing information.27 ,28
This report builds on previous work regarding the value of patient-centred information management. For the topic of allergies to medications, earlier work had shown that parents' responses to a structured, in-person verbal interview could produce information that improved upon allergy histories documented by nurses at triage.19 The present study reframes that structured interview within a computer-based approach and confirms the improved validity that comes from parents actually documenting the knowledge they possess instead of verbally reporting information that may or may not be documented accurately by a third party.29
High-quality data are an essential attribute of an information infrastructure that supports effective and safe care.26 Our results mirror findings from multiple clinical environments that report how patient-derived data can improve the validity of documented information relevant to risk screening, behavioural health and chronic disease management.6–16 It is important, however, to note that current scientific evidence for the effectiveness of patient-driven, IT-based mechanisms of clinical data collection on outcomes of care is sparse.30 ,31 The ParentLink study did not show an impact on rates of medication errors or outcomes for guideline-specific conditions during intervention periods but, notably, was implemented using paper-based output which had limited influence on providers' actions.17 ,18 Few healthcare systems currently have the capacity to embed patient-generated data streams within an electronically mediated care environment, where the gains from improved data quality can be tightly coupled to decision-making steps.
A structured interview and documentation process can promote accuracy in patient reporting. ParentLink collects information from parents using a ‘directed recall’ strategy.32 This stands in contrast to the open-ended nature of many fields in electronic medical record (EMR) systems used by health professionals. The significant rate of missing data in electronic systems from this unstructured approach limits downstream use of data to drive decision support specific to test ordering and prescribing of medications.
There is a trade-off between the completeness of information recorded by parents using ParentLink and some inherent subjectivity to the answers parents provide. For the data element of mental status, parents more often than ED clinicians report on their child's state as ‘not normal.’ Whether or not these data represent false reports of abnormal mental status from overly concerned parents, or a more sensitive determination of children who display subtle changes only picked up by parents, cannot be determined. It is also possible that any discrepancy in agreement is a result of improvements in mental status over time.
The present study reports on a patient-focused intervention that combined a structured approach to history-taking with a computer-based channel of communication. Independent of any technology, a provider-focused intervention that supported a structured style of interviewing or provided for structured documentation using templates can produce improvement in the quality of data.33 ,34 However, there are potential system-level efficiencies to be gained through applications like ParentLink and other patient-centred examples of health IT that distribute the time and effort burden of collecting and documenting information across all reporters (patients, nurses and doctors) who have a stake in optimising the processes and outcomes of care.35 Further efficiencies will emerge from patient-driven data streams as structured information is mapped to SNOWMED-RT and HL7 terminologies and then integrated into real-time electronic decision support.
This study does have several important limitations. Although our study benefits from a prospective, comparative approach across two EDs, we limited the scope of presenting chief complaints, and our results may not generalise across all patients. The results of this study are not a report on the effectiveness of a systematic implementation. We limited our analysis to 100 common medications, and our findings may not generalise to all medications used to treat children. Our evaluation of documentation necessarily relies on the written record, and therefore may not reflect the full array of questions asked and answers given during a provider–patient interview if providers' documentation is not comprehensive. However, for data to be useful within electronic decision support, and for information to have value for longitudinal care, actual documentation remains essential, and our measurement's focus on the record itself reflects this concern.36 The comparison of providers' documentation during control and intervention periods does not suggest that the availability of the ParentLink output decreased providers' documentation of key historical data.
Conclusions
Early capture of relevant and accurate information in electronic form directly from parents is possible during real-time emergency care. ParentLink supported parents in providing electronic information that met or exceeded the quality of data currently documented by ED nurses and physicians. Structured data collection produces information of superior quality compared with unstructured documentation by providers. ParentLink is one mechanism that can support such a structured approach.
Acknowledgments
Carts and batteries used during the study were donated by InfoLogix Systems, Hatboro, PA.
References
Footnotes
Funding This work was supported by grant R01 HS014947 to Dr Porter from the Agency for Healthcare Research and Quality.
Competing interests None.
Patient consent Obtained from the parents.
Ethics approval Ethics approval was provided by the Children's Hospital Boston, Boston, Massachusetts, USA and South Shore Hospital, Weymouth, Massachusetts, USA.
Provenance and peer review Not commissioned; externally peer reviewed.