One of the key recommendations of the Institute of Medicine's (IOM) report, To Err is Human, 15 years ago was for greater attention to incident reporting in healthcare, analogous to the role it has played in aviation and other high-risk industries. With the passage of time and maturation of the patient safety field, we conducted semistructured interviews with 11 international patient safety experts with knowledge of the US healthcare and meeting at least one of the following criteria: (1) involved in the development of the IOM's recommendations, (2) responsible for the design and/or implementation of national or regional incident reporting systems, (3) conducted research on patient safety/incident reporting at a national level. Five key challenges emerged to explain why incident reporting has not reached its potential: poor processing of incident reports (triaging, analysis, recommendations), inadequate engagement of doctors, insufficient subsequent visible action, inadequate funding and institutional support of incident reporting systems and inadequate usage of evolving health information technology. Leading patient safety experts acknowledge the current challenges of incident reports. The future of incident reporting lies in targeted incident reporting, effective triaging and robust analysis of the incident reports and meaningful engagement of doctors. Incident reporting must be coupled with visible, sustainable action and linkage of incident reports to the electronic health record. If the healthcare industry wants to learn from its mistakes, miss or near miss events, it will need to take incident reporting as seriously as the health budget.
- Adverse events, epidemiology and detection
- Significant event analysis, critical incident review
- Patient safety
- Qualitative research
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
- Adverse events, epidemiology and detection
- Significant event analysis, critical incident review
- Patient safety
- Qualitative research
Despite mounting international evidence of harm caused by the healthcare delivered,1–3 there was little collective action worldwide in the 1980s and 1990s to improve the safety of providing healthcare. In 1999, the Institute of Medicine (IOM) published its landmark report, To Err is Human.4 The report created an international sense of urgency to reduce patient harm from healthcare. It recommended sweeping reforms to the healthcare system in order to ensure patients’ safety. Drawing from aviation and other high-risk industries, the report specifically recommended that healthcare organisations institute incident reporting. In doing so, healthcare organisations could learn from adverse events, mitigate contributing factors, prevent future errors and ultimately make patients safer.
Incident reporting proliferated during the ensuing years. The reporting systems adopted various formats, but predominantly required front-line workers to submit reports about situations in which a patient had been harmed or had the potential to be harmed. In the follow-up of adverse events, these systems used either an independent, external body or an internal department to investigate and identify key issues that might need resolution. Some countries developed national adverse event reporting systems (eg, Denmark,5 Australia6 and the UK7) so that findings could be applied widely, and have a national effect.8 Today, although there is no national incident reporting system in the USA, nearly 90% of hospitals track adverse events to comply with the Centers for Medicare and Medicaid Services (CMS) Quality Assessment and Performance Improvement Program,9 and many others maintain or subscribe to their own incident reporting systems.
Over the last 15 years, literally millions of incident reports have been submitted around the world. In the UK alone, 1.5 million reports are submitted each year to their National Reporting and Learning System.7 However, as yet, there is little evidence that incident reporting is associated with delivering safer healthcare.10 During this time, clinicians, hospital administrators and accrediting bodies have gleaned useful insights about the challenges of incident reporting. For example, incident reports detect only a small percentage of relevant patient safety issues11 ,12 as most reports concern mundane events. Additionally, very few doctors submit adverse event reports.13 Instead, incident reporting is largely undertaken by nurses,14 and incident reporting systems largely fall under their governance within healthcare organisations.15 Doctors’ reluctance to report incidents appears to be multifactorial, and includes time constraints, medicolegal fears, a lack of clarity about what to report and a paucity of feedback regarding previously submitted reports.16 Other limitations include the general lack of disseminating information derived from incident reports13 and difficulties in using the data once disseminated.
Given little evidence linking incident reporting to improved patient safety, this report aimed to seek the wisdom and reflections of leading international patient safety experts on the opportunities realised, and the opportunities and challenges not met by incident reporting since the release of the IOM report.
Study design and sample
We conducted semistructured interviews with leading international authorities on patient safety who were familiar with the US healthcare system and met at least one of the following criteria: (1) involvement in the development of the IOM's original recommendations, (2) responsible for the design and/or the implementation of incident reporting systems and (3) conducted research on incident reporting or patient safety research at a national level. Among 17 individuals invited to participate in the study, 11 accepted (65% response rate, see online supplementary appendix 1). Four did not reply to the invitation, one declined to take part based on a conflict of interest and one individual was unable to schedule an interview within the project time frame. Of the 11 participants, seven were from the USA. The local institutional review board (IRB) deemed that the study was exempt from IRB oversight.
Based on information from the incident reporting literature, the study team members developed a semistructured interview protocol focused on gathering interviewees’ perspectives and comments on voluntary incident reporting's role, impact and challenges as well as the future prospects of incident reporting. Between January and June 2014, the lead researcher interviewed 11 experts individually, 10 in person at their place of work and one by telephone. The interviews, 60–90 min in duration, were digitally audio-recorded, and transcribed verbatim. All but one of the interviewees consented to the release of their names on the condition that the comments would be kept anonymous.
For analysis, the transcripts were imported into NVivo V.10 (QSR International, Cambridge, Massachusetts, USA), a software package for managing and analysing qualitative research data. The lead researcher, a priori, developed initial codes based on pre-existing data, which framed the interview questions. Once the data were collected, an iterative analytical process was undertaken. The lead researcher conducted the primary analysis of the transcripts using a combination of inductive and deductive coding techniques.17 This involved developing a thematic coding framework based on codes such as culture of reporting, creation of a learning environment, adequacy of follow-up of adverse event reports previously identified in the published literature and also on those identified through systematic review of the interview transcripts. A second researcher independently read the interview transcripts, and reviewed and modified the coding framework in consultation with the lead researcher to reveal key challenges. In discussion with the other three team members, the coding framework was further developed and refined. Finally, the data underwent a thematic analysis using the five key challenges identified to facilitate summarisation and cross referencing of emergent themes.
All the participants were recognised international experts on various aspects of patient safety. Three participants contributed to the development of To Err is Human. Three participants participated in the development of national incident reporting systems (UK, USA and Australia), one participant helped establish a state-wide incident reporting system and the remainder were either national patient safety or incident reporting researchers (see online supplementary appendix 1).
All of the respondents described the excitement generated by the publication of To Err is Human and the hope that it would lead to safer patient care. A number of participants articulated disappointment about the lack of meaningful progress since that time in delivering safer patient care. Nonetheless, the interviewees expressed, universally, the idea that healthcare organisations need to learn from adverse events if patient safety is to improve. The interviewees conceptualised voluntary incident reporting as a tool to help facilitate learning about and preventing patient harm. For instance, they thought that adverse events could serve as patient safety teaching aids or case studies for medical students. Additionally, they suggested that insights gleaned from analyses of adverse event reports could be shared across healthcare organisations to proactively identify threats to patient safety. As two interviewees pointed out, however, adverse event reporting systems can only facilitate learning and prevent harm if good data are collected. Indeed, on a number of occasions, adverse event reporting systems were described as little more than data-collection tools to generate a hypothesis that can trigger further investigation of the adverse event and lead to action if required.
In analysing the data, it appeared that no new ideas were revealed at the completion of all 11 interviews. Five key challenges emerged as reasons why incident reporting systems had been unable to reach their full potential: (1) reports were inadequately processed; (2) there was inadequate engagement of clinicians, particularly doctors, to report incidents; (3) there was insufficient action enacted in response to reports; (4) there was inadequate funding and institutional support of incident reporting systems and (5) reporting systems were not taking full advantage of the evolving health information technology developments. Box 1 outlines these themes and provides a summary of relevant quotes. The following sections provide a detailed discussion of each challenge.
Emergent themes and summary interpretation of key quotes*
Inadequate report processes
The overwhelming sentiment from all the participants was the unexpected large volume of reports generated in all incident reporting systems. There were often insufficient resources to deal with this volume of reports, leaving reports inadequately triaged, clustered, analysed or acted upon.
Lack of adequate medical engagement
A consistent theme was the lack of engaging doctors to report, own or lead the incident reporting process. This lack of engagement resulted in reporting bias frequently from the nurses and skewing the data. The key incidents, which would reflect medical decision-making, included diagnostic errors and hand offs.
A key message from many participants that reporting could not be considered without linking it to action. In the years following the Institute of Medicine's (IOM) report, much attention had focused on reporting, but not the action or feedback loop to the reporter. Many participants believed that the lack of visible action led to under-reporting of meaningful incidents.
Inadequate funding and institutional support
It was acknowledged by many participants that there had been an under-resourcing of incident reporting systems whether nationally, state or at the local level. The lack of fiscal support had unintended consequences such as not being able to deal with the volume of reports, which inevitably led to a delay in analysing the reports and distilling recommendations for dissemination. There were also concerns raised over accountability of the incident reporting systems and their ensuing actions.
Failure to capture evolving health information technology developments
A repeated theme for the future of incident reporting systems was the use of technology to improve the reporting of incidents, to analyse the incidents and disseminate the results. In the USA, concern was raised that there had been inadequate resourcing or thought of linking the electronic health record to incident reporting.
*Illustrative quotes are embedded in the text.
Inadequate report processing
All of the interviewees commented on the unanticipated large volume of adverse event reports being submitted. Regardless of the type of reporting system (ie, national, healthcare organisation, specific area of risk) or the country in which the reports originated, there were vast numbers of reports being submitted. Although the engagement of those reporting incidents was interpreted positively, the respondents believed that the large number diminished the quality of data being collected. Many participants attributed the volume to, among other factors, ambiguous definitions of ‘adverse events’. This has led to reports of many mundane events that have few system-wide implications. For instance, one participant commented:
So, there were many, many, many incidents being reported which weren't deemed to have generalisable learning for the entire NHS.
Participants indicated that among problems with handling the large volume of reports included the inability to confidently prioritise the analyses without the risk of missing serious events and the duration taken to conduct the analyses. As one respondent indicated, “There was an ambiguity, basically, over whether this [event] was a serious incident, or not.” Furthermore, prioritisation was hampered by the absence of a widely used taxonomy of analysis and the inadequate numbers of trained analysts. A participant believed that healthcare organisations “have not analysed [incident reports] in any coherent or organised way, in spite of the fact that we have a classification that can do that.” A number of respondents noted that the IOM report had probably intended for the incident reporting system for healthcare to mirror that of the aviation industry’s successful approach. However, over the course of 15 years, interviewees thought that healthcare had replicated neither the methods nor successes of the aviation reporting systems. Respondents agreed that following the approach of aviation should have established, among other strategies, effective report triaging processes. By not doing so, one interviewee commented that healthcare had created at times ‘a bit of monster’ by gathering up all the incidents, overwhelming the reporting systems, but leading to uncertain gains.
Lack of adequate medical engagement
Respondents reported that in areas notable for significant improvements in patient safety (eg, central venous catheter-related blood stream infections, routine use of pulse oximetry, wrong-site surgery and medication safety), doctors had been engaged in reporting incidents and/or designing and implementing interventions to reduce patient harm. The majority of respondents believed that doctors, from the outset, had never been adequately engaged or interested in incident reporting. One participant stated, “I think most doctors didn't know it [incident reporting system] existed. I think it was largely used by nurses.”
Indeed, an absence of doctors heading up incident reporting systems was purported as a reason why doctors, in general, do not take ownership of incident reporting systems. For example, a respondent noted, “The missing link is owning the data. And that requires ownership of senior leadership in an organisation. And that includes senior clinical [medical] leadership.” Further explanations given for poor physician engagement included mistrust of the hospital reporting systems due to fear of how reports would be used, uncertainty about what to report and a paucity of peer-reviewed literature substantiating incident reporting.
Without doctors submitting adverse event reports, interviewees stated that reporting defaulted to the nursing staff. The experts commented that by doctors abrogating their responsibility in reporting, the reports submitted were biased and missed important medically specific issues such as diagnostic errors. As evidence, one respondent noted that “diagnostic errors, hand-off errors, computer glitches that are all affecting the doctors’ work every single day but aren't appearing in the incident reports. So it gives you a very biased snapshot of what's going on in the organization.”
Respondents described frameworks developed to support medical engagement, including the introduction of legislation to legally protect the information reported. However, one interviewee believed that the current legislation, protecting adverse event information, in fact, stifled the reports’ utility: “the current legislation of the United States and some associated regulations tend to retard sharing of information. It’s so protected that we can’t, you know, you strip out any kind of, all the identifiers, and you’re left with nothing.” It was also stated that if protection of information were to continue, healthcare systems would have to demonstrate visible improvements in care resulting from the protected information.
A significant issue raised was the lack of visible action as a consequence of reporting an adverse event, and was considered to be one of the major weaknesses of incident reports. The failure to see visible action was believed to have a strong negative influence on the commitment by front-line workers, particularly doctors, to reporting adverse events. A participant remarked, “I think [lack of action] does the opposite of a learning system. You produce this kind of sclerosis where people know you have to report the incident, but they [doctors] equally know nothing will happen.”
Ideally, following the investigation of a report, feedback from the analysis would be delivered to the reporters as well as to relevant people in the organisation to close the loop. As illustrated by the following comment from one participant, it was articulated by participants that reporting back the analysis of the incident and its subsequent outcomes to the reporters would strongly link the reports to improvements in safe patient care: “I’m saying there’s no point in looking at reporting, you’ve got to look at the whole kind of feedback loop. To have a reporting system without any means of action is absolutely pointless.” Yet, many interviewees acknowledged that in the development of the IOM's report, there was insufficient conceptualisation of how healthcare organisations would learn, act and remedy identified errors. In particular, as one interviewee observed:
I think there was a recognition that just reporting served a little bit of value but wasn't all that valuable; there needed to be an infrastructure developed that could hear the reports, that could analyze them, that could look for patterns and that could create action plans that ultimately would solve problems. But I don't think we gave a whole lot of thought as to what all those stages looked like, what the structure was going to look like, how to manage the volume.
Inadequate funding and institutional support
Interviewees described how a lack of funding, particularly by federal governments, constrained the potential usefulness of voluntary AERS. In particular, participants attributed a lack of federal US funding to the absence of a US national reporting system. As one interviewee observed, “… the [US] federal government really has not done very much in the country. It’s under-funded the one agency that does this—the Agency for Health Research and Quality—never has an adequate budget.” In the UK, although there was fiscal support for a national reporting system, funding was reported to be inadequate to deal with the hundreds of thousands of reports each year. An interviewee stated, “I think their resources were limited, and they could only deal with a few kinds of problems every year. As I said, it took quite a long time to get from something going in to getting an alert out of the other end.”
In addition to poor financial support, participants noted that too few adequately trained and skilled analysts to deal with the deluge of reports further contribute to the failure of incident reporting systems. Participants discussed how the reporting systems lacked leadership in general and medical leadership in particular. One participant commented on the lack of robust governance structures to manage the large volumes of reports. The lack of governance led to confusion as to how the large volumes of reports would be addressed, which could then lead to a reduction in harm to patients. This participant, as evidenced by the following quote, believed that the area of medication safety offered a good example of how to establish a good governance structure for addressing safety incidents:
[Our organisation] we've gotten thousands of product and practice changes that have saved lives, helped companies develop new products, help them market products that are designed for safety, help the economy, help the FDA in their mission, help professionals [pharmacists].
Such governance structures had strong clinical leadership and provided clear lines of responsibility as to who was going to analyse the event reporting data, respond to the issues identified and evaluate the changes.
Failure to capture evolving health information technology developments
There was a general consensus among respondents that the future of adverse event reporting had to take full advantage of the rapidly developing electronic health records and related technologies. Many of the participants commented that the digitisation of submitting an adverse event report had significantly eased the process of report submission. Others also commented that the dissemination of relevant adverse event information had been facilitated by the era of electronic technology. Believing that an incident reporting system needed to be computerised, one interviewee stated, “You have to be able to disseminate data and one click gets it to all of the people that need to know and allow it then to be disseminated in a digital way.”
Several interviewees advocated for the automation of voluntary reporting using sophisticated data interpretation technologies of the electronic healthcare record, such as natural language processing, thereby automating adverse event reporting. This would allow healthcare organisations to be less reliant on voluntary reporting for useful data. As an example, one respondent said
You get natural language processing running in the background that says you just mentioned a delay, is that something you want to report about, and pop goes the thing you know, to try to prompt people at the right time to report, based on certain key words or something.
A few participants raised concern that there had been inadequate engagement of those overseeing adverse event reporting systems and those installing electronic health records to ensure strong linkage between the two systems. As one participant remarked, “We aren’t spending enough energy to link the EHR to … trigger systems, auditing tools.”
This report describes the accumulated wisdom of 11 leading international patient safety experts about the role, impact, challenges and the future prospects of incident reports supporting safer patient care. These leaders still viewed incident reports as an important tool for surfacing instances of poor healthcare delivery and improving patient safety. However, their reflections make it abundantly clear that 15 years after the original IOM recommendations, we are still missing elements vital to the success of incident reporting.18
By understanding the five challenges, it is possible to explore how the processes and systems of incident reporting can be optimised to increase the likelihood of safer patient care. These findings in concert with information from related literature and insights from some of the authors suggest that there are some practical improvement areas for healthcare organisations and policy-makers. While each of these areas is important to address, none is likely to be sufficient alone to improve patient safety. Rather, this may be a case where the whole is greater than the sum of its parts. Indeed, as seen in the aviation industry, a system is strongest and most effective when all the pieces fit together. In isolation, these reporting systems cannot be expected to resolve unsafe care without effective action in the same way that an MRI report cannot fix cancer without appropriate treatment.19 ,20
Potential areas of improvement
Overall governance of reporting
Our finding regarding the importance of governance for reporting systems has not been obvious in the published literature. Over the years, confusion as to the roles and responsibilities in response to adverse events has left many reporting systems under-resourced and paralysed with overwhelming amounts of data and little visible action. Healthcare organisations need to bring clarity to the roles and responsibilities for incident analysis and ownership of the related actions to allow incident reporting systems to carry out their roles adequately.
Improve medical engagement
Although there is no shortage of reports being submitted, reports are rarely submitted by doctors, limiting the types of events reported and thereby learnings for the organisation. Efforts are needed to allow doctors to feel truly safe when reporting,18 provide a strong medical leadership13 ,21 that values medical reporting and literature to support results-based adverse event reporting.10 Another avenue to engage doctors is to locally demonstrate that reporting has directly led to tangible action, which has been identified as a key issue for doctors.15 ,22 ,23 An increasing number of healthcare organisations have sought out the use of electronic health records either by desire or legislation.24 Electronic health records might be linked directly to electronically submitted adverse event reports. In addition, the event could be tracked forward by the reporter, just as a parcel in the post can be tracked electronically. The ability to track what has happened to a report could provide reporters, including doctors, with immediate feedback and provide a sense that the report has led to action. With the inevitability of automation of voluntary incident reporting systems and the use of data interpretation technologies of the electronic healthcare record such as natural language processing, healthcare organisations will be less reliant on voluntary reporting for useful data.
Reduce incident report volume
Many of the problems associated with the large volume of reports cited in this study have been previously identified by others—wide variation in the quality of the reports from the large number of reporters, thereby minimising their usefulness,25 the timeliness of analysis7 and ability to prioritise the analyses.26 This study documents the ongoing need to address the overwhelming volume of reports generated in routine collection of incident reports. There are several ways in which this can be addressed:
At the hospital level, reporting can be proactively targeted towards specific types of adverse events, new systems or services. By using tools such as failure mode and effects analysis27 or hazard analysis and critical control point,28 which can be modified for healthcare,29 potential vulnerabilities in new systems and services can be determined by a multidisciplinary team. Once identified, hospitals and healthcare workers can be prompted to actively report against these vulnerabilities, which can be looped back to the multidisciplinary team to actively engage in resolution of the issues discovered.
Establish processes for low-frequency event reporting
There need to be opportunities to encourage the reporting of low-frequency, but serious or potentially serious, adverse events. This allows one-off events that have not been identified a priori to be thoroughly investigated through a root cause analysis. Root cause analyses use a structured method to analyse serious adverse events to identify underlying problems that increase the likelihood of errors. Examples can include equipment failure with life-threatening complications (intra-aortic balloon pump/delivery of walled oxygen).
Robust triage processing
The robust triage of events requires a common conceptual analytical framework for adverse events and a universally accepted data-collection tool30 with further research to allow healthcare organisations to manage their reports meaningfully. One proposed and potentially helpful way to address the volume of reports would be to categorise the reports as either having reliability issues,31 the failure of an embedded system or resilience issues, the failure of an organisation to adapt to a new or unexpected demand.32 Doing so could separate mundane events that are deviations from the normal reliable processes and those events that are reliant on the individual or team or organisation to respond to a new or unexpected demand, reflecting the organisation's resilience. Those events deemed high risk could then be singled out so that a meaningful root cause analysis by skilled staff can be undertaken to ensure appropriate learnings can be made.
Learning from reports
The interviewed experts emphasised the importance of adequate analysis, visible action and feedback loops to encourage doctors to report, echoing calls from the authors of previous publications.15 ,22 ,23 The next generation of reporting systems must be visibly linked to effective action that can be easily monitored and evaluated to provide the evidence that doctors need in order to engage. One strategy that hospital leaders can use to help ensure that incident reports are linked to action and improved safety is to code events into specific areas, such as medication errors or wrong-site surgery. By identifying a specific problem, an enduring interdisciplinary team can be created to describe the ideal system, be responsible to review adverse events specific to the problem, develop interventions and monitor improvements.
It is important to note the limitations of this study. A potential weakness could be that only a limited number of participants were specifically incident reporting experts due to the interviewee selection method, which allowed for broader inclusion of general patient safety experts, thereby perhaps limiting the usefulness of the results. There was no attempt to separate or specifically recruit participants representing many national, regional and hospital reporting systems either as an end user, researcher, developer or overseer. These perspectives could have provided a richer data source, specific to incident reporting, relevant to developers and end users. As a consequence, our results have to be taken in a broader context of incident reporting systems in healthcare. Also, researchers did not seek input from individuals who do not speak English, thus, missing perspectives that again could have enriched the data. In conducting only 11 interviews, important information may have been missed. This concern is tempered by the fact that at the conclusion of these 11 interviews, no new ideas were being generated. Another potential weakness may be due to the predominance of the lead researcher interpreting the data. However, the further coding review by a second researcher and three senior researchers confirming the relevance of the codes diminished the risk of bias.
In the years following the release of the IOM's report, adverse event reporting systems have proliferated in countries such as the USA, UK and Australia. Although significant resources have been invested, the investment has been both insufficient and misdirected, and there has been little data to link voluntary incident reporting to safer healthcare. If healthcare organisations are to learn from adverse events, they need to reflect on the successes of aviation reporting systems that have been so effective at improving aviation safety. In aviation, there is a culture of reporting even at the front line. Once the report is submitted, the front-line workers can choose to resolve the issue themselves, but in the knowledge that the report goes to a central, highly independent unit. This unit, an investigation unit, brings together an integrated picture of safety, and will intervene where appropriate. In predetermined situations, there is a statutory requirement for airlines to report to the Civil Aviation Authority, which can further independently investigate an event and make solutions mandatory for the airlines to be able to continue to fly. Importantly, aviation recognises that reporting is only one aspect of creating aviation safety.
If healthcare organisations are to learn from adverse events and allow patients to be safe, they must have an appropriate safety culture and a true accountability to deliver safe patient care. It is imperative to ensure that adverse event reporting is more than fulfilling regulatory requirements. With more thoughtful implementation, incident reporting still has the potential to truly improve the safety of our patients.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
- Data supplement 1 - Online supplement
Twitter Follow Imogen Mitchell at @IA_MITCHELL
Contributors All the conditions have been met for the authorship with regard to: substantial contributions to the conception or design of the work, or the acquisition, analysis or interpretation of data; drafting the work or revising it critically for important intellectual content; final approval of the version published; agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding Commonwealth Fund (Harkness Fellowship).
Disclaimer The Commonwealth Fund had no role in: the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The views presented here are those of the authors and should not be attributed to The Commonwealth Fund or its directors, officers or staff.
Correction notice This article has been corrected since it was published Online First. The word ‘Incidence’ has been added to the title and the surname of the second author of reference 32 has been corrected.
Competing interests None declared.
Ethics approval Institutional Review Board, Johns Hopkins Bloomberg School of Public Health.
Provenance and peer review Not commissioned; internally peer reviewed.
Data sharing statement Our data are currently not being shared, and there are no additional data other than what has been reported upon.