Article Text

Identifying patient safety problems associated with information technology in general practice: an analysis of incident reports
  1. Farah Magrabi1,
  2. Siaw Teng Liaw2,
  3. Diana Arachi3,
  4. William Runciman4,
  5. Enrico Coiera1,
  6. Michael R Kidd5
  1. 1Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
  2. 2Centre for Primary Health Care and Equity, University of New South Wales, Sydney, New South Wales, Australia
  3. 3School of Public Health and Community Medicine, UNSW Medicine, University of New South Wales, Sydney, New South Wales, Australia
  4. 4The School of Psychology, Social Work & Social Policy, University of South Australia, Adelaide, South Australia, Australia
  5. 5Faculty of Health Sciences, Flinders University, Adelaide, South Australia, Australia
  1. Correspondence to Dr Farah Magrabi, Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2109, Australia; farah.magrabi{at}mq.edu.au

Abstract

Objective To identify the categories of problems with information technology (IT), which affect patient safety in general practice.

Design General practitioners (GPs) reported incidents online or by telephone between May 2012 and November 2013. Incidents were reviewed against an existing classification for problems associated with IT and the clinical process impacted.

Participants and setting 87 GPs across Australia.

Main outcome measure Types of problems, consequences and clinical processes.

Results GPs reported 90 incidents involving IT which had an observable impact on the delivery of care, including actual patient harm as well as near miss events. Practice systems and medications were the most affected clinical processes. Problems with IT disrupted clinical workflow, wasted time and caused frustration. Issues with user interfaces, routine updates to software packages and drug databases, and the migration of records from one package to another generated clinical errors that were unique to IT; some could affect many patients at once. Human factors issues gave rise to some errors that have always existed with paper records but are more likely to occur and cause harm with IT. Such errors were linked to slips in concentration, multitasking, distractions and interruptions. Problems with patient identification and hybrid records generated errors that were in principle no different to paper records.

Conclusions Problems associated with IT include perennial risks with paper records, but additional disruptions in workflow and hazards for patients unique to IT, occasionally affecting multiple patients. Surveillance for such hazards may have general utility, but particularly in the context of migrating historical records to new systems and software updates to existing systems.

  • Information technology
  • Patient safety
  • General practice

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Information technology (IT) has many benefits in clinical medicine.1 The computerisation of general practice is near universal in many countries and is on the rise in others.2 ,3 At the same time, problems with IT systems can increase the likelihood of new, often unforeseen, errors that can affect the safety and quality of clinical care and may lead to patient harm.4 ,5 Risks to patients can arise from problems with the IT systems themselves, the way they are implemented and how they are used.4

While previous studies have looked at the use of computers for clinical purposes6 and evaluated their impact on the quality of care,7 less is known about the problems with IT in general practice that pose threats to patient safety. Prior work has mostly focused on exploring the design of software packages used by general practitioners (GPs) and on examining variation in system function. For example, one UK study tested four common prescribing packages in a laboratory setting, and the best system only detected 7 out of 18 clinical scenarios with potential for significant errors.8 In Australia, the National Prescribing Service (NPS) tested prescribing software for common drug–drug interactions with the potential to harm patients.9 Only three of the six prescribing systems examined by the NPS alerted users to all 20 of the major drug–drug interactions tested. Other investigations have focused on the safety features available in software. A survey of 300 GPs in the UK found that respondents’ awareness of safety features and their deficiencies was poor.10 Another study interviewed a range of stakeholders to identify the ways to improve the design and use of software.11 Similarly, a recent US study involving 12 software packages identified a range of safety features and training requirements to prevent prescribing errors.12 However, the problems encountered by GPs in using computers and other IT in routine clinical work, and their effects on care delivery and patient safety, have not been examined to date.

Incident reports are a crucial source of early information about low-frequency safety problems.13 In general practice, they have proved invaluable in identifying emerging risks and harm to patients across many safety domains.14–18 In previous studies we examined incidents associated with IT from a state-based incident reporting system in Australia as well as those reported to the US Food and Drug Administration.19 ,20 These studies have improved our understanding of the types and consequences of safety problems associated with IT in hospitals. This current TechWatch Study focuses on the use of IT in routine general practice by soliciting incidents involving problems with computer systems and associated peripheral devices (eg, printers), or with their use.21 The types of problems encountered were identified and their consequences were examined.

Methods

Setting

Ninety-six per cent of Australian GPs use electronic health records and 94% prescribe electronically; 65% operate a paperless practice.22 At present, clinical software is not built to any common safety standard and there is no systematic operational oversight of software used in care provision. The National E-Health Transition Authority (NEHTA) develops health IT standards and specifications to improve safety. Its Conformance, Compliance and Declaration process defines the tests vendors should undertake for conformance testing. However, it is beyond NEHTA's remit to test the safety or compliance of clinical software or its operation. In Australia, medicines are subsidised by a government programme (the Pharmaceutical Benefits Scheme, PBS). Authority required medicines are medicines that can only be prescribed after approval from the Department of Human Services or the Department of Veterans’ Affairs.

Sampling strategy

From previous studies, we were confident that a sample of 200 GPs would provide sufficient data to examine incidents involving IT and their consequences.15 ,20 We estimated that a mail-out to 2000 GPs would be sufficient, allowing for a response rate of 10% (participation rates in online incident reporting had been 26%,15 and 40% for a mail survey6). The sample size was doubled, and 4000 GPs were invited, to allow for GPs who had died, ceased practice, reduced their practice hours or were unknown at the listed address.

Recruitment of participants

Participants were approached via a mail-out to 4000 GPs on behalf of the investigators from the Australian Government's Department of Human Services, and via a call for volunteers that went out in journals and newsletters.

Procedure

GPs who responded to the invitation were asked to register for the study and report incidents involving IT in their practice. An incident was defined as an event or circumstance that could have resulted, or did result, in unnecessary harm to a patient;21 and could be reported online via the TechWatch website or to a trained operator by telephone. Participants were issued with a username and password to access the website and complete an online tutorial about how to report. An online help manual was available.

An online prestudy survey gathered demographic information about sex, age, practice location and brand of clinical software package.

Incidents could be reported by answering six questions about the task being performed, response of the software and outcome of the clinical situation (ie, consequences; see online supplementary appendix table A).

An online poststudy survey was used to assess computer and information security procedures in line with the Royal Australian College of General Practitioners (RACGP) computer and information security guidelines by measuring responses to eight statements using a three-point scale (yes, no and don’t know; see online supplementary appendix table B). Participants were also asked to estimate the number of hours they spent on solving computer problems per week that affected the delivery of care to patients in their practice. All participants were offered an honorarium of $A200 on their completion of the poststudy survey.

Classification of problems involving IT

Incidents were categorised using an existing classification for safety problems associated with health IT.23 Problems were first divided into those primarily involving human factors or technical problems, and then assigned to one or more subclasses. Human factors problems related to interaction of humans with IT.24 We examined errors in the use of software (use errors) as well as sociotechnical contextual variables that contributed to incidents (eg, training, cognitive load and clinical workflow). For technical problems, hardware and software issues were examined and characterised (see online supplementary table C).

Clinical errors

We next sought to examine clinical errors arising from the problems based on their underlying mechanisms; a clinical error is an error (flawed plan or flawed execution of a plan) with actual or potential consequences for a patient.21 Actual and potential clinical errors were categorised into:

  1. Errors that were unique to IT: a clinical error caused by an IT problem, for example, a prescribing error due to a drop-down menu, system downtimes.

  2. Errors that existed with paper records but were made more likely with IT, for example, ordering medications for the wrong patient when interrupted.

  3. Errors that had always occurred but were more likely to cause harm with IT, for example, a GP relies on the medication list in a discharge summary which does not match a specialist's notes but is more easily accessible in an electronic system.

  4. Errors that were no different with use of IT or paper records, for example, a failure to use the latest protocol or guideline.

Consequences

Consequences were examined using a standard approach and are as follows:19 ,20 ,23

  1. Potential or actual harm to a patient: An incident that reached the patient,21 for example, a patient had a severe allergic reaction to prescribed medication even though allergy was entered in the patient's electronic medical record.

  2. An arrested or interrupted sequence or a near miss: An incident that was detected before reaching the patient,21 for example, a prescription in a wrong name noticed and corrected while printing.

  3. An incident with a noticeable consequence but no patient harm: Problems that affected the delivery of care but no harm to a patient, for example, time wasted waiting for a printer to function correctly.

  4. An incident with no noticeable consequence: Problems that did not directly affect the delivery of care, for example, an electronic backup copy of patient records was corrupted, but this was detected and the copy was not needed.

  5. A hazardous event or circumstance: Problems that could potentially lead to an adverse event or a near miss, for example, a prescribing package fails to display a patient's allergy status.

  6. A complaint: An expression of user dissatisfaction, for example, a user found that training to use new software was inadequate.

Clinical processes impacted

Clinical processes were examined using categories from the Threats to Australian Patient Safety study (practice systems, investigations, medications, non-medication treatments and communication).15

Analysis

Two investigators independently examined the free-text descriptions to classify IT problems, examine clinical errors, assess consequences and identify the clinical process in which the incident originated. More than one problem category could be chosen if necessary. Clinicians (STL and EC) were consulted when required. Inter-rater reliability analysis was performed to determine consistency between coders.25 If an incident was assigned to more than one problem category, the primary category (the one most directly related to any actual or potential consequences) was used for kappa score calculations. Inter-rater reliability for the primary classification was κ=0.80 (p<0.001, 95% CI 0.71 to 0.89).25 Inter-rater reliability for classification of the direct consequence was κ=0.79 (p<0.001, 95% CI 0.69 to 0.90). Descriptive analyses were undertaken of incidents by problem type, consequences and clinical process.

Ethics and other approvals and statutory immunity

Ethics approval for the study was obtained from University and the RACGP human research ethics committees. Approvals were also obtained from the Statistical Clearing House of the Australian Bureau of Statistics and the External Requests Evaluation Committee of the Australian Government Department of Human Services. The identities of all participants were protected by gaining statutory immunity for TechWatch as a quality assurance activity from the Federal Minister for Health under Part VC of the Health Insurance Act 1973 (Commonwealth of Australia).

Results

Demographics of participants and incident reporting

There were 87 GPs in the final sample, including 42 GPs from the 4000 invited via the mail-out and further 45 GPs recruited via the call for volunteers. The majority of participants were aged 45 years or older (n=70, 80%; table 1).

Table 1

Demographics of TechWatch participants compared with the general population of Australian GPs in 2011–2012

Over the 19-month study period (May 2012 to November 2013), 94 incidents were submitted with the majority being submitted via the study website (n=63, 67%). The number reported by each participant ranged from 0 to 6 (mean=1.08, SD=1.42; median=1); 40 GPs (46%) did not report any incidents.

Clinical processes impacted by IT incidents, consequences and patient harm

After removal of one report because the incident had been reported twice by the same GP and three others because they did not relate to IT problems, 90 incidents remained. An examination of clinical processes in which the incidents originated showed that practice systems (n=43, 48%) and medications (n=26, 29%) were most frequently reported. Eleven incidents (12%) were associated with investigations, nine (10%) with communication and one (1%) with a non-medication treatment.

Forty-two per cent of the incidents (n=39) had an observable impact on the delivery of care but were not associated with patient harm. A few (6%, n=6) reported potential for patient harm (box 1; see online supplementary appendix table C). Four of these were medication-related; these involved use errors and an incorrect dose when clinical records were moved from an old software package to a new system (record migration). One involved a patient who was injected with double the dose of a medication when there was a delay in scanning a new desensitisation schedule into the electronic records. Another involved a test assigned to the wrong patient due to an error in patient identification, and another was linked to a delayed diagnosis because a test result filed in the electronic record was missed. No discernable harm was reported in four of the six incidents where there was potential for patient harm.

Box 1

All reported incidents where there was potential for patient harm and examples of near misses

Errors that were unique to information technology (IT)

Migration of historical records: A patient's dosage information was not transferred correctly from one software package to an updated package, and they were prescribed (and took) twice the intended dose of meloxicam. There was no adverse outcome as the patient rang their doctor the day after first taking the medication and the dose was corrected.

Use error, drop-down menu: A general practitioner (GP) mistakenly prescribed and gave the wrong hepatitis vaccination to a child. ‘Gave HAB [Hepatitis A+B] instead of HBV [Hepatitis B]. HAB rarely given to children and prescribing error’. The software drop-down menu was long and contained a mix of generic and brand names. The error was detected by the GP and discussed with the mother of the child.

Use error: A GP intended to prescribe 4 mg trandolapril for an elderly male patient, but mistakenly prescribed Amaryl 4 mg (glimepiride). The GP reported that the, ‘software picked up similar generic appearing drug names’. On taking the medication, the patient went into a hypoglycaemic coma and had seizures. He was resuscitated in an ICU and admitted to hospital for a week.

Errors that were no different with the use of IT or paper records

Hybrid records system: A patient was injected with double the dose of a medication. This resulted from the use of an out-of-date dosing schedule because of a delay in scanning a new desensitisation schedule into the electronic records. The patient was at risk of an allergic reaction.

Use error, patient identification: A chlamydia test and pap smear were assigned to the wrong patient, that is, specimens were mislabelled with details of another patient with the same first and last names, because practice staff did not check the date of birth and address when the patient arrived at the clinic. The date of birth and address were also not checked by the GP placing an order for the tests. Practice staff detected the error when the chlamydia test returned a positive result and they contacted the wrong patient.

Use error, missed test result: A patient with severe abdominal pain suffered a delayed diagnosis because an abnormal scan that was filed in her electronic record without follow-up was not detected. The patient visited several doctors who were unable to diagnose her pain. When admitted to hospital, the patient's emergency scan revealed a large kidney mass and the patient needed a nephrectomy. The GP reported that the ‘abnormal scan was filed 5–6 months before her hospital surgery’. No further details were available.

Examples, near miss incidents (n=24)

Error that was unique to IT

Use error, drop-down menu: A GP mistakenly selected the wrong strength of an antibiotic from the options on a drop-down menu. The GP noticed the error and corrected it.

Error that existed with paper records but were made more likely with IT

Use error, wrong patient: A patient was given the wrong prescription (that was meant for another of the GP's patients) because multiple files were open on a GP's desktop. The GP detected the error in time and the prescription was corrected before the patient left the practice.

Error that had always occurred but was more likely to cause harm with IT

Retrieval of records: A GP changed a patient's medication based on their pathology results, which were easier to access than a discharge summary. A drug hypersensitivity was noted on a discharge summary but not on the pathology results. The GP was a locum who was unfamiliar with the patient and had limited time to review a complex discharge summary during a consultation as they were overloaded with work. The error was detected and corrected by the GP before the patient took the medication.

Twenty-four near miss events were reported (27%); these often involved prescribing errors involving the wrong patient, drug, formulation, route or strength. Most of these errors were detected and corrected by the GPs themselves, often during the same visit. While most other prescribing errors were subsequently detected by pharmacists, errors involving the wrong patient and formulation were also detected by practice staff, patients and carers. Other near misses related to test results and specialist letters that were mistakenly assigned to the wrong patient or doctor. These errors were detected by the GPs themselves or by a colleague. Fifteen incidents were associated with potentially hazardous circumstances. Six did not have a noticeable consequence (6%), and there were no complaints.

Effects on care delivery and strategies for dealing with IT issues

IT incidents affecting the delivery of care disrupted clinical workflow, wasted time and caused frustration. Waste of time was identified as a major consequence in 30 out of 46 incidents that affected care delivery. The time spent by GPs in dealing with IT issues was considerable. In the poststudy survey, which was completed by 49 participants (56%), GPs estimated spending about 2 h per week on solving IT problems. Yet, conformity with the RACGP computer and information security guidelines was high, with 86% of respondents indicating implementation of all procedures in their practice (see online supplementary appendix table B).

GPs solved IT problems by trouble shooting on their own, seeking assistance from their practice manager and colleagues, contacting IT support and by using help desks provided by vendors. Problems with software updates were generally referred to vendors, either directly by GPs or via practice managers. IT support was consulted about issues with network connections, email and problems with downloading test results.

Hybrid record systems, where some patient information was kept electronically and some on paper records, were reported to be used as a workaround for problems with IT. For example, prescriptions were handwritten when the network was down, software stopped responding or when a printer failed. Yet, hybrid records were also used as a pre-emptive measure to prevent IT issues. In some practices, prescriptions for newly approved drugs were handwritten because updates to the drug database could not be applied until the software package was updated. In these practices, the philosophy was to wait for the updated software to be available (whereafter bugs were fixed) so that software was tested and fully functional before use.

Problems that gave rise to errors that were unique to IT

Among the incidents, 170 contributing factors were identified (see online supplementary appendix table C). Some problems with software design and build meant that software did not adequately support clinical tasks (box 2, eg, clinical documents and letters were not displayed in chronological order) or behaved in an unexpected manner (eg, the previous patient's notes were displayed). Deficient user interfaces gave rise to use errors, for example, a drop-down menu with an excessive number of options that were placed too close to each other.

Box 2

Examples of problems that gave rise to errors unique to information technology (IT)

System availability: All the general practitioners (GPs) in a practice could not access patient records due to a server problem.

Display of records: There were errors in the display of electronic records. A previous patient's notes were displayed alongside the current patient's details.

Electronic retrieval of records: Problems locating information were a result of the software package being unable to display clinical documents and letters in chronological order. In another package, discharge letters could not be located because dates were displayed incorrectly. A third incident related to practice management software that did not support correct identification of patients who had the same first and last names (eg, John Smith born 1989 and John Smith born 2001). In another, test results could not be located during a consultation.

Prescription autocomplete: A doctor intending to prescribe clomipramine 50 mg (antidepressant) for a male mistakenly prescribed clomiphine 50 mg (medication that induces ovulation) due to an autocomplete feature in the software.

Prescriptions incorrectly archived: A previously prescribed medication (irbesartan for microalbuminuria) did not appear on the current medication list as a software package automatically moved prescribed medications to ‘old scripts’ unless ‘long-term’ was selected at the time of prescription. The patient had stopped taking the medication and the GP was unaware that the prescription needed to be continued. There was potential for an error if the GP had not reviewed the previous medication list.

Reason for visit automatically summarised: A GP excised a skin cancer and then prescribed an antacid requested by the patient. The software automatically summarised the reason for the visit based on the prescription (hyperacidity). This could have potentially led to a failure to review the results of the pathology.

Use error, drop-down menu: A GP mistakenly selected chloramphenicol eye drops for a child instead of ear drops when using a drop-down menu which had multiple items that were placed too close to each other. The child's mother noticed the error when she went to give the drops and contacted the GP.

Other errors unique to IT were due to software-related issues involving system transitions including updates to software packages and the drug database. Participants reported a range of problems with applying and configuring software updates (box 3). The frequency with which software needed to be updated was also highlighted as an issue, ‘every 3 months <name of software package> has an upgrade and almost every month PBS has a drug update’. Participants reported that updates often resulted in difficulties in retrieving patient records, investigations and correspondence.

Box 3

Routine updates to software were reported to be highly problematic

Applying software updates

Software could not be updated and a general practitioners (GPs) time was wasted attempting to trouble shoot because the vendor's instructions did not state that antivirus software needed to be switched off.

Prescriptions could not be generated electronically and needed to be handwritten because software was unavailable due to a problem with applying an update.

Data were lost and time was wasted when a computer crashed during a consultation. This was linked to a major system upgrade including new hardware, software and an update of the database in which patient records were stored.

Configuring updates

Software could not be configured to user preferences after an update had been applied. A busy GP was unable to familiarise herself with the most relevant changes because the documentation was too long and not user friendly. ‘Reminder to check kidney test in two weeks appears at bottom of notes (did not realise for weeks that I could sort chronologically to have it appear on top/ordered by date/doctor/and topic)’.

Retrieving records and correspondence after software updates

GPs were unable to access 10 months of patient-related correspondence following a software update. ‘Time was wasted searching for records and reconstructing past history, possibly taking unnecessary or inappropriate actions, and neglecting patient concerns…’. Many practices were affected by this problem.

Following an update of the encryption software, a message's sender name was displayed as a code in all correspondence. There was no subject heading and neither was it possible to search by the name of the specialist. A GP reported that time was wasted because they opened every coded letter to locate the correct one.

Retrieving test results after an update

Pathology results could not be downloaded and the consultation was delayed. There was potential for failure to follow-up if the patient had left the practice before the results were available.

Investigation results were wrongly marked as ‘normal result/no action required’ and filed under the wrong patient following an update. ‘I do not know which patient to chase to retrieve their abnormal results which have mistakenly been marked “normal result/no action required”’.

Pathology results could not be accessed after they had been initially reviewed by a GP. Results could not be located in the patient file as it was not clear who the patient was. The reporter worked around the problem by downloading all the results again. ‘No known adverse result to date. I have generally been able to track down the results with a lot of effort and duplication of work (HOURS) e.g. re-downloading all results again and re-checking’.

Another type of system transition that was reported to be problematic was record migration. In one case, the process reportedly took over 18 months due to problems in migrating from a legacy software package. Drug reference and PBS authority codes were out of date. A workaround was to manually enter PBS authority codes, which was time consuming. The migration of records also introduced errors. For example, a patient was issued a prescription and took twice the prescribed dose of meloxicam because the dosage of the medication was incorrect following the migration of records to a new package.

Problems that generated errors which existed with paper records but were made more likely with IT

GPs’ use of software was influenced by human factors issues (box 4; see online supplementary appendix table C). Use errors were generated when cognitive resources devoted to using software were inadequate such as when multitasking or due to a slip in concentration. Influences in the work environment such as distractions and interruptions similarly contributed to errors which existed with paper records but were made more likely with IT.

Box 4

Examples of problems that gave rise to errors which existed with paper but were made potentially more likely with information technology (IT)

Slip of concentration: Avanza (mirtazapine) was prescribed instead of Avandia (rosiglitazone) due to a slip in concentration. A pharmacist detected the error because the patient did not suffer from diabetes and contacted the doctor to issue a new prescription.

Multitasking, multiple patient files open: A general practitioner (GP) mistakenly prescribed a medication for the wrong patient when two patient files were opened up simultaneously on the computer screen. The GP noticed the error and corrected it.

Interruption: A doctor wrote a prescription for the wrong patient when interrupted by a phone call during a consultation. At the end of the call, the doctor returned to the wrong patient record. The error was detected by a pharmacist and returned to the doctor.

Problems that generated errors which were no different with use of IT or paper records

Although problems with patient identification posed challenges, these errors were no different than those with paper records and were often propagated electronically. For example, laboratory tests were ordered for the wrong patient, that is, another patient with the same first and last names, because practice staff and the GP did not check the date of birth and address when booking the appointment or when the patient arrived at the clinic. The date of birth and address were also not checked by the GP when ordering the tests. Practice staff detected the errors when one of the tests returned a positive result and they contacted the patient. Another contributor to such errors was hybrid record systems, which were similar to data being scattered across many different paper records. For example, test results and letters received on paper needed to be scanned in and subsequently filed into the electronic records system by practice staff. Delays and errors in processing these documents had an impact on patient care (boxes 1 and 5).

Box 5

Examples of problems that gave rise to errors that were no different with information technology (IT) or paper records

Printed health summaries: ‘Authority prescriptions’* were not included as part of the current medications list in the health summary generated by a software package. They needed to be entered manually and the reporter indicated that this step was easily forgotten. There was potential for medication errors because the patient was on a higher dose of pain medication and other medications.

Patient identification: The wrong patient was booked for a follow-up appointment when there were two patients with similar names in the practice system. In another case, electronic notes were updated and a prescription was written for the wrong patient because another patient with a similar name who attended the practice was added to the computer waiting list by reception staff. The patient noticed the error and reported it to the doctor who corrected the notes and issued a new prescription.

Hybrid record systems: A letter from a specialist recommending an urgent referral of a patient to another specialist was scanned and allocated to a general practitioner (GP) who had left the practice. The error was detected by a further GP who was following up on their former colleague’s correspondence. In another incident, scanned reports from investigations belonging to multiple patients went missing when uploaded into the software, with the risk that important information might have been missed. The practice decided to store paper copies for an extra week so that all scanned reports could be checked before paper copies were destroyed.

*Medicines that can only be prescribed after approval from the Department of Human Services or the Department of Veterans’ Affairs.

Discussion

While IT has many advantages, this study shows that problems with it can and do give rise to hazardous circumstances for patients and may disrupt the delivery of care and lead to patient harm. While the broad categories of problems we identified are similar to those previously reported in hospitals,19 ,20 ,23 we found that routine updates to software can be particularly problematic in general practice. Issues with software updates generated clinical errors that are unique to IT and can affect many patients at once. System downtimes, and errors in protocols and order sets,23 are other examples which may manifest as large-scale adverse events where a single malfunction may harm or increase the risk of harm to numerous patients.26

Many of the reported errors have always existed with paper records. For example, prescribing errors involving the wrong patient, drug, formulation, route or strength have long been well recognised. Of these, some, such as selecting the wrong formulation, route or strength from a drop-down menu are unique to IT. Others such as ordering medications for the wrong patient when interrupted by a phone call may be more likely with IT (box 4). Also certain errors may be more likely to cause harm with IT rather than paper systems. For instance, problems with confusing and contradictory medication lists have always existed with paper records, but if electronic records provide easy access to a ‘structured’ discharge summary, a GP may rely on it rather than accessing and reading a cardiologist's letter, which may contain vital information but might not be as readily accessible. Other errors are just as likely and no different to those that occur with paper records, for example, failure to use the latest protocol or guideline (box 1). The relative likelihood of these different classes of errors, and their potential to cause harm, needs to be examined in further comparative studies.

Our study highlights the effects of common types of problems found with the design, build and use of clinical software packages. Many design issues are not new and have been discussed previously in the general practice literature.10–12 We found that deficient user interfaces and unexpected behaviour of software impeded clinical workflow and resulted in errors that could cause patient harm (boxes 2 and 4). Importantly, most design and build problems can be addressed by ensuring that systems are user friendly and fit clinical workflow, and by addressing quality issues in processes to build software.27 ,28 At present, clinical software in Australia is not built to any common safety standard and there is no systematic operational oversight of software that is used in care provision. Our results therefore reinforce previous calls for safety standards as part of sound governance of health IT at a national level.29–31

In other nations with different approaches to IT safety, some problems have already been addressed.32 England is reported to have a comprehensive programme explicitly addressing the safety of unregulated software.32 However, the safety of the most common types of IT such as health records and order entry without decision support is not being explicitly addressed in most other nations. Some of the types of problems we identified are similar to those previously detected in Australian hospitals and England's national programme for IT, as well as those reported to the US Food and Drug Administration.19 ,20 ,23 Although the lessons learned may be applicable elsewhere,13 ,14 there is no systematic way for them to be applied.

At the practice level, we found deficiencies in local procedures around the implementation and use of electronic records. While the challenges of implementation are well known and have been previously reported in hospital settings, our analysis shows the need for ongoing surveillance of software in general practice, particularly during system transitions, which proved to be particularly risky.4 ,33 These included routine updates to software packages and drug databases, migration of records to new systems and installation of new operating systems, all of which need to be managed at the practice level in partnership with system vendors. Maintaining data integrity when migrating historical records to new systems is technically challenging, and errors in patient data, if left undetected for long periods, can have catastrophic consequences and, potentially, involve large numbers of patients.5 ,23 Such problems are also likely to generate clinical errors and have consequences that are unique to electronic records.

The hazards associated with partial implementation need to be identified and managed for improved safety. For instance, hybrid record systems created opportunities for error. This is consistent with findings in hospitals.20 At present, 27% of Australian general practices have a hybrid record system.3 Although 70% operate paperless systems, test results (eg, pathology, imaging) and letters are usually received on paper and need to be scanned and filed into electronic record systems by practice staff; this is also akin to operating a hybrid system. Many such problems with managing test results and letters may not be solely an IT issue but involve practice systems to make sure that the right test results and letters are filed for the right patient and go to the right GP. Well-managed IT can enhance access to such documents and provide audit trails.

Yet, another example of a procedure that needs to be managed locally is patient identification. Identification problems have existed with paper records but may be exacerbated when software does not support identification of patients using approved identifiers recommended by the RACGP standards.34 One way of practicing to address such system safety issues is by undertaking a formal safety assessment of their procedures for managing patient information and addressing the hazards identified. Such formal safety assessments are already established in other high-risk industries such as aviation and are increasingly used in healthcare in some countries.32

At an individual GP level, system use errors were an important contributor (see online supplementary appendix table C). Use errors can be reduced by making the user interface of software more intuitive and by designing out error-prone features. Another approach is to incorporate specific requirements for safe use of IT into existing accreditation of general practices. Practices may be asked to demonstrate that the users of systems are proficient in the safe use of the appropriate software and have regular checks and renewal of their IT credentials, particularly when there are major changes to software and workflow. Such training, which must be tailored to the needs of different practice roles and experience levels, might also incorporate knowledge about the risks of multitasking and handling of interruptions which are inherent to clinical work.35 For instance, general practices can be organised to triage calls and ‘warehouse’ other activities to minimise interruptions in the consulting room. Interruptions should be avoided or delayed when undertaking safety critical tasks, such as prescribing medications.36

A major consequence of some incidents was disruption to care delivery. On average, 2 h were spent every week solving IT problems by the GPs who participated in this study. If this was replicated nationally, then, with 22 600 practicing GPs nationwide, over two million person-hours would be lost every year in Australian general practice due to incidents involving IT.3 Quantification of the effects of IT problems on the likelihood and nature of errors and on delays in clinical processes, and their impact on patient outcomes, should be the subject of future research.

While our study has gathered some instructive information, it has limitations. The incidents we studied are self-reported by GPs, with all the inherent limitations of self reporting.37 The low response rate led to a smaller sample than planned and there is a risk that participants were a self-selected cohort who may have been predisposed to reporting incidents, although their demographic characteristics were broadly similar to the overall population of Australian GPs (table 1). Second, as most GPs are not expert in technology, many incidents may not have been detected, and those that were detected may not have been reported.38 For example, in one study clinicians often incorrectly assumed that an IT system was functional during a downtime.39 Furthermore, some clinical errors in the care of complex patients may be difficult to detect, for example, prescribing errors in elderly patients with multiple comorbidities, and who are taking many other medications. It is therefore inevitable that problems with IT were under-reported. Studies using the global trigger tool have shown that only 2%–8% of events which harm patients are the subject of an incident report.40 Furthermore, reports provide a ‘snapshot’ of only some events and this profile itself is subject to bias from a number of sources including a bias towards reporting incidents that appear interesting or unusual. As reports do not represent a systematic sample they cannot be used to examine the frequency of safety problems. Nevertheless, new aspects of known problems may come to light, and new, unforeseen problems may be identified for the first time,41 potentially allowing the timely application of remedial strategies at systemic as well as local levels.

The incidents were reported over a significant period, providing data about the nature of safety problems with IT in general practice. Importantly, most of the problems detected are amenable to mitigation and amelioration with careful system design, build, implementation and use. More broadly, this study demonstrates the feasibility of monitoring incidents on an ongoing basis which would allow researchers to track the evolving causes of IT-related harm in general practice, and to guide improvements in software design and its use. Progress can be demonstrated, even for low-frequency events, when certain problems have been successfully addressed, by qualitative changes in the nature and consequences of incidents.42 A representative sample of GPs is not required to characterise problems or to improve the safety and usability of clinical IT in a professional manner.

Acknowledgments

We thank: the general practitioners who generously gave their time to participate; Drs Janice Wiley, Ben Ticehurst, Health Whitson and Dianne Chambers from the University of Notre Dame Australia for sharing their expertise to pilot test the incident reporting questionnaire; and staff at the University of New South Wales (UNSW) Medicine Computing Support Unit for their assistance in developing and hosting the TechWatch website. The study was commenced when A/Prof. Magrabi and Prof. Coiera were at UNSW. They are now at Macquarie University.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors FM, MRK, STL and EC conceptualised the study; FM led the data analysis, drafted the paper and is responsible for the integrity of the work. She is the guarantor. DA helped with data analysis. All authors participated in writing and revising the paper. All aspects of the study (including design; collection, analysis and interpretation of data; writing of the report; and decision to publish) were led by the authors.

  • Funding This research is supported in part by grants from the Australian National Health and Medical Research Council (NHMRC): Project Grant 630583; and Centre for Research Excellence in e-Health 1032664.

  • Competing interests None declared.

  • Ethics approval Macquarie University, University of New South Wales, Flinders University and the Royal Australian College of General Practitioners human research ethics committees.

  • Provenance and peer review Not commissioned; externally peer reviewed.