Impact of providing patients access to electronic health records on quality and safety of care: a systematic review and meta-analysis
=====================================================================================================================================

* Ana Luisa Neves
* Lisa Freise
* Liliana Laranjo
* Alexander W Carter
* Ara Darzi
* Erik Mayer

## Abstract

**Objective** To evaluate the impact of sharing electronic health records (EHRs) with patients and map it across six domains of quality of care (ie, patient-centredness, effectiveness, efficiency, timeliness, equity and safety).

**Design** Systematic review and meta-analysis.

**Data sources** CINAHL, Cochrane, Embase, HMIC, Medline/PubMed and PsycINFO, from 1997 to 2017.

**Eligibility criteria** Randomised trials focusing on adult subjects, testing an intervention consisting of sharing EHRs with patients, and with an outcome in one of the six domains of quality of care.

**Data analysis** The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were followed. Title and abstract screening were performed by two pairs of investigators and assessed using the Cochrane Risk of Bias Tool. For each domain, a narrative synthesis of the results was performed, and significant differences in results between low risk and high/unclear risk of bias studies were tested (t-test, p<0.05). Continuous outcomes evaluated in four studies or more (glycated haemoglobin (HbA1c), systolic blood pressure (SBP) and diastolic blood pressure (DBP)) were pooled as weighted mean difference (WMD) using random effects meta-analysis. Sensitivity analyses were performed for low risk of bias studies, and long-term interventions only (lasting more than 12 months).

**Results** Twenty studies were included (17 387 participants). The domain most frequently assessed was effectiveness (n=14), and the least were timeliness and equity (n=0). Inconsistent results were found for patient-centredness outcomes (ie, satisfaction, activation, self-efficacy, empowerment or health literacy), with 54.5% of the studies (n=6) demonstrating a beneficial effect. Meta-analyses showed a beneficial effect in effectiveness by reducing absolute values of HbA1c (unit: %; WMD=−0.316; 95% CI −0.540 to −0.093, p=0.005, I2=0%), which remained significant in the sensitivity analyses for low risk of bias studies (WMD= −0.405; 95% CI −0.711 to −0.099), and long-term interventions only (WMD=−0.272; 95% CI −0.482 to −0.062). A significant reduction of absolute values of SBP (unit: mm Hg) was found but lost in sensitivity analysis for studies with low risk of bias (WMD= −1.375; 95% CI −2.791 to 0.041). No significant effect was found for DBP (unit: mm Hg; WMD=−0.918; 95% CI −2.078 to 0.242, p=0.121, I2=0%). Concerning efficiency, most studies (80%, n=4) found either a reduction of healthcare usage or no change. A beneficial effect was observed in a range of safety outcomes (ie, general adherence, medication safety), but not in medication adherence. The proportion of studies reporting a beneficial effect did not differ between low risk and high/unclear risk studies, for the domains evaluated.

**Discussion** Our analysis supports that sharing EHRs with patients is effective in reducing HbA1c levels, a major predictor of mortality in type 2 diabetes (mean decrease of −0.405, unit: %) and could improve patient safety. More studies are necessary to enhance meta-analytical power and assess the impact in other domains of care.

**Protocol registration** [http://www.crd.york.ac.uk/PROSPERO](http://www.crd.york.ac.uk/PROSPERO) (CRD42017070092).

*   patient safety
*   health policy
*   patient-centred care
*   information technology

## Introduction

Providing patients with access to electronic health records (EHRs) may improve quality of care by providing patients with their personal health information, and involving them as key stakeholders in the self-management of their health and disease.1 With the widespread use of these digital solutions, there is a growing need to evaluate their impact, in order to better understand their risks and benefits, and to inform health policies that are both patient-centred and evidence-based.

According to the Institute of Medicine (IOM), there are six domains of healthcare quality: patient-centredness, effectiveness, efficiency, safety, timeliness and equity.2 Patient-centred care is based on the provision of services that respect and respond to individual patients’ preferences and needs, and incorporates these aspects in clinical decisions and processes.2 3 Effective healthcare services result ultimately in measurable improvements in health outcomes,4 while ensuring the prevention of errors and adverse effects, *ie*, ensuring patient safety.2 Other dimensions of quality care delivery include minimising waste of resources (ie, efficiency), minimising delays in the provision of care (ie, timeliness) and avoiding differences in the provision of services to all groups of healthcare users (ie, equity).2 

Despite the claims on the theorised benefits of providing patients with access to EHRs, there is still a considerable lack of evidence of their demonstrated impact. Though evidence suggests that these interventions improve patient satisfaction and communication5 6 no clear benefits were found on effectiveness.5 Previous studies5 6 were also unable to find a beneficial effect on efficiency measures, such as number of face-to-face visits and telephone appointments.

Five landmark reviews provided a comprehensive characterisation of the literature published until 2013.5–9 One of them5 included studies evaluating the impact of both paper-based and electronic records, a heterogeneity that challenges the identification of individual benefits of the digital approach. The authors of previous systematic reviews highlight the paucity of published papers, and a tendency to include small and methodologically less robust studies,5 with a high risk of bias.9 In fact, only one systematic review specifically including randomised trials was published in 2012, having found only two studies investigating the impact on effectiveness.7 Recent discussions around patients’ rights and data ownership have acted as strong drivers to allocate resources to interventions capitalising on EHRs with patient access.10 Therefore, it is plausible that the more recent literature has provided new evidence to shed light on this subject.

This work builds on the previous landmark reviews, and aims to capture recent, highest quality evidence (ie, randomised trials) in order to clarify the impact of providing patients access to EHRs. The main objective of this systematic review was to assess the impact of these interventions on the six dimensions of quality of care.

## Methods

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines11 were followed in conducting this systematic review (online supplementary file 1). The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42017070092) and is available as an open access paper.12 Any differences between the protocol and review are described in online supplementary file 2.

### Supplementary data

[[bmjqs-2019-010581supp001.pdf]](pending:yes) 

### Supplementary data

[[bmjqs-2019-010581supp002.pdf]](pending:yes) 

### Search strategy

A systematic search of the literature published between 1997 and 2017 was performed on Current Index to Nursing and Allied Health Literature (CINAHL), Cochrane, Embase, Health Management and Policy Database (HMIC), Medline/PubMed and PsycINFO, using free terms and controlled vocabulary, whenever supported.12 The reference lists of relevant articles (including systematic reviews), and grey literature (including PROSPERO, reports of relevant stakeholder organisations (NHS Digital, AMIA, eHealth at WHO, International Society for Telemedicine and eHealth), and conference proceedings (last 5 years) of related conferences (American Medical Informatics Association, MedInfo, Medicine 2.0, Medicine X)) were also screened.

#### Study selection criteria

We included randomised trials only (see online supplementary file 2) that met the following criteria: (1) Focused on adults subjects (eg, patients, carers). (2) Included an intervention consisting of sharing EHRs with patients (either isolated or as part of a multicomponent intervention, that could include the identification of discrepancies in records, messaging systems, access to educational material, or other). (3) Had an outcome evaluating at least one of the six domains of quality of care. Studies were excluded if they (1) Included participants aged 16 years and under. (2) Had an intervention consisting of health reminders only. (3) Only reported cognitive outcomes (eg, intent) or other subjective measures only (eg, subjective perception of health and/or well-being). The detailed screening strategy is described in the study protocol.12 

#### Data extraction

One investigator extracted information from the included studies into a standardised computer-based spreadsheet, which was reviewed by a second investigator for consistency. The data collected for each study included: name of the first author, year of publication, number of participants, participants’ characteristics and setting, date of the intervention, study duration, study design, intervention characteristics, domain of healthcare quality assessed, main outcomes (specifying if primary or secondary), effect size (means (SD) or % for every group, whenever possible; or difference between groups, if the only information available), statistical significance, overall quality score.

#### Risk of bias assessment

Risk of bias was evaluated using the Cochrane Risk of Bias Tool.13 Two investigators reviewed all eligible studies in order to appraise their risk of bias (ALN, LF; ALN, LL). A third investigator resolved disagreements (LL, LF). A study was considered as ‘overall low risk’ if scoring low risk for at least 50% of the criteria evaluated; otherwise, the study was considered having an ‘overall high/unclear risk’.

#### Data synthesis and meta-analysis

A narrative synthesis of results was performed by domain of quality of care (IOM framework).2 For the meta-analysis, continuous outcomes representing the same variable and reported in at least four studies were pooled using random effects. This was the case for HbA1c (reported as the percentage of glycated haemoglobin over the total, %), and for systolic and diastolic blood pressure (SBP and DBP, respectively; both reported in mm Hg). All effect sizes are shown as absolute difference in means (DM) (weighted mean difference (WMD)) and classified as negative when in favour of the intervention, and positive when in favour of the control. Heterogeneity was assessed using I2 (<30%: low; 30%–60%: moderate; 60%–90%: substantial; >90%: considerable).13. The presence of publication bias was evaluated by a funnel plot. Comprehensive meta-analysis V.2.3. was used for statistical analysis.

#### Sensitivity analysis and subgroup analysis

For each domain of quality, we described the proportion of studies showing beneficial effects in both ‘low risk’ and ‘unclear/high risk of bias’ groups. Sensitivity analyses were conducted, excluding high/unclear risk of bias studies (for HbA1c and SBP), and short-term interventions (lasting less than 12 months) for HbA1c. Further information is provided in online supplementary file 2.

#### Patient and public involvement

Our research question emerged from the implementation evaluation of the Care Information Exchange ([https://www.careinformationexchange-nwl.nhs.uk/](https://www.careinformationexchange-nwl.nhs.uk/)), a portal/EHR with patient access available to 2.4 million people in North-West London. Lay partners will be involved in summarising the research findings into lay summaries and reports.

## Results

The database search retrieved 6594 citations (figure 1). Titles and abstracts were screened, and 1698 duplicates were excluded, as well as 4801 articles that did not meet the inclusion criteria. After the full-text screening of the remaining articles (n=95), 72 additional papers did not meet inclusion criteria and were therefore excluded. The kappa statistic measuring intercoder agreement in title and abstract screening was 0.40 (fair agreement). Screening of reference lists of systematic reviews revealed 13 additional studies that met our predefined criteria. A total of 36 papers was obtained, which included 20 randomised trials (17 randomised controlled trials (RCTs) and 3 cluster randomised trial (CRTs)).

![Figure 1](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/29/12/1019/F1.medium.gif)

[Figure 1](http://qualitysafety.bmj.com/content/29/12/1019/F1)

Figure 1 
Flow diagram of included studies. CRT, cluster randomised trial; RCT, randomised controlled trial.

### Description of included studies

The 20 included studies involved a total of 17 387 participants (table 1). Publication year ranged from 1999 to 2013 and study duration varied between 3 months and 32 months. Participants included had a range of health conditions, including type 2 diabetes (n=7),14–20 heart failure (n=2),21 22 arterial hypertension (n=2),23 24 cancer (n=1),25 type 1 diabetes (n=1),26 fertility issues (n=1)27 and pregnancy.28 Five studies included service users in general, without focusing on a specific health condition.29–33 

View this table:
[Table 1](http://qualitysafety.bmj.com/content/29/12/1019/T1)

Table 1 
Characteristics of the included studies

### Summary of risk of bias assessment

Overall, 50% of the studies included (n=10) were considered good quality, scoring low risk in at least half of the domains evaluated in the risk of bias assessment (figure 2).15 18 20 22 23 26–28 30 32. Four studies stood out with an overall low risk of bias for most of the domains evaluated.15 20 23 27 Due to the nature of the intervention, most studies scored a high risk of bias regarding blinding of participants and personnel; one study showed unclear risk.33 Blinding of the outcome assessment also showed a high risk of bias in several studies.14 17 21 22 24–26 28 29 31 32 Only three studies15 20 23 provided information on trial protocol registration.

![Figure 2](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/29/12/1019/F2.medium.gif)

[Figure 2](http://qualitysafety.bmj.com/content/29/12/1019/F2)

Figure 2 
Risk of bias assessment cells were colour-coded in orange for high risk of bias, in green for low risk of bias and in grey if risk of bias was unclear.

### Interventions and retention rates

Although all interventions provided participants with web-based access to EHRs, the content made available varied greatly (table 1). Content available to participants included access to previous medical history and risk factors,15 20 24 25 28 29 31 test results,14 16–19 21 26 31–33 medication lists,23 24 30 31 33 list of allergies,23 24 current health conditions,23 31 and clinical encounters and physician notes.26 31 One study specifically mentioned the existence of a functionality to download EHR data.31 In all studies, the patient access to EHRs was part of a complex intervention with other components. Intervention components included educational materials,14 18–20 22 24–26 28 30–32 generation of personalised action plans/messages,15 26 31 32 self-management training,17 and medication and appointment reminders.15 16 Twelve studies included secure messaging systems.14 17 20–24 27 29–31 33 Two studies provided incentives (either financial,29 or use of the portal after the study),22 and one explicitly mentioned that no incentives were provided.18 Retention rates were calculated as the proportion of randomised patients in each study that completed follow-up. Three studies did not provide enough information to adequately calculate retention rates (total and per arm).14 19 32 Among the other studies, only one28 had a retention rate below 60%.

### Comparisons

In most studies, the comparator was usual care (ie, no patient access to EHRs).14–22 24 26 27 29 31 32 In three studies, the comparisons were active controls.28 30 33 Two studies comprised three arms,23 25 which are described in further detail in table 1.

### Outcomes

Most papers assessed outcomes covering more than one domain (median=2). The domain most frequently assessed was effectiveness (n=14), and the least frequently evaluated were timeliness and equity (n=0). Patient-centredness, safety and efficiency were evaluated, respectively, in 11, 4 and 5 studies. A detailed overview of the outcomes evaluated is provided in online supplementary file 3.

### Supplementary data

[[bmjqs-2019-010581supp003.pdf]](pending:yes) 

#### Patient-centredness

Eleven studies evaluated the impact of sharing EHRs with patients on patient-centredness, including CRTs24 30 31 and eight RCTs.19–22 25–28 While six studies found a beneficial impact in at least one patient-centredness outcome,20 24–26 30 31 it is important to note that the exact measure of patient-centredness varied considerably across studies. Although patient satisfaction improved in two studies20 25 (46% vs 40%, p=0.04% and 27.7% vs 24.5%, p<0.0001, respectively), two other failed to show a significant effect.22 27 One study31 showed an increase in patient activation, as measured by the Patient Activation Measure34 (47 vs 45, p=0.0014), but these results were not replicated in a similar study.24 Self-efficacy scores improved in one study26 using the Diabetes Empowerment Scale35 (+0.14 vs −0.16, p=0.04), but no differences were found in two other studies22 27 using the Kansas City Cardiomyopathy Questionnaire (KCCQ) and the General Self-Efficacy Scale.36 Patient empowerment was accessed by the Patient Empowerment Scale37 in two studies21 24 but a significant improvement in mean scores was found only in one (41.2 vs 40.1, p=0.019).24 Three studies evaluated health literacy (ie, patients acknowledging to have learnt something new),19 25 28 but only one found the intervention to be beneficial (96% vs 74%, p=0.02). Six out of 11 studies (54.5%) scored an overall low risk of bias. The proportion of studies showing a significant positive effect for at least one of the outcomes evaluated was 50% in low risk of bias studies, and 80% in the remaining studies.

#### Effectiveness

A total of 14 studies appraised the impact of providing patients with access to EHRs on effectiveness, including 2 CRTs24 31 and 12 RCTs.14 15 17–20 22 23 25 26 32 33 Ten out of 14 studies (71.4%) demonstrated a positive impact on effectiveness-related outcomes.15 17–20 22 23 25 31 32 These studies evaluated the impact on a wide range of health conditions, including depression and anxiety,25 heart failure,22 cardiovascular risk (Framingham Score),20 obesity,15 23 smoking status,15 adherence to preventive services31 32 dyslipidaemia,17 18 20 24 33 diabetes14 15 17–20 26 33 and hypertension.15 17 18 20 23 24 33 In one study using the Hospital Anxiety and Depression Scale,38 patient access to EHRs did not change patients’ depression scores, and patients in the general computer information group were more anxious than the ones accessing personal records (DM=+18%, 95% CI 3.7 to 26.5, p=0.001).25 One study found a dramatic improvement in symptom stability scores, assessed by the KCCQ (DM:+17, 95% CI 9 to 29, p<0.001).22 Two studies found an improvement in LDL-cholesterol levels.17 20 No significant changes were observed on triglycerides,17 high-density lipoprotein (HDL)-cholesterol,17 total cholesterol,18 body weight,15 23 smoking status15 or total cardiovascular risk.20 Adherence to preventive services improved in the two studies evaluating this aspect31 32 (ie, use of low-dose aspirin (84.4% vs 67.6%, p<0.0001), complete immunisation (95.5% vs 87.2%, p=0.044), and uptake of cancer screening (increases ranging from 10.3% to +14.3%, all p<0.05)).While two studies specifically evaluated adherence to pneumococcal immunisation,31 32 only one found a beneficial effect.31 

Seven out of 14 studies scored an overall low risk of bias (50.0%). The proportion of studies showing a positive effect was 85.7% in the low risk of bias group, and 57.1% in the remaining studies.

##### Meta-analysis

Data from RCTs evaluating HbA1c and SBP were pooled together, and the respective meta-analyses performed. The six studies evaluating HbA1c17–20 26 33 comprised 950 participants, from which 894 completed follow-ups. Meta-analyses showed a beneficial effect in effectiveness by reducing HbA1c (unit, %; WMD= −0.316; 95% CI −0.540 to −0.093, p=0.005, I2=0%) (figure 3), which remained significant in sensitivity analyses for low risk of bias studies (WMD= −0.405; 95% CI −0.711 to −0.099) (online supplementary figure 1), and long-term interventions only (WMD= −0.272; 95% CI −0.482 to −0.062) (online supplementary figure 2). It is important to note that the study showing a high risk of bias,19 was also the one showing the smallest study sample. The funnel plot indicates asymmetry (online supplementary figure 3), suggesting potential publication bias.

### Supplementary data

[[bmjqs-2019-010581supp004.pdf]](pending:yes) 

### Supplementary data

[[bmjqs-2019-010581supp005.pdf]](pending:yes) 

### Supplementary data

[[bmjqs-2019-010581supp006.pdf]](pending:yes) 

![Figure 3](http://qualitysafety.bmj.com/https://qualitysafety.bmj.com/content/qhc/29/12/1019/F3.medium.gif)

[Figure 3](http://qualitysafety.bmj.com/content/29/12/1019/F3)

Figure 3 
Forest plots of effect sizes and 95% CIs representing the effect of interventions providing patients access to EHRs in HbA1c, SBP and DBP, using a random-effects model. The area of each square is proportional to the study's size, and therefore to its weight in the meta-analysis. For each study, CIs are represented by horizontal lines; a vertical line representing no effect is also plotted. The meta-analysed measure of effect is plotted as a diamond, the lateral points of which indicate CIs for this estimate. DBP, diastolic blood pressure; EHRs, electronic health records; HbA1c, glycated haemoglobin, SBP, systolic blood pressure.

The four studies evaluating the impact on blood pressure17 18 20 23 (comprising 1308 participants, of which 1021 completed follow-ups) were pooled in a meta-analysis, and showed a significant beneficial effect in SBP (unit: mm Hg; WMD=−1.416; 95% CI −2.814 to −0.018, p=0.047, I2=0%) (figure 3). However, significance was lost after removing the high/unclear risk of bias study17 (WMD=−1.375; 95% CI −2.791 to 0.041) (online supplementary figure 1). No significant effect was found in DBP in the meta-analysis (unit: mm Hg; WMD=−0.918; 95% CI −2.078 to 0.242, p=0.121, I2=0%) (figure 3), nor in the sensitivity analysis for low risk of bias studies only (WMD=−0.916; 95% CI −2.089 to 0.257) (online supplementary figure 1). The funnel plots appear symmetrical for SBP and DBP (online supplementary figures 4 and 5), indicating a similar proportion of studies in each direction of the effect size.

### Supplementary data

[[bmjqs-2019-010581supp007.pdf]](pending:yes) 

### Supplementary data

[[bmjqs-2019-010581supp008.pdf]](pending:yes) 

#### Safety

All studies19 22 29 30 showed a beneficial effect for at least one of the outcomes evaluated (online supplementary file 3). Two studies evaluated adherence, including general adherence to medical regimens22 (using the General Adherence Scale from the Medical Outcomes Study (MOS)39 and medication adherence.22 29 General adherence (MOS Scores) improved with the intervention22 (+2.3, 95% CI −3.7 to 8.3, p=0.01), but no significant changes were found in adherence to medication.22 29 A beneficial effect was observed in all studies evaluating medication safety,19 29 30 including a higher likelihood of reporting discrepancies (53% vs 24%, p<0.01),29 to change medications19 29 (88.3% vs 67.2%, p<0.01; and 84% vs 23%, p=0.002, respectively), and a resulting slightly lower proportion of patients with medication discrepancies (29% vs 30%, p=0.01).30 Two out of four studies scored an overall low risk of bias, and the proportion of studies showing a positive effect was the same in both risk groups (100.0%).

#### Efficiency

The impact of providing patients with access to EHRs was assessed in five studies.16 18 22 24 31 As less than four studies assessed the same construct, meta-analysis was not performed, and a descriptive analysis is provided. Number of hospitalisations per subject was lower in one study (0.17 vs 0.20, p=0.01),16 while total number remained unchanged in another (22 vs 21, p=1.00).22 Length of stay (in days) did not change in two studies (+0.2 vs –0.3, and 0.42 vs 0.34, respectively),18 24 but was shorter in another (0.99 vs 1.1, p<0.01).16 In the three studies evaluating the number of emergency visits, total numbers were either reduced,16 increased22 or remained unchanged.24 Number of primary care visits was lower in one study (2.9 vs 4.3, p<0.0001),31 but no changes were observed in another (0.0 vs –0.2).18 Two out of five studies scored an overall low risk of bias, and the proportion of studies showing a positive effect was 50.0% and 66.7% in low-risk and high/unclear-risk groups, respectively.

#### Timeliness and equity

While none of the studies assessed either timeliness or equity as primary outcome, three studies21 24 32 evaluated the predictors of usage of EHRs by patients. Earnest *et al* 21 did not find any associations between usage and race, symptom scores or number of visits; two studies found significant associations between usage and higher education,32 number of illnesses,32 younger age,24 clinic attended by the patient24 self-reported computer skills,24 and higher number of internet-use items.24 

## Discussion

### Key findings in context of published literature

This work systematically appraised the impact of EHRs with patient access across the six domains of quality of care as defined by the IOM:2 patient-centredness, effectiveness, efficiency, safety, timeliness and equity.

Regarding patient-centredness, results were inconsistent. More than half of the studies included in this domain showed a significant positive effect for at least one outcome, but no clear effect was found for specific outcomes, such as patient satisfaction, patient activation, self-efficacy, patient empowerment or health literacy. These results are line with previous studies5 6 8 that found mixed evidence about the impact in patient-centred outcomes. While providing patients access to EHRs is envisaged as a key strategy to deliver patient-centred care, the diversity of outcomes evaluated, and scales and tools used, hinders pooling of results and the use of meta-analytical approaches. It is critical, therefore, to identify and standardise measures and constructs to evaluate patient-centredness, to allow the application of meta-analytical methods in this domain.

A few studies included showed a positive impact in effectiveness in a range of outcomes (ie, anxiety, cardiac symptoms, LDL-cholesterol), but no significant improvements were found for triglycerides, HDL-cholesterol, total cholesterol, body weight, smoking status or total cardiovascular risk. Two additional studies not captured by our search also suggest that providing patients access to EHRs may improve glaucoma control40 and quality of life in patients with asthma.41 A positive effect was also found in adherence to several preventive services (ie, use of low-dose aspirin, cancer screening), an approach that can be particularly relevant in the context of cancer screening, where higher expected adherence rates have the potential to reduce cancer incidence and mortality.42 However, the number of studies published per outcome is limited, and further research is needed to increase meta-analytical power and explore the size and impact of the potential effect in specific health conditions.

Our meta-analysis showed a beneficial effect on HbA1c reduction, which remained significant after removing low/unclear-risk studies, or studies in which the intervention lasted less than 12 months. In 2013, Goldzweig *et al* identified several examples of improved outcomes for patients with chronic diseases (including hypertension and diabetes).8 In 2012, Ammenwerth *et al* 7 performed a systematic review of studies published between 1990 and 2011 and concluded that there was insufficient evidence to document a beneficial effect in effectiveness in patients with access to EHRs. However, by then only two studies (out of the four included in the review) investigated the effect on health outcomes. Our meta-analysis demonstrates a mean reduction in absolute values of HbA1c of 0.316% (95% CI −0.540% to −0.093%), with a low heterogeneity (I2=0.0) reflecting the specificity of our inclusion criteria. These results have important clinical implications, since an absolute reduction of 1 point on HbA1c levels (expressed in the same unit considered in our meta-analysis) is associated with a significant reduction of deaths related to diabetes (−21%), myocardial infarction (−14%) and microvascular complications (−37%).43 Visual inspection of the funnel plot suggests a potential publication bias, with studies with a lower precision (higher SE) reporting a greater beneficial effect. However, the meta-analysed effect remained significant after removing the study that stood out with a smallest sample size.19 

Although our meta-analysis found a beneficial effect in SBP, statistical significance was lost in sensitivity analysis for low risk of bias studies only; no significant effect was found in DBP. It must be noted, however, that the number of studies included is low, and further evidence is needed to establish robust conclusions.

For the efficiency domain, most studies included found either no change, or a reduction of healthcare usage (in primary care visits,31 or inpatient or emergency contacts).16 Ammenwerth *et al*,7 have also previously suggested a significant reduction in office visit rates. Further studies are required to clarify the impact on this dimension and pave the way to meta-analytical approaches that can provide further insights on the effect size in the various dimensions of healthcare usage.

Our work suggests that the intervention improves general adherence, but not medication adherence—however, a strong body of evidence showed a positive effect in medication safety. A previous study has suggested that patients find this approach valuable, and reported either unchanged or improved relationships with their clinician when using it.44 Further studies should further explore patients’ willingness and ability to report errors in their records, and also which specific groups are most likely to benefit. These results are in line with the findings of Mould *et al*, de Lusignan *et al* and Ammenwerth *et al*, who previously suggested that these digital solutions positively impacted patient safety.6 7 9 

Finally, we found no studies specifically focusing on the impact on timeliness or equity. Uptake of portals may differ by patient-specific factors, with lower use by racial and ethnic minorities, patients with lower education level or literacy, thus leading to digital-led health inequities.8 Davis Giardina *et al* 5 reported that, up to 2012, no studies had assessed any of these domains. Eight years later, these aspects remain unexplored.

### Strengths and limitations

Five landmark reviews have been published to date evaluating the impact of EHRs with patient access on different aspects of quality of care.5–9 Only one systematic review had focused on randomised trials, having found two studies investigating the impact on effectiveness.7 

Our systematic review included studies published between 1997 and 2017 and retrieved a total of 20 randomised trials. This study has several strengths: a predefined, openly available protocol was followed12 (with any changes described in online supplementary file 2); only randomised trials were included; focused exclusively on EHRs; and impact was assessed in all domains of quality of care, with meta-analysis performed whenever possible.

Only half of the studies included had an overall low risk of bias score. A possible approach to improve blinding in web-based interventions, or to test the impact of specific components, could be using A/B testing, a technique used for website optimisation that compares variation against a standard experience, and determines which variant is more effective.45 

## Conclusion

Our results suggest that providing patients with access to EHRs can improve patient safety and effectiveness. More methodologically robust studies are necessary to increase the strength of these conclusions, and to enhance meta-analytical power. For EHRs with patient access to be broadly used, it is important to focus on interventions that enhance adoption and measure usage, and issues of equity in both aspects need to be addressed by policy makers when implementing such programmes.46 

## Footnotes

*   Twitter @ana\_luisa\_neves

*   Contributors Conception and design of the work: ALN, AWC. Database searching: ALN, LF. Full text screening: ALN, LF. Outcome data extraction: ALN, LF. Risk of bias: ALN, LL, LF. Data analysis and interpretation: ALN, LL, LF, AWC, EM. Critical revision of drafts for important intellectual content. ALN, AWC, LF, LL, EM, AD. Final approval of the version to be published: ALN, AWC, LF, LL, EM, AD.

*   Funding This work is supported by the National Institute for Health Research (NIHR) Imperial Patient Safety Translation Research Centre. Infrastructure support was provided by the NIHR Imperial Biomedical Research Centre. The study funder(s) did not play a role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. In addition, researchers were independent from funders, and all authors had full access to all of the data included in this study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

*   Competing interests None declared.

*   Patient consent for publication Not required.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. Data supporting the findings of this study are available within the article and its supplementary materials.

[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).

## References

1.   Dickey LL . Promoting preventive care with patient-held minirecords: a review. Patient Educ Couns 1993;20:37–47.[doi:10.1016/0738-3991(93)90115-D](http://dx.doi.org/10.1016/0738-3991(93)90115-D) pmid:http://www.ncbi.nlm.nih.gov/pubmed/8474946 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/0738-3991(93)90115-D&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8474946&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1993KM11700005&link_type=ISI) 

2.  Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press (US), 2001.
    
    

3.   Scholl I , Zill JM , Härter M , et al . An integrative model of patient-centeredness - a systematic review and concept analysis. PLoS One 2014;9:e107828. [doi:10.1371/journal.pone.0107828](http://dx.doi.org/10.1371/journal.pone.0107828) pmid:http://www.ncbi.nlm.nih.gov/pubmed/25229640 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1371/journal.pone.0107828&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=25229640&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

4.   Arah OA , Klazinga NS , Delnoij DMJ , et al . Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. Int J Qual Health Care 2003;15:377–98.[doi:10.1093/intqhc/mzg049](http://dx.doi.org/10.1093/intqhc/mzg049) pmid:http://www.ncbi.nlm.nih.gov/pubmed/14527982 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1093/intqhc/mzg049&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=14527982&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000185882600003&link_type=ISI) 

5.   Davis Giardina T , Menon S , Parrish DE , et al . Patient access to medical records and healthcare outcomes: a systematic review. J Am Med Inform Assoc 2014;21:737–41.[doi:10.1136/amiajnl-2013-002239](http://dx.doi.org/10.1136/amiajnl-2013-002239) pmid:http://www.ncbi.nlm.nih.gov/pubmed/24154835 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2013-002239&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=24154835&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

6.   de Lusignan S , Mold F , Sheikh A , et al . Patients' online access to their electronic health records and linked online services: a systematic interpretative review. BMJ Open 2014;4:e006021. [doi:10.1136/bmjopen-2014-006021](http://dx.doi.org/10.1136/bmjopen-2014-006021) pmid:http://www.ncbi.nlm.nih.gov/pubmed/25200561 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNC85L2UwMDYwMjEiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.   Ammenwerth E , Schnell-Inderst P , Hoerbst A . The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res 2012;14:e162. [doi:10.2196/jmir.2238](http://dx.doi.org/10.2196/jmir.2238) pmid:http://www.ncbi.nlm.nih.gov/pubmed/23183044 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2196/jmir.2238&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=23183044&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

8.   Goldzweig CL , Orshansky G , Paige NM , et al . Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013;159:677–87.[doi:10.7326/0003-4819-159-10-201311190-00006](http://dx.doi.org/10.7326/0003-4819-159-10-201311190-00006) pmid:http://www.ncbi.nlm.nih.gov/pubmed/24247673 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-159-10-201311190-00006&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=24247673&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000328732700016&link_type=ISI) 

9.   Mold F , de Lusignan S , Sheikh A , et al . Patients' online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract 2015;65:e141–51.[doi:10.3399/bjgp15X683941](http://dx.doi.org/10.3399/bjgp15X683941) pmid:http://www.ncbi.nlm.nih.gov/pubmed/25733435 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjUvNjMyL2UxNDEiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10.  Blumenthal D , Squires D . Giving patients control of their EHR data. J Gen Intern Med 2015;30:42–3.[doi:10.1007/s11606-014-3071-y](http://dx.doi.org/10.1007/s11606-014-3071-y) pmid:http://www.ncbi.nlm.nih.gov/pubmed/25480725 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1007/s11606-014-3071-y&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

11.  Moher D , Shamseer L , Clarke M , et al . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1. [doi:10.1186/2046-4053-4-1](http://dx.doi.org/10.1186/2046-4053-4-1) pmid:http://www.ncbi.nlm.nih.gov/pubmed/25554246 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1186/2046-4053-4-1&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=25554246&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

12.  Neves AL , Carter AW , Freise L , et al . Impact of sharing electronic health records with patients on the quality and safety of care: a systematic review and narrative synthesis protocol. BMJ Open 2018;8:e020387. [doi:10.1136/bmjopen-2017-020387](http://dx.doi.org/10.1136/bmjopen-2017-020387) pmid:http://www.ncbi.nlm.nih.gov/pubmed/30104310 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC84L2UwMjAzODciO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

13.  Higgins JPT , Thomas J , Chandler J , et al . Cochrane Handbok for systematic reviews of interventions. 2nd edn. Chichester, UK, 2019.
    
    

14.  Fonda SJ , McMahon GT , Gomes HE , et al . Changes in diabetes distress related to participation in an Internet-based diabetes care management program and glycemic control. J Diabetes Sci Technol 2009;3:117–24.[doi:10.1177/193229680900300113](http://dx.doi.org/10.1177/193229680900300113) pmid:http://www.ncbi.nlm.nih.gov/pubmed/20046656 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1177/193229680900300113&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=20046656&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

15.  Holbrook A , Thabane L , Keshavjee K , et al . Individualized electronic decision support and reminders to improve diabetes care in the community: compete II randomized trial. CMAJ 2009;181:37–44.[doi:10.1503/cmaj.081272](http://dx.doi.org/10.1503/cmaj.081272) pmid:http://www.ncbi.nlm.nih.gov/pubmed/19581618 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTgxLzEtMi8zNyI7czo0OiJhdG9tIjtzOjIwOiIvcWhjLzI5LzEyLzEwMTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

16.  Khan S , Maclean CD , Littenberg B . The effect of the Vermont diabetes information system on inpatient and emergency room use: results from a randomized trial. Health Outcomes Res Med 2010;1:e61–6.[doi:10.1016/j.ehrm.2010.03.002](http://dx.doi.org/10.1016/j.ehrm.2010.03.002) pmid:http://www.ncbi.nlm.nih.gov/pubmed/20975923 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/j.ehrm.2010.03.002&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=20975923&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

17.  McMahon GT , Gomes HE , Hickson Hohne S , et al . Web-Based care management in patients with poorly controlled diabetes. Diabetes Care 2005;28:1624–9.[doi:10.2337/diacare.28.7.1624](http://dx.doi.org/10.2337/diacare.28.7.1624) pmid:http://www.ncbi.nlm.nih.gov/pubmed/15983311 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo5OiIyOC83LzE2MjQiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18.  Ralston JD , Hirsch IB , Hoath J , et al . Web-Based collaborative care for type 2 diabetes: a pilot randomized trial. Diabetes Care 2009;32:234–9.[doi:10.2337/dc08-1220](http://dx.doi.org/10.2337/dc08-1220) pmid:http://www.ncbi.nlm.nih.gov/pubmed/19017773 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo4OiIzMi8yLzIzNCI7czo0OiJhdG9tIjtzOjIwOiIvcWhjLzI5LzEyLzEwMTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

19.  Quinn CC , Clough SS , Minor JM , et al . WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther 2008;10:160–8.[doi:10.1089/dia.2008.0283](http://dx.doi.org/10.1089/dia.2008.0283) pmid:http://www.ncbi.nlm.nih.gov/pubmed/18473689 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1089/dia.2008.0283&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=18473689&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000256611200002&link_type=ISI) 

20.  Tang PC , Overhage JM , Chan AS , et al . Online disease management of diabetes: engaging and motivating patients online with enhanced resources-diabetes (EMPOWER-D), a randomized controlled trial. J Am Med Inform Assoc 2013;20:526–34.[doi:10.1136/amiajnl-2012-001263](http://dx.doi.org/10.1136/amiajnl-2012-001263) pmid:http://www.ncbi.nlm.nih.gov/pubmed/23171659 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2012-001263&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=23171659&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

21.  Earnest MA , Ross SE , Wittevrongel L , et al . Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inform Assoc 2004;11:410–7.[doi:10.1197/jamia.M1479](http://dx.doi.org/10.1197/jamia.M1479) pmid:http://www.ncbi.nlm.nih.gov/pubmed/15187074 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M1479&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=15187074&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

22.  Ross SE , Moore LA , Earnest MA , et al . Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004;6:e12. [doi:10.2196/jmir.6.2.e12](http://dx.doi.org/10.2196/jmir.6.2.e12) pmid:http://www.ncbi.nlm.nih.gov/pubmed/15249261 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2196/jmir.6.2.e12&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=15249261&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

23.  Green BB , Cook AJ , Ralston JD , et al . Effectiveness of home blood pressure monitoring, web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA 2008;299:2857–67.[doi:10.1001/jama.299.24.2857](http://dx.doi.org/10.1001/jama.299.24.2857) pmid:http://www.ncbi.nlm.nih.gov/pubmed/18577730 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/jama.299.24.2857&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=18577730&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000257021800023&link_type=ISI) 

24.  Wagner PJ , Dias J , Howard S , et al . Personal health records and hypertension control: a randomized trial. J Am Med Inform Assoc 2012;19:626–34.[doi:10.1136/amiajnl-2011-000349](http://dx.doi.org/10.1136/amiajnl-2011-000349) pmid:http://www.ncbi.nlm.nih.gov/pubmed/22234404 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2011-000349&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22234404&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

25.  Jones R , Pearson J , McGregor S , et al . Randomised trial of personalised computer based information for cancer patients. BMJ 1999;319:1241–7.[doi:10.1136/bmj.319.7219.1241](http://dx.doi.org/10.1136/bmj.319.7219.1241) pmid:http://www.ncbi.nlm.nih.gov/pubmed/10550090 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMTkvNzIxOS8xMjQxIjtzOjQ6ImF0b20iO3M6MjA6Ii9xaGMvMjkvMTIvMTAxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

26.  McCarrier KP , Ralston JD , Hirsch IB , et al . Web-Based collaborative care for type 1 diabetes: a pilot randomized trial. Diabetes Technol Ther 2009;11:211–7.[doi:10.1089/dia.2008.0063](http://dx.doi.org/10.1089/dia.2008.0063) pmid:http://www.ncbi.nlm.nih.gov/pubmed/19344195 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1089/dia.2008.0063&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=19344195&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000264891900002&link_type=ISI) 

27.  Tuil WS , Verhaak CM , Braat DDM , et al . Empowering patients undergoing in vitro fertilization by providing Internet access to medical data. Fertil Steril 2007;88:361–8.[doi:10.1016/j.fertnstert.2006.11.197](http://dx.doi.org/10.1016/j.fertnstert.2006.11.197) pmid:http://www.ncbi.nlm.nih.gov/pubmed/17416366 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/j.fertnstert.2006.11.197&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=17416366&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000248716000015&link_type=ISI) 

28.  Shaw E , Howard M , Chan D , et al . Access to web-based personalized antenatal health records for pregnant women: a randomized controlled trial. J Obstet Gynaecol Can 2008;30:38–43.[doi:10.1016/S1701-2163(16)32711-6](http://dx.doi.org/10.1016/S1701-2163(16)32711-6) pmid:http://www.ncbi.nlm.nih.gov/pubmed/18198066 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/S1701-2163(16)32711-6&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=18198066&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

29.  Chrischilles EA , Hourcade JP , Doucette W , et al . Personal health records: a randomized trial of effects on elder medication safety. J Am Med Inform Assoc 2014;21:679–86.[doi:10.1136/amiajnl-2013-002284](http://dx.doi.org/10.1136/amiajnl-2013-002284) pmid:http://www.ncbi.nlm.nih.gov/pubmed/24326536 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2013-002284&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=24326536&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

30.  Schnipper JL , Gandhi TK , Wald JS , et al . Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial. J Am Med Inform Assoc 2012;19:728–34.[doi:10.1136/amiajnl-2011-000723](http://dx.doi.org/10.1136/amiajnl-2011-000723) pmid:http://www.ncbi.nlm.nih.gov/pubmed/22556186 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2011-000723&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22556186&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

31.  Nagykaldi Z , Aspy CB , Chou A , et al . Impact of a wellness portal on the delivery of patient-centered preventive care. J Am Board Fam Med 2012;25:158–67.[doi:10.3122/jabfm.2012.02.110130](http://dx.doi.org/10.3122/jabfm.2012.02.110130) pmid:http://www.ncbi.nlm.nih.gov/pubmed/22403196 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjUvMi8xNTgiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

32.  Krist AH , Woolf SH , Rothemich SF , et al . Interactive preventive health record to enhance delivery of recommended care: a randomized trial. Ann Fam Med 2012;10:312–9.[doi:10.1370/afm.1383](http://dx.doi.org/10.1370/afm.1383) pmid:http://www.ncbi.nlm.nih.gov/pubmed/22778119 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTAvNC8zMTIiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

33.  Grant RW , Wald JS , Schnipper JL , et al . Practice-linked online personal health records for type 2 diabetes mellitus: a randomized controlled trial. Arch Intern Med 2008;168:1776–82.[doi:10.1001/archinte.168.16.1776](http://dx.doi.org/10.1001/archinte.168.16.1776) pmid:http://www.ncbi.nlm.nih.gov/pubmed/18779465 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/archinte.168.16.1776&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=18779465&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000259030600008&link_type=ISI) 

34.  Hibbard JH , Mahoney ER , Stockard J , et al . Development and testing of a short form of the patient activation measure. Health Serv Res 2005;40:1918–30.[doi:10.1111/j.1475-6773.2005.00438.x](http://dx.doi.org/10.1111/j.1475-6773.2005.00438.x) pmid:http://www.ncbi.nlm.nih.gov/pubmed/16336556 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1111/j.1475-6773.2005.00438.x&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=16336556&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000233170100013&link_type=ISI) 

35.  Anderson RM , Funnell MM , Fitzgerald JT , et al . The diabetes Empowerment scale: a measure of psychosocial self-efficacy. Diabetes Care 2000;23:739–43.[doi:10.2337/diacare.23.6.739](http://dx.doi.org/10.2337/diacare.23.6.739) pmid:http://www.ncbi.nlm.nih.gov/pubmed/10840988 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo4OiIyMy82LzczOSI7czo0OiJhdG9tIjtzOjIwOiIvcWhjLzI5LzEyLzEwMTkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

36. 1.  
    2.   Weinman J , 
    3.   Wright S , 
    4.   Johnston M  
    
     Schwarzer R , Jerusalem M . Generalized self-efficacy scale. In:  Weinman J ,  Wright S ,  Johnston M  , eds. Measures in health psychology: a user’s portfolio. Causal and control beliefs. Windsor: NEFR-Nelson (United Kingdom), 1995.
    
    

37.  Bulsara C , Styles I , Ward AM , et al . The psychometrics of developing the patient empowerment scale. J Psychosoc Oncol 2006;24:1–16.[doi:10.1300/J077v24n02\_01](http://dx.doi.org/10.1300/J077v24n02_01) pmid:http://www.ncbi.nlm.nih.gov/pubmed/17046803 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=17182474&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000240391200001&link_type=ISI) 

38.  Zigmond AS , Snaith RP . The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361–70.[doi:10.1111/j.1600-0447.1983.tb09716.x](http://dx.doi.org/10.1111/j.1600-0447.1983.tb09716.x) pmid:http://www.ncbi.nlm.nih.gov/pubmed/6880820 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1111/j.1600-0447.1983.tb09716.x&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=6880820&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1983QV25600001&link_type=ISI) 

39.  DiMatteo MR , Sherbourne CD , Hays RD , et al . Physicians' characteristics influence patients' adherence to medical treatment: results from the medical outcomes study. Health Psychol 1993;12:93–102.[doi:10.1037/0278-6133.12.2.93](http://dx.doi.org/10.1037/0278-6133.12.2.93) pmid:http://www.ncbi.nlm.nih.gov/pubmed/8500445 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1037/0278-6133.12.2.93&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=8500445&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=A1993LZ27100003&link_type=ISI) 

40.  Kashiwagi K , Tsukahara S . Impact of patient access to Internet health records on glaucoma medication: randomized controlled trial. J Med Internet Res 2014;16:e15. [doi:10.2196/jmir.2795](http://dx.doi.org/10.2196/jmir.2795) pmid:http://www.ncbi.nlm.nih.gov/pubmed/24429379 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2196/jmir.2795&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

41.  Ahmed S , Ernst P , Bartlett SJ , et al . The effectiveness of web-based asthma self-management system, my asthma portal (MAP): a pilot randomized controlled trial. J Med Internet Res 2016;18:e313. [doi:10.2196/jmir.5866](http://dx.doi.org/10.2196/jmir.5866) pmid:http://www.ncbi.nlm.nih.gov/pubmed/27908846 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.2196/jmir.5866&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

42.  D'Andrea E , Ahnen DJ , Sussman DA , et al . Quantifying the impact of adherence to screening strategies on colorectal cancer incidence and mortality. Cancer Med 2020;9:824–36.[doi:10.1002/cam4.2735](http://dx.doi.org/10.1002/cam4.2735) pmid:http://www.ncbi.nlm.nih.gov/pubmed/31777197 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 

43.  Stratton IM , Adler AI , Neil HA , et al . Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observational study. BMJ 2000;321:405–12.[doi:10.1136/bmj.321.7258.405](http://dx.doi.org/10.1136/bmj.321.7258.405) pmid:http://www.ncbi.nlm.nih.gov/pubmed/10938048 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjEvNzI1OC80MDUiO3M6NDoiYXRvbSI7czoyMDoiL3FoYy8yOS8xMi8xMDE5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

44.  Bell SK , Gerard M , Fossa A , et al . A patient feedback reporting tool for OpenNotes: implications for patient-clinician safety and quality partnerships. BMJ Qual Saf 2017;26:312–22.[doi:10.1136/bmjqs-2016-006020](http://dx.doi.org/10.1136/bmjqs-2016-006020) pmid:http://www.ncbi.nlm.nih.gov/pubmed/27965416 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI2LzQvMzEyIjtzOjQ6ImF0b20iO3M6MjA6Ii9xaGMvMjkvMTIvMTAxOS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

45. 1.  
    2.   Sammut C , 
    3.   Webb G  
    
     Kohavi R , Longbotham R . Online Controlled Experiments and A/B Tests. In:  Sammut C ,  Webb G  , eds. Encyclopedia of machine learning and data mining. Boston: Springer (US), 2017.
    
    

46.  Yamin CK , Emani S , Williams DH , et al . The digital divide in adoption and use of a personal health record. Arch Intern Med 2011;171:568–74.[doi:10.1001/archinternmed.2011.34](http://dx.doi.org/10.1001/archinternmed.2011.34) pmid:http://www.ncbi.nlm.nih.gov/pubmed/21444847 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1001/archinternmed.2011.34&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=21444847&link_type=MED&atom=%2Fqhc%2F29%2F12%2F1019.atom) 
    
    [Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000288864100014&link_type=ISI)