How to obtain the P value from a confidence interval
BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d2304 (Published 08 August 2011) Cite this as: BMJ 2011;343:d2304
- Douglas G Altman, professor of statistics in medicine 1,
- J Martin Bland, professor of health statistics2
- 1Centre for Statistics in Medicine, University of Oxford, Oxford OX2 6UD
- 2Department of Health Sciences, University of York, Heslington, York YO10 5DD
- Correspondence to: D G Altman doug.altman{at}csm.ox.ac.uk
We have shown in a previous Statistics Note1 how we can calculate a confidence interval (CI) from a P value. Some published articles report confidence intervals, but do not give corresponding P values. Here we show how a confidence interval can be used to calculate a P value, should this be required. This might also be useful when the P value is given only imprecisely (eg, as P<0.05). Wherever they can be calculated, we are advocates of confidence intervals as much more useful than P values, but we like to be helpful.
The method is outlined in the box below in which we have distinguished two cases.
Steps to obtain the P value from the CI for an estimate of effect (Est)
(a) P from CI for a difference
If the upper and lower limits of a 95% CI are u and l respectively:
1 calculate the standard error: SE = (u − l)/(2×1.96)
2 calculate the test statistic: z = Est/SE
3 calculate the P value2: P = exp(−0.717×z − 0.416×z2).
(b) P from CI for a ratio
For a ratio measure, such as a risk ratio, the above formulas should be used with the estimate Est and the confidence limits on the log scale (eg, the log risk ratio and its CI).
Notes
All P values are two sided.
All logarithms are natural (ie, to base e).3
“exp” is the exponential function.
The formula for P works only for positive z, so if z is negative we remove the minus sign.
For a 90% CI, we replace 1.96 by 1.65; for a 99% CI we use 2.57.
(a) P from CI for a difference (no transformation needed)
The simple case is when we have a CI for the difference between two means or two proportions. For example, participants in a trial received antihypertensive treatment with or without pravastatin. The authors report that pravastatin performed slightly worse than a placebo. The estimated difference between group means was 1.9 (95% CI −0.6 to 4.3) mm Hg.4 What was the P value?
Following the steps in the box above we calculate P as follows:
SE = [4.3 − (−0.6)]/(2×1.96) = 1.25;
z = 1.9/1.25 = 1.52;
P = exp(−0.717×1.52 − 0.416×1.522) = 0.13.
In this paper, the authors did indeed publish a P value of 0.13,4 as we have estimated from their confidence interval.
(b) CI for a ratio (log transformation needed)
The calculation is trickier for ratio measures, such as risk ratio, odds ratio, and hazard ratio. We need to log transform the estimate and confidence limits, so that Est, l, and u in the box are the logarithms of the published values.
For example, in a meta-analysis of several studies comparing single versus bilateral mammary artery coronary bypass grafts Taggart et al presented a hazard ratio of 0.81; 95% CI 0.70 to 0.94.5 They did not quote the P value.
Following the steps in the box we calculate P as follows:
Est = log(0.81) = −0.211
l = log(0.70) = −0.357, u = log (0.94) = −0.062
SE = [−0.062 − (−0.357)]/(2×1.96) = 0.0753.
z = −0.211/0.0753 = −2.802. We take the positive value of z, 2.802.
P = exp(−0.717×2.802 − 0.416×2.8022) = 0.005.
Limitations of the method
The formula for P is unreliable for very small P values and if your P value is smaller than 0.0001, just report it as P<0.0001.
The methods described can be applied in a wide range of settings, including the results from meta-analysis and regression analyses. The main context where they are not correct is small samples where the outcome is continuous and the analysis has been done by a t test or analysis of variance, or the outcome is dichotomous and an exact method has been used for the confidence interval. However, even here the methods will be approximately correct in larger studies with, say, 60 patients or more.
Notes
Cite this as: BMJ 2011;343:d2304
Footnotes
Contributors: JMB and DGA jointly wrote and agreed the text.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.