Main

In evidence-based medicine and dentistry randomised controlled trials (RCTs) are regarded as the most reliable method of evaluating the effects of healthcare interventions.1,2 RCTs are also considered the 'golden standard' for providing research evidence from clinical research.3 The strength of evidence has been given an order of hierarchy,3,4 where results from correctly conducted systematic reviews are regarded as most valuable, followed by RCTs.2,3,4 However, the value of a systematic review is largely dependent on the quality of the included RCTs.2,5,6 High quality systematic reviews are available for some interventions, but for many routine interventions in dental practice evidence is scarce.7 Systematic reviews are only as up to date as the latest included trial, therefore well conducted RCTs are the most valuable source of primary scientific evidence.

Evidence-based practice aims at integrating research evidence with clinical expertise and patient values.3 Skills in finding and appraising the evidence for interventions in everyday routine practice, for new therapies or in unusual patient situations, are required from the individual practitioners.3,8 There are several biomedical databases available for locating the evidence, such as the Cochrane Library, including several 'evidence-based' databases, such as the Cochrane Controlled Trials Register (CCTR), the Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effectiveness (DARE).9 The Medline database is the largest single database for biomedical references, indexing abstracts from about 4,000 journals worldwide.8,9 The PubMed version of Medline can be readily accessed via the internet, free of charge, at the National Library of Medicine (NIH, Bethesda, U.S.A. www.ncbi.nlm.nih.gov), making it a useful tool for all practitioners with an internet connection. In a previous study we showed that the proportion of RCT-Ds available on Medline had continuously increased between the years 1969 and 1999 from less than 0.1% to about 5% of all publications in dental research.10 However, faulty indexing and inherent software limitations may affect the search results of RCT-Ds in different areas of dental research,8 biasing the search results and making it harder to locate relevant RCT-Ds. Therefore, this study was performed with the aim of elucidating the validity of Medline searches for RCT-Ds related to different areas of dental research.

Material and methods

Medline searches

The Medline database (Entrez PubMed, www.ncbi.nlm.nih.gov) was searched (Feb 2001) for RCT-Ds published in 1999. Inclusion criteria in the searches were: 1) Medical Subject Headings (MeSH-terms): 'dentistry', 'endodontics', 'oral surgical procedures', 'orthodontics', 'pediatric dentistry', 'periodontics', 'prosthodontics', 'public health dentistry', and 'surgery, oral', 2) publication year 1999, and 3) publication type 'randomized controlled trial'.

Validity of the Medline searches for different areas of dental research

Abstracts of all RCT-Ds (MeSH-term 'dentistry') from 1999 were examined for relevance to different areas of dental research, and in doubtful cases the article was retrieved. All RCT-Ds were located to the following domains; 'endodontics', 'oral surgery', 'orthodontics', 'pediatric dentistry', 'periodontics', 'prosthodontics, excluding restorative treatments', 'prosthodontics, including restorative treatments', 'public health dentistry' and 'others'. In each domain the RCT-Ds were cross-tabulated against the results from the Medline searches with the corresponding MeSH-terms. For all the Medline search results, using the manual examination as control,11 we calculated the sensitivity (the proportion of correctly included RCT-Ds in a given area), specificity (the proportion of correctly excluded RCT-Ds in a given area), positive predictive value (PPV, for Medline also known as precision;12 (that is the proportion of RCT-Ds included by the search that belongs to the given area)), negative predictive value (NPV; the proportion of RCT-Ds excluded by the search that does not belong to the given area) and the accuracy (the proportion of correct results in the searches for a given area).11,12 Furthermore, the total proportion of false inclusions was calculated (RCTs belonging to other areas than dental research, MeSH-term 'dentistry').

Results

Medline searches

The Medline searches gave a total of 259 search hits for RCT-Ds published in 1999 (MeSH-term 'dentistry'). The proportion of false inclusions was 3.9%. The numbers of RCT-Ds located with different MeSH-terms are given in Table 1.

Table 1 Table 1

Validity of the Medline searches for different areas of dental research (Table 1)

The validity of the Medline searches (MeSH-terms) for different areas of dental research was highest for endodontics, with 1.00 sensitivity and 0.99 specificity, whereas the lowest validity was seen for pediatric dentistry, with 0.05 sensitivity, and for oral surgery, with 0.03 sensitivity, when the MeSH-term 'surgery, oral' was used. When the searches for oral surgery were conducted with the MeSH-term 'oral surgical procedures', the sensitivity increased to 0.85.

When all restorative treatments were included in prosthodontics, a lower sensitivity was obtained than when direct restorations were excluded in the manual examinations. However, in the first scenario, the proportion of false inclusions was lower, increasing the specificity.

When search results obtained with the faulty indexed MeSH-terms 'pediatric dentistry' and 'surgery, oral' were excluded, the highest PPV values (precision) were seen for periodontics and prosthodontics, including restorative treatments (Table 1).

Discussion

We found an overall adequate validity for locating RCT-Ds with MeSH-term searches on Medline, although variations between the different areas of dental research were seen. The MeSH-term searches for pediatric dentistry and public health dentistry had poor sensitivity, indicating faulty indexing on Medline. In addition, the MeSH-term 'surgery, oral' was inappropriate for locating RCT-Ds about oral surgery.

The sample in this study included all RCT-Ds published in 1999 that were available on Medline, strengthening the reliability of our results.

The capability of the Medline searches for locating RCT-Ds in a given area, given by the sensitivity values,11,12 ranged from 0.85 to 1.00 for the majority of the searches. The MeSH-terms 'pediatric dentistry', 'public health dentistry' and 'surgery, oral' had poor sensitivities, indicating faulty indexing. Thus, these MeSH-terms may not be regarded as suitable for searching RCT-Ds.

During the study it became apparent that the MeSH-term 'surgery, oral' was no longer adequate for searching publications about oral surgery and that the MeSH-term 'oral surgical procedures' should be used instead (www.nlm.nih.gov/mesh/MBrowser.html). This may be noteworthy since 'surgery, oral' is located in the MeSH tree under 'dentistry' [G02.163] / 'specialties, dental' [G02.163.876] together with other MeSH-terms related to clinical areas of dental research, whereas the term 'oral surgical procedures' [E06.645] is situated elsewhere, under 'dentistry' [E06].

The capability of correctly excluding RCT-Ds in a given area of dental research, given by the specificity values,11,12 was somewhat high in the searches, ranging from 0.81 to 1.00. However, in most of the searches the specificities were not absolute, and false exclusions occurred. When search results with higher specificity are required, comprehensive search strategies and filters should be used.8,9,12

The PPV values (or precision) of the searches indicated some uncertainties in the probability that an RCT-D included by the MeSH-term search truly belongs to a given area of dental research, leading to false inclusions.11,12 This, however, is a smaller problem than the opposite situation, since it is much easier to manually exclude false inclusions than to find those that were completely missed. This is mirrored by the specificities and the NPV values which were above 0.90 in most of the searches, making it plausible to conclude that RCT-Ds excluded by the searches do not belong to the given area of dental research.

In total, the proportion of correct results in the searches for each area,11,12 given by the accuracy values of more than 0.90 in most of the searches, indicate an overall satisfactory, though imperfect, validity for locating RCT-Ds.

In order to derive evidence for clinical practice it is important to be able to locate RCT-Ds in a relatively short time, since a practitioner seldom has hours to spend on literature searches. A practitioner can afford to miss a few RCT-Ds if there are several available about the same topic, but it is important to be able to appreciate the possible amount of false exclusions in the searches. Moreover, one must be aware that correct search strategies are crucial for retrieving relevant references within a reasonable time.9,11 For this purpose different search filters are available,9,11,12 such as 'clinical queries' on PubMed (www.ncbi.nlm.nih.gov/entrez/query/static/clinical.html). In addition, evidence-based summaries made by others, as found in Evidence-Based Dentistry (www.stockton-press.co.uk/ebd), may prove helpful for locating the current best evidence.

Conclusion

Our results suggest that MeSH-term searches provide an over all adequate method for rapid location of RCT-Ds on Medline for most areas of dental research. For some areas, however, faulty indexing or software limitations have compromised the search validity. Thus, for those areas the corresponding MeSH-terms may not be regarded as an appropriate way of locating RCT-Ds. Furthermore, when higher stringency is required in the searches, for example when systematic reviews are to be made, false exclusion cannot be tolerated, and more comprehensive strategies and search filters need to be used.8,9,12