Article Text
PDF
Abstract
Most of what I need to know about clinical research methods, I learned from the Bible.
- quality improvement
- Bible
Statistics from Altmetric.com
It is a challenge to teach the essentials of clinical design to health profession students who can easily be bored by this topic. Here is a way to make this class relaxed and enjoyable. The teacher hands out the first 21 verses of the biblical book of Daniel and asks the students to read them and describe the design of Daniel’s clinical trial to improve health. To refresh the memory of some of our readers, these verses are reprinted in box 1.
Box 1 Book of Daniel, Chapter 1
1 In the third year of the reign of Jehoiakim king of Judah came Nebuchadnezzar king of Babylon unto Jerusalem, and besieged it.
2 And the Lord gave Jehoiakim king of Judah into his hand, with part of the vessels of the house of God: which he carried into the land of Shinar to the house of his god; and he brought the vessels into the treasure house of his god.
3 And the king spake unto Ashpenaz the master of his eunuchs, that he should bring certain of the children of Israel, and of the king’s seed, and of the princes;
4 Children in whom was no blemish, but well favoured, and skilful in all wisdom, and cunning in knowledge, and understanding science, and such as had ability in them to stand in the king’s palace, and whom they might teach the learning and the tongue of the Chaldeans.
5 And the king appointed them a daily provision of the king’s meat, and of the wine which he drank: so nourishing them three years, that at the end thereof they might stand before the king.
6 Now among these were of the children of Judah, Daniel, Hananiah, Mishael, and Azariah:
7 Unto whom the prince of the eunuchs gave names: for he gave unto Daniel the name of Belteshazzar; and to Hananiah, of Shadrach; and to Mishael, of Meshach; and to Azariah, of Abednego.
8 But Daniel purposed in his heart that he would not defile himself with the portion of the king’s meat, nor with the wine which he drank: therefore he requested of the prince of the eunuchs that he might not defile himself.
9 Now God had brought Daniel into favour and tender love with the prince of the eunuchs.
10 And the prince of the eunuchs said unto Daniel, I fear my lord the king, who hath appointed your meat and your drink: for why should he see your faces worse liking than the children which are of your sort? then shall ye make me endanger my head to the king.
11 Then said Daniel to Melzar, whom the prince of the eunuchs had set over Daniel, Hananiah, Mishael, and Azariah,
12 Prove thy servants, I beseech thee, ten days; and let them give us pulse to eat, and water to drink.
13 Then let our countenances be looked upon before thee, and the countenance of the children that eat of the portion of the king’s meat: and as thou seest, deal with thy servants.
14 So he consented to them in this matter, and proved them ten days.
15 And at the end of ten days their countenances appeared fairer and fatter in flesh than all the children which did eat the portion of the king’s meat.
16 Thus Melzar took away the portion of their meat, and the wine that they should drink; and gave them pulse.
17 As for these four children, God gave them knowledge and skill in all learning and wisdom: and Daniel had understanding in all visions and dreams.
18 Now at the end of the days that the king had said he should bring them in, then the prince of the eunuchs brought them in before Nebuchadnezzar.
19 And the king communed with them; and among them all was found none like Daniel, Hananiah, Mishael, and Azariah: therefore stood they before the king.
20 And in all matters of wisdom and understanding, that the king enquired of them, he found them ten times better than all the magicians and astrologers that were in all his realm.
21 And Daniel continued even unto the first year of king Cyrus.
The old King James version is reproduced here to be sure that copyright laws are not violated. The instructor, wishing to maintain religious neutrality, may make available other translations sponsored by different religions and the serious scholar will want to read this in the original Hebrew and Aramaic.
In box 1 you have read the first known health related controlled clinical trial. These historical events occurred 2600 years ago and were written down about 2200 years ago.1 The following components of a clinical trial are here:
-
An hypothesis is defined in advance and tested. Pure food is healthier than the king’s meat. Pulse, by the way, are beans, peas or lentils.2 Today, hypotheses have to be clearly and publicly stated and written in advance before the trial. This is done to make sure the study design will best answer the hypothesized question and to avoid the human temptation of restating the hypothesis after the trial results are known.
-
There is an experimental group receiving the intervention and an unchanged control group. Today, such trials are often presented as diagrams (fig 1).
-
The control and experimental groups were chosen to be similar—Hebrew children. Exclusion and inclusion conditions are defined—“children in whom was no blemish but well favoured”. Note that these conditions were defined before the trial started, but this will have the same effect.
-
As in most clinical trials, there is the issue of representativeness of the select study population to all people. This raises the issue of generalizability to the whole population.
-
There is a predefined start and stop time period for the trial which was 10 days. This avoids the bias that could occur if “we quit when we are ahead”.
-
At the end of the trial there is an observer who measures the predetermined outcome in both control and experimental groups and a comparison between these two groups is made to test the study hypothesis. Usually this requires some method of averaging the results within each group to make a comparison. Perhaps the outcome differences were so large in this study that averaging may not have been a problem.
-
Potential sources of observer bias are described. For the Prince of the Eunuchs, a negative result would “make me endanger my head to the King” and the Prince had a “tender love” for Daniel. For this reason, having Melzar as the observer might have reduced the risk of this bias affecting the measurement of the results.
-
Design bias. The control group does not eat average Babylonian food, but meat and wine from the king’s table, which presumably is the best to be had. This is a bias which makes it harder to get a positive result in the trial and is therefore biased against the hypothesis and makes the positive result all the more convincing.3 In today’s drug trials this would be called an “equivalence trial”. The new drug is compared with the next best existing drug. This is a harder challenge than comparing the new drug with a placebo control. The USA Food and Drug Administration requires the latter and not the former type of trial unless the placebo control is unethical. Thus, new drugs enter the US market without clear knowledge that they are better than existing (often less expensive) drugs.
-
After the trial is carried out and the hypothesis is supported (“Their countenances appeared fairer and fatter in flesh”), a policy is changed (verse 16). The results of the trial were published, albeit four centuries later. Was this a peer reviewed publication?
Process diagram of Daniel’s clinical trial; v = verse.
ISSUES OF TRIAL MANAGEMENT
Daniel gave unwritten informed consent to participate in this trial. Could he do so as a captive child? Should the children in the control group give informed consent to participate in this trial even though nothing changed for them? After the trial had shown that this pure food was healthier, should the food have been changed for all the other children?
There is a confused definition of the role of the principal investigator. Is it the Prince, Melzar, or Daniel? We think it was Daniel, but he was also a participant. Lack of a clear organizational structure, no literature review, no discussion of the significance, no budget discussion, the absence of women and other minority groups in the trial, and the use of vulnerable subjects (captive children) would make this study totally unacceptable today and the grant proposal would not make it as far as peer review. Scientific journals are not supposed to publish unethical studies. The editors of this journal are taking an heroic decision to reprint these verses from the Bible. What do your students think of this editorial decision?
WHAT HAVE WE LEARNED SINCE DANIEL?
-
Random assignment to experimental and control groups to assure their similarity. This had to wait until the 1930s.
-
Blinding of subjects and observer and placebo control had to wait until Paris in 1784.4
-
The use of numbers to evaluate a clinical trial. This had to wait until 1721.5 The use of quantitative, reliable measures of quality of life is a development of the last 50 years.
-
Statistical tests of significance and power calculations to rule out chance as an explanation of observed outcome differences. This had to wait until the 20th century. In short, Daniel’s trial was “state of the art” for over two millennia.
THE FATAL SCIENTIFIC FLAW
The fatal scientific flaw of Daniel’s trial is that we do not have enough information to replicate it today. We do not have enough knowledge of the experimental and control diets or other environmental factors related to health then.
This is a problem for a large number of randomized clinical trials today in which a new drug is compared with current practice. Ask yourself and your students how would someone replicate today’s randomized controlled trials 250 years from now if they could not replicate our current health care practices?
THE ROLE OF THEORY
Randomization has allowed most clinical research to be devoid of theory. Current research asks whether treatment A is better than B, controlling for everything else. Without theory we have no ability to generalize such results beyond the study and its population. What are the underlying unspoken theories behind Daniel’s clinical trial? Here are just some possible theories your students might propose.
-
The King’s food was contaminated and the children constantly drunk on his wine.
-
Daniel’s pure food was nutritionally better.
-
Pure (Kosher) food is healthier for Hebrew children.
-
Actually the pure food (beans and water) was less nutritious and God produced a miracle.1
-
This study was reported 400 years after the event and is just a good story.
-
The sample size (experimental group n = 4) was too small and their results are due to chance.
Invite your students to design experiments to test these theories. A good study should allow us to reject some of these theories and support others until we have eliminated all but one theory. For example, if we could really replicate this study today many times and always found “no difference”, this would support theories 4 and 5, and allow us to reject theories 1, 2, 3, and 6. Theory is important because it provides testable generalizable knowledge.
In evaluating theory 6 above, it is essential to know probability theory. Assume that there are a total of 100 children in the two arms of Daniel’s trial and assume, on average, that only four in 100 children on any given day are clearly “fairer and fatter” than the rest. The probability of picking one of these children at random is then 0.04 or 1 in 25. Under the assumptions of independence and non-replacement, the probability of picking all four such children is (4/100)(3/99)(2/98)(1/97) or one chance in 3 921 225. Under these assumptions, chance cannot explain these results. Change the assumptions and the expected results would be different. Perhaps Daniel first met his friends working out at the King’s fitness center so that the choice of his friends for the experimental group was not independent (random). This is an example of why health profession students would benefit from knowledge of statistics. As a teacher, much as you might want to apply Bayesian statistics to the definition of miracles in evaluating theory 4 above, we suggest you leave this fascinating topic for later. There is no point in overdoing it in the first class.
If you have read this far, you can decide for yourself if learning about research methodology can be entertaining. Your students might agree that “most of what I need to know about clinical trials I learned from the Bible”. Daniel deserves to be a hero of quality improvement. He could also have been a martyr if a higher power had not miraculously closed the lions’ mouths in their den and thereby saved him from being eaten alive.6
Acknowledgments
The authors wish to thank Professor Sara Debanne and Jose Lozada for their advice and help.