References
Ebel RL, Frisbie DA. Essentials of educational measurement. Englewood Cliffs, New Jersey: Prentice-Hall, 1986.
Ory JC, Ryan KE. Tips for improving testing and grading. Newbury Park, California: Sage Publications, 1993.
Pickering C. Against multiple-choice questions. Medical Teacher 1979; 1: 84–6.
Newble DI, Baxter, A, Elmslie G. A comparison of multiple choice and free response tests in examinations of clinical competence. Medical Education 1979; 13: 263–8.
McGuire C. Perspectives in assessment. Academic Medicine 1993; 68(Supplement): S3–8.
Case SM, Swanson DB. Extended-matching items: A practical alternative to free-response questions. Teaching and Learning in Medicine 1993; 5: 107–15.
Schuwirth LWT, van der Vleuten CPM, Donkers HHLM. Open ended questions versus multiple choice questions: An analysis of cuing effects. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 486–91.
Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.
Van der Vleuten CPM, Newble DI. How can clinical reasoning be tested? The Lancet 1995; 345: 1032–4.
McGuire CH, Babnott D. Simulation technique in the measurement of problem solving skills. Journal of Educational Measurement 1967; 4: 1–10.
Hodgkin K, Knox JDE. Problem centered learning: The modified essay question in medical education. Edinburgh: Churchill-Livingstone, 1975.
Feletti GI, Saunders NA, Smith AJ. Comprehensive assessment of final-year medical student performance based on undergraduate programme objectives. Lancet 1983; 2(8340): 34–7.
Barrows HS, Tamblyn RM. The portable patient problem pack (P4). A problem-based learning unit. Journal of Medical Education 1977; 52: 1002–4.
Williams RG, Vu NV, Barrows HG, Verhulst S. Profile of the Clinical Reasoning Test (CRT): An objective measure of problem solving skills and proficiency in using medical knowledge. In: Schmidt HG, De Volder ML, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 81–90.
Norcini JJ, Meskauskas JA, Langdon LO, Webster GD. An evaluation of a computer simulation in the assessment of physician competence. Evaluation in the Health Professions 1986; 9: 286–304.
Bligh TJ. Written simulation scoring: Comparison of nine systems [dissertation]. Urbana-Champaign (IL): University of Illinois, 1980.
Swanson D, Norcini J, Grosso L. Assessment of clinical competence: Written and computer-based simulations. Assessment and Evaluation in Higher Education 1987; 12: 220–46.
Norman G, Bordage G, Curry L et al. A review of recent innovations in assessment. In: Wakeford RE, editor. Directions in clinical assessment. Report of the First Cambridge Conference. Cambridge: Cambridge University School of Clinical Medicine, 1985: 9–27.
Elstein A, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge Massachusetts: Harvard University Press.
Friedman R, Korst D, Schultz J, Beatty E, Entine S. Experience with the simulated patient physician encounter. Journal of Medical Education 1978; 53: 825–30.
McLeskey C, Ward R. Validity of written examinations. Anesthesiology 1978; 49: 224.
Marshall J. Assessment of problem-solving ability. Medical Education 1977; 11: 329–34.
Newble DI, Hoare J, Baxter A. Patient management problems: Issues of validity. Medical Education 1982; 16: 137–42.
Norman GR, Feightner JW. A comparison of behaviour on simulated patients and patient management problems. Journal of Medical Education 1981; 55: 529–37.
Norcini JJ, Swanson DB, Grosso LJ, Shea JA, Webster Gd. Reliability, validity and efficiency of multiple choice questions and patient management problem item formats in the assessment of physician competence. Medical Education 1985; 19: 238–47.
Bordage G, Page G. An alternative approach to PMPs: The “key features” concept. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Heal-Publications, 1987: 59–75.
De Graaff E, Post G, Drop M. Validation of a new measure of clinical problem-solving. Medical Education 1987; 21: 213–218.
Powles ACP, Wintrup N, Neufeld VR, Wakefield JH, Coates G, Burrows J. The triple jump exercise: further studies of an evaluative technique. Proceedings of the 20th Annual Conference on Research in Medical Education, Washington: American Association of Medical Colleges, 1981: 74–9.
Fiedman CP, Murphy GC, Smith AC, Mattern WD. Exploratory study of an examination format for problem-based learning. Teaching and Learning in Medicine 1994; 6: 194–8.
Boud D. The role of self-assessment in student grading. Assessment and Evaluation in Higher Education 1989; 14: 20–30.
De Grave W De Volder M. Peer-evaluation and problem-based learning. In: Schmidt H, De Volder M, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 116–22.
Magzoub M. Studies in community-based education [dissertation]. Masstricht: University of Limburg, 1994.
Case SM, Swanson DB, van der Vleuten CPM. Student assessment in problem-based learning curricula. In: Boud D, Feletti G, editors. The challenge of problem-based learning. London: Kogan Page, 1991: 260–73.
Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 1991; 66: 762–9.
Blake JM, Norman GR, Smith EKM. Report card from McMaster: Student evaluation at a problem-based medical school. Lancet 1995; 345: 899–902.
Harden R, Gleeson F. Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education 1979; 13: 41–54.
Rothman AI, Cohen R. (Editors). Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 292–321.
Reznick R, Blackmore DE, Cohen R et al. An objective structured clinical examination for the licentiate of the Medical Council of Canada: From research to reality. Academic Medicine 1993; 68(Supplement): S4–6.
Van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine 1990; 2: 58–76.
Vu NV, Barrows HS. Use of standardized patients in clinical assessments: recent developments and measurement findings. Educational Researcher 1994; 23: 23–30.
Swanson DB, Norman GR, Linn RL. Performance-based assessment: Lessons from the health professions. Educational Researcher 1995; 24: 5–22, 35.
Van Thiel J, Kraan HF, Van der Vleuten CPM. Reliability and feasibility of measuring interviewing skills using the revised Maastricht History Taking and Advice Checklist. Medical Education 1991; 25: 224–9.
Swanson Db. A measurement framework for performance-based tests. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 13–45.
Stalenhoef-Halling BF, Van der Vleuten CPM, Jaspers TAM, Fiolet JFBM. The feasibility, acceptability and reliability of open-ended questions in a problem-based learning curriculum. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 552–7.
Erviti V, Templeton B, Bunce J, Burg F. The relationships of pediatric resident recording behavior across medical conditions. Medical Care 1980; 18: 1020–31.
Rethans JJ, Sturmans F, Drop MJ, Van der Vleuten CPM. Assessment of performance in actual practice of general practitioners by use of standardized patients. British Journal of General Practice 1991; 41: 97–9.
Shavelson RJ, Baxter GP, Gao X. Sampling variability of performance assessments. Journal of Educational Measurement 1993; 30: 215–32.
Klein as cited in: Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.
Shavelson RJ, Mayberry P, Li W, Webb NM. Generalizability of military performance measurements: Marine Corps rifleman. Military Psychology 1990; 2: 129–44.
Frijns PRAM, Van der Vleuten CPM, Verwijnen GM, Van Leeuwen YD. The effect of structure in scoring methods on the reproducibility of tests using open-ended questions. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 466–71
Van Ham I, Gerritsma J. The assessment of clinical competence in general practice with chart stimulated recall. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk, 1990: 306–9.
Van der Vleuten CPM, Norman GR, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of reliability. Medical Education 1991; 25: 110–8.
Streiner DL. Global rating scales. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer, 1985: 119–41.
Streiner D. Clinical ratings - ward evaluation. In: Shannon S, Norman G, editors. Evaluation methods: A resource handbook. Hamilton: The Program for Educational Development, McMaster University, 1995: 29–31.
Hastorf AH, Schneider DJ, Polefka J. Person perception. Reading, Massachusetts: Addison-Wesley, 1970.
Ross M. Relation of implicit theories to the construction of personal histories. Psychological Review 1989; 96: 341–57.
Norman GR, Swanson DB, Case SM. Conceptual and methodological issues in studies comparing assessment formats. Teaching and Learning (in press).
Norman GR, Smith E, Powles A, Rooney P, Henry N, Dodd P. Factors underlying performance on written tests of knowledge. Medical Education 1987; 21: 297–304.
Jean P, Schuwirth L, Van Santen M, Van der Vleuten C. Do problem analysis questions (PAQs) and true/false questions (TFQs) measure different skills? Medical Education (in press).
Maatsch J, Huang R. An evaluation of the construct validity of four alternative theories of clinical competence. Proceedings of the Twenty-fifth Annual Conference on Research in Medical Education, American Association of Medical Colleges. Washington, DC, 1986.
Maatsch J. Model for a criterion-referenced medical specialty test. Final Report Grant No. HS-02038-02, Office of Medical Education Research and Development Michigan State University, 1980.
Van der Vleuten CPM, Van Luijk S, Beckers HJM. A written test as an alternative to performance testing. Medical Education 1989; 23: 97–107.
Ward W. A comparison of free-response and multiple choice forms of verbal aptitude tests. Applied Psychological Measurement 1982; 6: 1–11.
Thissen D, Wainer H, Wang X. Are tests comprising both multiple-choice and free-response items necessarily more unidimensional than multiple-choice tests? An analysis of two tests. Journal of Educational Measurement 1994; 31: 113–23.
Ramsey PG, Carline JD, Inui YS et al. Predictive validity of certification by the American Board of Internal Medicine. Annals of Internal Medicine 1989; 110: 719–26
Solomon et al. 1990 as cited in Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.
Norman GR, Davis DA, Painvin A, Rath D, Ragbeer M. Comprehensive assessment of clinical competence of family-general physicians using multiple measures. Proceedings 28th Conference on Research in Medical Education. Washington: American Association of Medical Colleges, 1989.
Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.
McGuire C. Written methods for assessing clinical competence. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 44–58.
Hager P, Gonczi A, Athanasou J. General issues about assessment of competence. Assessment & Evaluation in Higher Education 1994; 19: 3–16.
Norman GR, Van der Vleuten CPM, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of validity, efficiency and acceptability. Medical Education 1991; 25: 119–26.
Van Thiel J, van der Vleuten CPM, Kraan H. Assessment of medical interviewing skills: Generalizability of scores using successive MAAS-versions. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 536–540.
Newble D, Jaeger K. The effect of assessments and examinations on the learning of medical students. Medical Education 1983; 17: 165–71.
Popham WJ. Measurement as an instructional catalyst. In: Ekstrom RB, editor. Measurement, technology and individuality in education. San Francisco: Jossey-Bass, 1983: 87–103.
Frederiksen N. The real test bias: Influences of testing on teaching and learning. American Psychologist 1984; 39: 193–202.
Entwistle N. Styles of learning and teaching. Chichester: John Wiley & Sons, 1981.
Stillman P, Swanson D. Ensuring the clinical competence of medical school graduates through standardized patients. Archives of Internal Medicine 1987; 147: 1049–52.
Gibbs G. Improving the quality of student learning. Bristol: Technical & Educational Services, 1992.
Shepard LA. Psychometrician's beliefs about learning. Educational Researcher 1991; 20: 2–16.
Snyder BR. The hidden curriculum. New York: Knopf, 1971.
Popham WJ, Cruse KL, Rankin SC, Sandifer PD, Williams PL. Measurement-driven instruction: it's on the road. Phi Delta Kappan 1985; 66: 628–34.
Semb GB, Ellis JA. Knowledge taught in school: What is remembered? Review of Educational Research 1994; 64: 253–86.
Van Luijk SJ, Van der Vleuten CPM, Schelven RM. The relation between content and psychometric characteristics in performance-based testing. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publications, 1990: 202–7.
Mischel W. Personality and assessment. New York: John Wiley, 1968.
Schmidt H, Norman G, Boshuizen HA. Cognitive perspective on medical expertise: Theory and implications. Academic Medicine 1990; 65: 611–21.
Norman G, Allery L, Berkson L et al. Research in the psychology of clinical reasoning: implications for assessment. Paper from the Fourth Cambridge Conference, Cambridge University School of Clinical Medicine, Cambridge, 1989.
Higgs J, Jones M (Editors). Clinical reasoning in the health professions. Oxford: Butterworth/Heinemann, 1995.
Norman G, Regehr G. Contemporary issues in cognitive psychology: implications for professional education. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 17–25.
Rethans JJ, Sturmans F, Drop MJ et al. Does competence of general practitioners predict their performance. British Medical Journal 1991; 303: 1377–80.
Norman GR. Reliability and construct validity of some cognitive measures of clinical reasoning. Teaching and Learning in Medicine 1989; 1: 194–9.
Newble DI, Raymond GA. The Pattern Completion Item (PCI): A potential measure of clinical problem-solving skills. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 191–2.
Nelson MS, Clayton BL, Moreno R. How medical school faculty regard educational research and make pedagogical decisions 1990; 65: 122–6.
Hays RB, Fabb WE, Van der Vleuten CPM. Reliability of the fellowship examination of the Royal Australian College of General Practitioners. Teaching and Learning in Medicine 1995; 7: 43–50.
Van Luijk SJ, Van der Vleuten CPM. A comparison of checklists and rating scales in performance-based testing. In: Hart IR, Harden RM, Des Marchais J, editors. Current developments in assessing clinical competence. Montreal: Can-Heal, 1992: 357–62.
Newble D, Dawson B, Dauphinee D et al. Guidelines for assessing clinical competence. Teaching and Learning in Medicine 1994; 6: 213–20.
Endler NS, Magnusson D, editors. Interfactional psychology and personality. Washington DC: Hemisphere, 1976.
Vand er Vleuten CPM, Schuwirth LWT, Ronteltap CFM. A cognitive psychological interpretation of a few remarkable psychometric findings. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 506–8.
Frijns PHAM. Scoringsmodellen voor open-vraag vormen (Scoring models for free-response formats) [dissertation]. Maastricht: University of Limburg, 1992.
Brailovsky C, Bordage G, Carretier H, Page G. Content validity of the key features' approach of the Medical Council of Canada's Exam. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 476–7.
Bordage G, Brailovsky C, Carretier H, Page G. Content validation of key features on a national examination of clinical decision-making skills. Academic Medicine 1995; 70: 276–81.
Ebel RL. Must all tests be valid? American Psychologist 1961; 16: 640–7.
Ebel R. Measuring Educational Achievement. Englewood Cliffs: Prentice-Hall Inc, 1965.
Ebel R. The practical validation of tests of ability. Educational Measurement: Issues and Practice 1983; 2: 7–10.
Kane M. The validity of licensure examinations. American Psychologist 1982; 37: 911–8.
Arnold L, Willoughby TL. The quarterly profile examination. Academic Medicine 1990; 65: 515–6.
Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW Fifteen years of experience with progress-testing. Medical Teacher (in press).
Van Berkel HJM, Nuy HJP, Geerligs T. The influence of progress tests and block tests on study behavior. Instructional Science 1995; 22: 315–31.
Cohen-Schotanus J. Effecten van curriculumveranderingen [dissertation, with English summary]. Groningen: University of Groningen, 1994.
Van der Vleuten CPM, Verwijnen GM. A system for student assessment. In: Van der Vleuten CPM, Wijnen WHFW, editors. Problem-based learning: Perspectives from the Maastricht experience. Amsterdam: Thesis-publ., 1990: 27–49.
Klass D, Clauser B, Fletcher E et al. Progress in developing a standardized patient test of clinical skills at the National Board of Medical Examiners: prototype two. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 324–6.
Reznick RK, Smee S, Baumber JS et al. Guidelines for estimating the real cost of an Objective Structured Clinical Examination. Academic Medicine 1993; 68: 513–7.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Van Der Vleuten, C.P.M. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ 1, 41–67 (1996). https://doi.org/10.1007/BF00596229
Issue Date:
DOI: https://doi.org/10.1007/BF00596229