Skip to main content
Log in

The assessment of professional competence: Developments, research and practical implications

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Ebel RL, Frisbie DA. Essentials of educational measurement. Englewood Cliffs, New Jersey: Prentice-Hall, 1986.

    Google Scholar 

  2. Ory JC, Ryan KE. Tips for improving testing and grading. Newbury Park, California: Sage Publications, 1993.

    Google Scholar 

  3. Pickering C. Against multiple-choice questions. Medical Teacher 1979; 1: 84–6.

    Google Scholar 

  4. Newble DI, Baxter, A, Elmslie G. A comparison of multiple choice and free response tests in examinations of clinical competence. Medical Education 1979; 13: 263–8.

    Google Scholar 

  5. McGuire C. Perspectives in assessment. Academic Medicine 1993; 68(Supplement): S3–8.

    Google Scholar 

  6. Case SM, Swanson DB. Extended-matching items: A practical alternative to free-response questions. Teaching and Learning in Medicine 1993; 5: 107–15.

    Google Scholar 

  7. Schuwirth LWT, van der Vleuten CPM, Donkers HHLM. Open ended questions versus multiple choice questions: An analysis of cuing effects. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 486–91.

    Google Scholar 

  8. Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.

    Google Scholar 

  9. Van der Vleuten CPM, Newble DI. How can clinical reasoning be tested? The Lancet 1995; 345: 1032–4.

    Google Scholar 

  10. McGuire CH, Babnott D. Simulation technique in the measurement of problem solving skills. Journal of Educational Measurement 1967; 4: 1–10.

    Google Scholar 

  11. Hodgkin K, Knox JDE. Problem centered learning: The modified essay question in medical education. Edinburgh: Churchill-Livingstone, 1975.

    Google Scholar 

  12. Feletti GI, Saunders NA, Smith AJ. Comprehensive assessment of final-year medical student performance based on undergraduate programme objectives. Lancet 1983; 2(8340): 34–7.

    Google Scholar 

  13. Barrows HS, Tamblyn RM. The portable patient problem pack (P4). A problem-based learning unit. Journal of Medical Education 1977; 52: 1002–4.

    Google Scholar 

  14. Williams RG, Vu NV, Barrows HG, Verhulst S. Profile of the Clinical Reasoning Test (CRT): An objective measure of problem solving skills and proficiency in using medical knowledge. In: Schmidt HG, De Volder ML, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 81–90.

    Google Scholar 

  15. Norcini JJ, Meskauskas JA, Langdon LO, Webster GD. An evaluation of a computer simulation in the assessment of physician competence. Evaluation in the Health Professions 1986; 9: 286–304.

    Google Scholar 

  16. Bligh TJ. Written simulation scoring: Comparison of nine systems [dissertation]. Urbana-Champaign (IL): University of Illinois, 1980.

    Google Scholar 

  17. Swanson D, Norcini J, Grosso L. Assessment of clinical competence: Written and computer-based simulations. Assessment and Evaluation in Higher Education 1987; 12: 220–46.

    Google Scholar 

  18. Norman G, Bordage G, Curry L et al. A review of recent innovations in assessment. In: Wakeford RE, editor. Directions in clinical assessment. Report of the First Cambridge Conference. Cambridge: Cambridge University School of Clinical Medicine, 1985: 9–27.

    Google Scholar 

  19. Elstein A, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge Massachusetts: Harvard University Press.

  20. Friedman R, Korst D, Schultz J, Beatty E, Entine S. Experience with the simulated patient physician encounter. Journal of Medical Education 1978; 53: 825–30.

    Google Scholar 

  21. McLeskey C, Ward R. Validity of written examinations. Anesthesiology 1978; 49: 224.

    Google Scholar 

  22. Marshall J. Assessment of problem-solving ability. Medical Education 1977; 11: 329–34.

    Google Scholar 

  23. Newble DI, Hoare J, Baxter A. Patient management problems: Issues of validity. Medical Education 1982; 16: 137–42.

    Google Scholar 

  24. Norman GR, Feightner JW. A comparison of behaviour on simulated patients and patient management problems. Journal of Medical Education 1981; 55: 529–37.

    Google Scholar 

  25. Norcini JJ, Swanson DB, Grosso LJ, Shea JA, Webster Gd. Reliability, validity and efficiency of multiple choice questions and patient management problem item formats in the assessment of physician competence. Medical Education 1985; 19: 238–47.

    Google Scholar 

  26. Bordage G, Page G. An alternative approach to PMPs: The “key features” concept. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Heal-Publications, 1987: 59–75.

    Google Scholar 

  27. De Graaff E, Post G, Drop M. Validation of a new measure of clinical problem-solving. Medical Education 1987; 21: 213–218.

    Google Scholar 

  28. Powles ACP, Wintrup N, Neufeld VR, Wakefield JH, Coates G, Burrows J. The triple jump exercise: further studies of an evaluative technique. Proceedings of the 20th Annual Conference on Research in Medical Education, Washington: American Association of Medical Colleges, 1981: 74–9.

    Google Scholar 

  29. Fiedman CP, Murphy GC, Smith AC, Mattern WD. Exploratory study of an examination format for problem-based learning. Teaching and Learning in Medicine 1994; 6: 194–8.

    Google Scholar 

  30. Boud D. The role of self-assessment in student grading. Assessment and Evaluation in Higher Education 1989; 14: 20–30.

    Google Scholar 

  31. De Grave W De Volder M. Peer-evaluation and problem-based learning. In: Schmidt H, De Volder M, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 116–22.

    Google Scholar 

  32. Magzoub M. Studies in community-based education [dissertation]. Masstricht: University of Limburg, 1994.

    Google Scholar 

  33. Case SM, Swanson DB, van der Vleuten CPM. Student assessment in problem-based learning curricula. In: Boud D, Feletti G, editors. The challenge of problem-based learning. London: Kogan Page, 1991: 260–73.

    Google Scholar 

  34. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 1991; 66: 762–9.

    Google Scholar 

  35. Blake JM, Norman GR, Smith EKM. Report card from McMaster: Student evaluation at a problem-based medical school. Lancet 1995; 345: 899–902.

    Google Scholar 

  36. Harden R, Gleeson F. Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education 1979; 13: 41–54.

    Google Scholar 

  37. Rothman AI, Cohen R. (Editors). Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 292–321.

    Google Scholar 

  38. Reznick R, Blackmore DE, Cohen R et al. An objective structured clinical examination for the licentiate of the Medical Council of Canada: From research to reality. Academic Medicine 1993; 68(Supplement): S4–6.

    Google Scholar 

  39. Van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine 1990; 2: 58–76.

    Google Scholar 

  40. Vu NV, Barrows HS. Use of standardized patients in clinical assessments: recent developments and measurement findings. Educational Researcher 1994; 23: 23–30.

    Google Scholar 

  41. Swanson DB, Norman GR, Linn RL. Performance-based assessment: Lessons from the health professions. Educational Researcher 1995; 24: 5–22, 35.

    Google Scholar 

  42. Van Thiel J, Kraan HF, Van der Vleuten CPM. Reliability and feasibility of measuring interviewing skills using the revised Maastricht History Taking and Advice Checklist. Medical Education 1991; 25: 224–9.

    Google Scholar 

  43. Swanson Db. A measurement framework for performance-based tests. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 13–45.

    Google Scholar 

  44. Stalenhoef-Halling BF, Van der Vleuten CPM, Jaspers TAM, Fiolet JFBM. The feasibility, acceptability and reliability of open-ended questions in a problem-based learning curriculum. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 552–7.

    Google Scholar 

  45. Erviti V, Templeton B, Bunce J, Burg F. The relationships of pediatric resident recording behavior across medical conditions. Medical Care 1980; 18: 1020–31.

    Google Scholar 

  46. Rethans JJ, Sturmans F, Drop MJ, Van der Vleuten CPM. Assessment of performance in actual practice of general practitioners by use of standardized patients. British Journal of General Practice 1991; 41: 97–9.

    Google Scholar 

  47. Shavelson RJ, Baxter GP, Gao X. Sampling variability of performance assessments. Journal of Educational Measurement 1993; 30: 215–32.

    Google Scholar 

  48. Klein as cited in: Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.

    Google Scholar 

  49. Shavelson RJ, Mayberry P, Li W, Webb NM. Generalizability of military performance measurements: Marine Corps rifleman. Military Psychology 1990; 2: 129–44.

    Google Scholar 

  50. Frijns PRAM, Van der Vleuten CPM, Verwijnen GM, Van Leeuwen YD. The effect of structure in scoring methods on the reproducibility of tests using open-ended questions. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 466–71

    Google Scholar 

  51. Van Ham I, Gerritsma J. The assessment of clinical competence in general practice with chart stimulated recall. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk, 1990: 306–9.

    Google Scholar 

  52. Van der Vleuten CPM, Norman GR, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of reliability. Medical Education 1991; 25: 110–8.

    Google Scholar 

  53. Streiner DL. Global rating scales. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer, 1985: 119–41.

    Google Scholar 

  54. Streiner D. Clinical ratings - ward evaluation. In: Shannon S, Norman G, editors. Evaluation methods: A resource handbook. Hamilton: The Program for Educational Development, McMaster University, 1995: 29–31.

    Google Scholar 

  55. Hastorf AH, Schneider DJ, Polefka J. Person perception. Reading, Massachusetts: Addison-Wesley, 1970.

    Google Scholar 

  56. Ross M. Relation of implicit theories to the construction of personal histories. Psychological Review 1989; 96: 341–57.

    Google Scholar 

  57. Norman GR, Swanson DB, Case SM. Conceptual and methodological issues in studies comparing assessment formats. Teaching and Learning (in press).

  58. Norman GR, Smith E, Powles A, Rooney P, Henry N, Dodd P. Factors underlying performance on written tests of knowledge. Medical Education 1987; 21: 297–304.

    Google Scholar 

  59. Jean P, Schuwirth L, Van Santen M, Van der Vleuten C. Do problem analysis questions (PAQs) and true/false questions (TFQs) measure different skills? Medical Education (in press).

  60. Maatsch J, Huang R. An evaluation of the construct validity of four alternative theories of clinical competence. Proceedings of the Twenty-fifth Annual Conference on Research in Medical Education, American Association of Medical Colleges. Washington, DC, 1986.

    Google Scholar 

  61. Maatsch J. Model for a criterion-referenced medical specialty test. Final Report Grant No. HS-02038-02, Office of Medical Education Research and Development Michigan State University, 1980.

  62. Van der Vleuten CPM, Van Luijk S, Beckers HJM. A written test as an alternative to performance testing. Medical Education 1989; 23: 97–107.

    Google Scholar 

  63. Ward W. A comparison of free-response and multiple choice forms of verbal aptitude tests. Applied Psychological Measurement 1982; 6: 1–11.

    Google Scholar 

  64. Thissen D, Wainer H, Wang X. Are tests comprising both multiple-choice and free-response items necessarily more unidimensional than multiple-choice tests? An analysis of two tests. Journal of Educational Measurement 1994; 31: 113–23.

    Google Scholar 

  65. Ramsey PG, Carline JD, Inui YS et al. Predictive validity of certification by the American Board of Internal Medicine. Annals of Internal Medicine 1989; 110: 719–26

    Google Scholar 

  66. Solomon et al. 1990 as cited in Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.

    Google Scholar 

  67. Norman GR, Davis DA, Painvin A, Rath D, Ragbeer M. Comprehensive assessment of clinical competence of family-general physicians using multiple measures. Proceedings 28th Conference on Research in Medical Education. Washington: American Association of Medical Colleges, 1989.

    Google Scholar 

  68. Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.

    Google Scholar 

  69. McGuire C. Written methods for assessing clinical competence. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 44–58.

    Google Scholar 

  70. Hager P, Gonczi A, Athanasou J. General issues about assessment of competence. Assessment & Evaluation in Higher Education 1994; 19: 3–16.

    Google Scholar 

  71. Norman GR, Van der Vleuten CPM, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of validity, efficiency and acceptability. Medical Education 1991; 25: 119–26.

    Google Scholar 

  72. Van Thiel J, van der Vleuten CPM, Kraan H. Assessment of medical interviewing skills: Generalizability of scores using successive MAAS-versions. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 536–540.

    Google Scholar 

  73. Newble D, Jaeger K. The effect of assessments and examinations on the learning of medical students. Medical Education 1983; 17: 165–71.

    Google Scholar 

  74. Popham WJ. Measurement as an instructional catalyst. In: Ekstrom RB, editor. Measurement, technology and individuality in education. San Francisco: Jossey-Bass, 1983: 87–103.

    Google Scholar 

  75. Frederiksen N. The real test bias: Influences of testing on teaching and learning. American Psychologist 1984; 39: 193–202.

    Google Scholar 

  76. Entwistle N. Styles of learning and teaching. Chichester: John Wiley & Sons, 1981.

    Google Scholar 

  77. Stillman P, Swanson D. Ensuring the clinical competence of medical school graduates through standardized patients. Archives of Internal Medicine 1987; 147: 1049–52.

    Google Scholar 

  78. Gibbs G. Improving the quality of student learning. Bristol: Technical & Educational Services, 1992.

    Google Scholar 

  79. Shepard LA. Psychometrician's beliefs about learning. Educational Researcher 1991; 20: 2–16.

    Google Scholar 

  80. Snyder BR. The hidden curriculum. New York: Knopf, 1971.

    Google Scholar 

  81. Popham WJ, Cruse KL, Rankin SC, Sandifer PD, Williams PL. Measurement-driven instruction: it's on the road. Phi Delta Kappan 1985; 66: 628–34.

    Google Scholar 

  82. Semb GB, Ellis JA. Knowledge taught in school: What is remembered? Review of Educational Research 1994; 64: 253–86.

    Google Scholar 

  83. Van Luijk SJ, Van der Vleuten CPM, Schelven RM. The relation between content and psychometric characteristics in performance-based testing. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publications, 1990: 202–7.

    Google Scholar 

  84. Mischel W. Personality and assessment. New York: John Wiley, 1968.

    Google Scholar 

  85. Schmidt H, Norman G, Boshuizen HA. Cognitive perspective on medical expertise: Theory and implications. Academic Medicine 1990; 65: 611–21.

    Google Scholar 

  86. Norman G, Allery L, Berkson L et al. Research in the psychology of clinical reasoning: implications for assessment. Paper from the Fourth Cambridge Conference, Cambridge University School of Clinical Medicine, Cambridge, 1989.

    Google Scholar 

  87. Higgs J, Jones M (Editors). Clinical reasoning in the health professions. Oxford: Butterworth/Heinemann, 1995.

    Google Scholar 

  88. Norman G, Regehr G. Contemporary issues in cognitive psychology: implications for professional education. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 17–25.

    Google Scholar 

  89. Rethans JJ, Sturmans F, Drop MJ et al. Does competence of general practitioners predict their performance. British Medical Journal 1991; 303: 1377–80.

    Google Scholar 

  90. Norman GR. Reliability and construct validity of some cognitive measures of clinical reasoning. Teaching and Learning in Medicine 1989; 1: 194–9.

    Google Scholar 

  91. Newble DI, Raymond GA. The Pattern Completion Item (PCI): A potential measure of clinical problem-solving skills. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 191–2.

    Google Scholar 

  92. Nelson MS, Clayton BL, Moreno R. How medical school faculty regard educational research and make pedagogical decisions 1990; 65: 122–6.

    Google Scholar 

  93. Hays RB, Fabb WE, Van der Vleuten CPM. Reliability of the fellowship examination of the Royal Australian College of General Practitioners. Teaching and Learning in Medicine 1995; 7: 43–50.

    Google Scholar 

  94. Van Luijk SJ, Van der Vleuten CPM. A comparison of checklists and rating scales in performance-based testing. In: Hart IR, Harden RM, Des Marchais J, editors. Current developments in assessing clinical competence. Montreal: Can-Heal, 1992: 357–62.

    Google Scholar 

  95. Newble D, Dawson B, Dauphinee D et al. Guidelines for assessing clinical competence. Teaching and Learning in Medicine 1994; 6: 213–20.

    Google Scholar 

  96. Endler NS, Magnusson D, editors. Interfactional psychology and personality. Washington DC: Hemisphere, 1976.

    Google Scholar 

  97. Vand er Vleuten CPM, Schuwirth LWT, Ronteltap CFM. A cognitive psychological interpretation of a few remarkable psychometric findings. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 506–8.

    Google Scholar 

  98. Frijns PHAM. Scoringsmodellen voor open-vraag vormen (Scoring models for free-response formats) [dissertation]. Maastricht: University of Limburg, 1992.

    Google Scholar 

  99. Brailovsky C, Bordage G, Carretier H, Page G. Content validity of the key features' approach of the Medical Council of Canada's Exam. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 476–7.

    Google Scholar 

  100. Bordage G, Brailovsky C, Carretier H, Page G. Content validation of key features on a national examination of clinical decision-making skills. Academic Medicine 1995; 70: 276–81.

    Google Scholar 

  101. Ebel RL. Must all tests be valid? American Psychologist 1961; 16: 640–7.

    Google Scholar 

  102. Ebel R. Measuring Educational Achievement. Englewood Cliffs: Prentice-Hall Inc, 1965.

    Google Scholar 

  103. Ebel R. The practical validation of tests of ability. Educational Measurement: Issues and Practice 1983; 2: 7–10.

    Google Scholar 

  104. Kane M. The validity of licensure examinations. American Psychologist 1982; 37: 911–8.

    Google Scholar 

  105. Arnold L, Willoughby TL. The quarterly profile examination. Academic Medicine 1990; 65: 515–6.

    Google Scholar 

  106. Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW Fifteen years of experience with progress-testing. Medical Teacher (in press).

  107. Van Berkel HJM, Nuy HJP, Geerligs T. The influence of progress tests and block tests on study behavior. Instructional Science 1995; 22: 315–31.

    Google Scholar 

  108. Cohen-Schotanus J. Effecten van curriculumveranderingen [dissertation, with English summary]. Groningen: University of Groningen, 1994.

    Google Scholar 

  109. Van der Vleuten CPM, Verwijnen GM. A system for student assessment. In: Van der Vleuten CPM, Wijnen WHFW, editors. Problem-based learning: Perspectives from the Maastricht experience. Amsterdam: Thesis-publ., 1990: 27–49.

    Google Scholar 

  110. Klass D, Clauser B, Fletcher E et al. Progress in developing a standardized patient test of clinical skills at the National Board of Medical Examiners: prototype two. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 324–6.

    Google Scholar 

  111. Reznick RK, Smee S, Baumber JS et al. Guidelines for estimating the real cost of an Objective Structured Clinical Examination. Academic Medicine 1993; 68: 513–7.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Van Der Vleuten, C.P.M. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ 1, 41–67 (1996). https://doi.org/10.1007/BF00596229

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00596229

Keywords

Navigation