Abstract
Assessment in ethics education faces a challenge. From the perspectives of teachers, students, and third-party evaluators like the Accreditation Board for Engineering and Technology and the National Institutes of Health, assessment of student performance is essential. Because of the complexity of ethical case analysis, however, it is difficult to formulate assessment criteria, and to recognize when students fulfill them. Improvement in students’ moral reasoning skills can serve as the focus of assessment. In previous work, Rosa Lynn Pinkus and Claire Gloeckner developed a novel instrument for assessing moral reasoning skills in bioengineering ethics. In this paper, we compare that approach to existing assessment techniques, and evaluate its validity and reliability. We find that it is sensitive to knowledge gain and that independent coders agree on how to apply it.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Accreditation Board of Engineering and Technology. (2006). Criteria for accrediting engineering programs.
Arras, John D. (1991). Getting down to cases: The revival of casuistry in bioethics. Journal of Medicine and Philosophy, 16, 29–51. doi:10.1093/jmp/16.1.29.
Beauchamp, T. L., & Childress, J. F. (2001). Principles of biomedical ethics (Vol. 5). New York, N.Y.: Oxford University Press.
Byrt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46, 423–429.
Campbell, D.T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. In N. L. Gage (Ed.), Handbook of research on teaching (pp. ix, 84). Chicago: Rand McNally.
Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22, 249–254.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.
Comunian, A. L. 2002. Structure of the Padua moral judgment scale: A study of young adults in seven countries. In 110th annual conference of the American Psychological Association. Chicago, IL.
Di Eugenio, B., & Glass, M. (2004). The Kappa statistic: A second look. Computational Linguistics, 30(1), 95–101. doi:10.1162/089120104773633402.
Fisher, F. T., & Peterson, P. L. (2001). A tool to measure adaptive expertise in biomedical engineering students. In American Society for Engineering Education. Mira Digital Publishing, Inc.
Gibbs, J. C., Widaman, K. F., & Colby, A. (1982). Social intelligence: Measuring the development of sociomoral reflection (pp. 191–211). New Jersey: Prentice-Hall.
Goldie, J., Schwartz, L., McConnachie, A., & Morrison, J. (2001). Impact of a new course on students’ potential behaviour on encountering ethical dilemmas. Medical Education, 35, 295–302.
Goldie, J., Schwartz, L., McConnachie, A., & Morrison, J. (2002). The impact of three years’ ethics teaching, in an integrated medical curriculum, on students’ proposed behaviour on meeting ethical dilemmas. Medical Education, 36, 489–497.
Goldin, I. M., Ashley, K. D., & Pinkus, R. L. (2006a). Teaching case analysis through framing: Prospects for an ITS in an ill-defined domain. In “Workshop on intelligent tutoring systems for ill-defined domains”, 8th international conference on intelligent tutoring systems. Jhongli, Taiwan.
Goldin, I. M., Ashley, K. D., & Pinkus, R. L. (2006b). Assessing case analyses in bioengineering ethics education: Reliability and training. In Proceedings of the 2006 International Conference on Engineering Education. San Juan, Puerto Rico.
Harris, C. E., Pritchard, M. S., & Rabins, M. J. (2000). Engineering ethics: Concepts and cases (Vol. 2). Belmont, CA: Wadsworth.
Hayes, R. P., Stoudemire, A., Kinlaw, K., Dell, M. L., & Loomis, A. (1999). Qualitative outcome assessment of a medical ethics program for clinical clerkships: A pilot study. General Hospital Psychiatry, 21, 284–295.
Hébert, P. C., Meslin, E. M., & Dunn, E. V. (1992). Measuring the ethical sensitivity of medical students: A study at the university of Toronto. Journal of Medical Ethics, 18, 142–147.
Jonsen, A. R., & Toulmin, S. E. (1988). The abuse of casuistry: A history of moral reasoning. Berkeley: University of California Press.
Kipnis, K. (2000). Medical ethics education in a problem-based learning curriculum. American Philosophical Association Newsletter on Philosophy and Medicine, 00(1), 128–132.
Kohlberg, L. (1981). The philosophy of moral development (Vol. 1). San Francisco: Harper & Row.
Kohlberg, L. (1984). The psychology of moral development (Vol. 2). San Francisco: Harper & Row.
Kuczewski, M. G. (1997). Fragmentation and consensus: Communitarian and casuist bioethics. Washington, D.C.: Georgetown University Press.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
Lind, G. (2000). Moral regression in medical students and their learning environment. Revista Brasileira de Educacao Médica, 24, 24–33.
Lynch, D. C., Surdyk, P. M., & Eiser, A. R. (2004). Assessing professionalism: A review of the literature. Medical Teacher, 26, 366–373.
Martin, T., Rayne, K., Kemp, N. J., Hart, J., & Diller, K. (2005). Teaching adaptive expertise in biomedical engineering ethics. Science and Engineering Ethics, 11, 257–276.
National Institutes of Health Office of Extramural Research. (2006). Frequently asked questions for the requirement for education on the protection of human subjects. May 9.
Olds, B. M., Moskal, B. M., & Miller, R. L. (2005). Assessment in engineering education: Evolution, approaches and future collaborations. Journal of Engineering Education, 94, 13–25.
Pence, G. E. (2000). Classic cases in medical ethics: Accounts of cases that have shaped medical ethics, with philosophical, legal, and historical backgrounds (Vol. 3). Boston: McGraw-Hill.
Pinkus, R. L. (1997). Engineering ethics: Balancing cost, schedule, and risk–lessons learned from the space shuttle. New York: Cambridge University Press.
Pinkus, R. L., Chi, M. T. H., McQuaide, J., Ashley, K. D., & Pollack, M. (1999). Some preliminary thoughts on reasoning with cases: A cognitive science approach. Symposium presented at the Association for Moral Education, November 20, Minneapolis, MN.
Pinkus, R. L., Gloeckner, C., & Fortunato, A. (2015). The role of professional knowledge in case-based reasoning in practical ethics. Science and Engineering Ethics. doi:10.1007/s11948-015-9645-1.
Rest, J. R., Narvaez, D., Bebeau, M. J., & Thoma, S. J. (1999). Postconventional moral thinking: A neo-Kohlbergian approach. New Jersey: Lawrence Erlbaum Associates.
Rudnicka, E. (2004). A review of instruments for measuring moral reasoning/values. In 10th international conference on industry, engineering, and management systems, Cocoa Beach, FL, pp 305–311.
Savulescu, J., Crisp, R., Fulford, K. W. M., & Hope, T. (1999). Evaluating ethics competence in medical education. Journal of Medical Ethics, 25, 367–374.
Shapiro, J., & Miller, R. (1994). How medical students think about ethical issues. Academic Medicine, 69, 591–593.
Shuman, L. J., Besterfield-Sacre, M., & McGourty, J. (2005). The ABET “professional skills”—Can they be taught? Can they be assessed? Journal of Engineering Education, 94, 41–55.
Sim, J., & Wright, C. C. (2005). The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy, 85, 257–268.
Sindelar, M., Shuman, L., Besterfield-Sacre, M., Miller, R., Mitcham, C., Olds, B., Pinkus, R. L., & Wolfe, H. (2003). Assessing engineering students’ abilities to resolve ethical dilemmas. In 33rd annual frontiers in education. Boulder, CO: IEEE.
Steneck, N. H. 2007. ORI introduction to the responsible conduct of research. August 3.
Venables, W. N., & Ripley, B. D. (2002). Modern applied statistics with S (4th ed.). New York: Springer.
Voss, J. F., & Post, T. A. (1988). On the solving of ill-structured problems. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 261–285). Hillsdale, NJ: Lawrence Erlbaum.
Weil, V. 1990. Engineering ethics in engineering education: Summary report. In conference at the Center for the Study of Ethics in the Professions at the Illinois Institute of Technology. Chicago.
Acknowledgments
The authors thank Christian Schunn, Janyce Wiebe, Diane Litman, and our reviewers for valuable feedback; Christos Theodoulou and Scott Nickleach working under the supervision of Dr. Alan Sampson for statistics consulting; Claire Gloeckner and Jessica DiFrancesco for coding and data entry; Mark Sindelar and Kaitlin Jones for coding, and Angela Fortunato for extensive assistance and advice. This work was supported by NSF Engineering and Computing Education Grant #0203307.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Goldin, I.M., Pinkus, R.L. & Ashley, K. Validity and Reliability of an Instrument for Assessing Case Analyses in Bioengineering Ethics Education. Sci Eng Ethics 21, 789–807 (2015). https://doi.org/10.1007/s11948-015-9644-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11948-015-9644-2