Teaching Ethics or Testing It? A Systematic Review of the Methodology Behind Ethical Decision-Making Research
Faculty Mentor Name
Dr. Carolynn Kohn
Research or Creativity Area
Social Sciences
Abstract
Three undergraduate students systematically reviewed 46 articles from a larger literature review of 367 articles on ethical decision-making. These 46 articles were selected if they included at least one manipulated variable. Fourteen codes were developed and operationally defined, organized around analytical categories: study design, single-subject or group designs, as well as individual versus aggregate result analysis; instructional approach, reflecting the specific design type, use of random assignment, and teaching or implementation model employed; scenario usage, encompassing vignette use, method of vignette distribution, and whether scenarios included solution-based prompting; and data representation, including tables, bar graphs, scatter plots, and other formats. Each coder coded each study separately before comparing agreement across coders.
Although most of the studies used scenarios, very few of them did so with the aim of teaching ethical decision-making skills, and even fewer used a decision-making model. Many of these studies also failed to consider how ethical decision-making skills can be generalized across multiple environments. Future research should consider conducting experimental studies that compare different instructional designs to evaluate their effectiveness in promoting ethical decision-making across various professional contexts.
Purpose
Numerous studies examine correlates of ethical decision-making among professionals and students across various fields. These studies employ a range of methods, including but not limited to: self-report, instructional design, and gamification. However, a recent literature review by our research team found that few of these studies focus on specific behaviors. Of these, fewer employ a design aimed at teaching participants how to make ethical decisions or utilize decision-making models. The purposes of this study were to (1) explore the reliability of coding various aspects of studies on ethical decision-making as well as (2) examine the frequency with which researchers incorporated (a) teaching skills to participants, (b) vignettes or scenarios, (c) ethical decision-making models, and (d) assessments of whether the skills generalized across different environments.
Results
Two articles included three or more studies. A consensus was reached to separate distinct studies within these articles, increasing the total sample from 46 to 52 studies. Once all 52 studies were coded, inter-rater reliability (IRR) was assessed by calculating the percentage agreement (number of agreements / total codes * 100). IRR was 84.5% (615/728*100). This IRR suggests there was a high level of consistency and shared understanding between coders, without which we would not be able to interpret the rest of the data. Discrepancies (15.5%) primarily occurred when coding the number of participants within groups, the use of random assignment, and whether scenarios were designed to guide participants toward resolving ethical dilemmas.
Among the 46 articles (and 52 studies) we found, the majority of studies used group designs (74.4%) over single-subject designs (25.6%), and two-thirds utilized random assignment (66%). Most studies were observational in nature (77.8%), with only 22.2% employing an instructional approach. Similarly, only 11.1% utilized a specific behavioral decision-making model or teaching guide for participants to follow. Regarding scenario usage, 89.1% of studies involved the experimenter presenting vignettes directly to participants.
Teaching Ethics or Testing It? A Systematic Review of the Methodology Behind Ethical Decision-Making Research
Three undergraduate students systematically reviewed 46 articles from a larger literature review of 367 articles on ethical decision-making. These 46 articles were selected if they included at least one manipulated variable. Fourteen codes were developed and operationally defined, organized around analytical categories: study design, single-subject or group designs, as well as individual versus aggregate result analysis; instructional approach, reflecting the specific design type, use of random assignment, and teaching or implementation model employed; scenario usage, encompassing vignette use, method of vignette distribution, and whether scenarios included solution-based prompting; and data representation, including tables, bar graphs, scatter plots, and other formats. Each coder coded each study separately before comparing agreement across coders.
Although most of the studies used scenarios, very few of them did so with the aim of teaching ethical decision-making skills, and even fewer used a decision-making model. Many of these studies also failed to consider how ethical decision-making skills can be generalized across multiple environments. Future research should consider conducting experimental studies that compare different instructional designs to evaluate their effectiveness in promoting ethical decision-making across various professional contexts.
