Skip to main content
SpringerLink
Log in

Moral reasoning and automatic risk reaction during driving

  • Original Research
  • Published:
Cognition, Technology & Work Aims and scope Submit manuscript

Abstract

Recent advances in autonomous vehicles promise to revolutionize the transportation system. This perspective has led to new research on a number of open questions, such as how the self-driving system should behave in unavoidable crash situations. Our study aims to contribute to this investigation. In most ongoing research, this question is presented as a moral dilemma, drawing on established research on the trolley dilemma. However, more recent studies have shifted the focus from morality to risk analysis. We investigated the dual contribution of moral judgment and risk analysis in subjects facing dangerous situations. To this end, we use virtual reality to recreate a driving situation that allows subjects to immerse themselves in the road environment. Our results show a strong dissociation between quick decisions, when risk analysis seems to suggest the best choice, and conscious decisions, when proper moral judgment is at stake.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Alaieri F, Vellino A (2016) Ethical decision making in robots: Autonomy, trust and responsibility. In: Agah A, Cabibihan JJ, Howard AM, Salichs MA, He H (eds) Social robotics. Springer, Cham, pp 159–168

  • Allen C, Smit I, Wallach W (2005) Artificial morality: top-down, bottom-up, and hybrid approaches. Ethics Inf Technol 7(3):149–155

    Article  Google Scholar 

  • Appiah KA (2008) Experiments in ethics. Harvard University Press, Cambridge

    Google Scholar 

  • Awad E, Dsouza S, Kim R, Schulz J, Henrich J, Shariff A, Rahwan I (2018) The moral machine experiment. Nature, 563(7729):59–64

  • Bartels DM (2008) Principled moral sentiment and the flexibility of moral judgment and decision making. Cognition 108:381–417

    Article  Google Scholar 

  • Bengtsson, P. (2018). Attuning the pedestrian-vehicle and driver-vehicle: why attributing a mind to a vehicle matters. In: W. Karwowski, T. Ahram (Eds) International conference on intelligent human systems integration. Springer, Cham, pp 19–22

  • Bergmann LT, Schlicht L, Meixner C, König P, Pipa G, Boshammer S, Stephan A (2018) Autonomous vehicles require socio-political acceptance an empirical and philosophical perspective on the problem of moral decision making. Front Behav Neurosci 12:31

  • Bjorndahl A, London AJ, Zollman KJ (2017) Kantian decision making under uncertainty: dignity, price, and consistency. Philosopher's Imprint 17(7)

  • Bojarski M, Testa DD, Dworakowski D, Firner B, Flepp B, Goyal P, Zhao J (2016) End-to-end learning for self-driving cars. arXiv abs/1604.07316

  • Bojarski M, YeresP, Choromanaska A, Choromanski K, Firner B, Jackel L, Muller U (2017) Explaining how a deep neural network trained with end-to-end learning steers a car. arXivabs/1704.07911

  • Bonnefon JF, Shariff A, Rahwan I (2016) The social dilemma of autonomous vehicles. Science 352(6293):1573–1576

    Article  Google Scholar 

  • Chen LC, Schwing A, Yuille A, Urtasun R (2015) Learning deep structured models. In International Conference on Machine Learning (pp. 1785–1794). PMLR

  • Crisman JD, Thorpe CE (1993) SCARF: a color vision system that tracks roads and intersections. IEEE Trans Robot Autom 9:49–58

    Article  Google Scholar 

  • Davnall R (2019) Solving the single-vehicle self-driving car trolley problem using risk theory and vehicle dynamics. Sci Eng Ethics 26:1–19

    Google Scholar 

  • Edmonds D (2014) Would you kill the fat man? The trolley problem and what your answer tells us about right and wrong. Princeton University Press, Princeton

    Google Scholar 

  • Faulhaber AK, Dittmer A, Blind F, Wächter MA, Timm S, Sütfeld LR, König P (2019) Human decisions in moral dilemmas are largely described by utilitarianism: virtual car driving study provides guidelines for autonomous driving vehicles. Sci Eng Ethics 25(2):399–418

    Article  Google Scholar 

  • Fayyad J, Jaradat MA, Gruyer D, Najjaran H (2020) Deep learning sensor fusion for autonomous vehicle perception and localization: a review. Sensors 20(15):4220

    Article  Google Scholar 

  • Foot P (1967) The problem of abortion. Oxford Rev 5:5–15

    Google Scholar 

  • Frank DA, Chrysochou P, Mitkidis P, Ariely D (2019) Human decision-making biases in the moral dilemmas of autonomous vehicles. Sci Rep 9(1):1–19

    Article  Google Scholar 

  • Gold N, Pulford BD, Colman AM (2014) The outlandish, the realistic, and the real: contextual manipulation and agent role effects in trolley problems. Front Psychol 5:35

    Article  Google Scholar 

  • Grasso GM, Lucifora C, Perconti P, Plebe A (2020) Integrating human acceptable morality in autonomous vehicles. In: Ahram T, Karwowski W, Vergnano A, Leali F, Taiar R (eds) Intelligent human systems integration 2020. IHSI 2020. Advances in intelligent systems and computing. Springer, Cham. https://doi.org/10.1007/978-3-030-39512-4_7

    Chapter  Google Scholar 

  • Greene JD, Sommerville RB, Nystrom LE, Darley JM, Cohen JD (2001) fMRI investigation of emotional engagement in moral judgment. Science 293:2105–2108

    Article  Google Scholar 

  • Greene JD, Nystrom LE, Engell AD, Darley JM, Cohen JD (2004) The neural bases of cognitive conflict and control in moral judgment. Neuron 44(2):389–400

    Article  Google Scholar 

  • Gross D, Ehlers W, Wriggers P, Schröder J, Müller R (2017) Statics-formulas and problems. Springer, Berlin

    Book  MATH  Google Scholar 

  • Hansson SO (2012) A panorama of the philosophy of risk. In Handbook of risk theory. Springer, Dordrecht, pp 27–54

  • Himmelreich J (2018) Never mind the trolley: The ethics of autonomous vehicles in mundane situations. In: «Ethical Theory & Moral Practice» vol. XXI, n. 3:669–684

  • Jorge CC, Rossetti RJF (2018) On social interactions and the emergence of autonomous vehicles. In International conference on vehicle technology and intelligent transport systems (pp. 423–430)

  • Kagan S (1989) The limits of morality

  • Kahneman D (2011) Thinking, fast and slow, Macmillan, Basingstoke

  • Kallioinen N, Pershina M, Zeiser J, Nosrat Nezami F, Pipa G, Stephan A, König P (2019) Moral judgements on the actions of self-driving carsand human drivers in dilemma situations from different perspectives. Front Psychol 10:2415

  • Kant I (1785) The categorical imperative

  • Keeling G (2020) Why trolley problems mater for the ethics of automated vehicles. In: «Science and Engineering Ethics», vol. XXVI, n. 1:293–307

  • Kastrinaki V, Zervakis M, Kalaitzakis K (2003) A survey of video processing techniques for traffic applications. Image Vis Comput 21:359–381

    Article  Google Scholar 

  • Koenings M, Young L, Adolphs R, Tranel D, Cushman F, Hauser M, Damasio A (2007) Damage to the prefrontal cortex increases utilitarian moral judgments. Nature 446:908–911

    Article  Google Scholar 

  • Kuutti S, Fallah S, Bowden R, Barber P (2019) Deep learning for autonomous vehicle control—algorithms, state-of-the-art, and future prospects. Synth Lect Adv Automot Technol 3:1–80

    Google Scholar 

  • LaViola JJ Jr (2000) A discussion of cybersickness in virtual environments. ACM Sigchi Bull 32(1):47–56

    Article  Google Scholar 

  • Li S, Zhang J, Li P, Wang Y, Wang Q (2019) Influencing factors of driving decision-making under the moral dilemma. IEEE Access 7:104132–104142

    Article  Google Scholar 

  • Lieberman MD (2007) Social cognitive neuroscience: a review of core processes. Annu Rev Psychol 58:259–289

    Article  Google Scholar 

  • Lucifora C, Grasso GM, Perconti P, Plebe A (2020) Moral dilemmas in self-driving cars. Rivista internazionale di Filosofia e Psicologia 11(2):238–250

    Google Scholar 

  • Marchand R (1992) The designer go to the fair II: Norman Bel Geddes, the general motors “futurama” and the visit to the factory transformed. Design Issues 8(2):23–40

    Article  Google Scholar 

  • Masaki I (ed) (1992) Vision-based vehicle guidance. Springer-Verlag, Berlin

    MATH  Google Scholar 

  • McCall JC, Trivedi MM (2006) Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans Intell Transp Syst 7:20–37

    Article  Google Scholar 

  • McCubbins MD, Turner M (2012) Going cognitive: tools for rebuilding the social sciences. In: Sun R (ed) Grounding social sciences in cognitive sciences. MIT Press, Cambridge, pp 387–414

    Google Scholar 

  • Meder B, Fleischhut N, Krumnau NC, Waldmann MR (2019) How should autonomous cars drive? A preference for defaults in moral judgments under risk and uncertainty. Risk Anal 39(2):295–314

    Article  Google Scholar 

  • Mikhail J (2007) Universal moral grammar: theory, evidence and the future. Trends Cogn Sci 11:143–152

    Article  Google Scholar 

  • Nyholm S, Smids J (2016) The ethics of accident-algorithms for self-driving cars: An applied trolley problem?. In: «Ethical Theory & Moral Practice» vol. XIX, n. 5:1275–1289

  • Ohn-Bar E, Trivedi MM (2016) Looking at humans in the age of self-driving and highly automated vehicles. IEEE Trans Intell Transp Syst 29:90–104

    Article  Google Scholar 

  • Patil I, Cogoni C, Zangrando N, Chittaro L, Silani G (2014) Affective basis of judgment-behavior discrepancy in virtual experiences of moral dilemmas. Soc Neurosci 9(1):94–107

    Article  Google Scholar 

  • Pomerleau, D. A. (1989). ALVINN: An autonomous land vehicle in a neural network. In Ieee intelligent vehicles symposium (pp. 305–313)

  • Rhim J, Lee GB, Lee JH (2020) Human moral reasoning types in autonomous vehicle moral dilemma: a cross-cultural comparison of Korea andCanada. Comput Hum Behav 102:39–56

    Article  Google Scholar 

  • Riaz F, Niazi M (2017) Towards social autonomous vehicles: efficient collision avoidance scheme using Richardson’s arms race model. PLoS ONE 12:e0186103

    Article  Google Scholar 

  • Richards DC (2010) Relationship between speed and risk of fatal injury: pedestrians and car occupants. Department for Transport, London

    Google Scholar 

  • Rumiati RI, Humphreys GW (2015) Cognitive neuroscience goes social. Cortex 70:1–4

    Article  Google Scholar 

  • Samuel S, Yahoodik S, Yamani Y, Valluru K, Fisher DL (2020) Ethical decision making behind the wheel–a driving simulator study. Transport Res Interdisip Perspect 5:100147

    Google Scholar 

  • Schwarting W, Pierson A, Alonso-Mora J, Karaman S, Rus D (2019) Social behavior for autonomous vehicles. Proc Nat Acad Sci USA 116:24972–24978

    Article  MathSciNet  MATH  Google Scholar 

  • Stanovich KE, West RF (2000) Individual differences in reasoning: implications for the rationality debate? Behav Brain Sci 23:645–726

    Article  Google Scholar 

  • Thomson JJ (1976) Killing, let die, and the trolley problem. Monist An Int Quart J General Philisophical Inquiry 59:204–217

    Google Scholar 

  • Thomson JJ (1985) The trolley problem. Yale Law J 94:1395–1415

    Article  Google Scholar 

  • Turner M (2001) Cognitive dimensions of social science. Oxford University Press, Oxford

    Google Scholar 

  • Vanderhaegen F (2016) A rule-based support system for dissonance discovery and control applied to car driving. Expert Syst Appl 65:361–371

    Article  Google Scholar 

  • Vanderhaegen F (2021) Weak signal-oriented investigation of ethical dissonance applied to unsuccessful mobility experiences linked to human-machine interactions. Sci Eng Ethics 27(1):1–25

    Article  Google Scholar 

  • VanRullen R (2017) Perception science in the age of deep neural networks. Front Psychol 8:142

    Article  Google Scholar 

  • Winfield AF, Blum C, Liu W (2014) Towards an ethical robot: internal models, consequences and ethical action selection. In: Mistry M, Leonardis A, Witkowski M, Melhuish C (eds) Advances in autonomous robotic systems, TAROS 2014. Lecture Notes in Computer Science. Springer, Cham, pp. 85–96. https://doi.org/10.1007/978-3-319-10401-08

  • Wilson H, Theodorou A, Bryson JJ (2019). Slam the Brakes: Perceptions of Moral Decisions in Driving Dilemmas. In AISafety@ IJCAI

  • Wood A (2013) Humanity as end in itself. In: Scheffler S (ed) On what matters, vol 2. Oxford University Press, Oxford, pp 58–82

    Google Scholar 

  • Yokoi R, Nakayachi K (2020) Trust in autonomous cars: exploring the role of shared moral values, reasoning, and emotion in safety-critical decisions. Hum Factors. https://doi.org/10.1177/0018720820933041

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cognitive Science, University of Messina, Via Concezione 6/8, 98121, Messina, ME, Italy

    Chiara Lucifora, Giorgio Mario Grasso, Pietro Perconti & Alessio Plebe

Authors
  1. Chiara Lucifora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giorgio Mario Grasso
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pietro Perconti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alessio Plebe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chiara Lucifora.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lucifora, C., Grasso, G.M., Perconti, P. et al. Moral reasoning and automatic risk reaction during driving. Cogn Tech Work 23, 705–713 (2021). https://doi.org/10.1007/s10111-021-00675-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10111-021-00675-y

Keywords

Search

Navigation