Mechanizing modal psychology

Machines are becoming more capable of substantively interacting with human beings as part of simple dyads and within the confines of our complex social structures. Thought must be given to how their behavior might be regulated with respect to the norms and conventions by which we live. This is certainly true for the military domain [1], but is no less true for eldercare, health care, disaster relief and law enforcement; all areas where robotic systems are poised to make tremendous impact in the near future. But how should we inculcate sensitivity to normative considerations in the next generation of intelligent systems? I argue here for an approach to building moral machines grounded in cognitive architectural considerations, and specifically in the dynamics of how alternatives are represented and reasoned over. After examining some recent results in the empirical literature on human moral judgment, I suggest some desiderata for knowledge representation and reasoning tools that may offer the means to capture some of the foundations of human moral cognition.