Euler CenterWork Composition Run the assessment
Library / Concept

The moral crumple zone

When an automated system fails, blame lands on the nearest human, who often had no real control.

TypeConcept: an idea the canvas inherits
Cells it touchesAuthority · Evidence · Mastery
Sources, and how strong they are
Primary source

Madeleine Clare Elish, Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction, Engaging Science, Technology, and Society, volume 5

2019

Madeleine Clare Elish coined the term in 2019 for the way responsibility for an automated system's failure gets attributed to the human operator closest to it, whose actual control over the system's behavior may be small or none. The name borrows from the part of a car that absorbs the impact so the rest of the vehicle survives.

On the canvas it is the Authority cell scored solid because a name exists, on top of an Evidence cell that cannot show what that person could have seen, and a Mastery cell that never checked whether they were qualified to intervene. The canvas lists it as an anti-pattern because it is what human oversight becomes when it is asserted in prose and never drilled.

Where it appears on the canvas Related entries
Correct this entryCorrections are made in public and credited by name.