↗

Crash: how computers are setting us up for disaster

The headline is slightly over-the-top. But this is nevertheless a fascinating long read on the paradox of automation — how our reliance on computers leaves us incompetent to act when we are needed the most.

First, automatic systems accommodate incompetence by being easy to operate and by automatically correcting mistakes. Because of this, an inexpert operator can function for a long time before his lack of skill becomes apparent – his incompetence is a hidden weakness that can persist almost indefinitely. Second, even if operators are expert, automatic systems erode their skills by removing the need for practice. Third, automatic systems tend to fail either in unusual situations or in ways that produce unusual situations, requiring a particularly skilful response.