Every time a person dies, writes Russian novelist Vasily Grossman in Life and Fate, the entire world that has been built in that individual’s consciousness dies as well: “The stars have disappeared from the night sky; the Milky Way has vanished; the sun has gone out… flowers have lost their color and fragrance; bread has vanished; water has vanished.” Elsewhere in the book, he writes that one day we may engineer a machine that can have human-like experiences; but if we do, it will have to be enormous—so vast is this space of consciousness, even within the most “average, inconspicuous human being.”
And, he adds, “Fascism annihilated tens of millions of people.” Trying to think those two thoughts together is a near-impossible feat, even for the immense capacities of our consciousness. But will machine minds ever acquire anything like our ability to have such thoughts, in all their seriousness and depth? Or to reflect morally on events, or to equal our artistic and imaginative reach? Some think that this question distracts us from a more urgent one: we should be asking what our close relationship with our machines is doing to us.
Jaron Lanier, himself a pioneer of computer technology, warns in You Are Not a Gadget that we are allowing ourselves to become ever more algorithmic and quantifiable, because this makes us easier for computers to deal with. Education, for example, becomes less about the unfolding of humanity, which cannot be measured in units, and more about tick boxes.
