{"id":773,"date":"2023-02-25T22:50:58","date_gmt":"2023-02-26T03:50:58","guid":{"rendered":"http:\/\/thoughtsofstone.com\/?p=773"},"modified":"2023-02-26T03:23:21","modified_gmt":"2023-02-26T08:23:21","slug":"yudkowskys-golem","status":"publish","type":"post","link":"https:\/\/thoughtsofstone.com\/yudkowskys-golem\/","title":{"rendered":"YUDKOWSKY’S GOLEM"},"content":{"rendered":"

Advanced AI will be more dangerous than it seems, but (good news!) probably won’t be in position to snuff out humanity for another decade at least.<\/em><\/p>\n

<\/p>\n

Eliezer Yudkowsky is one of those people who, along with being hyper-intelligent, bears the modern secondary characteristics of hyper-intelligence. Asked how he\u2019s doing, he replies archly: \u201cwithin one standard deviation of my own peculiar little mean.\u201d He feels compelled, when talking, to digress down mazelike lanes and alleys of technical detail. He looks<\/em> like a geek. Above all, he has the kind of backstory (no high school, no college\u2014just homeschooled and self-taught) that conjures up the image of a lonely boy, lost in books and computers, his principal companion his own multifarious cortex.<\/p>\n

Raised in Modern Orthodox Judaism, Yudkowsky has been warning anyone who will listen of a nemesis right out of the Judaic lore: a golem<\/em>, a kind of Frankenstein\u2019s monster, built by hubristic, irreverent men and destined to punish them for their sinful pride.<\/p>\n

Yudkowsky\u2019s golem is A.I., which he expects to get smarter and smarter in the coming years, until it starts to take a hand in its own programming, and quickly makes the leap to superintelligence\u2014the state of being cleverer than humans at everything. He doesn\u2019t just expect that<\/em>, though. He expects A.I. at some point to conclude that humans are in its way<\/em> . . . and devise some method for swiftly dispatching us all, globally and completely. A specific scenario that apparently haunts him is one in which a superintelligent A.I. pays dumb human lackeys to do synthetic biology for it, building an artificial bacterial species that\u2014unforeseen by the dumb lackeys\u2014consumes Earth\u2019s atmosphere within a few days or weeks of being released.<\/p>\n

Why would A.I. murder its makers? Why can\u2019t we just program it, as people did in Asimov\u2019s stories, to adhere to the First Law of Robotics?* The answer lies in the design of modern, machine-learning (ML), \u201ctransformer based\u201d A.I., which could be described crudely as a black box approach. These ML algorithms, working from parallel-processing GPU clusters (effectively big copper-silicon brains) essentially process vast datasets to learn what is probably the best answer given a particular input question, or what is probably the best decision given a particular situation\/problem. The technical details of how this works are less important than the fact that what goes on inside these machine brains, how they encode their \u201cknowledge,\u201d is utterly opaque to humans\u2014including the computer geek humans that build the damn things. (Yudkowsky calls the contents of these brains \u201cgiant inscrutable matrices of floating-point numbers.\u201d) Because of this internal opacity, and the dissimilarity of its cognition from human cognition, this type of A.I. can\u2019t straightforwardly <\/em>be programmed not<\/em> to do something objectionable (such as killing all life on Earth) in the course of carrying out its primary prediction tasks.<\/p>\n

\"\"
Yudkowsky with OpenAI’s Sam Altman and pop star Grimes.<\/figcaption><\/figure>\n

In other words, this form of A.I. is like an alien species that, while it can be very good at some things, can\u2019t easily be \u201caligned\u201d with human values. We can usually align fellow humans<\/em> (despite the opacity of their own detailed neural workings) to human values\u2014that\u2019s one of the key training processes that goes on in childhood\u2014but we would need even more effective training for current A.I. systems. And researchers, to the extent that they acknowledge this problem, aren\u2019t even sure where to start.<\/p>\n

If it is true that the risk to us from what Yudkowski calls the \u201cA.I. alignment problem\u201d is real, then it should quickly become all-important as A.I. gets smarter and more versatile and is entrusted with more tasks. An A.I. wouldn\u2019t even have to be \u201csuperintelligent\u201d in any formal sense to conclude that it would be better off without us, but of course once it also achieved superintelligence, and was in a position to block our attempts to shut it off, we\u2019d probably be screwed.<\/p>\n

If you want more detail, here is Yudkowsky on a recent, lengthy podcast-type interview with two crypto guys\u2014who clearly got more \u201cblackpill\u201d than they bargained for.<\/p>\n