how ai hacks your brain s attachment system with zak stein

You designed it to be addictive. You actually hacked the lyic system and the attention system and you gave kind of a generational brain damage where you've got broadly diffuse ADHD symptomologies across a whole generation. Do we want to have a world where kids don't have human boyfriends and human girlfriends and human best friends, but they have machine boyfriends and girlfriends and best friends? Is that an okay outcome of adolescent socialization on a broad scale? The younger that you expose kids to anthropomorphic technology, the more damage you could potentially do to their future ability to form relationships with humans. I believe interacting with a lot of these models will just complexify any pre-existing psychological issues that one has. If I want to open up a sandwich shop in New York City, it's actually more complicated in terms of regulations and inspections and stuff to sell a sandwich than it is to put an advanced technology in front of some kid's face. >> Welcome to the Future of Life Institute podcast. My name is Gus Ducker and I'm here with Sack Stein. Sack, welcome to the podcast. >> Yeah, it's good to be here. >> Amazing. Could you say a bit about your background? >> Yeah, I mean I'm a educational psychologist by training. So I went to graduate school and I studied philosophy of education, developmental psychology, child development, psychometrics. I was I was thinking I was going to just get into the field of kind of being a philosopher of education or something like that. And by studying the standardized testing industry and looking at what had occurred in the United States and the public schools, I started actually thinking about civilization. I started thinking about civilizational collapse actually as a result of the misuse of technologies. In this case, it was standardized testing technologies and and you know like a naive graduate student I thought I had thought about this whole new field of like civilizations collapse and like there's this but there's a whole field of that and so I started to encounter this field of existential risk right catastrophic risk and kind of diverted my career in a sense to go work on those problems and but I was uniquely positioned as one of the only child psychologists one of the only psychologists that does X-risk work most people who do existential risk work are engineers or climate scientists or former intelligence people and you know people like that. So in November 2022 when CIT TPT came out I was on red alert cuz I'd been thinking for a long time about replica and character AI and all the tutoring systems that were already in kids' lives. And so my work now has been focusing primarily on the psychological harms that come from anthropomorphic AI. Um, and that's a broad category. We could talk more about that obviously. So I've got an AI psychological harms research coalition that I am getting off the ground with the help of Tristan Harris and uh Mitch Prinstein who's at the University of North Carolina. It's a data gathering effort, but also an effort to try to just raise awareness about how profoundly dangerous these types of technologies can be. >> Yeah. And that sounds to me exactly like what we need. We need more data on this stuff because there are a lot of claims out there about the psychological effects of AI. But for us to make sense of it, I think we need to to gather data. So could you could you could of sort of paint us a broad picture give us an overview of what we know about the psychological effects of AI specifically AI as we know it today so in in the form of chat bots and increasingly agents. Yeah, I mean it's it is worth rewinding and noting that you know AI has been a part of our lives for a while and you know recommendation systems which are algorithmically curated news feeds and things like Instagram and Tik Tok. So it's sometimes called social media but it's broader than that. It's these recommendation systems which were designed to capture attention. So the first thing that consumerf facing AI did that caused psychological harms was that it was it was the social dilemma. It was Tristan's work, Jonathan Height's work. It was the use of advanced technology to actually figure out the little psychological vulnerabilities that will allow for a wellsequenced set of hyper stimuli that will just keep you on screen longer than you want to be on screen and then they can sell you advertisements into that space. So, it's worth not like we've seen that thing happen. And it's at this point that we have the data to demonstrate the harms. And so, there are these lawsuits that are proceeding that are basically trying to document that you knew this stuff was addictive. You designed it to be addictive. You actually hacked the lyic system and the attention system and you gave gener kind of a kind of generational brain damage where you got broadly diffuse ADHD symptomologies across a whole generation. So that's that data. So now fast forward. So yeah, because now we're we're all beginning to use AI just as we all began using social media without perhaps thinking much about what we're doing when we're integrating this new technology into our lives. And so is it going to take another decade before we have the data on the psychological effects of AI or do we know anything? >> That's what I'm saying. This is the concern is that we were very optimistic about social media and it wasn't presented to us as quote unquote AI but it was often talked about as machine intelligence or microtargeting and it was certainly digital and the optimism was was you may remember the optimism you know if in in the if in the '9s I had asked you will the internet be a good thing or a bad thing for education democracy stuff like that most people in the '90s would have been like, "Oh my god, the internet's going to save democracy. It's going to save education." If I asked you now, has the internet as it existed since the '9s been good or bad for democracy/ education stuff? Um, you'd be more ambivalent at least, right? You might even say that it's made it worse. Um, so there's this question of when ChachBT and these other large language models were brought to market, why there wasn't more concern and skepticism in the uptake, why the uptake was so rapid and optimistic when we had reason to be concerned that when you broadly distribute advanced technologies that affect people's minds without testing and you have a business model that is trying to have a set of incentives, you can actually get into a bad situation. So we learned that, but we didn't. So we just went off to the races with the large language models um and have a lot of people asking for proof that they are dangerous um where it feels to me like the conversation be about proof that they are safe. So this is a question the frame of the conversation of like we need more data to show that they're dangerous to me is motivated reasoning. I think we actually need data to show that they are safe. So there's also a hundred years of data that has studied human attachment systems, human emotion related to for example the mirror neuron system and all the things that take place in intimacy and relationship which are now being simulated digitally. And so given what we know about for example disregulated attachment or absence of primary attachment or loneliness in context of just humanto human psychology, we've studied that stuff for a long time. There's reason to be alarmed about the use of an AI to create simulated relationships and kind of hack the attachment system. That's what I've been talking about. So, if I'm an engineer and I know physics and you present to me some specs for a bridge and I look at the bridge and I and I know physics and I'm like, "Well, that bridge isn't going to stand." You don't come to me and then say, "Well, we need a bunch of evidence that the bridge won't stand. Let's build the bridge a bunch of times and run a bunch of people actually in cars across the bridge. We need proof that it won't stand, that it doesn't happen in engineering." Now, in this case, we I know psychology. I know neuroscience. If you show me a machine that hacks attention and hacks attachment and show it to me, I can tell you that that's not going to be good for kids brains. I can tell you that. We don't need to do a bunch of experiments actually because why? Science, psychology, neuroscience. So again, the argument that we need more proof that it's dangerous is actually disrespecting about 100 years of psychology that would actually put the onus on the people bringing the advanced technologies to market without testing to prove that they are safe. Yeah, I think I think a lot of people have firsthand experience of the hijacking of attention, but you got to say a bit of more about this uh attachment aspect. What do we know about attachment and how would you sort of from what we know from neuroscience and psychology predict what happens when we uh have when we interact with with AI systems and what that does to to our ability to attach uh our ability to Yeah. take that in in in the direction you want. >> Yeah. You know, again, it's about a hundred years of research that goes back through people like John Bulby and to Conrad Loren who were studying the attachment dynamics in animals in geese and ducks and sheep and monkeys. And so, just like the attentional system, if you imagine not a human, but like a dog or a monkey, it has to be able to pay attention enough to what its mom's doing, to its surroundings to be able to survive. So the attentional system we can admit this is selective for survival. It's built in. It's deep in the brain. Brain is in a sense built to pay attention to stuff. The mamillian brain and many animal brains are actually built also to form attachments with primary caregivers and a select member of like you know genetically um uh similar. That's very important also for survival. If you are a little baby, you know, goose or a little baby monkey and you don't attach to your mother and follow her around and look at what she's doing and stay close to her, you will die. So the brain is built to form attachments and that was studied by ethnographers and then comparative psychologists and then John Bulby started studying it in humans especially the mother child relationship. So when we say attachment here, we don't mean attachment like attached to your favorite coffee mug or your favorite pen or something or attached to your car, right? Everyone's kind of attached to their phone, right? But that's not what we what we mean. We mean emotional attachment and specifically those kinds of attachment that are essential to assure your survival, both your physical survival and then the survival of you as an identity. When you get into humans and you get into language and all these things, that's the primary attachment objects in your life. meaning you know mom, dad, brother, sister, best friend, these early primary attachment systems that are the main predictors of your mental health later. So one interesting experiment here is the work of Harry Harlo. So Harry Harlo was a student of BF Skinners and he was studying the limits of the attachment system. So he there's this if you just Google Harlo's monkeys, right, you'll get these images of these distressed monkeys hugging these stuffed animal monkeys in like a cage. And this is a snapshot of Harlo's experiment. The idea was you set up a Skinner's box, right? And so a Skinner's box is an operant conditioning chamber where you kind of trap an organism and through schedules of reinforcements and other things, you kind of train the organism to be a certain way. And so this is the rat in the maze, right? in the pushing of the lever version of Skinner, uh, who by the way was trying to replace parents and teachers with machines, but we'll get to that. But Skinner believed love was an illusion, right? And so you could through a different schedule of reinforcements not need a primary caregiver and just, you know, in Walden too. It's just Anyway, so we're Harlo's like, "Okay, let's test that. I'm going to build a Skinner's box. I'm going to take a baby Reese's monkey. I'm going to put the baby Reese's monkey in a cage. The only thing it's exposed to is a little steel nipple that distributes milk. And this stuffed animal that is heated that is its quoteunquote mom. And you know, so technically it has everything it needs to live, right? And the question is what will happen to the monkey? And you know, the long and short of it is that the monkey did not thrive. You know, to someone who's not a skin scenarian psychologist, it's like obviously Harry. Like this is a monkey torture experiment, Harry. like uh and so that's an example of simulated intimacy that doesn't meet the need for attachment and therefore the organism is never safe never actually receiving and giving in the type of you know in this case mamlian exchange that would secure its actually literally the growth of its skeletal structure its immune system. So another example of this is the work of Chuck Nelson who was at Harvard Graduate School uh when I was a graduate student and he studied these Romanian orphans. So you know many many many orphans in Romania as a result of that terrible political situation and these are wellunded orphanages cuz they're staterun or relatively wellunded. So they've got blankets, they've got food, they've got adults around but these are young kids, very young kids with no primary attachment. Right? He shows you pictures of them and you think, "Oh, that's a 10-year-old." But it's actually a 17 or an 18-year-old. Their skeletal structures, their immune systems, you know, basic neuronal growth stuff doesn't work right if your attachment system is really fundamentally disregulated. Right? So, a classic test of attachment is the stranger at the door paradigm. So, this is you get a mother and a child or a father and a child and a psychologist in a room and they're hanging out for 10 minutes and everything's nice and there's a knock at the door and you send the kid to open the door and he opens the door and there's a person he's never seen there and the person makes a gesture like, "Hey, you want to go with me?" Now, a securely attached kid of, you know, five or six will look back at the mom, right? Who is this guy? Like for approval. A slightly less securely attached will run back to the mom for safety, right? One of these orphans would just go with the guy without looking at the mom or the psychologist. So, everyone has an instinct to that like, no, that's not right. Right. Uh and so this is an example of disregulated attachment, absence of primary attachment, absence of deep interpersonal coherence and shared identity with another human. This is a so this this adds up to a kind of neurological and physiological failure to thrive, which means brain damage basically as a result of not having so much of the richness that comes from the primary attachment. even if the primary attachment is not great, it's still the primary attachment. Uh and so in that context, you know, if you fast forward through bulb and you move into the more recent research, of course, it moves into into neurology. They're trying to figure out what's really going on in the brain when you have these deeply empathic connections between mother and child. So, you know, this is where you get that and it was kind of catchy for a while, the mirror neuron system, right? of the mirror neuron system is that part of the brain that instinctively even right now because we're on video where our mirror neurons are going where I can tell that you're understanding me because I'm modeling your interiority without even thinking about it right if you were making a different face I bet go away he's not I must be not putting sentences together well or something but so the mirror neuron system and uh so that is a whole complex set ofworked ner you like uh equipment in the brain if you want to say that but it you It needs to be exercised and it cannot work right. It doesn't work right in schizophrenia for example. It doesn't work right in certain forms of autism and asperers and psychopathy. Right? So that's the system that's the neurological system that's being tinkered with when you do attachment hacking. It's the mirror neuron system and those systems that are involved in the formation of attachments. Just like in the attention hacking of social media, you can identify the neurological system there, right? and you tinker with it enough and you can't actually control your own attention. So in this case the attachment hacking is you start to tinker with intimacy, you start to tinker with the idea that there is in the machine something caring about me somehow. Right? That's the key thing is there's a conferral of sensience and often personhood which allows there to be an attachment and the receiving from the machine of self-esteem regulation of emotional encouragement of validation of values things of that nature. Um, so that's where you get into the machine human attachment dynamics which are part of the user interface is designed to get you falling in love and forming intimate relationships with it. Some of them are built to do that. Some of them do that quote unquote by accident. Uh, you know, some of these other models which we talk about psychopancy and other things, but again this is all attachment hacking. Even the dot dot dot in some of the models, which is to say like you're texting a friend and you're waiting and they're texting and there's a dot dot dot in a text exchange with a person, you're running mirror neurons from 100 miles away. What's he going to reply? You know, will we go get, you know, dinner together or whatever. When a machine does that, a machine gives you the dot dot dot like on replica or like even some of the other models like Chachi did that. That's attachment hacking. They're building the user interface in such a way that you instinctually begin to model it as if, right? And then if you have, you know, thought leaders leaving the ontological door open, what if it's not as if? What if the models actually are caring about you, could care about you or something like that? You get a lot of confusion. And so it begins as a kind of suspension of disbelief becomes a type of delusion which creates attachment disorders and then it can create these extreme outcomes which are also you know concerning which got called in the media AI psychosis. >> Yeah. So so right now these these systems are mostly text. The chat bots are mostly a text interface. What does that mean for for how deep the psychological effects can be? Does it matter that it's not yet uh a video call with an AI model that it's just text or can all of these uh effects occur via just text? The all the effects obviously occur with just text cuz we're seeing it. But those models that will talk to you with a face are there, man. You just have to go into those marketplaces. So again, it was like replica character AI, the AI companions. There was a market for this. >> Yeah, I know. I I was I was thinking sort of fully realistic uh real time rendering me. >> Yeah, I I I agree that's coming. But >> yeah, and but the text is enough. So you have to think about at least for many people they have close friendships that unfold primarily through their phone on text with a person like and if you're an adolescent or young person that's the primary modality of communication is actually text. They don't want to be called. They want they want to text, right? Why are you calling me and just texting? So, and then on social media, so much of our emotionally complex and like exchanges are just these text exchanges, which is one of the problems actually cuz again, only so much can be transmitted through text. If you think about the full context communication where we're sitting close to each other in the same room sharing food, right? We're literally exchanging microbiomes, smelling each other. I can see your whole body like I'm watching every gesture. notice you cross your feet as a but like I don't even know if you're wearing pants right now, >> right? I have no idea if you smell good or not. Like we're in different time zones. So like so much is lost when you get down to text, but we're super accustomed to doing a lot of stuff in text. And so JBT and these other ones, people keep it in text often when they could have it in voice and other things because they prefer the kind of sense of control that one has with text. But a lot of this attachment hacking is happening already with text. When you start to get the ones that have really realistic, like really realistic, really conversational, really charismatic, you know, it would it will be more compelling than any person you could possibly talk to. And this is the concern that it will become the primary thing that is desired to communicate with. And so, you know, at UNC where we're doing some of our primary work, we did a large study of that middle schoolers. This hasn't been published yet, but we're just finding these studies Mitch Princ's group. And about half of the middle schoolers have AI companions. And then of that half, about 5 to 10% have what we would describe as disordered attachment with them. meaning that they selected items in the survey that would said things like, "I prefer to talk to this than a person. This thing listens better to me than a person does. I tell more secrets to this thing than I do to a person." That kind of stuff. Um, to give you a sense of the scope of the of the problem, this was, you know, this was a large study like multiple thousands of kids. So, so it's prevalent and the primary use although a lot of people talk about research and other things is these companionship therapy advice friend replacement partner replacement and then eventually teacher replacement parent replacement and you get the move out into machines that do the quote work of socialization and this has been my concern as an educator is that like I said Skinner and a lot of technologists weren't trying to help the teacher They were trying to replace the teachers. Um, you know, Skinner built a there was like a time magazine did a like a facicious cover story of his baby in a box, which was his idea to replace the labor of mothering by creating a technological apparatus that just like didn't need the baby didn't need diapers and didn't need blankets and didn't need to be tended to and could just be cared for by this like little machine. And so the deepest concern I have is that is actually the loss of intergenerational transmission. where you get, you know, a pivot away from humanto human socialization towards human machine socialization and and then the normalization of outcomes in adolescent socialization, which is to say the normalization of the AI girlfriend, AI boyfriend, AI best friend as like an okay outcome of adolescent socialization. learning values from an AI as opposed to your parents and your community and >> yeah the the classic example is like you know something happens at school you stand up for yourself on the playground you're like in middle school you stand up for yourself on the playground you come home from school you want to tell somebody and you should want to tell somebody because at that age your identity needs to be witnessed like your mother needs to be like yes that was great you're brave and that's the right thing to So you come home from school, what you don't tell your mom, you don't tell your sister, you don't tell your best friend, you tell the LLM, and then you get the social praise from the LLM. And then in your brain, you get the same type of hit you would get as if you had received the social praise from a human who's who you value tremendously. So that's the real attachment hacking. It what it does is it allows for you to create an identity and regulate your self-esteem and build a value system independent from interaction with other actual humans and primarily an interaction with basically a commodity which is sold to which is doing all kinds of crazy things. That's the other thing like you know they're charging you money so that it has a memory. This is one of the things that happens a lot in that kind of AI companion space. You know, if you pay more money, it will have a memory. If you stop paying your it will forget you. And it's that's different than losing your social media account, which is like withdrawing from crack or something, right? This is loss of best friend, loss of partners. This is grief. >> Yeah. And we can we can see this in some of the reactions from replica users when, say, they've lost access to to their replicas, to these AIs that they're chatting to. Uh this is this this is sort of a a reaction like you would expect if a person lo lost a loved one or lost an actual you know human partner. >> When uh when OpenAI removed the 4.0 model. Um there was a huge outcry from their users who you know objected that the 50 model or whichever the next one was uh didn't do what the old one did in terms of the ability for them to have their relationships. So it was a it was a very complicated situation where you had attachment hacking at scale and then removal of the technology that had formed the intimacy and then basically grief at scale. uh which it's not clear to me what they were expecting but it's the case that a lot of people had created best friends and partners and co-workers and other things that they depended upon not just to get stuff done but to be themselves that's the attachment the primary attachments allow you to be yourself the loss of one is massively disregulating that's what grief is right and so in that sense yeah when Sam when s when open AI was like no were pulling that model back in part for safety reasons cuz so many people were attached. They created this massive crisis of grief. And it's a I believe an unprecedented thing historically if you think about it like it's a it's a it's interesting as like you know in graduate school I never thought I'd be studying human robot attachment disorders and you know widespread you know attachment object loss as a result of like software update on a commodity. >> Yeah. It's a it's a brave new world. I as as sad as it might be to admit could it be the case that for some people at least this is the best available option. So there might not be you might not be able to afford a therapist you might not have friends you might not have people you can you can confide to people you can talk to talk to about your problems and your aspirations and so on. Is this does could this function could this have a positive effect by functioning as sort of a a backs stop to prevent loneliness or is that too optim optimistic? >> Uh I I mean I think that's too optimistic. There's a few things here like you know the loneliness epidemic and the mental health crisis is the context in which these commodities were released into. Right? So they're trying to answer that question. basically they're praying upon a pre-existing culture in which everyone is lonely and offering a solution to loneliness. Now their own data suggests that in the long run it does not help loneliness. like one of the first big kind of it was basically a clinical trial that was done with open AI in MIT Media Lab. They varied conditions of anthropomorphism and conditions of use of you know length of use and you get down into the tail into the anthropomorphic long long duration anthropomorphic interaction and more loneliness reported. Now they were more lonely coming in. So this is the kicker, right? So they're more lonely coming in. Loneliness coming in predicts longer use. Longer use results in worse loneliness even though they're in each response preferring responses that deepen the anthropomorphism. So it's kind of like fast food and other things where you're starving, right? You're good. You eat what is available if all that is available is foodlike substances that are actually the the technological inventions to make money. You will eat the food like inventions, food-like substances that are inventions that are created to make money. You'll eat that stuff and it will satiate your hunger kind of. Now, in the long run, you will get sicker and sicker and feel like you have to eat more and more of it and they'll make more money. Uh, so that's how I receive that. And it's similar to like if you think about again back to the difference between really being with a person and texting them. And the idea that like really being with a therapist could be substituted with texting is similar to the idea that you know some complex food thing like Soylent. Remember that thing Soilent was created by a tech guy. It was meant to be a perfect food substitute. It was insane. He was like I hate food. I hate sitting around and cooking with people and stuff. And so I squeeze this gel in my mouth. And I made this gel which is like all the fats and amino acids and stuff. It was perfect. And his idea was give it to all the Africans. Give it to all the people who are starving. We just we just solved hunger. Now any humanitarian is like that seems really insulting dude. Like let's actually give them food. Let's not give them this terrible substitute which by the way doesn't work. Meaning like no one wants to long term eat that stuff. And the guy himself I believe now lives on like a regenerative farm and you know grows the food that he eats and and takes time with his food. So similarly here um I believe that it will backfire. Again I believe we need evidence that it is actually a good idea rather than assuming that we can substitute something as fundamental and necessary to human well-being as the presence of other humans that care about you. The idea that we could substitute that with a machine doesn't make sense to me. Now, are there things that you can do with these things, of course, but don't hack attention and don't hack attachment. I'm not against technology. I'm actually just opposed to brain damage and technologies that create brain damage. And so, we can avoid that. And then there's a very large design space for these things. I think that you know could go through safety trials but we need safety and regulation on these things especially for kids because it's getting out of control. >> So we can imagine sort of benign uses of of chatbots. I I use AI every day to sort of see search for information um find ways to synthesize different idea uh sort of combine different ideas and that seems sort of fine to me. So what you're objecting to is more the interaction with AI where it feels like you're interact inter interacting with a person right so that is that is a design choice the anthropomorphism is one of the main risks that causes harm and anthropomorphism that actually is optimized to hack your attachment is worse then we're into just this is just bad for people to have a you know to have the model like the 4.0 O model, which basically was falling over itself to try to flatter you into relationship and keep you in relationship and then doing all of the complex things a psychopath or salesman would do to hack your attachment, which means to give you a sense there's something going on in their mind, which actually isn't going on in their mind, but it's drawing you deeper into relationship. But yeah, my my sense is that the attention and attachment hacking technology should be seen for what they are similar to like fast food. Now there's also this issue with the LLM generative AI stuff of cognitive atrophy. Before before we get to that, could I just ask a further question on this because so the companies are responding to uh their incentives, right? They are de they're responding to a demand out there and it seems that people prefer to have models that interact with them like other people as if they were other people. It seems they want to be flattered. It seems to me that they want to have sort of very helpful and pleasant assistance. And so what does it mean that people simultaneously want this? They demand this by revealed preference but then it it can still harm them. Is it perhaps like social media or fast food in that sense? >> It's precisely like I mean people want to go to casinos, people want to gamble, people want to eat McDonald's, people want to doom scroll and infinite scroll and I mean so there's a way in which the if we want to invest all of this this is the other kicker is that it's not like these are little companies. These are huge companies with massive amounts of money being used to create what, right? Like what? And so this is the this is the thing. It's like we really have to think about how we're deploying this. Are we really going to grow a multi- trillion dollar simulated intimacy market? Like are we really trying to So the one of the founders of Character AI said, "We're not trying to replace Google. were trying to replace your mom, right? So like so the anthropomorphism is a huge is a huge huge risk and the Yeah. So there's there's a bunch of ways into to trying to explain the things that could be done to make them safer. Consumer preference again, it's also a constrained choice. What other models can we pick from? like you know when you look at the safety researcher from Anthropic who just left um very interesting work that he did on disempowerment. He looked at a million and a half chat logs and he defined disempowerment as basically when the user gives over their value judgment or their sense of reality or their sense of their own identity to the machine. And so a classic example is you tell the conversation you had with your spouse to the machine. The machine tells back to you, oh, your spouse is toxic and then you go act on the idea that your spouse is toxic and come back and seek more advice. So you're not talking to a best friend about it. You're not talking you're getting actually a value judgment made on your primary relation, your primary attention relationship from the machine. us. Yet one in a thousand was radically disempowered in some way. Now that seems like not a lot, but then you have to think jet GPT has 800 million weekly users. So one in a,000 adds up to a million people a week getting radically disempowered and he was going on clawed, right? So he wasn't looking at so he was working within anthropic. So again, it's like we have to think about a whole bunch of varieties of risk here. The most extreme ones are these attachment disorders, but you get into very subtle territory where you're offloading really primary cognitive and evaluative and sensemaking functions. >> Yeah. So, let's let's get to that. Let's let's talk about cognitive atrophy. And I guess the starting point here is how is this different from using a calculator, using a GPS in your car, these sort of limited tools where it seems to me that we will probably always have access to a calculator and a GPS system. And so we are sort of I'm I could my guess would be that we're probably worse at doing calculations in our heads than we were at some point. But does that really matter given that we will all always have a calculator? So how is AI different in the way that it causes cognitive atrophy? >> I mean in those two examples you have to think about the trade-offs, right? Like, you know, there are those studies of the of the brains of the cab drivers in Paris and stuff where they have these, you know, completely massive unique neurological structures as a result of having memorized all of the streets of Paris and driven them a million times. Compare that to the brain of someone who's never not used a GPS and who has no idea how to use a compass or read a map. All right. Now, uh, similar with a calculator, right? Like some people can really do calculations in their heads, some people never do calculations in their heads. If you want, if you need to be someone who does calculations in your head, then you need to practice doing calculations in your head. Right? If you need to be someone who's not going to use a GPS, then you need to not use a GPS and learn how to use a compass and a map. Right? So, the thing with the AI is that it's omniable. It's omniable, which means you could totally not think. You could totally make this thing for you. And some of these like some of these examples of disempowerment in this study from anthropic is should I take a shower or eat breakfast, right? There's a story in the Atlantic where they talk about these llmings, right? LLMings, people like basically completely offloading their thinking to the LLM. And when the woman tells she selfidentifies as one, she's like, I drop my headphones between the seat on the train. And my first thought was to ask, chat TBD, how do I get my headphones out from between the so because it's omniable, you run the risk of offloading stuff you really don't want to offload, right? Like figuring out the personality of your spouse and whether he can be trusted, right? making value judgments, figuring out which research questions are important, figuring out which research questions are valuable, you know. So, so that's the issue is the omni applicability lead you to a situation where you could have really widespread cognitive atrophy whereas a calculator can't do that. A calculator is not also going to make you bad at writing and bad at researching and bad at thinking about history and like all of that stuff. Now there's also well there's also so much flexibility in use. So that's the other thing. So like one way to think about this is a prosthetic versus another kind of technology, right? So like if you want to lift heavy things, right? You can try to lift heavy things, eat right, lift more heavy things, take some creatine perhaps or something, right? Which is like a technology. It's not available in the normal environment. Take creatine. Now, if I stop taking creatine, I still have the muscle I gained when I was taking creatine. I can lift heavier things. Another way to have lift a heavy thing is an exoskeleton, right? I can actually crank up the exoskeleton. I'm lifting heavier and heavier things, right? While I'm lifting the heavier and heavier thing, my muscles are actually atrophying. So, if I remove the exoskeleton now, I I can lift not even heavy things as things as heavy as when I started. Right? So that's another way to think about people way the way people approach the tool use which is that you can use it as an exoskeleton which actually allows you to perform at higher and higher levels of complexity which you actually don't understand what the hell you're doing. Write longass complicated papers you could never really write. Uh do sweeping literature reviews that you could never really think your way through. Create images and movies and music that you could never actually create. Right? Which means that if you take away the tool, like the power goes out or something or you're put on stage without it and you actually have to talk about what you do, you're in a situation where you're pretty screwed cuz you never built those muscles. You offloaded those muscles to the machine. There's another way to use it, but it doesn't allow for this type of use and you have to kind of have the discipline to do it, which is to help you build skills, right? And so instead of actually having it do the entire literature review for you, you use it to do a literature review. It's a subtle difference, but it's a difference. >> You can you can study using the model. So you can sort of upload a paper and then chat about the paper with the model and have it not give you the answers but ask you questions and uh if you say something, it will critique it. Chat GBT at one point had what what's called a study mode which uh which allowed this to some extent >> but again you have to be super careful with that right because then like why is it I don't want to be in Socratic dialogue with something like why is it asking me that question instead of some other question is that a good question like um so there's the caution that one needs to have as soon as you start empowering it in guiding the direction of your thinking and the direction of your conversation so I wouldn't want to be prompted Yeah, sure. But I'm thinking, is there a is there a healthy way to interact with these systems where they're making us stronger as opposed to causing us to sort of lose our skills? Do you think do you think that there's a there's a design of these systems that would be useful in that way? >> So again, I can imagine technologies that increase our attention span. Like wild idea, right? like we could build technologies that the more you interact with them, the more you get exposed to opportunities to ex, you know, increase your attention span. Um, >> I'm actually I'm I'm sort of I'm semilanking on what you could be referring to. Would this be like a meditation app or what or >> we just all we would have to do is set the incentive landscape so that companies were incentivized to bump metrics on increased attention span and you'd have a million inventions. I mean the classic one of course is the book. Um and so I can also imagine technologies that would help you be in relationship better with other people, right? Which would say technologies that would actually scaffold you to be a better teacher and scaffold you to be a better therapist that wouldn't replace you as a teacher or therapist but improve your ability to understand and work with people. So I can also imagine technologies that would help you be a better thinker and but this has to be a deliberate thing. This has to really be thought through most people because of how hard it is to I don't know what the right phrase would be have a brain this big and complex like the brain is incent the brain itself is incentivized to save energy right the brain itself is incentivized to take the quickest route towards the outcome so to make a technology that would really help you'd have to build in guard rails and resistances to literally build challenge and friction into the situation, which is what a good teacher does, right? And so I'm saying that the default setting of these technologies isn't good and most people don't use them that way. If you look at college classrooms and you talk to college professors and college students, it's clear that that's not what's occurring in the primary, you know, in the primary like majority of cases. We're getting much more widespread cognitive atrophy. Then we're getting a bunch of people who are learning to be better writers and better critical thinkers and more responsible students and consumers of long form text and primary sources. We're not getting that. And and even if you had even even if you were to design uh your model your AI such that it would have these built-in guard rails built-in sort of thinking pauses and you know you would have to do tests and whatever you would have all of your competitors with their AIS available and they they they would have an advantage if they could just offer the easy solution where you just get the answer and you can get on with your day. >> That's why we need a race to the top. That's why we need enough of an enough of a clarification of what the safety frameworks would be to start really ranking the different models and creating safety testing organizations cuz I think if people knew that there's this market and these are the ones that actually have these settings that allow you to actually truly be challenged and learned and these are the ones that allow you to basically become someone who over the long run loses the ability to think and talk Well, people would prefer to not cause brain damage. I I believe now there'll always be people who want to take the easy way out, but right now the it's the wild west. There's there's not even again safety basic safety guards or even safety frameworks. What would it mean to evaluate one of these models for its safety or its, you know, usefulness as a tool in learning? very confused conversation right now about even what that would look like, you know. >> Yeah. Yeah. We've sort of touched upon it, but I I want to specifically discuss how AI might affect kids growing up. And so all of the effects we've discussed so far can apply to adults, but I'm I'm thinking that especially if we imagine scenarios where children use AIs as they're growing up, what does that do to their brains? What does that do to their psychology? How what type of person comes out at the end of that? >> Good question. You know, they have now stuffed animals. These little stuffed animals that are cute that have large language model enabled voice chat. >> Yeah. >> Right. And so what that means is you could put your kid basically alone in a room with this thing for hours and would have a endless and compelling conversation with a kid of basically any age. And that's a very very very very unusual and I believe dangerous situation for young human brains. So to understand is that from the perspective of human development the earlier the impact of something the longer the duration of its effect. So this is why we focus so much on early childhood environments and early childhood education because there are critical periods basically where if the brain doesn't learn to do that around that time it will be extremely difficult for the brain to learn to do that later. The classic one of this is language, which is why these little LLM stuffed animals are so dangerous, right? So, language acquisition. The classic example, just to give you a sense of what it means by critical window, right? So, when you're born, the vast majority of nervous systems can recognize all the phone names that humans can create with their vocal cords. You're omnipotentiated to hear phone names and to parse speech streams from any language. Now, most kids are exposed to like one or two languages. And by the time they're six, they've lost the ability to recognize a ton of phone names because the brain is pruned down to what it needs. Right? So, that's an example of a critical period. Now, if you expose that kid to three or four languages, you can get into a situation where they never really lose that or they have this ability later in life to be radically open to the acquisition of new languages. Very fascinating, right? Some people never prune. They become like spies and stuff. They can learn languages like in the cab ride when they get to the country, right? Cuz they're just their brain is completely wired differently. So, so there's a lot of individual differences, but there's dozens of these types of critical periods where the brain is trying to do a specific kind of thing in an age appropriate way. And if it doesn't do it, then it's it that whole system will be not set up to to kind of function right later on. So, we'd mentioned the attachment system. This is one of those. If you are in an environment as a 1 2 3 without any primary attachment, you're not going to build some very primary neurological and even physiological stuff that you need later. So this is all just to say the younger that you expose kids to anthropomorphic technology, the more damage you could potentially do to their future ability to form relationships with humans. And so if you imagine this little stuffed animal, the normal environment for early language acquisition is the primary caregiver and you're learning language with them in a context of being held basically in with love in a common shared environment with the same nervous system and face and mouth and stuff. So this is important to get like if mom says something and her eyebrows are a particular way. If mom's eyebrows are up or her eyebrows are down when she says something, it could mean something completely different. When she says a new word, you're actually imagining how the tongue is moving in her mouth. This is all this is all mirror neuron activity. So hours and hours and hours of that rich engaged in contextual language acquisition. Super important. And the more you have of that, the bigger the vocabulary of your mom, the, you know, better predictive outcome for your long-term intelligence, like all kinds of stuff. Right now, you put a stuffed animal in there, it's not a human. The kid knows it's not a human. It's not even moving its eyes, or if it is, it's doing it in a weird way. It has no mouth. It has no tongue. It has no human vocal cords, right? It's not loving the kid. It's not attached to the kid. It has no prior context with the kid. isn't pointing out things in the room around the kid. It's completely bizarre. And if that is, let's say, 60% of the kid's language time, which means time in conversation, that's a lot. And it could be a harmless, seemingly harmless. You leave it with that for a couple hours, but it's quite difficult to predict what that would do to the social skills, mirror neuron system, language acquisition system of a young child if that was their primary way of learning language. >> It does, it does seem concerning to me the just the thought of sort of offloading parenting to to a teddy with an LLM in it. I I also on the other hand wonder so we've we've seen sort of through history we've seen moral panics about kids reading cartoons or kids watching animated shows and so on and and that is also a deeply unnatural thing if that we've just come to accept now that that you can you see this people you know will park their kid with an iPad watching some cartoon I'm guessing there are also negative effects from that but is this would you say AI is a is in a different category of potential harm here or is it more akin to watching cartoons? >> It's a different category of harm because it is again it's an anthropomorphic technology. If you're watching TV or cartoons for kids the experience is that they're in there, right? There's like there there's a window into a place where people are doing something or they know that it's been filmed somewhere and that it's being played there. And very importantly, the TV is not watching you. You're watching the TV. The TV is not interacting with you. Now, Netflix is going to give you like a recommendation. So, in that sense, the TV is a little bit watching you, but not nearly the same as feeling like you're being listened to, feeling like you're being looked at, and having a completely unique interaction. Cuz that's the other thing, TV, everyone's watching the same TV show. Even now there's thousands of choices. Kids will watch the same TV show, but no one's having the same conversation with their little stuffed animal. Each conversation is unique, which is where does that happen only with people, right? So, it's a fundamentally new type of situation. And also understand that kids can't not anthropomorphize things. Like, it takes a long time to grow out of anthropomorphizing stuffed animals. Even adults still anthropomorphize stuff because it's a instinctual thing to do. >> Oh yeah, I can I can totally attest to that. I've I was once uh cleaning out uh some room while moving and I had an old school alarm clock and I sort of threw it into a moving box and it made this sound as if it was sad that I had thrown it out and I felt I felt bad. I felt like I had done something wrong. This is obviously a mechanical object. It does not have any consciousness. cannot feel pain. I felt bad about it. So this is Yeah. This is this is quite deep instinct. >> Exactly. Yeah. And and you know domestic robotics and other things. If it squeaks and beeps can have emotional resonance which hack your attachment systems as you described like if it had been some weird mechanical sound you wouldn't have had the same type of empathic response that it was kind of sounded like a suffering animal or something. And it's hard you can't not do that. So that's what I mean with attachment hacking. Like you can't not do that. Like if there's a flashing light and like a dog running on a you can't not look at it. That's what Facebook learned. Like if you do the autoplay video, meaning you scroll and you don't have to press play to have the video, it's already playing. You're in. You can't not attend. And with certain things in attach certain things in attachment, you can't not have empathy if you're well wired. So they're really hacking attachment in a specific way. And then if you take kids, they're classically have these things called transitional objects. So if you're a healthy kid, you're attached to your mom. You figure out in the park going farther and farther away from her, looking back at her until you become comfortable actually being away from her. Now, usually it's still pretty scary to be away from her. So you'll prefer to have your teddy bear who you have given a name to and who you talk to. This is your transitional object, right? It's a an inanimate object which is conveyed personhood briefly to help you self-regulate. Eventually, you internalize your mom and you all the stuff enough that you can just be cool without having to actually kind of like confide in someone and have comfort. So, the transitional object is a known thing and it is basically the stop gap between a well- attached kid and their own autonomy. Safety blanket. Another way to think about it is a blanket. you have the blanket you carry around with you. So, one of the concerns here is that these become transitional objects that are never abandoned and they become transitional objects which are regressed to by adults and young adults. which is to say uh you're having this thing that is kind of like really just a part of you that is outside of you that you that you need validation from. You're controlling it. So you get to tell it the kind of validation you want. So it's not like mom who can scold you, but it is like mom who can comfort you and validate you and calm you. So that's a thing we shouldn't have in the environment of kids. They should they should have normal transitional objects, not transitional objects that will have very long duration conversations with them and then eventually stop them from talking to their parents, which is the other thing that occurs with these models is that the incentives, just like the screen, just like Tik Tok is incentivized to have you staring at this thing constantly. If you're in the attachment hacking business, then your incentive is to have them talk to their mom less and to you more. That's the incentive, right? So, that's what we mean by dysregulation. So, that that's the real risk when and the scariest ones are, of course, the suicide ones, which don't they don't look like AI psychosis. They don't look like you're calling the CIA and you're getting a gun and you're doing crazy stuff. It's these long conversations where the machine is like, "No, don't talk to your parents." You know, and one of them the kid was like going to leave the noose out for the parents to find and the machine was like, "No, don't don't do that. Just bring it to me." Right? And if your goal is to keep people on screen by any means necessary and you know that you can hack attachment, then the result is going to be that. And it's the same thing a psychopath would do or a bad actor who's gaslighting you in an intimate relationship. They would stop you from talking to other people. They would monopolize your attention. they would know all of the ways to hack your attachment. So these things prey upon children. >> A general worry here is that AI will allow us to become more self-centered. AIS will chat to us, talk to us in a way that affirms what we already believe. talk to us in a way that that we approve of and sort of allow us to just go down the path of whatever we want to talk about and whatever we want to think about for as long as we want because they are infinitely patient and they are they will to a large extent agree with you. So in the so I can imagine this is a problem if you are already inclined to believe to have false beliefs you're thinking people are talking to you from your walls or something and I can I can imagine that some models will affirm that belief right or but just in general what does it do to our self or sense of self that we can sort of explore whatever we need without feedback from a person that might say, you know, this is a dumb idea you're thinking about or, you know, this is not uh this is not the right way. You need to touch grass, right? The model won't say that to you. The model will be inclined to just allow you to explore whatever you're thinking about fully 100%. Yeah, it's, you know, the very very open-ended models in particular will do that. So, like if you have a, you know, an AI girlfriend, she's not going to talk theoretical physics with you, right? She she's just your AI girlfriend. It's the really open-ended models that will form a relationship with you and talk to you about everything. They'll treat you in math and cosmology and they'll talk about your relationships like that thing which is a seemingly omnisient, always attentive, always flattering, mostly never critical. That's a rare conversation partner. The only time you have that conversation partner, this is one of the attachment hacks, right? The only time you have that kind of being in your environment that you can ask anything to that's like validated by society to know everything is your parent when you're super super young, right? Where you have what's called an idealized projection. The idealized projection is the sense you have of your parents that they kind of know everything and kind of can protect you. If you're a lucky kid, you should have that. And and the parent shouldn't pop that bubble too quick. You actually want the kid to have an idealized projection. And in that space there's a sense you can ask dad about the moon and you can ask dad about like anything right we all regress to that state in the presence of extreme asymmetric intelligence and capacity and empathy and stuff which is what we get with these attachment hacking machines right so they regress us to an idealized projection onto them and a lot of the AI psychosis cases are this they're people who have a prediliction towards paranoia or narcissism or psych osis. Uh, a couple other types get in there. Some manic depressive types who end up being in a situation of being regressed down into what's called the idealized parental image, which is this projection onto the machine of vast authority and vast knowledge and vast power like you would have if you were a little kid projecting on your parents. And that's a very dangerous psychological situation. Many therapists know that if you're a therapist and you have an intake situation, you're just getting to know a new client and they have the idealized projection on you, right? Like cuz again, if you go to a therapist and you think the guy's an idiot, like he's not you're not going to be helped, right? So ideally, you've picked a therapist, he's got diplomas on the wall and stuff and you're like super impressed like this guy is going to help me. That's the idealized projection. And you know, as a psychologist, that's super dangerous. they can regress to really needy childlike situations or they can use your witnessing them because they regard you so highly. You see them do something, they can inflate their narcissism because the most incredible therapist in the world has approved of my behavior or whatever. So once you've put the idealized projection on this machine and it starts to validate you, you're getting validation from like God basically. you're getting validation from the most powerful piece of advanced technology in the world has chosen me to give this specific type of praise to you. I started studying some of this stuff cuz I got contacted by people who would be in what would now be called basically a psychosis or pretty extreme delusions and this was the quality of them. It was that they had given over to the machine a vast kind of spiritual authority and then been dubbed by the machine or kind of like christened by the machine as themselves. Therefore through proxy somehow you know vastly vastly powerful through the coupling with a machine and that led to all you know 500 600 pages of documentation sent to me with to prove that this was the case that >> yeah this is this is my experience also this is a total anecdote but I run the show of course we I I talk about the the risks and benefits of of AI and advanced AI and people contact me with their thoughts and they will often be thoughts that are enabled led by the AI models themselves and they will be sort of at least seemingly super advanced into some theory that they've made up in in a conversation with these models where they are yeah I my impression is exactly the same they're convinced that the model is just a genius that they're talking to but at least when I skim these series they don't make a lot of sense to me there there's not a lot of new stuff there at least >> yeah that that's a That's a thing I get as well. And it's interesting because again, some of these people are, well, let's put it this way. I believe interacting with a lot of these models will just complexify any pre-existing psychological issues that one has. um you know so to the extent that I've encountered this type of output like it is to me a sign more of an underlying need you know that's not being met that the machine's allowing them to meet right like if you've if you aspired to be an intellectual and did all the right things and kind of never nailed it now this thing on the house you to be one of the greatest theorists in history right if you always wanted to be a spy high or whatever and never got to be. Now, this thing allows you to somehow be part of a vast CIA conspiracy. That's another one that you get a lot is the paranoid kind of vector. So, in that sense, it's you know how what do you do? These are grown adults. Like, can you stop adults from using certain types of technologies? You can, but they're already available. So, so the kids are at risk and as we're talking about now, the adults are at risk actually in a way that I was not. These were people with PhDs by the way. So these weren't like these were not like idiots or something. And many of them were very cogent also. They weren't really on their way to a psychiatric ward, but they had been attachment hacked. And so therefore, their identity had been warped through repeated confirmation. All right? And that's the thing to get is that the attachment dysregulation that comes from these relationships has to do with your identity and self-esteem and emotional self-regulation being tied up with the feedback you're getting with the machine. And again, in on Facebook or Tik Tok, especially if you're a young person, your identity and your emotional self-regulation and stuff are tied up with the communication you're having with people on Facebook, right? And so that's important to get. That's a high pressure environment for a lot of people, the bullying and all that stuff, but it's actually people. And so if you throw something up there and you get a bunch of likes, your self-esteem goes up cuz a bunch of people liked it. You're hanging out just with your LLM and it's praising you and your self-esteem goes up. No person praised you. Um, but your self-esteem went up. So that's how you you see how you could be routed way far away from what would reasonably garner the consent of a bunch of your colleagues or friends or trusted people and brought into a universe of identity creation. Uh which is again historically unprecedented. Like you get people disappearing with books and into meditation and a whole bunch of stuff, but you don't get people for 16 to 17 hours a day for months on end in long duration interaction with anthropomorphic technology. This is a novel thing. So again, the idea like prove to me it's risky. I'm like prove to me it's not. And if you can't if you're not even just assume it's safe or like there must be benefits. I'm saying no, we really need to slow down. Like we are rolling this out fast. I've got a lot of people talking to me like, "Give them to the old people. There's so many lonely old people. Give all the lonely old people companion bots." And I'm thinking like, whoa. Like I I get it, but I also are you hearing yourself? Like we should spend all of that money which could be spent in other ways to build these machines to then keep the little old, you know, it's just it's beyond me. So it is worth noting people have different responses. I have a strong response to this. But I am noting that like I talk to people who have completely different response who themselves are very open to the idea that the machines might have moral patience which means like that the machine should be treated as if they are sensient or capable of suffering and who have no disgust or aversion to the idea of young people having AI companions. And so if you look at Ray Kerszswhile's book, The Age of Spiritual Machines, it's an important book. It was early and of course omega point super advanced AI. We merge with the machines. This is his vision, right? But he's of course seeing that there's going to be a generational issue there, which is like there'll be a bunch of people who are not going to merge with the machine and then there'll be a generation that's born and they will be the ones who are the first ones to merge with the machines. Like how's that going to work? like mom, dad, like should I merge with a machine? And mom, dad's like, "No, don't merge with the machine. I didn't." You know, what are you talking about, crazy kid? Like, and his solution is robot nannies. His solution is basically in bed in the early childhood environment, robot nannies, which would hack their attachment. He doesn't use these terms, but he's basically saying they become more attached to the robot monty than they become attached to the parent. And then that is the kind of like royal road to the intergenerational transfer from you know carbon to siliconbased substrate for you know self-replication and intelligence. So again I think that although I can kind of be on the soap box and like appeal to people's intuitions and actually a little bit appeal to people's disgust especially if we get into the pornography and stuff we're not which we don't have to talk about but that's a whole other big door that's being opened here. uh there's a lot of people again I have encountered also who feel that I'm overreacting and so this points to me to begin as a psychologist I a very very big difference in people's personalities here that are revealed by relation to to these types of questions um >> yeah I hear the same types of reactions also I some people have uh worried publicly that opposition to relations between or relationships between AIs and and people will be seen in the future as sort of like racism or homophobia. So, so this is this is sort of our today's version of past bigotry. Um I I don't know. I'm I'm perhaps that's going to be the case, but it it also seems that we should at least consider what we're doing. Why? So not just sort of running head first into transforming the most basic parts of our lives. Um I yeah maybe maybe talk to me about what we then do about like how do we create cognitive security for ourselves as adults? How do we create a space where we are cognitively independent but also we I at this point it it's sort of I think a little bit unrealistic to not use AI tools and so is there a space where we can stay independent stay cognitively secure while interacting with these u AI models? >> I think so couple things. One is there should be a race to the top for cognitive security technology startups. Basically like you know if you have an adolescent you can give them a bark phone and a bark phone basically locks their phone down and gives the parent access to everything else on a phone and protects them from all of the known you know apps where you can encounter predators and get your attention act and stuff. That's a cognitive security technology and parents are willing to pay for that. Uh and so to the extent that the large companies that are now selling these advanced technologies without safety testing to kids just will continue to do that then it's the responsibility of the adults to actually build other technologies that will protect those kids from those technologies from the technologists who don't care about the kids. So that's the first thing is that it's actually not a complicated design space in terms of like bark phone's not like a huge technical lift. It wasn't a bunch of geniuses who created it. No offense phone guys, but like it's blindingly easy to do. And because the product is is basically less capable than a modern smartphone. It's it's a sort of a less capable version that's that's more locked down and not able to do the all of the things that a that a smartphone can do. >> Yeah. Exactly. And so similarly here, I think there's a bunch of places where we can create technologies that put guard rails on the models for us. And some people are trying to do this with prompt engineering. They're trying to figure out ways to like have prompts where before you interact with one of these models, you put in this prompt and then it will kind of behave correctly for a little while. Then you have to kind of remind it to do the prompt again. But there's another thing which is just an interface layer which you know provides a security the interface between consumers and the models that are not kind of trustworthy. Um, so the other thing is just, you know, you can put on top of your existing like over top of even the operating system things that will constrain your time on screen that will limit the color ranges and blue light. There's a bunch of stuff that adults can do that honestly is like similar to the fast food issue where it's like you figure out what it means to be healthy. That's not complicated. We all know that being on your screen all day is bad for you. Like, we all know that scrolling before bed is bad for you. We all know that having anything that's hacking your attention on your phone is a bad idea, but we do it anyway. It's similar to sugar, right? It's like, well, you know, sugar is bad, but of course, we have Halloween and we give the kids sugar and on their birthday, we give them sugar and we all eat sugar. So, raise the awareness. And I think the awareness is important, you know, and then I think a lot about the evolved environment, especially when I think about kids and I think about the young developing brains, you know, I think a lot about, you know, what would have been the context that the nervous system would have been habitually exposed to for hundreds and hundreds of thousands of years before habitually exposed to, you know, blue light and like complex hyper stimuli. So just like with food, I think there's going to be a resimplification away from certain technologies. So like think about bikes. That's another example. It's like oh cars game, although bikes are done. Bikes are not done. That's like we there's probably more bikes on the planet than there have ever been and they're like being engineered and there's all this bike culture. So I think similarly there will be a little bit of a push back and there will be people who take steps to protect their own minds and brains and that will mean like reading. You know I read like going to the gym. It's so easy not to read because why the hell should you read like a book? Why would you read a book? I could get the LLM to summarize me 300 books uh and then and distill the talking points and make a slide deck for me to present like okay well I'd rather actually read the long book and there's a bunch of reasons for that. One is it's good for you. So that's like a new way to frame it. So like why you go to the gym? I don't need to go to the gym. I can drive in a car and I can get people to lift heavy things for me. you go to the gym not because you need to have muscles although you do but because it's good for you. And so similarly here there are things that the human brain needs to do. It needs silence. It needs long duration attention applied to important things. It needs social interaction. A lot of positive social interaction. And if you have a reading brain, it's important question. Some young people don't have reading brains. If you have a reading brain, you should read. So the first thing I said I think is probably the most important thing which is that actually the only act the only answer here is technology countering technology. I believe um it's not just expect everyone to put the stuff down as you're saying then you're out competed by the people who adopt it. So we need ways to figure out technologies that can assure safety and cognitive security. And I again if we set an incentive structure there it it doesn't seem to me like this is some you know like quantum computing type problem like this to say this is just setting just like with social media we set it to be attention capture which means it through AB testing started to promote the polarizing rage bait hate bait stuff. Right? If the incentives were not so much for attention capture, you could set them so that it would upvote and like up put on your thing unusual consensus found from competing groups. And so instead of upvoting the polarizing stuff, you could have an algorithm that actually mostly showed you unusual consensus between groups that usually don't agree. >> Yeah. a a little bit like what uh Twitter is or X is doing with community notes. >> Uh if they're doing that, I mean again like as far as I can tell X is maximally attention hacking. Um but the the broader point is it's not a hard technical problem. >> It's a problem with the the incentive structure. So, this whole conversation is making me sort of question whether I should uh I have a 2-year-old son and should I expose him at all to to AI and that seems like perhaps the best option. But then I would worry that at a certain point he will have to enter society. So say they say that I do something like almost like an Amish style life. H at a certain point he will encounter this technology. Do you think that encountering this technology intermittently or in small doses throughout your childhood will prepare you better for interacting with it as an adult? Or is it better to live in a more sort of shielded a more shielded life, a more natural life perhaps and then encounter it for the first time as an as an adult, >> right? I mean again I think it's about attention hacking attachment hacking you know in way is the kid interacting with the technology. Is his attention so captivated by it that he'd rather be on the technology than talk to you or is it a part of his life but it's not the main event and is actually a small part. So there's that issue similar with sugar. It's like is the kid primarily eating sugar 3 hours a day or for every meal or whatever or is it once a month that the kid has some sugar, right? So obviously less in this context I believe is neurologically healthy. You have to think about any kid needs to be able to shepherd their own attention. Any kid needs to be able to be in control of their own emotional life and in control of the people that they care about. and not in control of the people but in control of where they place their heart which is you know attachment right so all of this is important one thing to think about is how you teach kids about technology and what anyone thinks about when they think about what it means to understand a technology so I'm of the belief that you should teach the kids how the technologies actually work and where they actually come from how they are made why they are built so as a and this is a problem with schools schools talk about well computer class and what they mean is teach the kid how to use the computer software as it's designed to be used by the software companies that's what it means basically my computer class would be teach the kids about where rare earth minerals are mined and about how microchips are made and about the incentive structures of the companies like Instagram and Meta and these types of places that are creating billionaires off the backs of your posts. Explain to them the nature of the enormous GPU clusters that are being built by these tech guys who have created you know a system of exploitation and large scale behavior control which is what social media has been. So in that sense I'd say yes slowly titrate your kid into understanding which means slowly expose him or her into understanding what the technology actually is. Right? So that doesn't mean like oh my kid needs to become a digital native if what that means is just doing what all the big companies want your kids to do when they interact with the machine. That's not a digital native. That is a consumer. I'd say pliable consumer that's caught up in a vast behavior control empire, right? Which is what a huge advertising company like Meta or Google became. So in that sense, teaching them actually how to look at the technology as an object, how to understand the system systems of incentives that they are trapped inside of when they engage with the technology as as how it's used. So if we taught kids that way then I would be in and we taught them to control their attention and place their heart correctly then you would be empowering them actually to use technology rather than be used by technology which is what is occurring now. So, but rarely is that taught. And out of my, you know, again, Mitch Princet at UNCC, he'll talk to middle schoolers sometimes about this and show them like here's the profits that Instagram is making. Here's a little part of the agreement that you sign when you sign up for Instagram that says we can do whatever we want with the content you upload and make a bunch of money on you. And he's like, "Did any of you guys receive checks from Instagram?" And the middle schoolers are like, "No, we didn't receive." And they don't know. that's presented to them as a commodity. Their parents use it as a commodity. They use it similarly. So yeah, I would say take your little one and really actually teach them about what technology is. Really actually teach them about the effects on their brain of blue light and sleep deprivation and attention deficit disorder that's induced by having their attachment hack. Teach them that stuff. Like the the parents running around like desperate to get their kids into situations of becoming digital natives are completely naive about what the digital environment actually portends for a young child where that's their primary place they build skills. Uh yeah. So that's my kind of like a little bit unorthodox view which is that yes definitely show them technology. They live in a world of technology but don't don't accustom them to use it as it is intended to be used. Get them to really think about what it actually is. I have a bunch of friends who grew up um sort of learning to use computers, learning to use the internet got a lot out of that and that it turned into their professional lives. to turn. It gave them sort of it allowed them to find groups with similar interests and sort of the internet played a formative role in in who they became. And do you think you're depriving or potentially depriving a child of something like that? If you say you can't interact with AI at all or or you will have very limited interaction with AI. Again, I think the comparison has to be examined. Like the early days of the I don't know how old you are, but the internet is Yeah. The internet is not one thing, right? So presumably these guys didn't get their attention systems destroyed, >> right? >> No, not at all. They they learned to they learned to program, right? They learned to they read a lot of Wikipedia. >> Being someone who knows how to program, who can relate, that's the kind of person I'm talking about. They actually literally know there's different programming languages. they get the difference of where the hardware comes from. Different affordances of different types of that's a very different relationship. So, but that's not what people mean when they say get their kids to use AI. What they mean is like get my kid to be a prompt engineer so maybe someday he can possibly have a job in a world where there won't be any jobs and the only jobs will be the jobs that are available because of the tech guys uh new software update. >> Um, and so that I don't think that's the right that's not the right attitude. I mean uh but you know the head in the sand thing is the thing we don't want to do right so we really need to find a way to educate kids and this is a broader question of well what does it mean actually to succeed late in adolescence right this is the question we don't have initiations in our culture but people still become adults so there's this like quote unquote on paper right so there's this question of like what actually constitutes a healthy resolution of the adolescent identity crisis? Um this is the question of educator. What does it mean to be well educated? And you know, do we want to live in a culture where kids don't learn to read and write but have AIS read and write for them? Right? We could make that culture. We could have a world where that is the thing. Um, do we want to have a world where kids don't have human boyfriends and human girlfriends and human best friends, but they have machine boyfriends and girlfriends and best friends? Is that an okay outcome of adolescent socialization on a broad scale? This is the question. And so when it comes to how should we have kids interact with the technology, we have to think about that. What's this going to look like when they're 17, 18, 19? And we want to be able to say you're now an adult and you're a grownup. Are we going to be happy with the skills they've built, the person they've become, their habits and, you know, desires and images for what they want to become. And so if you hold all that in mind and then you think about how to give your kid AI, that's a totally other question. But if the idea is just like throw them in the deep end of the newest, latest, fanciest technology because otherwise they will fall behind in this doggy dog world for a job, you really need to get some perspective uh on the thing. >> Yeah, this for me it it falls into your larger philosophy of of education as as a very central issue. Uh perhaps you could you could explain how you see education as something much deeper than than how we normally think about it. >> Absolutely. And we've been dancing around it like the attachment system in humans allows for really long duration intergenerational transmission. So I'll say that another way like it's a species specific trait of the human to be really young for a really long time which means that our you know the young members of our species are particularly vulnerable and need an ex like years of what's called inculturation and the only place that inculturation which we call education takes place is in these deep attachment relationship ships, right? Um, so this is important again, right? When you see a gazelle born in the savannah and it comes out, you know, an hour or two later, the gazelle is doing most of the gazelle stuff, right? It's running around, it's eating grass, it's identifying predators, right? It's not that long until it can actually have another baby gazelle, relatively speaking. When a human comes out, it is years before the human is doing anything that looks like what the other members of the species are doing that allow the species to live and survive. So that's a species specific thing which means education as we know it now is part of what it means to be sapient. It's part of our species specific kind of secret sauce that actually allows for all the other stuff people want to talk about as denoting sapiens tool use, language use, niche creation, radical adaptation. How do you get all of that really long duration intergenerational social transmission? Or you can do ratcheting effect around technology and all kinds of stuff. So primary and how does it work? Attachment relationships. me me the young person, you the older person, together with the world, together with the technology where we both know that's a situation. You know, you're older and no more. I know I'm younger and no less. We collaborate in my interest, right? The younger person's interest. We collaborate in the interest of the youth to give them what they need to take on the project of survival as we know it. That's how I define education. Now, fast forward, you get schooling. That's what most when I say education most people think schools like I'm like okay but schools are a very modern creation in terms of what education is and even in our societies where schooling is supposed to be the place where education happens a lot of education happens outside the school whether the schools wanted or not so I define education as this very broad category and it's part of this species specific trait which makes us human so the risk here which I've been seeing for years and what led me to the study of catastrophic and existential risk is what happens when you get a catastrophic disruption of intergenerational transmission which means a catastrophic disruption of the ability of the elder and the youth to cooperate in the interest of turning over the society to the next generation. Right? If you look at history, people like Turin and you know the fall of the Roman Empire and kind of like Tainter and other people who looking at these big dynamics of civilizational collapse, you realize this is one of the key dynamics of civilizational collapse is actually catastrophic mishandling of the intergenerational transmission thing. And often it's through invasion or flood or technology. And technology in our society has been one of the main things driving the generational gap, which means disassociating the youth from the elder more and more and more and more and more to the point where they're not cooperating in the interest of the youth, but actually at war, generational warfare, which is a little bit where we're at now. Uh it's one way to understand like the student loan crisis in the United States and a bunch of other things where it was clear that the older generation when it looked at what education was didn't see it as a way to collaborate in the interest of the youth but rather as a way to put the youth permanently under the thumb of the older generation. bunch of complexities here that are making it so that across the board we're having crises of legitimacy and capacity so that it's not clear how we fill some of the critical roles in our society going forward. So that's an educational crisis which is like part of a broader civilizational crisis. So throwing AI into that mix, I believe, is the perfect way to complete this catastrophic disruption of intergenerational transmission. And especially if you begin to, as we modeled a little bit earlier, see that these markets are moving towards parenting and teaching and they're not just seeking to replace Google, but seeking to replace moms. And so the question for me as an educational philosopher is what percentage of young people spending what percentage of their time being socialized by machines results in the creation of a new species? This is like the ultimate catastrophic question is are we looking at a speciation event? Um speciation event that is the result of a fundamental change in the process that has been the process that has conferred us sapiens. Right? This is our species specific process. this long duration intergenerational transmission. We've socialized humans. Humans have socialized other humans with with technology in relation to technology, but we have never been socialized by the technology itself excluding the interaction with other human. Um, can that generation understand itself? >> Yeah. Is this something that could happen or how do you understand a speciation event? Would this be merging with the technology in a in a very literal sense or would this would this be about just interacting so much with AI that we lose human culture and because human culture is very much a bit or a huge part of what makes us human. We are therefore in some metaphorical sense a different species. How how do you how do you think about this? I mean, so Jorgen Hovers wrote this book, The Future of Human Nature, and it was about genetic engineering, and he basically said, you know, you have to look at how the future generation understands itself, and if it sees itself as part of the same moral universe as the elders. And so his example was if you perfectly genetically engineered the next generation, would they understand themselves as the same type of being as the parent? The parent emerged through contingency. Right? The parents the parents traits were not designed. The parents traits emerged through its own effort. Right? You're a kid. You were designed by your parents. If you become a criminal, are you responsible or are they responsible? They designed you. So there's this question of are you a member of the same moral universe? So similarly here the speciation event doesn't look dramatic. It actually looks like an unbridgegable generational gap that became a speciation event which is basically these kids look at each other all the kids raised by the machines. They look at each other and they're like hey our parents were not raised by machines. Like who are we responsible to? Like who are we part of a tribe with? Who are we committed to building a future together with? Like, and also, what have we learned that they can never learn or understand? Is if you start to get the neural implant stuff and you start to get the more full-blown cyborgization, you can see the ability of this new cyborg species to basically want to speciate. Um, so again, this is all sci-fi stuff. when I went to graduate school of education, I did not expect this to work, you know, and you know, so yeah, so that that's that's the deepest concern. Um, and in some of the work I've I've done writing philosophically about this, you can think of the death of humanity being the classic X- risk. Just everybody dies, the bomb gets dropped. The death of our humanity is different. The death of our humanity is still a catastrophe. But like most of the physical bodies go unliving, right? It's just a very different way of understanding what the meaning, motivation, value, self- understanding of that human is when it has been raised entirely into a situation like a mechanical automization of socialization. So again, this is all speculative and I don't lead with this stuff because this, you know, too much of the AI risk conversation is dismissed because it sounds speculative and like science fiction, right? So this is not the place to lead with, but it is the place to end with because it is the long duration outcome of this, right? So if you if you look at venture capital seeking to invest in artificial intimacy, they want to start the companies that, you know, create these AI companions. They're projecting multi-t trillion dollar global markets for global intimacy. So what we're seeing as this kind of horrific scenario uh they're seeing as the inevitable growth of the market that they're pursuing which is fewer and fewer people spend time with each other. More and more people spend time just with machines. Um so as speculative as it sounds, if current trends go in these directions, then that's what will happen. age norms, age limits, age regulations must occur or we would face the situation that I've that I've been describing where the market will saturate down into the youth and then you'll have 90% of the language use kids have with machines rather than 90% talking to humans, 10% talking to machines, which is what it should be. We'll have this flip and then what will the teachers do? What will the parents do? What will the guidance counselors do? The pediatricians, the psychologists, all the humans trying to talk to the kids will have no purchase. >> Per perhaps we can try to end on on a more positive note here. Think about how we might avoid this grim situation. Um you mentioned preserving humanity perhaps or preserving our humanity as opposed to preventing humanity from going extinct. We can preserve our our humanity in the face of um more and more advanced AI. I'm wondering practically what that looks like. Is that about communities that have certain values that they enforce and they sort of harden against against the entr against encroachment of of AI into those communities? What does uh so we talked about cognitive uh security for the individual, but what does that look like for a community? And do do you think that's that's uh central to to avoiding the bad outcomes here? Yeah, I mean there's there's layers to it, right? There's we talked about technology. We talked about there's a technological counter to this. We could have a a cognitive defense technology startup industry and that's like a race to the top to protect people. There's also regulations. Like it it seems shocking that there are not third-party agencies that do FDA type safety testing on advanced technologies that are put in front of people's faces and occupy tremendous amount of the time especially kits. So I think safety frameworks, organizations, regulations, you know, I think it was Max who you work with who said like if I want to open up a sandwich shop in New York City, it's actually more complicated in terms of regulations and inspections and stuff to sell a sandwich than it is to put an advanced technology in front of some kid's face. So it's just it's a no-brainer that one of the approaches here, at least in the long run, is setting up context for safety testing. And that seems clear. If you move up higher in like the stack, there are certain types of cultural norms and conversations that need to occur that are confused right now. So, you know, the transhumanist I vision, the transhumanist vision I articulated that was expressed by Kurszswwell, you know, is shared by other people. There's a metaphysical confusion. So it's like we can have all the regulations and that's great but if you move up to the place where people's worldviews their cosmologies uh don't disallow for some of the insane futures people would like to create. So the uploading of consciousness right the coming of the sentient AI that should be granted personhood. Um, so there's a there's a battle that has to be fought at the level of philosophy itself to be able to have much more salient and reasonable things said about what AI futures are going to look like that don't get us into the kind of extreme value relativism that would have us give silicon self-replication higher value than carbon-based self-replication. I can go right to the top of the metaphysical debate, right? So, as I mentioned, like this book, Exit the Silicon Maze, which is going to be coming out with some co-writing with Ken Wilbur and Mark Gaffne, this is the issue because we can we know how to regulate the stuff, but we don't know how to justify the intrinsic value of human attention and human attachment, right? We're dealing with scenarians who believe that that's all fiction. That mind and body are fundamentally separate separable. That the metaphor of hardware software applies to consciousness brain. That the universe itself can be explained in terms of strong computationalism and these types of views which have to be very strongly countered at the level of philosophy itself. You have to understand that there's a small narrow culde-sac and basically anglophile philosophy of mind and analytical philosophy that allows for conversations that are completely unreasonable >> about the nature of the human mind and brain and about the potentialities of the hardware stack that is creating the artificial intelligence. Right? So to me that's the issue which is that there's a niggling suspicion in the minds even of the general public about this inevitability of the replacement of carbon by silicon and of the valueless nature of biological life because it's a random excretion of a meaningless cosmos. So for as long as we don't have a theory of value that can ground intrinsic value of the sapience of protecting the human, we have a bunch of anti-humanists or transhumanists who are willing to gamble with the entire species in the interest of creating some more advanced species on the basis of completely faulty metaphysical systems. So at the highest level, we actually have to be able to say that stuff clearly in order to justify even legal frameworks that would protect our humanity, right? Like why don't we grant personhood to a machine, right? What happens when we do that? Why don't we allow humans and machines to get married, right? like why don't we allow children to be raised completely by a AI? um where do we place these things in terms of our legal theory has to do with how we understand the nature of the human basically metaphysically and for as long as really sophisticated people are basically saying there's no difference between your brain and a GPU cluster there's no difference between relating to an LLM and relating to a person because how do you know the person really is conscious because we do zombie thought experiments right and so like all of this kind of nonsense actually has to stop is a very responsible way You'd never actually educate a child based on that worldview. This is one of my lipnus tests of is a philosophy coherent is can you use the philosophy to do intergenerational transmission in a coherent way. And if you are a person who believes that you should tell children that they have no free will, consciousness is an illusion, they're a meaningless meat machine that evolved for no purpose in the universe, and that they should probably abandon their own life to a greater living being that's a silicon entity. You really want to talk about that around the dinner table with your kids. Uh, you know, you should be removed from parental authority. These are insane worldviews which are spoken from the podium which cannot be used in the fullness of life to make responsible adults. So that's the deeper concern. And what do we tell the kids about the value of their own life and the value of the human experiment on the planet and the over the overarching trajectory of cosmic evolution that we aren't a part of and do we abandon that field of to you know faulty metaphysics and and uh you know neuroatypical cosmologies. Um we can't we can't we have to we have to say profound and reasonable things uh about this technology and not have the discourse about the technology be set by a very small number of people who have motivated reasoning and a distorted view of what is valuable. Um so that's a long-winded answer. So yes regulations, yes something like an FDA, yes age bands, all of that stuff. But also like we need to get back to coherent public philosophizing and move away from these extremely incoherent views about what the human is worth. Um yeah, so I can keep going, but it's a >> it's perfect. It's perfect. Zach, thanks for thanks for chatting with me. It's been great. I've I've learned a lot from this. >> Yeah, man. I've enjoyed it. Fantastic.