Why we can't fix bias with more AI w/ Patrick Lin

my morning routine goes something like this I wake up hit snooze a couple times finally get out of bed and make myself a cup of tea the tea is probably the most important part and then I sit down with my tea and look at Twitter to see what everyone's talking about and yes I'm still calling it Twitter now in Feb of this year I got out of bed brewed my tea and started scrolling and that's when I saw that Google had launched an AI image generator inside their chatbot called Gemini and Twitter was on fire people were calling Gemini all kinds of things racist woke biased and everything in between and the thing was everyone seemed to have a completely different take on what was happening so what actually happened well Google launched their image generator inside Gemini and folks started using it one of the initial tweets from the user at and wokeness showed screenshots for Gemini's response to a bunch of different prompts ranging from the founding fathers of America to Viking to Pope while you might expect that each of these prompts would yield pictures of mostly men and mostly white men at that Gemini would prove you wrong the founding fathers Vikings and popes Gemini generated were people of color and in the instance of the Pope some of them were women okay so this is obviously a pretty significant problem of historical inaccuracy other users shared screenshots of Gemini's responses to their own prompts and some of them were pretty bad actually among the most egregious pictures of people of color in Nazi uniforms definitely not the kind of diversity we're looking for then Google weighed in praka ragavan a senior vice president at the company wrote in a blog post that the Google team tried to get ahead of quote some of the traps we've seen in the past with image generation technology such as creating violent or sexually explicit images in other words they were trying to correct for AI bias which had become a major topic of conversation in AI ethics circles but this resulted in pissing everyone off to some on the left it was advancing a kind of colorblind identity politics that glossed over the history of Oppression to some on the rights it was over representing minority groups and advancing some kind of conspiratorial big Tech woke agenda in fact Elon Musk always the provocator called Gemini both woke and racist go figure everyone was shouting about the bias baked into the system but in doing so they ended up revealing their own biases in other words the lens through which they see the outputs of these AI systems I'm belaval sadu and this is the Ted AI show and on this episode we're tackling one of the thorniest issues out there bias in AI so look some of these images that Gemini generated are really bad I'm not excusing those but I get what they were trying to do essentially they ended up overcorrecting for some of the major misses that AI image generators like Dolly mid journey and stable diffusion have stumbled through in the past like in late 2023 a group of universities put out a joint study about their findings on text to image generators they found that when they prompted for pictures of surgeons and surgical trainees they got some trouble results the vast majority of images were white men in surgical gear which isn't actually reflective of the demographic breakdowns of Surgeons today another investigation by The Washington Post on bias and AI generated images showed similarly troubling Trends The Prompt Muslim people revealed men in turbin and other head coverings The Prompt Attractive people yielded pictures of only young light-skinned folks and the promp productive person generated pictures of men sitting at desk most of them white now there's been so much talk about bias in AI from flaws in the outputs to flaws in the training data in cases like the Gemini Scandal we risk historical inaccuracy and miseducation undeniably real problems and in other cases like the study in the investigation I just mentioned bias and AI can keep perpetuating harmful stereotypes that don't actually reflect reality so if everyone agrees bias and AI is a problem then why isn't anyone fixing it to help us unpack back this I sat down with Patrick Lynn a professor of philosophy at California poly Technic State University and the director of the University's ethics and emerging Sciences Group which tackles topics like Ai and predicted policing the future of autonomous vehicles and cyber security in space he's been examining the ethics of technology for a long time and is thinking a lot about bias in AI where it comes from and how it impacts us on a daily basis so when we say the ethics of AI we're talking about a huge almost NeverEnding topic right can you explain for our audience why AI ethics is such a vast topic so when we're talking about human ethics you know uh your ethics and my ethics we could do that because we like to think we have free will and we can make choices and some of the choices we make are um ethical or unethical or neither but when you talk about machines some people rightly point out hey they're not moral ages they don't know what they're doing so how could it be held to any kind of ethical standard so quickly right off the bat I would say that's a wrong interpretation a wrong understanding of Technology ethics or AI ethics ethics could be about you know not so much about the technology as an agent but about how the technology is designed it could be about the ethics of the technology developers it could also be the ethics of the technology users it's about a whole ecosystem of developers us users stakeholders unintentional stakeholders environmental interest um you know and so on when it comes to AI now ai I mean you know at a very high level you could think of AI as you know an automation of a decision making process right so AI decides well what is this image I'm looking at what is this text I'm seeing and it makes a decision on predicting the the next words right so it's a it's a decision engine of sorts and because it's a decision engine it could be used to replace decision makers if AI can be uh integrated in society and a lot of these decision-making roles then that already implicates countless domains right AI in agriculture in chemistry and education in Warfare uh it's hard to imagine a single domain where AI cannot be applied to this means that you know you're really looking at the entire universe of ethical issues potentially for AI ethics that's a great Point especially as AI permeates all these verticals and domains as you say the surface area for this bias to manifest itself is also very very broad right and so today you and I are going to talk about bias in Ai and there's a bunch of interesting examples in there just a few that we've come across one is racial bias and facial recognition right some facial recognition systems have been shown to have higher error rates for people with darker skin tones potentially leading to false identifications right Amazon scrapped an AI recruiting tool after discovering that it was penalizing resumés that contained the world women's after downgrading graduates of all women's colleges reflecting gender bias in the training data what are some concrete examples of bias that you've encountered in the AI space if we start out with the you know technology dour which is llms you know like chat GPT and you know AI writer or chatbots um we could already see bias in their outputs I'm thinking about Google Gemini's recent debacle where it wants to diversify the ethnicity of Nazis and the founding fathers of the United States right who are all white um uh but but you know if you think that wait a minute uh anti-bias anti-discrimination means you got to mix it up with the with the ethnicities and the genders then that's how you get some false negatives but I think the Big Stake examples are still related to AI bias in hiring which you mentioned but also AI bias in Bank lending and criminal sentencing and I would even include things like AI policing so these are potentially life and death decisions even a bank loan decision could be a life and death decision if you're denied a loan for a mortgage then that could mean you lose your house you could become homeless and you know homeless homeless unhoused folks tend to have a shorter lifespan than than um other folks right so this so these are Big serious decisions and even if the AI doesn't look specifically for gender um ethnicity age it can still deduce a lot of this information from from uh other data so for instance um a banking AI you know making a loan decision could could could be programmed or trained to ignore ethnicity right ignore race um but it could still discriminate in its outputs in its impacts so for instance it might say it might uh you know given us trading data it could say oh you know what uh borrowers from a certain zip code you know have a high uh uh rate of default so we're going to just not give loans to people in the zip code but guess what it turns out that zip code is is full of minority neighborhoods all right so it is a proxy for race or ethnicity which is discriminatory for almost any given AI application you could probably come up with some kind of you know some kind of weird case uh of bias I mean we talk about Healthcare AI if a medical AI is trained primarily on say white patients then it might misdiagnose someone who you know of African descent or Asian descent or Jewish descent and by the way this isn't just a you know a white versus other thing I mean if you look at facial recognition uh uh projects in China for instance where they train their AI mainly on Chinese faces they have a hard time recognizing differentiating white faces right just because white faces are underrepresented in their data set so so it's not it's not that AI is inherently racist uh in One Direction it depends on the training data absolutely it brings up the concept of implicit bias also and how it might surface in AI in ways that we don't expect one example that I've been really fascinated by recently is if you type in just a word or token Thief into any text to image model you're going to get an image that resembles a character from the video game Assassin's Creed or the 2014 video game Thief rather than you know the stereotypical depiction of a thief wearing that mask with a money bag slung over the shoulder or worse a racist caricature you get this person in a CA in a cape uh and that's what the model thinks a thief is right and so this seems to reflect biases present in the training data which in this case over represents video game imagery how can we account for and mitigate these sort of implicit biases in AI systems you know so we ensure that we're not embedding or reinforcing problematic stereotypes let's say from Media or pop culture implicit biases by definition are hidden they're under the surface I I mean you you've lived in it for so long you don't even realize it's there you know I mean you must have heard the joke where you know there's two fish and one one fish asks the other fish hey how's the water and the other fish says what's water right I mean it's just so pervasive that you're not even aware it exists I me that is part of the problem recognizing bias when you see it and and um and AI bias is a popular problem because people understand bias they can imagine that you know they could be on the wrong end of an AI decision someday you know no matter who you are no matter how privileged you are um that that could be you so that's why AI bias out of all the various issues in AI ethics you know might be the the most um the most uh well-known one most widespread one um but the big trick is how do you get rid of AI bias bias is such a tricky problem because of you know I think how humans are just simply hardwired and constructed um I mean you know think about our brains right we're not just flaw machines we're stereotyping machines that's what we're built to do you know we're built to uh we're built for one shot learning we're built to learn very quickly um and I mean early on you know Humanity's uh history this is a this was critical for survival so imagine you know imagine you're the first caveman who've ever come across a carrot you think o what is this weird orange thing I wonder if it's poisonous or not you nibble it you eat it you survive right it's it's natural to make a judgment that anything that looks like this is also going to be edible right so that's that that's a form of stereotype it could also go too far especially when you're talk about people individuals have so much uh variation from one one person to the next even inside the same groups you know whether you're talk about ethnics group religious groups or um or whatnot um but also another tricky thing about bias and and stereotypes is that you know it seems that there's some kernel of Truth in some in the stereotypes right otherwise they wouldn't be stereotypes but you know but to but to make such a broad judgment and start making decisions based on stereotypes that uh seems AC cross a line I think that's a really good point right like as you say bias has existed since time Memorial it's almost intrinsic to our nature it's perhaps a simplistic way of you know looking at patterns and extrapolating based on that and so it's going to be really hard to solve this problem in the AI space right and we can't use more AI to solve it because AI doesn't know right from wrong like what even is right what is the truth right since it can't detect what is and isn't biased or racist or misogynist um you know the logical fix is to train the AI on less biased data but where do we find all this less biased data because the data is generated by humans that bring their own biases to the party MH so this is no small task right and I think you've set up the problem appropriately so to your mind how can we even start to fix this bias problem in artificial intelligence we don't understand bias well enough we do have a intuitive superficial understanding of bias you know you might think of bias or discrimination as just treating people differently because they're different because of their different gender or ethnicity or religion and these are generally legally protected categories that's the usual understanding of what bias is but if that's all you have you're going to get it wrong we need a deeper more nuanced understanding of bias if if we're going to truly uh tackle the problem What are problems that arise when our definition of bias and AI isn't sufficiently nuanced one example would be this if you think it's inappropriate if you think it's discriminatory and biased to treat people differently because of an age or gender um you know if that's all you think bias is it's going to give you a lot of false positives right so here's an example that shows that it might be okay to discriminate on age and on gender and on ethnicity at the same time right so imagine I'm a filmmaker and I'm interviewing actors uh um for the title role of Martin Luther King Jr right I'm going to reject every single Asian teenage girl who auditions for that role and I'm and I'm rejecting them for a job precisely on the basis of their age their gender and their ethnicity but that seems okay you know at least if I'm trying to make a historical accurate biopic right it seems legitimate um that I could I could filter out applicants based on their profile if they don't match the age ethnicity and gender of of what I'm what I'm U aiming for you can't simplistically say thou shalt not discriminate based on protected categories and hope for the best right so clearly there are a lot of problems what role do you think subjects like ethics philosophy social sciences play in the training of these AI researchers and developers that are building these next Generation systems oh I think it's huge I have great respect for Science and technologists I wanted to be one of them when I was growing up before I accidentally found philosophy they fundamentally want are curious they want to know how things work but more than that they want to change the world for the better but here's the problem uh you also got to understand the world in order to make those kinds of interventions technology isn't you know it doesn't really do a great job in solving social human problems only humans can really solve social problems technology and AI they're tools they they could do some things they could you know alleviate some of the symptoms of these problems but they have a hard time getting at the root of the problem take the human very human problem of drunk driving right now it's hard to change culture it's even harder to change drinking culture in America but one thing we can do is make cars safer right so we can make uh cars more survivable if you get in an accident they're saving more lives but are they doing anything to the drunk driving problem you know are they making any progress in rolling back drinking culture I would say no right in fact it might be worse it they could be encouraging more drinking and more drunk driving if you know your car is safer and you're more likely to get home uh in one piece and you're less likely to kill random pedestrians or other drivers then that's an incentive to drink more because you know things will be okay so we've talked about the problems AI creates I want to switch gears a bit and talk about Solutions mhm I'm wondering if you see folks or companies or organizations out there doing anything to fix the problem of bias and AI well I mean I do see a lot of organizations say they're working on things to fix AI it's not entirely clear how they're doing it some of this proprietary information um but still you know again I would still be skeptical that that uh their Solutions are going to do a whole lot in solving the problem the one move they they they're thinking of is to throw more AI at it and this is the you know this is exactly the problem of if all you have is a hammer then everything looks like a nail right if you're an AI you know programmer or developer of course you're going to think AI is going to be able to solve that problem and that's what you're going to try but I think with bias um uh that's a different kind of challenge for one reason bias is a social construct so the challenge facing uh developers and making it AI that can detect bias is the same kind of challenge uh with developers who think they can make AI that can detect pornography or can detect an unethical situation right pornography ethics they're social constructs too they're very squishy they're very hard to Define they resist definition the US Supreme Court had famously uh concluded we might not be able to Define pornography but we know it when we see it right and I think humans are like that with bias too like we we could proba we can recognize bias when we see it most of the time right in my Martin Luther King example you know film example you could recognize that I'm not being malicious I'm not I'm not doing anything inappropriate um but a machine might not be able to so machines are not great with ambiguity there's no law of nature that says technology will solve all your problems right I mean it's it's it it's made life easier in a lot of ways it's made us more secure in a lot of ways but it still hasn't solved hunger um racism you know think about all the societal ills if it could why aren't we working on that I mean all we have now are apps that make life more convenient here's an app that could find me a ride here's an app that could find me a place to crash T night here's an app where someone will do is chore for five bucks we're putting Ai and all all our best Minds on to these projects to do things that someone else's mom will do for you they're not they're not like these great world shaking uh applications so back to bias I I think the Temptation here is just to throw more data at it a couple ways we can go here uh yes you could curate your data sets to ensure that AI is being trained on uh examples where there's no bias but when you're talking about millions and billions of of you know examples in a in a large training set that's not a very feasible solution it's definitely not scalable um so so so if you want a scalable solution seems that you would need to create an AI train it program it to identify bias when it sees it but to do that it needs to be crystal clear on what bias is what discrimination is if you try to look up the definition of bias you're not going to find a really good one they say things like discrimination is the unfair treatment of people now you have to Define what fairness means or unfairness means right but I think so that work hasn't been done and if uh if developers don't understand the nature of bias then they're going to have a hard time fixing the problem you know I'm trying to imagine an AI that can understand all the nuances of every situation here you know of of whether this is relevant or this factor is not relevant and I have a hard time imagining um uh that that could be done it sounds like what you're saying is we're not going to have one model to Ru them all right it seems like it's a lot about context I want to spend a little time on that um there's a lot of discussion about National AI models that are tailored to cultural contexts that vary from region to region right the concept being that bias and AI isn't a one-size fits-all solution and what constitutes bias can vary significantly based across you know across different cultures regions and Nations what's considered acceptable or unacceptable offensive or benign can differ based on local Norms values histories and sensitivities what do you think about so do you think we should have a a plurality of models that are tuned to various cultural context and could that be one near-term solution to address this rather thorny issue of bias as a general approach I think it makes sense because there's no one set of values to rule them all right there's no one ethical Theory to rule them all and there are variations and ethics uh from culture to culture and many of them are reasonable variations right I mean others are not reasonable others are just plain offensive right so if a culture that doesn't uh want women and children to be educated and thinks it's okay to throw acid on them uh to prevent them from going to school that's bad I I I don't see any reason to respect that kind of um you know those kinds of values but other other differences could be reasonable so for instance uh you know in Asian cultures elderly tend to be more valued than kids right so for instance let's imagine a AI app that does triage in a hospital right that does Hospital admissions and and and these hospitals in these big cities are generally overworked so it's got to figure out a a priority list for uh um you know for the for the patients to be seen um in one culture you know let's say in Asia it might it might give bonus points if you're older if you're elderly it might move you up the priority list right and that seems okay neither here nor there and other cultures they might have the opposite value they might you know treat their uh children as kings and queens they and they put uh a premium on them in which case an a hospital AI or triage AI in that culture would would move younger people up on the priority list I think we would want to respect uh diversity and these variations especially since there's no one uh one ethics no one culture to rule them all you know I I certainly don't think you know we we have it right here in America same with same with just about every other culture but if if I were a AI company looking to roll out products worldwide now I'm thinking wait I got to localize my my products you know my Ai and these products that means I got to train my AI from data that comes from those geographies and cultures for every Market I want to play in right and that um sounds like a lot of work I mean it could be a deal breaker you know the diverse the diversity uh model makes sense uh in in theory but in practice how do you implement that I I don't know so I have to say do you think we're going to be stuck in the same wacko Loop where the problem grows and multiplies exponentially especially as all these major Labs chase this current Paradigm we're on which is let's throw more data and more compute at it do even bigger training runs or is there a chance that we could get this under control I do see potential fixes but they hard fixes and people don't want to hear about them because they're about human labor things that human beings need to do the work we got to put in to solve this problem what should individuals do to address bias in AI right look this this this is a hard problem because it's a social problem it's a it's a human problem and I think it would be a mistake to put all your eggs in one basket and hope that technology can solve this unfortunately I think it's going to take a lot of hard societal level work it's worth trying out even if only technology can paper over the symptoms you know that might be okay for now right just let just as safer safer cars aren't fixing the problem drunk driving but they're saving lives you know that might be enough so um you know I would say good luck to the developers but I think you know if you really are serious about tackling bias you got to understand what it is thank you so much Patrick this is clearly a complex problem and I really appreciate you taking the time to break it down and explain to us why it isn't a very simple one siiz fits-all solution so thank you for your time and um yeah we really appreciate it welcome thanks for having me on so Patrick said something that I think is really important for us to recognize in order to solve the bias in AI we have to solve the bias in ourselves which is a pretty tall order right especially when as he says bias is pretty much implicit to human nature and I tend to agree with Patrick that throwing more AI at already flawed AI systems isn't necessarily going to solve the problem for us because here's the thing AI is a reflection of who we are after all it's trained on us our art our memes movies jokes history music math science philosophy it's complicated because we're complicated it's flawed because we're flawed it's biased because we're biased but I also don't want to throw up my hands and say we can't fix this or there's nothing we can do because I think there are some things that we are actually in control of when it comes to bias in AI particularly our responses to this nent technology now much has been said about large companies creating transparency around their training data and that's a welcome step but even training data transparency presents its own challenges for starters these data sets are enormous we're talking billions of images and trillions of words it will be a massive effort to comb through it all find all the flaws and make the necessary changes so it's a big big problem with elusive Solutions but I want to offer up a couple of solutions that I think could at the very least help the first is something I mentioned in my interview with Patrick the idea that we could create more Nuance in our AI models by keeping it Regional like generative AI systems in Singapore probably should not behave identically to generative AI systems in California in fact the Singaporean government called for AI sovereignty addressing the fact that quote Singapore and the Region's local and Regional cultures values Norms differ from those of Western countries where most large language models originate end quote I believe that AI sovereignty can help preserve our diversity whether that's state to state or country to Country the second might seem at first glance to be a little too obvious you might not like it at first but hear me out what if we gave ourselves a bit more agency in this issue and committed to getting better at using tools like Chachi BT mid journey and Gemini let me give you an example I want to share a process I go through in my mind every time I prompt an image generator or a chatbot or what have you first I remember that I'm using a flawed tool just because it's Ai and it's supposed to be really smart it's not always going to give me the most accurate results second once I get a response to my prompt I scrutinize it in the same way I scrutinize the news I read I'm skeptical about the source of the results knowing that these AI systems are train trained on imperfect data third I take a beat before I'm even tempted to jump on Twitter and post a spicy screenshot of this messed up response to my prompt I pause and I think about what I need to do to get a better response and I revise it accordingly maybe I'll respond to chat GPT with something like hey not all nurses are women can you show me some images of nurses that are more reflective of the actual demographics of the nursing field because if these generative AI tools are only as good as the we feed them they're also only as good as the prompts we give them and yes it is up to these big tech companies to make better products and give us more and more transparency about how they're trained they should also give us more transparency into how and when they're trying to solve for bias within their systems for example if Google had told us they're trying to address some of the bias in Gemini it may not have solved the problems in the images generated by their system but it at least would have helped perhaps to know why the AI system was generating those images in the first place but also we need to be educated consumers and users of these tools and know that the better we are at identifying their flaws the better we will be at prompting them for better responses that's why developing AI literacy is so crucial we need to understand how these systems work how they learn and how they can go wrong sometimes horribly wrong and we need to stop taking their outputs as gospel or a factual reflection of reality it's as crucial as being literate about our own biases if what an AI system generates is not consistent with our values we can absolutely take control and shape it for the [Music] better the tedi show is a part of the Ted audio Collective and is produced by Ted with Cosmic standard our producers are Ella feder and Sarah McCrae our editors are B bansang and Alejandra Salazar our showrunner is Ivana Tucker and our associate producer is Ben Montoya our engineer is Asia polar Simpson our technical director is Jacob winnink and our executive producer is Eliza Smith our fact Checker is Julia Dickerson and I'm your host belaval sidu see y'all in the next one [Music]