AI Horizons: Ethics, Risks, and the Road Ahead
Released on 12/07/2023
Hey, there's two people. [chuckling]
Well, it's my great honor to be here today
with two women whose advocacy
and research have contributed heavily
to my work over the course
of the better part of the past decade
covering how artificial intelligence can harm humanity.
Their efforts continue to inform how policymakers
and regulators in the U.S. and abroad,
you know, are taking steps to ensure
that AI does not discriminate or violate human rights,
and to avoid the reification of power.
Dr. Joy Buolamwini is founder
of the Algorithmic Justice League
and author of the new book,
got a copy here, signed copy,
thanks to my partner, Kaylee, Unmasking AI.
She's also co-author of Gender Shades.
Gender Shades was a research project which concluded
that commercially available face recognition algorithms
misidentified women with dark skin more often
than other people.
That research and Dr. Joy's advocacy
led to bans on face recognition
in several major U.S. cities, including San Francisco.
Margaret Mitchell is Chief Ethics Scientist at Hugging Face.
Previously, she worked at Microsoft Research,
and was the founder and co-lead
at the Ethical AI team at Google.
Her work on the stochastic parrots research paper,
critical of the negative impact of large language models
on individuals, the environment, and society.
Time Magazine named her, earlier this year,
one of The Most Influential People of 2003.
Please join me in welcoming our panel.
[audience applauding]
Margaret, or excuse me, Meredith Whittaker,
the President of Signal Foundation,
was scheduled to join us today,
but she was feeling under the weather and unable to attend.
Margaret, something that stood out to me
in getting to know you better and other members
of the ethical and responsible AI research community
is hearing stories
about how there was sort of a seminal event
that led people to feel motivated to enter this field.
What was that moment, or series of seminal moments, for you
that woke you up to the urgent need
to address how AI can harm people?
Yeah, that's interesting.
I hadn't realized that most people had a seminal moment.
But for me, I was working at Microsoft Research
on a computer vision-to-language generation system,
so translating images to language
in order to describe it or tell stories about it.
And I gave it a sequence of images about a massive explosion
that had happened at an oil factory,
where 30 workers were wounded, actually.
So I gave it the sequence
and said sort of, you know, Tell me about this.
I'm anthropomorphizing a bit.
And it took a look at the sequence.
It saw the view
that the photographer had taken in the image,
where there were purples and pinks in the sky
with this massive explosion, all this fire,
all this, you know, harm, all this intensity.
[Khari] Yeah.
And it said, This is a beautiful view.
This is awesome!
And I realized at that moment
that it was completely disconnected
from the realities of what an explosion would mean.
And that's because of the data.
Because the data had lots of, like, pretty sunsets, right?
Everyone takes pictures of beautiful sunsets
or awesome sunsets.
And so it had learned
that if you see these purples and pinks,
and these, like, really intense colors in the sky,
it's something beautiful and awesome.
And it made me realize,
people weren't thinking about the connection
between what was in the data
and what the system would learn and say.
And the system was very disconnected
from the realities of human life.
And no one was thinking about it at the time.
Everybody was just trying
to make this technology get better, right,
like, not paying attention to the data.
So I was sort of like,
Well, if no one's thinking about this, I guess I have to.
And then, I turned a page
to really examining the kinds of biases
that are in data and how they affect people.
[Khari] And Dr. Joy, what was that moment like for you?
Well, that is the moment
that is on the cover of the book, as you see.
I was at MIT as a graduate student.
I had no intention
of creating anything like the Algorithmic Justice League.
I wanted to make a fun art installation.
Long story short, the face tracking software
I got to make that project work didn't really work so well
on my face until I, literally, put on a white mask.
And even before the mask was already on my face,
it was being detected.
But when I took the mask off,
my dark skin face wasn't being detected.
And so that was the moment,
this moment of coding in white face
at this epicenter of innovation.
Going to MIT was my dream school, after Georgia Tech.
Go Yellow Jackets, right? [Khari chuckling]
And so I was fascinated, amused,
annoyed, many mixed feelings.
But I thought, You know, I can't make a conclusion
just based on this one experience.
But it was the motivation
to really start testing out AI systems, some open source,
some from tech companies that all of you have heard of,
to see if my experience was unique
or indicative of a much larger pattern.
And it was what I call my first knowing encounter
with the coded gaze.
Yeah. Yeah.
It's been a real busy year for AI.
I think we could say that. I think that's safe to say.
What existential risk do you see on the horizon,
and how can we get ahead today?
So when I think of x-risk, I think more of the excoded.
Those who stand to be exploited,
extorted, even exterminated.
You now have AI systems that are being used
to select military targets with known collateral damage,
meaning precious human lives being knowingly decimated
with algorithmic precision.
And so, we're already in a world where AI is dangerous
when we're thinking about that context.
And I also think about the ways
in which AI can kill people slowly,
so when you think of this notion of structural violence.
So we know the bombs. We know the guns, right?
We know that type of violence.
But there's also a certain type of violence that happens
when you don't have access to adequate healthcare.
There's a type of violence
when you don't have the economic opportunity
to actually gain the resources that would better your life.
And so, this also impacts people's lives,
and impacts people's livelihoods,
and those types of examples.
Let's say,
somebody not getting the medical insurance coverage
they need.
So, they get kicked out
of the hospital sooner than is necessary.
We're seeing that happen
with the lawsuit against UnitedHealthcare,
where they're saying the algorithm's, you know, inaccurate
about 90% of the claims.
[Khari] Right.
And so that's what I think about, the excoded,
the people being harmed right now.
Yeah, that's the nH Predict AI tool, I believe. Yeah.
Margaret, any thoughts?
Yeah, I mean, Joy always says what I'm thinking,
and says it much better than I could ever say it.
[Khari chuckling]
So, I guess I'll just try and add onto that to say
that, you know, you said on the horizon.
And I think this is a little bit speaking
to this misunderstanding
that there's gonna be some future point
where suddenly there's an existential risk.
You know, suddenly everyone in the world gets blown up
or something like that.
But what that misses
is that there's paths leading up to that point.
It's not like something is just gonna suddenly,
fundamentally change and a lot of people die.
People are dying now, today, due to deployed AI drones
that are making the decision to kill people, right?
That's happening now.
And so it might be on a smaller scale than the entire world.
And actually, you know, disproportionately,
it's, you know, people of color,
people who are in lower poverty areas.
But we need to just pay attention
to the relationship between how we might be at a point
where everything is exploding,
or, you know, everyone's being bombed,
or whatever it is, and work backwards,
something that I like to call back sight,
to what is the steps that lead to that?
What's the path?
And it turns out we're on that path,
once you work through it.
So, you know, there's existential risk now,
and unless we, you know, take it more seriously,
it's just gonna get worse.
Sorry.
Yeah, I know there was a lot of schism, rift,
whatever words you want to use,
between existential risk and existing risk this year,
a lot of debate around that.
Do you feel like for members of the ethical
and responsible AI community
that the word existential has become like a loaded term?
Well, I think it might be.
I'm pretty sad about that,
'cause my dad was an existentialist, actually.
So I'm like, I want to. [chuckles]
[Khari chuckling]
I don't want to lose this term.
It's an important term for me.
But I do think that, you know, like everything in language,
it evolves with respect to different populations
and how they sort of claim it.
And I feel like that terminology is being claimed
by, predominantly, privileged, white men
to mean a certain kind of view of technology
where they ignore, you know, the work
of sort of everyone else.
And so, that's really problematic.
But I, actually, just put out a piece today
where I tried to sort of explain
if we think about this as a right to existence,
then we can actually unify, I think, a lot of the views.
And maybe, we don't have to lose the term existential
to mean something, [chuckles]
something that only a privileged class can kind of speak to.
So I hope to not completely lose it.
Yeah, can you talk a little bit more about the,
I know that we had talked previously
about, you know, this piece that you released today.
Can you talk a little bit more
about a rights-based approach to AI regulation?
What that looks like?
Yeah, well, there was a moment where I realized
that a lot of large tech corporations were advocating
for a risk-based approach to AI.
[Khari] Mm-hmm.
And as soon as you have a lot of corporations
aligning and agreeing on something,
it should sort of set up red flags.
Like, What am I missing? Right?
'Cause usually, they're motivated
by somewhat different goals than helping humanity.
So I was sort of thinking through it a bit more
and the sort of disconnect
between what regulators are aware of in terms of being able
to like, anticipate risks of given technology,
even if they're not working on that technology,
and how that like fuels narratives
from larger tech corporations
that regulators can't understand.
So if it's in the realm of just risks,
then it might be a little bit easier
to make the argument that like,
Oh, well you can't actually say what the risks are,
because you're not in the weeds of the technology.
But governments are set up to protect rights, right?
Human rights, civil rights, cultural rights.
[Khari] Yep.
And so I started connecting the descriptions of risks
to the kinds of rights
that governments are set up to protect,
and realizing that they mapped together relatively well.
And then working through it further,
it just sort of became clear to me that a lot of the harms
that are coming from AI, that will come from AI,
are directly tied to risks.
Like, you know, or sorry, directly tied to rights,
like a right to non-discrimination,
a right to equality of opportunity,
a right to existence, these kinds of things.
The UN recently put out a piece
talking about the different rights
specifically relevant to AI.
So it just sort of, everything fell up from there.
It became really clear
that regulation might be more straightforward
from a rights-based approach.
Awesome.
Dr. Joy, I saw you on conversation
with OpenAI CEO, Sam Altman, last month
here in San Francisco.
Did any of his responses surprise you?
I don't think I was necessarily surprised
by his responses.
He reminds me of so many people
that I meet in the tech world,
and kind of this privileged optimism
about what AI can do and how AI can be beneficial.
And in that conversation,
I was really looking for more specificity.
And I think we got there in some areas.
One area, ironically enough, in that conversation
we were talking about was the future of work.
You know, and that was 10 days
before all of the interesting changes
[Khari] that happened. Yes, yes.
And a piece that we started to explore a bit
was something I brought up called the apprentice gap.
And so, bear with me.
I'd recently picked up playing guitar again.
[Khari] I heard. [chuckling]
I was in Rolling Stone, and I was feeling good.
So I went and I got a little yellow guitar
to match my dress [Khari chuckling]
in the Rolling Stone theme.
And as I was playing it,
I realized I still had my calluses.
Right? You know?
And I realized the work that I had put in years prior
was still helping me out.
So it made me start thinking of this notion
of professional calluses and what we gain
when we're doing, maybe, the scaled
or the mundane essentials within any profession,
and then what happens when we have AI systems
automating some of this work, the entry level work.
Do we risk, potentially, being in the late age
of the last masters or the last experts?
And also understanding our own human development
as not just being output, particularly as an artist.
What does it mean to go through that process of creation,
finding the right word, figuring out how to express yourself
and learning something in the struggle to do it?
So that was part of the conversation that we were having.
And I was also happy to hear, and we've been hearing this
from many [chuckles] tech organizations,
saying they want to be,
they don't want to be self-regulated.
They want to have regulations.
And so we have to, of course, think
about corporate capture there,
but we know self-regulation isn't going to work.
We saw it with social media.
And so it's helpful to, at least, not have that resistance
that has been a signature of the past.
Yeah, I know something that stood out to me,
I don't recall if it came up in that conversation,
but certainly in the book, in learning more about you,
is that your dad is a scientist and your mom's an artist.
And that makes so much sense.
You're a poet and a computer scientist.
You know, I'm interested in,
I think we've talked a bit about this,
but you know, how should tech companies be held accountable
for the impact AI systems have on society?
So I can speak to sort of the rights-based approach,
where you want to be able to demonstrate
that you're respecting people's rights,
so showing that your systems don't discriminate,
showing that your systems give rise
to an equality of opportunity for different people
from different subpopulations.
And I think that if you don't show that,
you have to be fined enough
for it to really matter for your bottom line.
'Cause currently, when there's any sort of wins
against, you know, the tech machine,
they're fined like peanuts.
And this is the kind of amounts
that they likely budget for already, right?
So if they're budgeting for $4 million in some fine
for, you know, you know, Q1 or whatever,
and it's only $2 million,
then it's like they've gained $2 million.
So you really need to make it matter
in terms of their bottom line,
because that fundamentally drives
what tech companies will do.
Yeah.
And you know, so often, I think the question,
and I think this is a great, you know, panel to ask this,
but so often the question in tech, I think,
is what we should build.
But I'm interested in asking both of you
sort of what form of AI would you like to see go away?
[chuckling]
What needs to get eradicated?
Yeah, probably no surprise coming from me,
facial recognition being used for face surveillance,
lethal autonomous weapons where we're giving machines
and automated systems the kill decision.
I do write about the campaign to stop killer robots,
because it's really about valuing human life.
And the more abstracted it becomes,
the less proximate we become
to actually understanding what it is to take a life,
the easier it becomes to do.
And so I would continue to urge
that we absolutely put a ban on lethal autonomous weapons.
And I remember when I first started getting into issues
of AI and society, you know,
there'd been a thousand researchers who'd signed a letter
saying we should ban lethal autonomous weapons,
well before, you know, I started this kind of research.
And so the escalating conflicts
around the world put this on the map even more.
Because the future of peace, as some would call it,
or the future of war,
means that we have AIs as angels of death at the moment.
And so, I think we should do everything to stop those.
Well said. Margaret, any thoughts?
Yeah, I mean, I can plus one that,
and add on, add an on,
and add on that there's a branch of AI
called effective computing
that deals with things like predicting people's emotions.
[Khari] Mm-hmm.
And then, there's related work
on predicting people's personality,
trying to predict things like criminality.
[Khari] Right.
And any AI work that is trying
to predict people's internal characteristics
shouldn't be advanced.
Not only because it probably doesn't work, [chuckles]
and it's really just reflecting the biases in the data
based on surface level characteristics,
but also because it only opens up more avenues
for discrimination.
Yeah, yeah.
You know, each of you have spent, you know,
the better part of the past decade
working on critical analysis of AI.
And, you know, this is the last question.
But how can people work together
to protect their communities, their families, and society?
What gives you hope?
And do you have any favorite stories of resistance?
There's a lot of things to do,
and it, like a lot of things in AI,
[Khari] I think it depends on context. Mm-hmm.
So I would say that one thing
that actually really changes what happens in AI
is headlines, and this is something
that I've learned from Joy really well.
I'd be fighting for something at Google and nobody cared.
And then, Joy would like make a headline about it
in a way that was much more powerful and much more clear,
and suddenly everyone at Google cared.
So they value the input from PR and headlines,
sometimes a lot more than their experts internally.
So to the extent that you can support the free press,
and if you're in tech, talk to the free press
to the extent that you can.
That makes a really big difference
in terms of what technology companies will focus on.
And it's part of why I appreciate your work so much,
[Khari] as well. Thank you.
Because you really like to dig
into, you know, the sort of serious problems
where we want to reshape how technology works.
And so I think that's a critical piece of it.
I don't know about hope.
Can you come back to me for hope, maybe?
[Khari and Margaret laughing]
[Khari] Think on that for a sec. Oh no.
[group laughing]
I'm happy to jump in here
[Khari] while you buffer on hope. Okay. [chuckling]
So hope buffering over here. [Margaret chuckling]
I am so much so thinking about the right to refusal.
Because oftentimes,
it can feel like technology's inevitable,
or we just have to use it because it's been released,
that kind of thing.
One area people can exercise the right to refusal
is at airports.
You have TSA rolling out facial recognition.
It's supposed to be opt in.
That's what it says on the website.
We at the Algorithmic Justice League
have people submit reports.
You know, http://report.ajl.org,
or in this case, http://fly.ajl.org,
where people are saying
that they didn't even know they could opt out.
And so sometimes, you have these situations
that in practice become coercive consent.
So I do think the right to refusal is important to exercise,
and it's important for people to know.
I also think it's worth exploring ways
in which AI tools can be beneficial.
Today, I was reading about PaidLeave.ai initiative
from Moms First U.S., headed by Reshma Saujani.
And it is talking about the ways
in which many people fear retaliation
or just feel overwhelmed
when it comes to actually getting benefits
or seeking ways to have paid leave,
whether it's for bereavement or other reasons.
And so now they've released a tool, right,
where you don't have to go through the PDFs
or try to hunt down that policy.
But you can ask, through this interface,
questions you might otherwise feel afraid to,
particularly, if you're an immigrant
or you have some other type of status
that might make reaching out a bit more precarious.
So when I see those sorts of examples, I'm inspired.
I know I look youthful. [Khari chuckling]
There are those who are younger, right?
So when I see organizations like End Code Bias
with high schoolers, you know,
talking about algorithmic justice,
pushing back on school surveys,
asking, Do we even need these systems in the first place?
That very much inspires me.
When I see the work
of, honestly, people like Meg Mitchell, right?
So, we have Dr. Mitchell here.
When I think of the work
of, you know, our colleague Dr. Timnit Gebru
and so many others who have put their careers on the line
to speak truth to power, to warn so many of these warnings
that people lost their jobs for.
Now, it's taken for granted as known risk. [chuckles]
You know, that wasn't always the case.
I'm hopeful that I was sitting, you know, one seat away
from President Biden talking about the story
of Robert Williams being falsely arrested
in front of his two young daughters,
and that we have governments around the world
that are actually attending to how we prevent AI harms.
I can assure you, when I started this work,
this was not getting that level of attention.
And oftentimes, I'd be like,
Oh, now you tell me a computer is a race?
Oh, come on. [Khari chuckling]
You know, that kind of thing. [chuckling]
So now everyone has to at least say,
And we know there's bias,
and we're dealing with the discrimination.
Now, we have to make that mean something.
But those are incredible narrative wins in the space
that ought to be celebrated
while knowing there's so much more work to be done.
Thank you for reminding us
of ways that we can empower instead of exploit,
and for ending on a hopeful note.
Join me please in thanking our panel, Dr. Joy Buolamwini
and Dr. Margaret Mitchell. [audience applauding]
WIRED25: Kai-Fu Lee and Fei Fei Li On What's Next for Artificial Intelligence
President Barack Obama on Bureaucracy VS. Moonshots
WIRED25: Accenture CTO Paul Daugherty On Reimagining The Future of Business
The Robot Will See You Now – AI and Health Care
President Barack Obama on How We'll Embrace Self-Driving Cars
UN Secretary-General António Guterres speaks with WIRED's Nicholas Thompson
Incite Presents Laura Boykin and Malkia Devich-Cyril in Conversation with Peter Rubin
WIRED25: Google CEO Sundar Pichai on Doing Business in China, Working with the Military, and More
Taking on Amazon, Google & Silicon Valley: EU Competition Chief Talks Tech with WIRED's Nicholas Thompson
Re:WIRED GREEN 2022: Going Electric Isn’t Enough—We Need a Clean Resilient Energy Grid to Run It