Cindy Moehring chats with Vivienne Ming, a self-described mad scientist, theoretical neuroscientist, AI expert, entrepreneur, author, and is well-known for her research and inventions. The pair discuss the responsible and ethical use of AI as a tool. AI is not able to solve problems that humans don’t know the answers to first. Therefore, the best practice for solving impossible business problems is combining creative people with sophisticated artificial intelligence.
Podcast
Resources From the Episode
- Ming’s Financial Times Article: “Human insight remains essential to beat the bias of algorithms”
- Ming’s article on Diversity-Innovation Paradox
- Socos Newsletter
- Remote Work
- This Is Not the Industrial Revolution
- Explore economic working papers of the National Bureau of Economic Research
Episode Transcript
Cindy Moehring 0:03
Hi, everyone. I'm Cindy Moehring, the founder and Executive Chair of the Business
Integrity Leadership Initiative at the Sam M. Walton College of Business, and this
is TheBIS, The Business Integrity School podcast. Here we talk about applying ethics,
integrity and courageous leadership in business, education, and most importantly,
your life today.I've had nearly 30 years of real world experience as a senior executive.
So if you're looking for practical tips from a business pro who's been there, then
this is the podcast for you. Welcome. Let's get started.
Hi, everybody, and welcome back for another episode of TheBIS, The Business Integrity School. I am really excited to tell you today that we have a self-described, described mad scientist with us, Vivienne Ming, and she is going to lead us through an incredibly insightful conversation that I know you all will be jazzed about. Hi, Vivienne. Nice to meet you and see you again.
Vivienne Ming 1:03
It's always wonderful to have really fascinating conversation. So it's great to be
here.
Cindy Moehring 1:09
Great. Let me tell you guys, just a little bit about Vivienne and then you'll understand
why it is such a treat to have her with us today. In addition to being a self-described,
mad scientist, she's a theoretical neuroscientist, an AI expert, an entrepreneur and
an author and she's frequently featured for her research and inventions. In the Financial
Times, The Atlantic, The Guardian, Courts, The New York Times, you won't believe this,
but she's actually now co founded her fifth company, we're going to hear a little
bit about that journey and what the current company does. Socos Labs, Socos is an
independent Institute exploring the future of human potential. Before that Vivienne
was a visiting scholar at UC Berkeley's Redwood Center for Theoretical Neuroscience.
And she has also invented some really cool AI systems of using AI for good, she's
created systems that do things like help treat her diabetic son and predict manic
episodes and bipolar sufferers weeks in advance, and even reunite orphan refugees
with extended family members. So welcome, Vivienne, my goodness, you, I have to say,
are living the dream of I know, many students today and many entrepreneurs, can you
just share with us a little bit about your personal journey? How did you get to where
you are today and founding your fifth company?
Vivienne Ming 2:28
Yeah. Well, you know, as you unravel my story, and we probably don't want it to take
the entire discussion time. But oh, it could, you'll see that most of what people
are probably imagining is actually the second half of my life. And strangely, I'll
start there, at the start of the second half. So I was an academic, I did my PhD work
at Carnegie Mellon, where I studied cognitive psychology and computational neuroscience,
really loved the idea of using machine learning, artificial intelligence, but to understand
people. And I didn't think of it as just a scientific curiosity, I wasn't expecting
a world to emerge in 2000, whereby in 2020, artificial intelligence becoming a dominant
technology in a lot of fields in the world, from medicine, to logistics, to just ad
targeting. So it was really dumb luck, that I happen to be walking into this field.
But it was also, I'm not an artificial intelligence researcher. I study brains. I
just happen to use machine learning to do that. And it turns out, brains are really
interesting to inspire new forms of machine learning. So for me, it's always been
a tool, something that you apply to a problem. I'm thrilled that people can geek out
about convergence proofs and complexity theory. I love stuff like that. But to me,
it's just fun. Ultimately, this stuff is meaningful, because someone might be alive
because you made a choice that otherwise what's possible. That's what really moves
me. So I was doing my PhD, I joined my staff, Stanford, Berkeley. And then my wife
and I had an idea to start an educational technology company. You know, I got my AI
chocolate and her education and psychology, peanut butter, which if you're old enough
to get the reference, then you're a good person. If you're not well, you can you probably
can guess what I think about you. So i i started this company, and I didn't really
have a background in the business space, I'd been a scientist, I'd been other things,
I'll come back around to that. But I, we wanted to make a difference. This could just
have been an academic experiment, but we saw a way to listen to machines talk to each
other. And from that, without ever giving that formula to them, we could actually
tell how well they would do in the class. And our deep belief was, why wait till a
student fails in class? Or even gets a B? If we could, as soon as possible, identify
where students had miscon, missed concepts, and I didn't want to replace the teacher.
Could we actually create a little dashboard that would allow teachers to explore what's
going on in the black box of their students heads? And I was so proud of myself, it
was cool. We publish scientific papers, not necessarily what you're supposed to do
with a startup. But that's what I did. Boy, were venture capitalists not interested
in education. They weren't interested. I mean, they thought artificial intelligence
was the coolest thing in the world, they have no idea what it was back then. And their
only thing they could see that would be worthwhile with it would be looking at credit
card fraud. So the only tool was if we completely start a new business plan, retooled
to focus on credit card fraud, and fire myself as CEO. One guy said, I've got two
million dollars for you, and I've got the perfect CEO, as I'm literally sitting across
the table pitching my company to him. So you know, it was, I have founded a bunch
of companies. Since then I've been the chief scientist of several others, it was a
good idea. I can say with great confidence. In retrospect, I may not have truly known
what I was doing. But we had a good idea that could have made a big difference. But
no one could see the value add of either me as an entrepreneur, or making a difference
for at risk kids and education. And you know what? They're right. Of course, we could
have made more money in fraud detection, than an education, education is not a great
place to go make money. There's money to be made there. There's a sustainable business.
But I love being a scientist, I love being an academic. So I wasn't chasing the next
billion dollar deal, I was looking at how something I can build would have a positive
impact on the world in a sustainable way. And could grow big, I mean, positive and
sustainable at the scale of a billion kids. But boy, was that a hard thing to get
across, before Coursera, before Khan Academy, before AI was an everyday concept in
the business world, these are hard things to get across. And it did not help, now
for the big reveal that it was co founded by a couple of married women, that one of
the cofounders despite having a couple of very fancy smancy PhDs and scientific publications,
had spent a fairly long time in the 90s homeless. You know, I am not anyone's, I am
not Hollywood Central Casting idea of what a geek tycoon looks like. I turned up,
most successful founders aren't that. That's right. But trying trying to go out there
and raise funding through that is hard. And, you know, in the end, we had a successes
we had a small acquisition wasn't my dream, but was an outcome. Once you make money
for people, suddenly doors open up. Every company I've done since then was so much
easier. But it also really opened my eyes to how hard, if you are even a little bit
different than the stereotype for an entrepreneur, how hard how, how much extra effort
goes into just passing through those first barriers. But I will admit, I got kind
of hooked. I started a net tech company, then I started another one. And then I got
started in a company doing sort of social connection in business instead of who do
you know, ala LinkedIn, it was who should you know? You go into the big conference
in your field. Who should you meet while you're there? That would really make a difference.
That was one of my coolest projects, though. It was the initial meeting where every
company has it but not that one. I got poached out of that to be the Chief Scientist
of a company called Gild, one of the first ever doing AI in hiring. And oh my goodness,
is that a fraud ethical field, but also incredible, from both a business and a technology
standpoint. So, I'll close with this, I've had the chance to start these companies,
I've been the Chief Scientist at three others. I've actually started my sixth company,
fairly recently. It's called Dionysus, and we're in public health, again, the things
you could do with artificial intelligence that would make money, lots of money, don't
include education or public health. But it turns out, if you want to make a big impact,
the overwhelming positive effect on mortality and morbidity over the last 40 years
has been from public health. I, you know, it's great that people have discovered drugs
and come up with new surgical techniques. But overwhelmingly more people are alive
today, because of improvements in public health. So we're looking at things like how
chronic stress directly, causally leads to ischemic heart disease, type two diabetes,
major depression, exacerbation of bipolar disorder, the things that ruin lives over
decades. You know, again, probably not the sexiest business idea. But this is an area
that will actually affect people in a positive way.
Cindy Moehring 11:30
That's right.
Vivienne Ming 11:30
And if I wanted to be a billionaire, I'd already be a billionaire. This, this is the
world I want my kids to grow up in.
Cindy Moehring 11:37
Exactly, exactly. And you're making a huge difference. So being in education, I have
to go back and ask you about that first business idea that you had not to take the
teacher out of the classroom, but to help them see where they were missing concepts.
What happened to that idea?
Vivienne Ming 11:51
Well, you know, some of it was me learning things, you know, as an entrepreneur, scientist,
who could totally geek out. And one of my advisors in grad school told me what his
advisor told him, which was always meant to keep, you know, my head up, science is
a story. And that story should be about an hour long. But science is a story. So it
means to one, of course, is be able to communicate the work that you do to more than
just the six other people in the world that understand it. But the other is that it's
not about one single discovery, or one paper, it's about the story that explains your
entire field. Everything you're doing is actually building that story.
Cindy Moehring 12:39
Right.
Vivienne Ming 12:40
It's like a story as a metahypothesis of the whole way the world works. So but but
still, you know, I am a scientist. I'm moderately clever. And I walked into this space
with the attitude of I'm so much smarter than everyone else. I've invented this image,
and actually identify students of a missed concept and reviews it. And the general
feedback we got was, Oh, my God, this is amazing. And terrifying.
Cindy Moehring 13:16
Yeah!
Vivienne Ming 13:17
And what the hell am I supposed to do with it? You know, I've already got up three
hour, Well, I've got to get paid for 40 hours a week, I work 70 hours a week. When
am I supposed to learn how to use this incredibly confusing thing? So, you know, one
of the things I learned the hard way over the years, both in philanthropic work and
in entrepreneurship is, one, you're never as smart as you think you are. Just throw
that out, right off the bat. But also, it wouldn't even matter if you were so if i
was 100% right. If they didn't use it, I didn't make a difference...
Cindy Moehring 13:59
Right.
Vivienne Ming 14:02
And so we found ourselves going back and back and back. I've worked on a lot of products
over the years, Education is the one has, in a sense never ended. From that original
piece of work. Analyzing students conversations, just literally we could pick up on
for example, a university students just talking to each other online, right, to see
whether they understood the material they were talking about and who might have missed
concepts and so forth. And we built dashboards and all this sort of stuff. We have
walked that back all the way to Mom and Dad, do you have 20 minutes free tonight?
If you do, here is the way we think you should spend those 20 minutes. Here's an activity
that we believe will have the biggest impact on your young child's life. So we've
instead of a dashboard for a teacher to explore, you know, I learned the lesson. They
want answers. They don't want more data. I mean, I wish people wanted to explore.
In this process, it's not thinking, what is the problem to be solved here? And the
problem is, you're teaching situations that you've got command constraints on your
time. Yeah. So instead of creating new things to do, how do I make the things you're
already doing easier and more effective? And let's be honest, if the choice is between
easier and more effective, we can send what increased the university admission rate
for your students by 50%. Or we'll cut your grading time in half, exact same product,
cut the grading time in half will win every single time. You know, it's an Yeah, the
university stuff is a nice to have. But the grading time, like they feel that. So
I guess the way I always describe it, is I invent broccoli. And then I have to spend
years figuring out how to bake my broccoli into a brownie. People get a brownie, they
have distribution channels for brownies, they got they know what it is. It wouldbe
great, if I could give people the raw broccoli, because it'd be good for them, that
would be good for them. But that doesn't actually fix the system that we're trying
to address. So we built this stuff. And it's evolved over the years, we run a system,
which is in desperate need of a recent overhaul, which we're working on right now
called Muse, and it does what I got to, which is we really resolve down to alright,
let's go, right who cares the most about the long term outcomes of a kid, and when
can you have the biggest impact? Early in their life directly to caregivers, a parent,
a grandparent, a foster whoever it might be. And make it super easy. We're gonna ask
you one question a day. And we'll give you one activity a day. And that's it. We won't
even allow you to do more than that. You know, the most at-risk kids, the most at-risk
families are the ones that are the least able to make use of the kind of technology
I'm talking about. So make this something. And this is a general truth, I think about
technology and product development in general. Build a product where your desired
outcome is not a possible outcome. It's not, I can imagine a world in which people
put in the extra effort, it is the inevitable outcome. People will inevitably choose
to make use of it. And it will have that positive. Because our world is particularly
in the technology space, is overpopulated with people who I think, you know, I'll
just pick an obvious name. I don't think in a million years, Mark Zuckerberg saw Facebook
as something that was going to erode public trust. And he didn't intend for it to
be and it doesn't have to be. And yet in the end, in certain places, it absolutely
has that impact. And I think he and many of the people building it are still in that
space of, I can imagine a world in which this is an undeniable, good. All right, but
we only live in a world where the inevitable outcomes dominate. And that's the kind
of product you have to build.
Cindy Moehring 18:45
Yeah, yeah. Very interesting. Okay, let's dive a little bit deeper into this whole
topic of responsible and ethical use of AI. I love how you framed it up at the beginning
about it being a tool. So that sort of like human-in-the-middle is what comes to mind
for me and the thought of not handing over to a machine a problem that you don't already
know the answer to and expect that somehow, it's just going to come up with the answer.
I think you actually wrote a great piece on that in the Financial Times, which I'll
drop in the show notes for everybody to read. But let's talk specifically about that,
about bias in our promotions and our hiring. We know it exists. We know that we all
have implicit biases as humans, right. So as much as we may try to root it out, it's
there. And the risk with AI seems to me is that it's only going to exacerbate that
because we haven't yet figured out how to completely root out bias in in hiring and
in promotions, right. A perfect example is what happened with Amazon and their, you
know, their secret tool they had built. So what do you think the answer is here to
this to this problem? And is it a problem that AI can truly help us with or not?
Vivienne Ming 20:01
You know, I will start again with the simple statement that: AI is just a tool. It's
not a magic wand. I am not Hermione Granger. I can't magic problems away using AI.
I'm gonna say a lot of AI people, at least in their early days and those early promises,
that's exactly what was being sold. You got a business problem or anything else. Here's
this magic, which will make it go away.
Cindy Moehring 20:29
Right. It didn't work.
Vivienne Ming 20:31
It's it's numbers and equations. So of course, it can't be biased, which is a truly
absurd notion. I know that machine learning as a complex field is can be a little
obscure and obtuse. But the fact is, when problems get complicated, there's a universal
truth, which is, in fact, all of these systems have bias. In fact, it's impossible
not to have bias, whether we're talking about human cognition, or complex machine
learning, or a rat, it doesn't doesn't matter. The definition of AI that's most useful,
if a little wonky is it's any autonomous system, making a decision under uncertainty.
There's no right answer. There are better answers. But there's no right answer. So
fundamentally, in a world with no right answer, there's bias. There's different ways,
for example, to measure what is a better and a worse answer. So right off the bat,
you can appreciate that, even if we were talking about two people, they might have
different criteria for what makes a great employee or what makes a great loan, or
what makes a great product recommendation.
Cindy Moehring 21:45
Yeah. Right.
Vivienne Ming 21:45
And so they're gonna recommend different things. Well, guess what? AI is just as idiosyncratic
as unique as the rest of us. Now, it is, it is that in a very different way. Let's
be clear, artificial intelligence is not the same as natural intelligence, maybe someday,
these things will converge. I do work in neural prosthetics, i.e. I build cyborgs.
So I have that vision of a world. Or maybe AI itself truly becomes intelligent. But
what AI can do that we are not so good at is it can take every single number into
account.
Cindy Moehring 22:22
Right.
Vivienne Ming 22:23
But that doesn't mean it's unbiased. It just means when a human recruiter looks at
a paper, and this has been well researched for literally a 100 years, they have about
as much attention as three variables, your name, your school and your last job. So
your average recruiter, that's it, you know, maybe five seconds, a quick look. And
you think that's terrible. But that's almost exactly what a grandmaster in chess does.
When they look at a chessboard, they don't look at 1000, different possible moves,
generally thinking, they are looking at three possible moves, just as the Grandmaster
turns out, those are often the three best moves, if you really crunch the numbers.
Whereas a lot of AI is going to look, you know, it's gonna do massive computation
and explore lots of possibility.
Cindy Moehring 23:20
Right. Right.
Vivienne Ming 23:21
But it also has its biases based on how it's been trained, much like us. So a recruiter
looks at three pieces of information, but I can make a machine learning system, and
I did at Gild that looked at 400,000 pieces of information, and didn't even take five
seconds, it took a couple 100 milliseconds to process that information.
Cindy Moehring 23:43
Right.
Vivienne Ming 23:44
So now we can see two very different things going on, you know, this massive glut
of information and trying to infer something unknown about the world, should I bring
this person in for an interview versus three pieces of information? And maybe let's
call it a lot of intuition. Yeah. You know, which is better? And I would contend neither.
The simple truth is that we know there is a lot of human bias in the hiring process.
It's fairly easy to measure that bias. I'm willing to bet that some meaningful percentage
of your viewers are sort of shaking their head and saying, oh, we're hearing about
bias in hiring again. There's bias and everything. Like I said earlier, we're human,
we have biases. I'm not saying you're a bad person. I'm just saying you're human.
Cindy Moehring 24:37
Right.
Vivienne Ming 24:37
We have, we have biases in favor of certain schools, we have biases in favor of a
depth of a voice. We have biases, sometimes not so much about whether someone's a
man or women but just how feminine or masculine they are. Turns out being a feminine
presenting man is is actually really bad. So there's all of these subtle biases that
play into our decisions, including on a resume. Turns out people really love an Ivy
League school doesn't mean that's not predictive, but they love it way more than the
value it actually predicts. So we've sort of encoded the simple rules heuristics to
get by. AI doesn't do it. AI can crunch much broader numbers. But here's the problem
with those broader numbers is, because it is math, it needs to get referenced against
something, it needs to be told what the right answer is. So I've got 400,000 numbers,
but what's the equation? What am I trying to generate? What am I regressing it against
a training a deep neural network? Well, for Amazon, it was will you get a promotion
in your first year at Amazon? And now we can get at what the problem might be, which
is, so where do they get those right answers from? They get it from Amazon's hiring
history. Well, we can all understand. The same is true of any tech company, Amazon's
hiring history, and its first year promotion rates are wildly biased in favor of men.
Cindy Moehring 26:10
Sure.
Vivienne Ming 26:11
And so strangely enough, when an AI is trained on that data, it becomes even more
biased than the original recruiters that generated that data. We see the same thing,
by the way, in health. Some very well intentioned doctors put together a model to
look for people that should be patients that should be brought back for checkup, after
a health heart scare, and they made a very reasonable assumption, the amount of money
spent on the patient in that subsequent years is an indicator of how sick they were.
So we really want to bring back the patients that we spent that were more sick prophylactically
before they become sick, so the ones that have more money spent. So here's the terrible
story. That model was implemented, it was used in practice. And then a paper came
out in nature, saying, Oh, by the way, much more money is spent on average on white
patients versus black patients for the exact same conditions. And there's a variety
of reasons which can even be purely regional, by patients, disproportionately, for
example, live in portions of the south where healthcare costs are lower than on the
coasts. So just simple things like that. I am sure there was well documented sources
of more explicit bias as well. But even simple things like regionality mean, more
money spent on average on white patients and black patients for the exact same conditions,
which meant white patients were brought back by this model at a much higher rate than
black patients. And so the authors of this paper went through and calculated 1000s
of people died, who wouldn't have had the model been trained more equitably. No one
intended for this model to have this bias. Nobody wanted these people to die. No one
was a villain here.
Cindy Moehring 28:09
Right, right.
Vivienne Ming 28:10
But it is a recognition that just because it's an equation, yeah, just because there
are lots of numbers, doesn't mean that it's right. All of these numbers have their
own implicit bias behind them. And you know, we so for me, in the end, it's a tool.
Where do we get the best results consistently over and over again, including, from
a business perspective? We get it when we create, take creative people and sophisticated
artificial intelligence and bring them together. When we substitute for people,
Cindy Moehring 28:45
Yeah, yeah, with models that have to be trained, right? I mean, you have to step back
and take that time for the human to, to train it. I heard that in some of what you
were saying, you know, both in the hospital example and the hiring example. If you
spend a little more time upfront with humans, kind of, monitoring, if you will, modeling
it, doing the training of the, of the, of the algorithm before it's released, then,
then it could be a better tool. Might not be perfect, but it could be better. Right?
Vivienne Ming 29:12
It may not be perfect, but then here is the other side of the story, which is going
to harken back to my comments about education. So we built that tool. We built a tool
that was transparent, which means it explains why it makes the hiring recommendations
it does or sourcing in our case. It's very specific and identifies specific things
that are driving its recommendations, why they are a fit for your company versus another
company. It even pre templated a personalized letter that you could send to build
a relationship with this candidate. And almost nobody used any of that. When we did
our user testing with our very fancy AI that was meant to make recruiters better,
when we did our user testing, we found that they gave you five seconds. And when they
opened up your profile in our system, they looked at your name, your school and your
last job. Not every recruiter, but the vast majority of them.
Cindy Moehring 30:16
So user adoption was a huge problem.
Vivienne Ming 30:19
Well people used it for some reasons that we embedded in here, you know, we made it
very easy to discover people that you may not have come across otherwise. But the
funny thing is, that's why people bought our system, it was the a whole reason. You
know, if you're an elite company, sure, you already know, everyone that's coming out
of Stanford and Harvard, or Cal or Michigan. It's everyone that's not, that is the
big question mark. And that's why people paid us a lot of money for our product.
Cindy Moehring 30:53
Okay.
Vivienne Ming 30:55
But then, once the recruiters got into the system, yeah, they hired you, because you
went to Stanford or Harvard. And that may feel like a little bit of a heartbreak.
But I come back to the fact that I own this. You know, I'm, why would they change
their entire way they have learned how to be a recruiter just because this thing comes
out, particularly in HR, which can be a very risk averse part of the business world,
like, you hire someone that 20 years from now becomes a senior executive, no one's
going to come back and give you a bonus. But if you hire someone who on day one is
a disaster, you take a risk on someone that doesn't have credentials then you are
at a real risk of losing your job. So that dynamic builds up. Nobody wants to learn
this new tool, nobody wants to invest. And when I say nobody, I'm being too strong,
there were people. So with all of these things, if I'm saying, we shouldn't substitute
for people, we should make them better. You should not only be better when you're
using a new technology, you should be better than where you started when you turn
it off again. So when I build a product nowadays, or my philanthropic work, or any
of it, that is always my starting question. How do I make certain that this person
is immediately better in some way that matters to them? And that even when they stop
using it, they remain better at whatever it is they were trying to do. Which is a
hard constraint. But again, it is those inevitable outcomes that are really meaningful.
If I can't solve that problem, I don't know that I've truly solved anything.
Cindy Moehring 32:44
Hmm. Interesting. So let's, let's, let's, let's riff on that for a minute. So let's,
let's take that, that idea of what you just said, I'm not going to be a better person
when I actually turn off that tool and stop using it, right. And so promoting inclusion
and getting people to, you know, think more open-mindedly all kinds of studies showing
all the benefits of that, right. And then we go into this COVID world where we've
all been working at home for the last two years, and you know, going on more, so not
everybody, but a lot of people are, right. So it would seem to me that that would,
could, potentially throw the a big juggernaut into a company's ability to follow through
on their desire of promoting inclusion for everyone when everyone's like working at
home and sort of siloed. So how, I think you may have done some research in this area,
I'm really interested in hearing your thoughts on that. And like, how have companies
dealt with that issue? And what did your research show about how to deal with that
issue?
Vivienne Ming 33:48
Yeah,
Cindy Moehring 33:49
Do you work better when you turn off your zoom?
Vivienne Ming 33:52
Exactly. You know, I I have this little organization here called Jocose Labs, and
it works as follows. People bring me a problem, or sometimes we come up with it internally.
And if I think my team and I can make a unique difference on that problem, then I
pay for everything. And we try and come up with a solution. And if we succeed, we
give it away. And those problems come to me often in the form of emails, like "Dr.
Ming, my daughter has 500 seizures a day. Or my son can't enter REM sleep," which
is doesn't sound like a huge issue, but it's astonishing that that kid was still alive.
Or in this case, Facebook and Amazon separately reached out to me and said, "We've
never had to do inclusion or innovation with everyone at home before. We've never
even had everyone at home much less these two specific things. What do we do?" And
I obviously I got this these messages like in March of 2020, uh, So you get the sense,
I'm a terrible entrepreneur. I mean, most of my time is spent running an organization
where people give me their problems, and I charge them nothing for the answers. But
in this particular case, that seemed like a problem we're solving because everyone
in the world was experiencing the same thing. And I thought, what I'd be able to do
is just leaf through a bunch of existing research and offer some insights. There's
nothing. You know, there's one notable case of a Chinese company, a large one that
went entirely remote. But essentially, there was no substantial data on remote work
in general, much less these two questions, innovation and inclusion.
Cindy Moehring 35:44
Right.
Vivienne Ming 35:45
So in the end, we just had to break some new ground. So we put some existing research
together, and then we ran some experiments with some of our partners. And in a way,
as a mad scientist, remote work is a fascinating experiment. Because unlike in the
office, I get to see everything you're doing. Every time you speak to someone else.
There's a record of how long it was, often about what it was you were talking about.
I've had the chance, you know, in COVID world, there's contract tracing. Well, actually,
researchers have been doing contract tracing to reconstruct social networks, you know,
in body social networks, inside schools and companies for a while, I've had the chance
to do this with some big companies as well. But here, it's super easy. So we're able
to see all of these interactions. And we found, let's start with the innovation. But
trust me, inclusion turns out to be fundamentally related to it. On the innovation
side, we've found that the most innovative, First off, there's an idea that you can
innovate, if you're not hanging out at the watercooler together, we found that there's
no special privilege to smashing everyone together on a university campus or a corporate
campus or anything else. In fact, in some ways, that can be a drag on the system.
Cindy Moehring 37:08
Interesting
Vivienne Ming 37:09
What we found, what predicted the most, the greatest innovation was small teams, flat
hierarchies within those teams. So even though there might be a senior vice president
and a new hire, you know, member of the National Academy of Science and an undergrad
student, when they're actually collaborating those hierarchies disappear. Because
otherwise, what's the point of having the grad student there?
Cindy Moehring 37:38
Right.
Vivienne Ming 37:38
If they're afraid to say something, then they're not adding value.
Cindy Moehring 37:41
Right.
Vivienne Ming 37:42
So flat hierarchies, on small teams. A lot of fascinating work with technology tools.
So collaborative documents, wikis, things like Google Docs, not so much chat. Turns
out chat doesn't seem to improve innovation in any way. It's, it's actually sharing,
like having a centralized shared mind.
Cindy Moehring 38:13
Right.
Vivienne Ming 38:15
And so we see these flat hierarchies, small teams, integrated, they have complementary
diversity, which is a bit of a term to unpack. So this is a well known thing in collective
intelligence, in general. Two of the biggest predictors of collective intelligence
at any scale, is how diverse that population is, for example, slightly more women
than men, slightly more women than men is universally a strong predictor of collective
intelligence from a small team to a whole community. And the other core ingredient
is psychological safety. Which I'm going to nuance a little bit and give it my own
specific definition. It is that I feel free to pitch a crazy idea knowing that even
if I'm wrong, it's okay. I'm not out of the team. I'm not low, less in people's eyes.
That is a huge predictor of collective intelligence is because that's the whole point
of being different is that chance to bring a different idea. And when you upset that,
then you realize the inevitable result is most of us are going to be wrong most of
the time.
Cindy Moehring 39:34
Right.
Vivienne Ming 39:34
And it's what we collectively arrive at that is truly important. So these are the
things of complimentary diversity. We are different in certain ways, but we have shared
qualities that build trust. Small teams, flat hierarchies. Turns out those exact same
things when we're tracking everyone on Zoom and Google Meets and Microsoft Teams or
whatever other craziness people are making use of, all of those same things become
predictors of inclusion. And it's a little complicated operationalizing it but let's
do one that I've worked on for with for a while, do people with similar work histories
have the same probability of promotion? Why don't you take say race or gender other
outcomes. So for example, gender, almost immediately women with similar work histories
to men are, sorry, let me put it in these terms. Women who are rated higher in performance
are also systematically rated lower in potential and less likely to get promoted into
management and executive positions.
Cindy Moehring 40:46
Interesting.
Vivienne Ming 40:47
And, and there's a lot of nuance there. But it's really replicated finding again,
and again, and it's a, it's a meaningful, and it's not like, women never get promoted.
You know, it's not that kind of a glass ceiling. But it's a substantial kind of drag.
It accounts for about 40% of the difference in promotion rates between men and women.
Cindy Moehring 41:09
Yeah, she's really good at what she's doing. So why would we not want her to be able
to keep doing that? She's so good at what she's doing.
Vivienne Ming 41:15
Yeah, absolutely. So you know, so we see these kinds of things creep in. So that becomes
an pretty reasonably objective way. If we're looking at people's work histories and
we're seeing very different promotion rates, despite very similar work histories.
Something weird is going on in this system. So it turns out the organizational structure
that best predicts similar promotion rates, small teams, flat hierarchies, complimentary
diversity, but one additional and fairly important element, which is then opened a
whole new can of worms for us, is equitable time on camera. And this is an interesting
thing is true of us in office subtly but it is a big effect on a camera. The amount
of time that your face is on the screen in front of me, not that you're in the Hollywood
Squares background, but that you're the principal speaker that I'm paying attention
to, count the minutes. And that all things being equal, that becomes a significant
predictor of work opportunities and promotion rates. Again, controlling for work history
and job performance. People that get more face time, are getting more promotions.
Cindy Moehring 42:44
Managers need to really think about that then like, not just to make, because if you
have a small team, and the makeup of the team isn't diverse, then you're not going
to have a diverse, you know, face on the camera either. So it starts with having the
right members of the small team, but then making sure that almost it's like equal
time, if you will, but that everyone is seen, truly seen, and heard for the same amount
of time. That's really interesting.
Vivienne Ming 43:09
Yeah. And, you know, I'm saying this, it's, this isn't, again, an explicit form of
bias. This is, in fact, a fairly low level phenomenon happening in our brain. It's
really a brain hack or bad kind of brain hack.
Cindy Moehring 43:21
Yeah, yeah.
Vivienne Ming 43:22
Where we're just sort of accumulated minutes, and our brain starts kind of up voting
people just because of accumulated minutes. But then it interacts. Because we're much
more likely to spend time with people that are similar to us. And yeah, similar in
race and similar in gender. Turns out, though, again, this is a pretty low level human
phenomenon, similar in smell, similar in terms of the intestinal flora and fauna.
I mean, you may be amazed along how many different dimensions, we sort of lazily prefer
similarity, even though it doesn't actually present, obviously, any business advantage.
I just told you, diversity actually includes, improves collective intelligence when
it's paired with trust. So this is the can of worms that are open for me. I'm a neuroscientist.
Why is it, if for decades now, we've been talking about for example, the value of
diversity? And diversity is so clearly, so replicably, from boards to lab experiments,
such a component of innovation. Why does it persist? So I wrote about something called
the Diversity-Innovation Paradox. It is well, it's very easy to demonstrate that as
you increase the diversity of a community, innovation improves. Some colleagues of
mine at Stanford computational linguists, made an AI that analyzed every single dissertation
written in every field since 1977. It may be the only entity in the entire world that
read my dissertation, because I sure as hell didn't. And it found the more of an outlier,
you were in your field, the more innovative your research was, the more novel the
scientific impact. But the less cited, and the less likely you were to get subsequently
promoted. So we have these deep biases that have persisted and persisted. And I really
wanted to know what was going on. So I started looking at the neuroscience of trust.
And this is something I think a lot of managers need to think hard about.
Cindy Moehring 43:22
I agree.
Vivienne Ming 45:23
What we found was, if something's, if someone's very similar to me, not only do I
have these circuits involving frontal networks, and reward centers in your brain,
like the nucleus accumbens, and your dopaminergic system, I have this complex network.
And when someone is very similar to me, not only does that network essentially activate,
and with no real effort, it's what we would call automatized, I actually get a little
reward, I get a little bit of dopamine, a little bit of endogenous opioid trusting
someone that's similar to me, is easy. And it even makes me feel good. As people get
progressively more different, different culturally, different in race, different in
socio economic status, we can track all of these differences, and it holds true, the
rewards disappear. And the effort, the effort to deploy that same level of trust,
to an otherwise similar person goes up. And those frontal circuits that are supplying
that effort, are the same ones you're using to get your job done. The same ones that
are firing, because you're trying to hit a deadline, the same ones that are under
pressure, because the board needs you to deliver, so you're competing for the exact
same resources, to do the thing you're trying to do, and to recruit the people to
help you do it. Well, now you begin to see why our problems persist. If I'm putting
all this effort into my job, as I should, and it pulls away the resources, I need
to see someone for who they truly are. Then I am going to essentially undertrust,
I'm going to undervalue people that could actually provide the value I need to my
teams. And I think until we can see that and work hard to overcome it. We're going
to continue to put together teams which are easy. They'll deliver, they'll get there.
But they, won't you'll actually miss out on all of the additional value that could
have been created.
Cindy Moehring 47:51
Yeah. Wow. That's an incredible piece of research that that that is, if somebody wanted
to go find and read about that a little bit more, Vivienne, where would that be?
Vivienne Ming 48:01
So we're putting together some free technical versions of this. But if you want to
have some fun, if they go to Socos.org, you'll find our little newsletter there. And
we write about all this stuff. In particular, this was in a piece appropriately titled
"Remote Work". But my dirty secret is it had sub chapters on, I mean, all these things
I write start out with, like, I'm going to write this little 600 word essay and 100,000
words later, and multiple chapters, I wrote one called "This Is Not the Industrial
Revolution" about artificial intelligence in the future of work. And again, that line
just came from something I said to a jerk on the stage at the Milken conference once
and, and this one similarly, so it ended up being like nine chapters long and there's
chapters on innovation and inclusion, which quite frankly, are true, regardless of
whether we're talking about remote work or not. Yeah, this idea of the Diversity-Innovation
Paradox of the, what we call you know, looking at distributed innovation, how you,
how you merge, get the right people to meet one another. And create those sparks.
And specifically on the neuroscience of trust, if you want to, if you can stand the
occasional dirty joke cuz I just can't help myself and the story of how we got there
with this work, then you can visit the site, find the piece on remote work, see some
of the other ones. And yeah, it was, I mean, it's, it's, again, I enjoy the things
I get to do. I mean, I must, cuz I pay a lot of money to do that.
Cindy Moehring 49:46
Yeah, you do! And it's incredible and you're so much value to society and leaving
an incredible legacy. It's just fabulous. So I've got to ask you one last question,
you've been so generous with your time. And I really appreciate it. I know the audience
does, too, you're putting out an incredible amount of great work. But when you need
to turn to other resources for inspiration and to learn a little bit more, and to
dig a little bit deeper, where do you go? What else could you recommend to the to
the audience from the expert as to, I don't know, podcasts or documentaries, or books
or white papers? What where do you go for inspiration?
Vivienne Ming 50:25
So I go to a few different places. But I'll say this, the first thing I do is I go
to the problem. As as a naive person, recognizing I don't know it, I don't know the
solution. I start with the problem. So the Make a Wish Foundation. I once, we once
did a little collaboration with them, which evolved into, you know, could we build
a little system that helped give a nudge to their wish granters, to such that when
that kid says, "I want to go to Disneyland", the wish granter could say, "Well, what
if you brought your three best friends with you will pay for that, too." And the reason
for the nudge is because it literally increases the survival rate of the kid. So can
we nudge the wish to increase survival rates? So that is a perfect example of where
I start on every project, which isn't the data because they didn't have any. And it
isn't so much the background, although there are a couple of existing papers to say
that granting the wish does improve survival rates for these kids. It was watching
the wish granter go out and ring doorbells. Right but think about the two places where
I've already admitted failure, the teachers and the recruiters. The problem was not
that I didn't have some cool ass AI that could save the world in some hypothetical
way. The problem was, I built something that they didn't want to use. And so until
you actually go out and observe the world, the actual human problem, not the data
problem.
Cindy Moehring 51:22
Right...
Vivienne Ming 51:23
...you don't understand what it is you're solving. Once you've done that, where do
I go? I read blogs of people I disagree with. Now, I don't mean people that are provocateurs.
And they're just, you know, speaking nonsense. But I read conservative economists,
whose views of the world are different than mine. But remember, when I said Science
is a story, if my story doesn't include their research, then my story isn't right.
So what it, what it explains my research and their research at the same time, I never
get to just say no. So I love obviously, I'm talking economists and scientists, that
may not be the thing for everyone else, but follow a couple of people on a blog or
Twitter, who are reasonable. But on the opposite side of an issue for me,
Cindy Moehring 51:48
Yeah, yeah.
Vivienne Ming 51:19
Because you'll get a very different view of what real insights into problems once
you confront that. And after that, oh, gosh, I mean, I read a lot of science and scholarship
in general, I read a lot of economics papers, the National Bureau of Economic Research
is just puts out tons of working papers, and you can find so many fascinating topics
covered in the economics world. Obviously, I still read a lot of of about neuroscience
because I sit on the boards of a couple of different neuro tech companies out there
trying to build cyborgs, although in our case, Alzheimer's, cerebral palsy, these
are the things we're trying to address. So essentially, I just keep my eyes open for
new problems.
Cindy Moehring 52:50
For new problems. I love that advice, though. Look, understand the problem, the human
problem that you're actually trying to solve, and then open your mind and your perspective
to differing points of view, so that you can make sure that your solution to that
well defined human problem actually is going to be received, you know, well.
Vivienne Ming 54:16
And let me be entirely clear, I am always right about everything. It's it's a it's
a curse, but I somehow bear this burden. So I'm not saying I just, you know, I'm I
accept everything everyone says, or can't we all get along? It's okay to disagree
with people. But you have to understand why you disagree with them. And frequently,
occasionally, not that I would ever admit it, they're actually right about something.
And I need to update the way I'm thinking about the world to explain why I didn't
understand this problem appropriately. Yeah. Because I think the the number one reason
big, new ideas fail, whether it's inventing or science or, you know, business in general,
is you didn't actually understand the problem you're trying to solve. That's right.
You thought you did, right. And generally speaking, it's because you thought it was
a simpler problem than it actually is. My story is, no matter how complex you think
the brain is, it's more complex than that. And everything, as far as I can tell, this
has generalized to anything involving human beings at all. So just get comfortable
with the problems being messy with the fact that maybe there aren't perfect solutions
to problems. The question is, can you make a positive difference? Even if it's an
imperfect solution? Sometimes that's all you have available to you.
Cindy Moehring 55:52
Right, right, right. I love it. We're going to end it there, Vivienne. Thank you so
so much. This has been an enlightening and uplifting conversation about responsible
and ethical use of AI. Yes, we talked about some problems that can be created. But
there's a lot of good that can be done in the world too, when it's done in the right
way. And when it's married together with humans, and you think through it in the right
way. And remember that it's a tool, as you said, we'll end where we started, AI is
a tool.
Vivienne Ming 56:19
Absolutely, let me just say AI, I wouldn't spend so many hours of everyday working
on this stuff, if I didn't think it can be a force for good in the world.
Cindy Moehring 56:32
That's right.
Vivienne Ming 56:33
We end up with problems when we think that it's going to solve our problems for us.
Yeah. And in general, I would generalize that, if you think someone else is going
to solve your problems for you. Then I, whenever someone comes to me and says, "Dr.
Ming, you know, what are we going to do about x?" I immediately think, well, apparently
nothing. Because you think someone like me is gonna walk along and magically solve
that problem for everyone else. We make progress when we all push towards something
that's going to change the world in a positive way. This is not a it's not a world
of superheroes. It's a world where we make effortful, incremental progress. And that
makes it look better.
Cindy Moehring 57:28
Yeah. Well said, thank you so so much for your time and sharing your thoughts and
wisdom and knowledge with us. We really, really appreciate it. Thank you.
Vivienne Ming 57:39
It was a blast.
Cindy Moehring 57:40
It was. Alright, Bye bye. Thanks for listening to today's episode of theBIS the Business
Integrity School. You can find us on YouTube, Google, SoundCloud, iTunes or wherever
you find your podcasts. Be sure to subscribe and rate us and you can find us by searching
TheBIS. That's one word T-H-E-B-I-S which stands for the Business Integrity School.
Tune in next time for more practical tips from a pro.
