This video discusses the current state of user experience (UX) research, arguing that it's facing a "reckoning" and needs to change. Jud Anon, a former research leader at Facebook and Airbnb, explains that much UX research is currently focused on "user-centered performance"—doing research for show rather than true learning. He introduces a framework of macro, middle-range, and micro research, suggesting that too much time is spent on ineffective middle-range research. The conversation emphasizes the need for researchers to be more business-oriented and for product teams to integrate researchers earlier and more deeply into their processes.
Key Takeaways
1
User-centered performance is when companies act customer-obsessed for show, not to truly learn and make different decisions.
2
A common example of user-centered performance is asking for a quick user study at the end of a product process just to validate assumptions, when it's too late to matter.
3
The user research field is undergoing a "reckoning" because it often doesn't drive the value or impact it should, with layoffs hitting UX research particularly hard.
4
The speaker introduces a framework for research: macro (big picture, strategic), middle-range (mid-altitude user understanding questions), and micro (technical usability, pixel-perfect improvements).
5
Too much research is currently focused on the "middle-range," which is interesting but often not impactful enough for the business.
6
To improve, research needs to be integrated much more fully into the product development process, from beginning to end, rather than being treated as a reactive service function.
7
Great researchers are multi-method, possessing skills in formative/generative research, evaluative research, rigorous survey design, applied statistics, and technical skills like SQL or prompt engineering.
8
Researchers need to be more business and profit-focused, understanding company strategy, OKRs, metrics, and conversion funnels to ensure their work drives maximum impact.
9
Product managers and designers often engage in user-centered performance by asking for research to justify decisions they've already made or are reluctant to make on their own.
10
Good research doesn't slow down the process; it speeds it up by preventing costly mistakes, and micro-level research can quickly drive significant business value.
No matching segments found
0:00
user- centered performance refers to
0:03
customer Obsession or user- centered
0:05
practice that is symbolic rather than
0:07
focused on learning it's hugely common I
0:10
would argue it's work we do to signal to
0:13
each other how customer obsessed we are
0:16
not because we want to make a different
0:17
decision if your like listeners are like
0:19
I don't do that I'm kind of like think
0:21
about it for a second this is extremely
0:24
common every time a PM comes to a
0:26
researcher at the end of a product
0:28
process and says can you just run a
0:30
quick user study you know just to
0:31
validate our assumptions that's user
0:33
centered performance it's too late to
0:35
matter we got to ship it right what they
0:37
want is to check the box one of my big
0:39
kind of mantras was we don't validate we
0:42
falsify we are looking to be wrong many
0:45
PMS many designers are not in that place
0:48
they do not want to be wrong they're
0:49
looking to validate and that's user
0:51
centered
0:54
performance today my guest is Jud Anon
0:57
Jud helped build the user research
0:59
practice at Facebook he was a longtime
1:01
head of research at Airbnb and his
1:03
direct reports have gone on to lead
1:05
research teams at figma notion slack
1:08
Robin Hood dualingo fair and other
1:11
amazing companies these days Jud spends
1:13
his time Consulting helping companies
1:15
with organizational challenges product
1:17
strategy design research hiring
1:19
onboarding and crisis Management in our
1:22
conversation we unpack a conclusion that
1:25
Jud has come to recently about how the
1:27
user research field is going through a
1:30
Reckoning and What needs to change both
1:33
within the user research field and how
1:35
companies leverage user research going
1:37
forward judge shares what the user
1:39
research field has gotten wrong over the
1:41
last decade how PMs and designers rely
1:43
on user research too often and to answer
1:46
the wrong questions where user research
1:49
will continue to provide significant
1:50
value and how to best leverage your
1:52
researchers why it's important for
1:54
researchers to think about the business
1:55
goals more versus just what the users
1:57
need what to look for when you're hiring
1:59
you user research researcher how PMs can
2:01
be better Partners to researchers and
2:03
also a phenomenon that I love that Jud
2:06
describes and often witnesses that he
2:08
calls user centered performance where
2:10
everyone acts like they care about the
2:11
user but they're just doing it for show
2:13
and already know what they want to do
2:15
this episode has a lot of spicy takes
2:17
and will probably upset some people but
2:19
Jud is sharing some real talk here that
2:21
I think we all need to hear with that I
2:23
bring you Jud Anon after a short word
2:26
from our
2:27
sponsors this time of year is prime for
2:29
career of reflection and setting goals
2:31
for professional growth I always like to
2:33
spend this time reflecting on what I
2:35
accomplished the previous year what I
2:36
hope to accomplish the next year and
2:38
whether this is the year I look for a
2:40
new opportunity that's where today's
2:42
sponsor teal comes in teal provides you
2:44
with the tools to run an amazing job
2:46
search with an AI powered Resume Builder
2:49
job tracker cover letter generator and
2:51
Chrome extension that integrates with
2:52
over 40 job boards teal is the
2:54
all-in-one platform you need to run a
2:56
more streamlined and efficient job
2:58
search and stand out in this competitive
2:59
market there's a reason nearly 1 million
3:02
people have trusted teal to run their
3:04
job search if you're thinking of making
3:05
a change in the New Year leverage teal
3:07
to grow your career on your own terms
3:10
get started for free at TL hq.com Lenny
3:14
that's TL hq.com SL
3:17
Lenny this episode is brought to you by
3:20
vanta helping you streamline your
3:22
security compliance to accelerate your
3:24
growth thousands of fast growing
3:25
companies like Gusto Comm Kora and
3:28
modern treasury trust van that to help
3:30
build scale manage and demonstrate their
3:32
security and compliance programs and get
3:35
ready for audits in weeks not months by
3:37
offering the most in demand security and
3:39
privacy Frameworks such as sock 2 ISO
3:42
2701 gdpr Hippa and many more vanta
3:46
helps companies obtain the reports they
3:48
need to accelerate growth build
3:49
efficient compliance processes mitigate
3:51
risks to their businesses and build
3:53
trust with external stakeholders over
3:56
5,000 fast growing companies use V to
3:58
automate up to 90% of the work involved
4:00
with sock 2 and these other Frameworks
4:02
for a limited time Lenny's podcast
4:04
listeners get $1,000 off vanta go to
4:08
v.com Lenny that's V
4:11
n.com Lenny to learn more and to claim
4:13
your discounts get started
4:18
today Jud thank you so much for being
4:21
here welcome to the podcast Lenny thanks
4:23
for having me it's my pleasure so we
4:25
actually work together atbb for many
4:27
years and as I Was preparing for this I
4:30
realized how many of the people that you
4:32
managed went on to do amazing things so
4:35
I'm just going to read a list of people
4:37
that worked for you and what they do now
4:40
so we had Matt Gallivan who now leads
4:42
research at slack we have Janna Bray who
4:44
leads research at notion Celeste Ridin
4:47
who leads research at Robin Hood Rebecca
4:49
gray who leads research at Fair Hannah
4:53
Pelley who I think was leading research
4:55
at dualingo Luis barel who leads
4:58
research at figma and then gnome who was
5:02
leading research at wealth front I think
5:04
he moved on to something else what a
5:05
freaking crazy alumni community and
5:08
group from this one team that you hired
5:10
and incubated no I've never looked at
5:12
that list but I'll tell you I have been
5:14
so privileged to work with all these
5:15
amazing humans I can't take credit for
5:17
it they're just outstanding people and
5:19
I'm glad the diaspora is out there
5:21
because like these people rock stars
5:23
okay so the main reason that I wanted to
5:25
do a podcast episode with you is that
5:28
you wrote this piece that was titled the
5:30
user research Reckoning is here which I
5:34
understand caused quite a stir in the
5:36
research community and I think adjacent
5:37
communities and let me just read uh one
5:40
of your takeaways at the top of your
5:42
post to give people a sense of what it
5:43
was about so you wrote the user research
5:46
discipline over the last 15 years is
5:48
dying the Reckoning is here the
5:51
discipline can still survive and thrive
5:53
but we better adapt and
5:55
quick before we get into the meat of the
5:58
piece could you share a bit about just
6:00
the reaction to this piece and maybe if
6:02
it was a surprise and what you expected
6:04
would happen when you put this out yeah
6:07
I I mean I was definitely surprised uh I
6:09
I I wrote it because I wanted to start a
6:11
conversation about something I was
6:13
thinking about I didn't really know who
6:14
would read it and and in the end it
6:16
turned out a lot of people read it uh I
6:18
learned that using the word Reckoning uh
6:21
may have been a mistake because it
6:23
inspires a lot of drama in a
6:25
conversation that I wanted to be really
6:27
productive and positive overall I would
6:29
say though that like the response was
6:31
was very positive um it it seemed to
6:34
resonate with a lot of people who
6:36
reached out to me I spent a lot of time
6:38
talking to teams to designers to
6:40
researchers but there were also a ton of
6:42
critiques I would say some of it was
6:45
like people thought I was throwing
6:46
research or researchers under the bus
6:48
like it's it's researchers fault we're
6:50
doing it wrong which I don't believe at
6:51
all and that I wasn't taking
6:53
responsibility as a as a research leader
6:55
or a design leader myself and like a
6:58
real uh that the most interesting one I
7:01
would say was the anti- capitalist crew
7:04
because one of my points that we'll talk
7:05
about is that I think researchers need
7:06
to be more profit focused and there are
7:08
a lot of people out there who think
7:10
that's not I think they think that's not
7:12
cool or not research's job and I'm kind
7:16
of like well what are we doing then if
7:18
we're not helping businesses succeed um
7:20
so but that was that was the most
7:21
surprising critique for sure I've I've
7:23
worked with some of those people who are
7:24
just like why are we growing why do we
7:26
focus so much on growth why do we need
7:28
to grow this business so maybe maybe
7:30
it's the wrong industry for them yeah
7:32
yeah I'm not a fan of that okay so let's
7:34
actually dig into the meat of your
7:37
message and the big takeaway and kind of
7:39
the conclusion of what you're finding is
7:40
happening us research and I know a lot
7:42
of this comes from a lot of us
7:43
researchers have been laid off at a lot
7:45
of companies it was one of the hardest
7:47
hit uh teams and so I think a lot of
7:49
this comes from that so yeah so let's
7:51
just start big and then see where it
7:52
goes so yeah I mean everybody who's
7:55
paying attention has noticed that there
7:57
have been a bunch of layoffs and I I
7:59
think back in the summer I was thinking
8:01
listen this seems to have be hitting ux
8:03
and ux research particularly hard is
8:06
there something going on is there a
8:07
bigger picture is it maybe the the
8:09
reason I use the word Reckoning is
8:11
because to me that's like hey a moment
8:13
to take stock um and triggered by the
8:15
fact that a lot of wonderful humans may
8:17
have lost their jobs and many more are
8:19
afraid of losing their jobs and so if
8:22
it's a sign the fact that research has
8:24
been hit so hard it's a sign of what and
8:26
so the thesis of my article is really
8:28
it's a sign that maybe the system is a
8:31
little more broken than we think and
8:33
that research is not driving the value
8:35
or impact that it should or could and
8:38
that's for a bunch of reasons I think
8:40
some of it is is stuff that research can
8:41
do better and a lot of it is how
8:43
research is integrated and positioned in
8:45
companies and at the root of all that I
8:48
think is that we're we're just doing too
8:51
much of what I would consider the wrong
8:53
type of
8:53
research and what I mean by the wrong
8:56
type of research is I have this
8:57
framework that it's in the article macro
9:00
middle range and micro research at least
9:03
three ways to talk about it and it's
9:05
pretty simple the intuition of what
9:06
those are so macro research is like big
9:09
picture strategic business focused
9:12
forward-looking Innovation you know look
9:14
at the market look at competitors you
9:16
know long-term research to understand
9:17
where the product should go n stuff like
9:19
that and then you have Micro research
9:22
which a lot of really technical
9:24
usability falls into this all the
9:25
beautiful stuff that researchers do to
9:27
enable like a really high quality
9:29
excellent pixel perfect thing to go out
9:30
the door laser focus research to
9:34
understand AB test results stuff like
9:36
that and then you have this middle range
9:39
which is this blob place where the
9:41
research questions are sort of uh middle
9:44
altitude and a lot of the core kind of
9:46
let's say user understanding questions
9:49
fall
9:50
here and a lot of what research is doing
9:52
is researching that space it's sort of
9:54
let's take a group of people and ask
9:56
some questions about how they think feel
9:58
behave how they're using a product or
10:00
not using a product and it's just this
10:05
devastating mix of really interesting to
10:07
many including researchers and not
10:10
impactful enough for the business that's
10:12
kind of the core thesis researchers do
10:14
it because it's interesting but honestly
10:16
and a thing we should talk about Lenny
10:17
is researchers also do it because it's
10:19
the kind of work we most often get asked
10:22
to do yeah that's exactly what I was
10:24
thinking that's what as a PM like that's
10:25
that's what I want to get answers to is
10:27
like how should we think about this one
10:28
product and I totally get this yeah it's
10:32
like the the questions turn out to be
10:33
really interesting and there are many
10:36
cases at many companies where it's super
10:38
impactful but the problem with that
10:40
those types of questions is they tend to
10:42
be really like they trigger all the
10:44
worst stuff that researchers experience
10:47
right so they yield results which are
10:49
interesting but sometimes hard to
10:51
operationalize um they trigger the post
10:54
hog bias really really really well where
10:56
like a lot of people can say confidently
10:59
like that was kind of obvious we knew
11:01
that already and they fulfill this kind
11:03
of need for us to feel and be customer
11:06
obsessed user centered without changing
11:09
anything and so doing too much of that
11:11
research to me is a symptom of a broken
11:14
system right and where um like companies
11:17
are really different from each other I
11:18
heard from so many after this article
11:20
and they're like well my company and my
11:22
industry is like this or not like this
11:25
but in Tech we spent the last many years
11:28
hiring hiring hiring researchers and but
11:30
maybe I'm sure most of your listeners
11:32
are familiar with the idea of a zerp you
11:34
know like maybe it was a zero interest
11:36
rates phenomenon where it was okay when
11:38
the money was easy to hire
11:40
researchers even though we were not
11:42
setting them up properly we were going
11:45
to set them up to fail we set them up as
11:47
a service function we didn't know what
11:49
research was for we didn't know how to
11:50
how to really drive impact with it and
11:52
and that's where the Reckoning comes
11:54
from it's like that era is over research
11:58
I think is more crucial than ever
11:59
researchers good great researchers are
12:01
more impactful than ever but it's it's
12:02
in a new space we're in a new space now
12:05
I want to make sure people understand
12:07
this this framework and specifically how
12:09
would you best describe the difference
12:11
between this middle range research and
12:14
macro
12:15
research so middle range research um is
12:19
usually focused on a more specific set
12:21
of research questions or a constituency
12:23
so if macro is like let's understand the
12:26
overall competitive landscape let's do a
12:29
conep car typee project where we really
12:31
look ahead let's get involved with
12:33
strategic planning which is a wonderful
12:35
thing for researchers to do um you know
12:38
do Tam studies other things like that
12:40
that stuff lives in the macro space the
12:42
middle range space is like um what's a
12:46
good example we want to know how Airbnb
12:48
hosts feel about their payment
12:52
options let's like a really interesting
12:54
reasonable question right and we can go
12:56
out and do research on that but it's not
12:58
that specific it's not really targeted
13:00
at a business problem yet it could be
13:04
right maybe that's a result of the
13:05
research but it yields these kind of
13:07
like middle range insights in which
13:09
we've learned things like well hosts
13:11
want flexibility about their you know
13:14
payment options I'm making this up but
13:15
you know and that's a good example where
13:17
it's like it's not that that's not an
13:19
interesting set of questions it's just
13:21
not quite pointed enough in order to
13:24
like re and it's not framed in the
13:26
language of like the funnel or the the
13:29
business strategy or the okrs it's not
13:32
quite enough aligned enough to that it's
13:33
too blob in that middle level and it
13:36
ends up not driving impact I think it
13:38
also leads to a lot of the things as you
13:41
describe people don't like about
13:42
research it delays everything you have
13:44
to wait for the research to be done to
13:45
like have an answer to make a clear
13:48
decision it also creates this issue that
13:51
people complain about that like PMs and
13:53
product teams don't want to just make a
13:55
decision on their own they're like I
13:56
will get this additional data point and
13:58
make sure research tells us this is the
14:00
right answer instead of just trusting
14:02
there got I guess maybe along those
14:03
lines this may be going off a little
14:05
track but what are your what's your
14:07
advice there for say product managers or
14:08
PMS or product teams to not necessarily
14:12
rely on research for that middle
14:13
research I think the reason why so many
14:16
PMS ask for those middle-range questions
14:18
is because they haven't really gotten
14:20
deep with their researcher in a way
14:22
which can leverage it for Maximum Impact
14:24
so if the question is like hey Jud you
14:27
just pointed out like a bunch of
14:28
problems like can you be more solutions
14:30
oriented well the solution is simple but
14:32
not easy to me it's that we need to
14:35
restructure the way we make products in
14:38
a way which integrates uh research much
14:41
more fully like it looks like consistent
14:44
relationships in which researchers and
14:46
the work and the insights they provide
14:48
are a part of the process from beginning
14:50
to end and like I think Glenny you as a
14:53
PM that's how you worked you know I
14:55
remember you like I know who you worked
14:57
with you worked with great researchers
14:59
but honestly most product processes are
15:02
not that way and so that's when you like
15:04
research is a service function it gets
15:07
called in right at the end it's reactive
15:10
in the sense that a researcher in the
15:11
room listening and participating in the
15:13
conversation could have a ton of impact
15:16
on framing exactly the right question
15:18
that will drive maximum business impact
15:20
maximum product Improvement at that
15:22
moment and then go do it quick and get
15:23
back and we're on to the next but they
15:26
weren't there the relationship wasn't
15:27
there they're not engaged in the in the
15:30
project from the beginning and and and
15:32
that's the number one root of the
15:33
problem as long as research is a service
15:35
discipline I think we're going to be
15:36
stuck in this spot when people might be
15:39
hearing this on the one hand it's
15:41
research has been not as helpful to
15:42
teams as they thought and researchers
15:45
have been spending time on the wrong
15:46
thing on the other hand your advice is
15:49
integrate research from the beginning
15:50
make them more involved throughout and I
15:52
think that might confuse people what's
15:55
your how should people think about like
15:56
research is actually more important you
15:58
should integrate them more deeply
15:59
there's a vicious cycle that's been
16:01
happening right is from from where I sit
16:03
and this is what I hear from you know
16:05
many many researchers and research
16:06
leaders which is a lot of companies
16:09
hired a lot of researchers with great
16:10
intentions didn't quite know how to
16:12
integrate them you know and research is
16:14
a is a ux research is a kind of a newer
16:17
discipline so maybe that's not
16:18
surprising we're still learning how to
16:19
use it cool let's evolve but a lot of
16:22
companies hired these people but they
16:24
hired them into kind of like a service
16:26
discipline very reactive not in the room
16:28
not in gred in the way I said and so
16:30
they had less input on the questions to
16:32
ask not or they're included but only at
16:34
the end right and then they're unable to
16:37
build those direct relationships to be
16:39
there in the room to actually like Drive
16:41
the questions and insert insights
16:43
because a good researcher is like the
16:44
repository of insights you need for
16:46
growth but they're not there they don't
16:48
participate in the decision so they end
16:50
up re doing research they have jobs to
16:52
do so they do research that is too
16:54
reactive it doesn't matter and then it
16:56
doesn't you know it's less impactful
16:58
researcher or um Executives kind of
17:00
conclude that therefore researchers are
17:03
not as impactful and then they get
17:05
sidelined or laid off and the cycle
17:06
continues so I think the short circuit
17:09
is the constant engagement if you take a
17:12
great researcher and you insert them
17:14
consistently in a product process I feel
17:17
confident that researcher will drive uh
17:20
product Improvement uh metrics impact
17:23
growth all the things that we you want
17:25
to see as a PM and a product leader it's
17:27
just that's the exception not the norm
17:29
these
17:30
days this may be a hard question to
17:32
answer but when people hear if you have
17:34
a great researcher here's how you
17:36
approach it what are signals that your
17:38
researcher is great versus not great
17:40
what are some things people could look
17:41
for to tell them like oh maybe I have
17:43
the wrong researcher in my team so the
17:45
best researchers I think are first of
17:47
all multi method the first iteration of
17:51
user research was primarily a
17:52
qualitative discipline but a strong
17:55
opinion that I have is that is largely
17:58
one of those models that needs to evolve
18:00
it's not that qualitative user research
18:01
is no longer important it's that the
18:04
best researchers have five tools I think
18:07
they have five tools and those five
18:08
tools are um number one what we would
18:11
call formative or generative user
18:14
experience research so looking ahead
18:16
Innovation focused really open-ended
18:18
maybe more ethnographic let's go out
18:20
into the field and talk to hosts and
18:22
guests on Airbnb um let's see people
18:24
using our product in the field stuff
18:25
like that so that's formative um the um
18:28
the second type is evaluative right so
18:31
more like usability
18:33
testing the third tool is a basic
18:36
rigorous survey design right it's the
18:38
best scaled way to get um responses from
18:40
communities small and large you can get
18:42
a lot out of really well-crafted surveys
18:45
but to do that you have to have the
18:46
fourth tool which is applied statistics
18:49
the best research know a little bit of
18:51
stats like you can't interact in a world
18:52
of AB testing without knowing basic
18:55
statistics and then in the old version
18:58
of this the fifth tool was um was SQL
19:01
because I think good researchers need to
19:03
be able to run their own queries these
19:05
days so much of that is dashboard it
19:07
that the fifth tool may now be um prompt
19:10
engineering you know which is the thing
19:11
we could talk about but I think it's
19:13
some maybe that maybe that's the fifth
19:15
tool is somewhere is it technical skills
19:16
that fall in between querying your own
19:18
data understanding it very well in
19:20
companies that are wash with data and
19:21
then interacting with generative AI
19:24
amazing that's such a cool list okay so
19:26
just a playback formative generative
19:28
Innovative uh skills to think bigger and
19:31
you and come up with new ideas um
19:34
usability yep yeah usability what how
19:37
did you describe it I have a different
19:38
word here evaluate evaluative right so
19:41
like we're evaluating products and doing
19:44
more really that's kind of the micro
19:46
level uh survey design being really
19:49
rigorous about it applied statistics and
19:51
then SQL dasboard prompt engineering
19:54
right maybe just one last question along
19:55
this thread also big question but any
19:58
advice for how to evaluate these skills
20:00
slash interview for them I know this is
20:02
its own like deep topic but any advice
20:04
for someone try to find this person you
20:06
know I've interviewed hundreds or
20:07
thousands of of researchers and the way
20:09
I usually approach that is you know you
20:11
want you want a researcher who's a who's
20:13
got a Swiss army knife because if all
20:15
you have is a hammer then everything
20:16
looks like a nail and so if you give in
20:19
the context of an interview let's say a
20:21
researcher a pretty juicy open-ended
20:24
research question and you want to see
20:27
how they handle it and a good answer is
20:29
usually multimethod right we're not
20:31
going to handle it in any one way we're
20:32
going to say well here's a couple of
20:33
ways we could deal with this here say we
20:35
could do this in a day or a week or a
20:37
month I mean we usually don't have a
20:39
month but sometimes big research
20:40
projects go on for that long and here
20:42
are the different sets of methods that
20:44
we can use so see where they go it's
20:45
actually pretty simple most researchers
20:48
are deeper in one than the other and
20:49
sometimes you can make up for those five
20:51
tools with the team right so you have
20:53
experts who are who are kind of t-shaped
20:55
but maybe deeper in one or several of
20:57
those those ways but when I built a team
21:00
at at uh meta and at Airbnb that was my
21:03
goal is individually as researchers
21:05
build up those tools and then as a team
21:07
build deep expertise that would fill all
21:09
the gaps coming back to the main premise
21:12
of your post one of your big takeaways
21:14
is researchers need to be much more
21:16
business oriented thinking about what
21:18
helps the business versus the user which
21:20
I think to a lot of researchers they
21:22
feel really weird can just talk about
21:23
your your kind of takeaways there so
21:26
much of user experience practice not
21:28
just research but design
21:30
too is focused on empathy right and very
21:34
user centered this is beautiful I'm not
21:37
saying that we should abandon that I
21:38
think what I'm saying is there's an
21:40
overlapping vent where you have the user
21:42
and profit or the business and what
21:45
researchers need to do is be way more
21:48
explicit about finding that overlap so
21:51
one thing I I I often when researchers
21:54
ask for advice they're like well what
21:55
should I do to be more business Prof
21:57
business or profit F focused I say
21:59
something
22:00
like did you read the last quarterly
22:03
report if it's a public company did you
22:05
listen to the shareholder call you know
22:08
um and they're probably like no you know
22:10
it's full of a bunch of you know
22:11
language I didn't quite get and I'm like
22:12
M yeah so there you go that's the
22:15
language you need to learn scour scour
22:18
your um you know Google drive folder
22:20
your internal folder and look for all of
22:22
the documents that are about this
22:24
quarter or this halfes or next half
22:26
strategy what are the okrs understand
22:28
the metrics and the funnel the
22:29
conversion funnel like know it back and
22:31
forward because then what you're doing
22:34
is you're proposing you if you're in the
22:36
active conversation you're saying cool I
22:38
hear you asking that research question
22:39
I've identified this is exactly the spot
22:41
in the funnel where I think we need to
22:43
do work right there's a there's an
22:44
opportunity here or that competitor is
22:46
eating our lunch um with this group of
22:49
users like I know that because I I read
22:51
the competitive report and I understand
22:53
it deeply so like those are skills that
22:56
many some researchers have and a lot are
22:58
built building these days but
22:59
historically like last 15 years it
23:02
hasn't been a thing we've been as as
23:03
focused on and I think that's an
23:04
evolution that needs to happen I think a
23:06
lot of PMS listening to this are going
23:08
to be like Hallelujah this is exactly
23:11
what I've been trying to convince people
23:13
of it's what I've been trying to
23:14
convince my researchers of design often
23:16
falls into this but Lenny the opposite
23:19
is true too like because you got to take
23:21
the average PM who lives in that land
23:23
all day every day and what they do is
23:26
not in the VIN you know I think those
23:28
are a people who are also like
23:32
performing customer centricity and
23:34
Performing user centeredness a lot when
23:37
they're really not interested and so
23:39
this is like not about researcher this
23:42
takes two sides fixing this broken
23:43
system takes everyone researchers PMS
23:47
designers everyone at a company but also
23:49
the way that organization is structured
23:52
and integrating itself in a different
23:54
way everybody's got to come to the table
23:56
such a good point and you have this
23:57
actual term that you call uh user
23:59
centered performance where it's the
24:01
performance of being user centered can
24:03
you talk about that and then just what
24:04
advice you give to PMS that hearing this
24:06
are like yes I love everything you're
24:08
saying and then not realizing maybe
24:09
they're too far in that extreme so user
24:12
centered performance is uh a term I made
24:15
up because it's fun to make up terms and
24:18
it refers to um customer Obsession or
24:21
user centered practice that is symbolic
24:24
rather than focused on learning right so
24:28
it's hugely common I would argue it's
24:31
work we do to signal to each other how
24:34
customer obsessed we are not because we
24:36
want to make a different decision and
24:38
like I if you're if your like listeners
24:40
are like I don't do that I'm kind of
24:43
like think about it for a second right
24:45
because there there this is extremely
24:48
common it shows up in explicit ways and
24:51
implicit ways so explicitly I would say
24:54
every time a PM comes to a researcher at
24:57
the end of a product process and says
24:59
can you just run a quick user study you
25:00
know just to validate our
25:02
assumptions that's user centered
25:04
performance it's too late to matter that
25:07
PM is not interested in being wrong at
25:11
all it's too late in the game for that
25:13
we got to ship it right what they want
25:15
is to check the box so any check the box
25:17
style research is a wild example of user
25:21
centered performance I would argue every
25:24
researcher has probably had to do
25:26
executive listening sessions you know
25:28
because a lot of PMS Founders product
25:30
people but designers too they want to
25:33
get close to the customer right and so
25:35
like can I do some focus groups I want
25:37
to be there I want to ask them questions
25:39
this is 97% performance it's
25:42
well-intentioned but it isn't focused on
25:45
learning it isn't going to drive better
25:47
outcomes or more impact and then there's
25:50
all these implicit ways that people
25:51
engage in that kind of user performance
25:53
too A lot of it comes down to cognitive
25:55
biases confirmation bias ego like one of
25:59
my big kind of
26:00
mantras was we don't validate we falsify
26:05
right we are looking to be wrong that is
26:07
the mindset you should use when you're
26:09
approaching insights and research I want
26:12
to be wrong I want you to do research
26:14
that shows we were off base in the
26:15
following ways tell me exactly how and
26:18
why in a way that allows me to fix it
26:20
quickly but many PMS many designers are
26:23
not in that place they do not want to be
26:25
wrong they're looking to validate and
26:27
that's user centered performance oh man
26:30
I think a lot of people are hearing this
26:31
and feeling feeling exposed
26:34
exposed I feel like you're like this
26:36
deep Thro person coming from sharing
26:39
these things people don't want to talk
26:40
about I love there's this quote in your
26:44
post I'm going to read product managers
26:46
love to ask for Middle range research
26:48
that they can use to justify decisions
26:49
they're reluctant to make on their own
26:51
user designers love to ask for Middle
26:53
range research because it fits their
26:54
model of what proper design process
26:56
should look like Executives love to to
26:58
ask for Middle range because they don't
26:59
really understand what research is for
27:01
it helps them do performative user
27:03
centeredness in the end they will decide
27:05
based on their own opinions there is an
27:09
important place for intuition in in
27:12
product development of course like the
27:14
best designers researchers product
27:16
people develop strong intuition for the
27:19
product but like you got to understand
27:22
intuition is where all of those biases
27:25
lie it's where all your blind spots are
27:27
and what great insights people do what
27:30
great researchers do when you're next to
27:31
them all the time is they'll expose you
27:33
I don't have to be the deep throat right
27:35
because you have somebody whose
27:37
professional job is keeping you honest
27:39
is probably the wrong way to to put it
27:41
but you know somebody who's who who
27:44
capabilities are an about expanding your
27:47
horizons making it so that your
27:49
intuition is constantly improving you
27:50
don't have to rely on it when you're
27:52
intuition and the evidence sort of
27:53
collide in a way that either affirms or
27:57
falsifies the product decision you made
27:59
now something really good is happening
28:01
so you know and the other thing that is
28:03
inherent in that quote is you know I at
28:06
Airbnb wore many hats over the years I
28:08
was head of research two different times
28:10
I was head of design for guest products
28:11
and my last job was I was head of the
28:13
design studi so ux research ux design
28:16
writing localization they all report it
28:18
to me so I've seen this from many
28:20
disciplinary angles in the ux field and
28:24
researchers aren't the only ones who are
28:26
guilty of this like I would say design
28:28
it has a ton of
28:30
performance and it comes from the fact
28:32
that we have like figured out user
28:34
Center design this process or design
28:37
thinking which which idio popularized
28:40
like that's what we're supposed to do
28:42
right right Bezos told us that we as PMS
28:45
had to be customer obsessed right so
28:46
that's what we're supposed to do it's a
28:48
really common and damaging thing when we
28:51
are not genuinely like we don't
28:53
genuinely have that growth learning
28:55
mindset and it's like easy to sideline
28:57
research Searchers like we don't need
28:58
them in that situation we we've got our
29:00
guts isn't the gut where like a great PM
29:03
a great founder needs to have that gut
29:05
and they do but they need to be open to
29:08
the fact that your gut is limited and
29:10
biased and and narrow and wrong
29:12
sometimes the two sides of this is trust
29:15
your gut opinion I don't need research I
29:16
don't need data I have opinions and my
29:18
own experience and I'm going to use the
29:20
product and let's just go with what
29:22
feels right to me versus pure data
29:24
driven research driven for designers
29:27
that are maybe listening for product
29:28
managers do you have any advice for just
29:30
like where to fall on that spectrum and
29:32
just how to best leverage research to
29:34
inform that opinion yeah I taught a
29:37
class at UC Berkeley this semester on
29:39
leadership and um we talked about that a
29:41
lot um because you know great leaders
29:44
develop intuition right they that's part
29:46
it's the it's the it's the pattern
29:48
matching part of experience right where
29:50
you're you develop puristic which allow
29:52
you to make good judgments even if you
29:55
can't quite explain where that judgment
29:56
came from right that that's what the gut
29:58
is but it's also like I said where bias
30:01
comes from where where cogn like all the
30:03
cognitive biases there areund there's a
30:05
list of 151 of them on Wikipedia I won't
30:07
name them but like all those thorny
30:09
things that that lead us astray you know
30:11
the behavioral economists and social
30:13
psychologist study those live in the gut
30:16
and so the advice is you know when you
30:18
are looking to check your gut you have
30:21
to do that
30:23
thing a lot of a lot of your listeners
30:25
have probably read uh Thinking Fast and
30:27
Slow system one system two right I have
30:30
it here right under my laptop actually
30:31
holding up my laptop screen that's so
30:33
appropriate Lenny um so the secret is
30:36
not that sexy it's system too right so
30:39
you engage that slow methodical process
30:44
in which you do analytic thinking as a
30:46
means of checking your gut slow in the
30:48
grand scheme of things right slow
30:49
meaning not a split second decision not
30:51
like months of analysis that's not what
30:53
I mean the other thing you can do and
30:56
there's really great research on this is
30:58
you bring in the wisdom of the crowd
31:00
right so the wisdom of the crowd is a
31:02
phrase a lot of people are familiar with
31:04
and it works in a specific situation the
31:07
wisdom of the crowd works when the
31:10
people involved with the decision are
31:12
bringing diverse sources of information
31:15
and judgment to the table obviously if
31:16
everybody has the same sources of
31:18
information then it doesn't matter how
31:19
many people are out there so if you want
31:21
to check your gut get a bunch of
31:22
different guts together get a bunch of
31:25
different people in the room um who can
31:27
bring evidence and intuition to bear and
31:30
have a open like direct and kind
31:32
conversation in which we might disagree
31:35
you know who's great at that
31:37
researchers leading those discussions
31:39
essentially and getting a bunch of
31:40
people's opinions yeah this is the
31:42
structural solution I'm talking about
31:44
Lenny is like I never asked for research
31:46
teams to have their own separate
31:49
okrs I said two things number
31:52
one what's the teams okay like shouldn't
31:55
the PMS the engineers the designers and
31:57
the reearch everybody should have the
31:58
same set of metrics for Success right
32:00
because either we're doing it together
32:01
or we're not and then I said my metric
32:04
for
32:05
success is when they won't that have
32:07
that meeting without you that's my
32:09
metric for Success if they cannot have
32:12
that decision me making meeting without
32:14
the researcher there that means you've
32:16
developed influence strong strong
32:18
trusting relationships you're an active
32:20
participant in the process not just sort
32:23
of somebody who provides input into
32:25
someone else's process and that is when
32:27
researchers can have huge impact I think
32:30
of the PM role in a similar way even
32:32
though people won't have these meetings
32:33
without PMS because they're often at the
32:35
center of a lot of the stuff but you
32:36
want to be a PM that people want on
32:38
their team there's a lot of teams that
32:40
are like we don't want need PMS we don't
32:42
need product managers they just get in
32:43
the way and I find that that's only the
32:46
case when the product manager is not
32:47
great and not really good at their job
32:49
because most gr PMS just make everyone's
32:50
life easier they do the grease I love it
32:54
you mentioned also that before we start
32:56
recording that the biggest challenge for
32:58
us researchers is the relationship with
33:00
their product manager can you speak to
33:02
that and what you've seen there I'm wary
33:05
of overgeneralizing but I can tell you
33:07
that from my experience and from what I
33:10
hear the product research or product
33:13
insights relationship is one of the most
33:15
challenged and I think it comes
33:18
from the fact that fundamentally many
33:21
researchers are just not included in the
33:23
process that that PMS are running and
33:26
then there I actually I kind of thought
33:29
I I I did some asking around before
33:32
before this uh podcast and so I I
33:35
thought you know there are some tropes
33:38
that researchers have about PMS that are
33:40
worth PMS knowing right like just like
33:42
four or five of them like the the things
33:45
that researchers know PM say which drive
33:48
us nuts because they're not true so the
33:53
first one is that research just slows us
33:56
down research is too
33:58
slow this is um a great
34:01
research team can do research in a day a
34:03
week or a month you know it just depends
34:06
on what you want to get out of it you
34:08
know like if you you know how much how
34:10
how much detail do you need how many
34:11
people do we need to talk to what is the
34:13
depth or breath do we need to go to
34:15
seven different countries to talk about
34:16
our constituencies in Latin America well
34:18
that's not going to happen overnight but
34:20
we don't often need that the other way
34:22
to look at that is that is it slower to
34:24
get it wrong and fix it than to take a
34:28
hot second to do the work to get it
34:30
right the first time you know so that's
34:32
BS like research good research doesn't
34:35
slow us down it speeds us up and also
34:37
just along those lines like a big part
34:38
of your premise is you don't need to do
34:40
as much research as people are doing
34:42
like this middle research that a lot of
34:44
the time is put into yeah research can
34:47
go super fast I think especially so the
34:49
macro level research I hope what it is
34:52
is tied to things like annual planning
34:55
processes we did a thing at air be
34:57
several years that we called the it was
34:59
like insights 2019 insights 2020 they
35:02
were concept car projects and we spent
35:03
quite a long time synthesizing the
35:06
entire Year's worth of insights from
35:08
every place we could get them and then
35:10
developing with designers and Engineers
35:12
like a concept car for five years in the
35:14
future so that's a long process but the
35:17
micro level there's so much business
35:19
value to be derived there so much
35:21
business value and it can go so fast
35:24
Lenny right it can go so fast you can
35:26
have you know you can have results in 48
35:28
hours on these things like we did a
35:31
thing at Airbnb
35:33
um there's a famous story which I'll
35:35
only tell in the abstract because I
35:36
don't want to out anything but it's like
35:38
the we call it the multi-million dollar
35:39
button and basically we did research
35:43
which uh revealed that people weren't
35:47
going down the purchase funnel because
35:49
they were afraid that the calls to
35:52
actions on the button was making them
35:53
afraid that it would initiate a purchase
35:55
when really it was just taking the next
35:56
step
35:58
right we changed the text on the button
36:00
with with help from our amazing uh
36:02
content design our ux writing team we
36:04
basically changed seven characters and
36:07
made Airbnb millions of dollars right
36:10
because because what we found out it
36:12
like was really simple it was just like
36:15
hey this button feels scary the CTA on
36:18
the button feels scary so that's a great
36:19
example of how micro and that happened
36:21
in like 48 hours we we would discover
36:23
that Insight or 20 overnight basically
36:25
and we were like hm maybe we should test
36:26
some ctas we did the conver the
36:29
conversion like we we added like 1%
36:32
which is really really hard to do you
36:34
know so that's a quick example of how
36:36
that type of quick research can drive a
36:37
huge amount of business value this
36:40
episode is brought to you by HRS many of
36:43
you already know HRS as one of the top
36:45
tools for search engine optimization
36:47
it's used by thousands of seos and
36:49
companies like IBM Adidas and eBay what
36:52
you may not know is that there's a free
36:53
version that was made with small website
36:55
owners in mind it's called called hre's
36:57
webmaster tools it's free and it can
37:00
help you bring more traffic to your
37:01
website hre's webmaster tools will show
37:04
you keywords that you rank for and
37:06
backlinks that you can get it also
37:08
performs automated site audits to find
37:10
what issues prevent your website from
37:11
ranking higher on Google every detected
37:14
issue comes with a detailed explanation
37:16
and advice on how to fix it visit
37:18
hrs.com
37:20
awt set up a free account connect your
37:23
website and start improving it that's
37:25
ahrefs.com
37:27
SL
37:29
awt so just to make this even clearer I
37:32
think this middle research zone is the
37:34
stuff that does slow people down I
37:36
imagine it's like what are the
37:38
challenges hosts have with payments on
37:39
your BNB like what you're basically
37:42
saying is spend your time doing the
37:44
micro stuff like usability research and
37:47
then the bigger stuff that's part of
37:49
overall planning that's part of the
37:50
planning cycle it's not like every
37:52
project you're working on you need to
37:53
have like a whole research project on
37:56
exactly the micro research should be
37:58
much more common a lot of researchers
38:00
think that there that that's sort of
38:02
scut work you know that like usability
38:04
is something Junior researchers do I
38:06
completely disagree like I think we need
38:09
to get back there as an industry and be
38:11
like when you make a product easier to
38:12
use when you like discover problems with
38:16
functionality business metrics we care
38:18
about will go up I've seen it happen
38:20
right but it's that's not just work for
38:22
interns and N grats that's for sure and
38:24
then the planning process like
38:26
absolutely if we're integrated from
38:27
beginning to end we can help and you
38:30
know the thing about that middle range I
38:31
think you're right like that's the stuff
38:33
that makes the the sort of stereotype
38:35
that research is is slow and a lot of
38:36
times it's also because it's just not
38:38
pointed enough the researcher can also
38:40
say in that moment I have studied the
38:43
business plan I know exactly where like
38:46
I've seen the metrics Trend I have an
38:48
idea about exactly where that's going to
38:50
go we still need to do that middle range
38:53
uh research the question is valuable but
38:54
it's now very pointed and that the time
38:57
is worth it amazing okay I want to hear
39:00
the rest of these stropes okay so
39:01
research is too slow is the first one
39:03
the second one I can do my own research
39:06
why do I need researchers and like
39:08
that's true as product people I hope I
39:11
hope you know you are uh engaging with
39:14
customers and and and listening well but
39:18
no offense garbage in garbage out you
39:20
know um like the thing uh the thing is
39:22
anyone can talk to a user that does not
39:25
constitute research or insights work you
39:27
know because uh one user can be powerful
39:31
but one user can be idiosyncratic and a
39:34
researcher knows how to get to the heart
39:35
of that really quick they know how to
39:37
take that conversation and understand
39:41
and situate it in a way which means like
39:42
sure like democratize research that's
39:45
happening there are tools out there that
39:46
will let anybody get customer feedback
39:48
voice a customer type stuff but a
39:51
researcher is there to help you turn uh
39:55
garbage into you know into something
39:58
that's not garbage so and avoid the bias
39:59
that can come from like you just
40:01
reaching out to your cousin's family and
40:02
then doing whatever they thought you
40:04
should do to the product so that's the
40:07
that's the second Trope the third one is
40:09
AB test everything right um and like
40:13
like AB tests are great but one of my
40:15
most painful things to do is to sit in a
40:17
room full of PMs and data scientists who
40:19
have just uh seen the results of an
40:21
experiment that like flipped to stat Sig
40:24
and then they're like cool I was
40:27
significantly down over this course of
40:28
time for these users and then they just
40:30
start speculating about why that is
40:34
because the ab test rarely tells you why
40:37
it changed in the way it did and then
40:39
this endless flywheel of AB testing goes
40:43
and I'm like hey you don't have to guess
40:45
I know somebody who can get you an
40:47
answer to the or at least evidence that
40:49
addresses the question of why did we see
40:51
the the test result we did in a very
40:53
short amount of time or you could use
40:55
your customers as skin pigs and throw
40:57
more experiments at them over and over
40:59
and spend a long time on it and and and
41:02
come to the same place in the end I
41:04
think a similar critique that PMS often
41:07
have is AB testing is conclusive
41:10
scientifically statistically us a
41:12
research is just talking a bunch of
41:14
people why would I trust that what is
41:17
your best way to help PMS realize that
41:20
this is actually very uh valuable data
41:23
and you should listen to it it's not
41:25
just this you know a story here and
41:26
there yeah no I think they're both right
41:29
AB testing is as close as we can get to
41:32
causal making causal claims about
41:34
products right research is usually not
41:36
oriented towards making causal claims or
41:38
it should not be but those causal claims
41:41
rarely tell you how and why things
41:43
happen and if you want to develop if you
41:45
want to not make that mistake again in
41:46
the future you need to know how and why
41:49
if you want to build a better product in
41:50
a way that doesn't just answer this
41:52
narrow question that an AB test answered
41:54
you need to know how and why and so you
41:57
kind of need both like beautiful
41:59
Partnerships between data scientists and
42:01
and research and insights people like
42:03
are are I think what we're going to see
42:05
in that next Evolution and if you set
42:07
that up that that sort of virtuous cycle
42:09
up if you set up the engagement where
42:10
those people are involved from the
42:12
beginning you don't make those mistakes
42:13
you get the cause of relationship which
42:15
is valuable for one reason and the hous
42:17
and wise which are valuable for other
42:18
reasons awesome okay I think there's two
42:22
more tropes you had so one of them is
42:24
like a simple one which is like everyone
42:25
loves to quote that that it turns out a
42:28
totally apocryphal Henry Ford quote you
42:30
know about if I'd asked my users it
42:32
turns out to the best of our knowledge
42:34
he did not say that and really yeah I
42:36
know isn't that sad know that I know
42:39
sorry to burst your bubble Lenny oh wow
42:41
um does anyone say anything I feel like
42:43
every quote is is aable now I I know
42:47
what is reality geez can't we well let's
42:49
just okay maybe he said that like he
42:51
certainly believed that that's what the
42:52
historians say but the reason that makes
42:55
researchers so angry is
42:57
because that's not research that's not
42:59
what researchers do a researcher who's
43:01
going to ask customers what they want is
43:04
a bad researcher you need a different
43:06
researcher like that's not I've never
43:08
done that in my career no one on my team
43:10
has ever run a study that's like that
43:13
you know so so that that just makes
43:14
researchers mad and then the last one is
43:19
about post talk bias it's we knew this
43:21
already that was obvious and um I think
43:24
a lot about this book which I would
43:27
recommend to um your listeners so the
43:30
author is a sociologist at upen named
43:32
Duncan watts and the title is everything
43:35
is obvious if you already know the
43:38
answer and it's about hindsight bias he
43:41
kind of like makes the argument that we
43:42
rely too much on
43:44
intuitionistic and pattern matching in a
43:47
way that sort of is inappropriate to our
43:49
experience and it's like it leads us
43:51
astray it's kind of like a form of self
43:54
gaslighting and it happens because we
43:56
end up sort of selectively remembering
43:58
things and then constructing narratives
44:00
around them in a way which makes us feel
44:02
like we already knew that when we in
44:03
fact did not and and he talks about this
44:06
other the one of those cognitive biases
44:08
called the the the narrative fallacy
44:10
which is is the idea that like people
44:12
love to make convenient simple stories
44:14
about the past like if I asked you about
44:16
your career Lenny and how you got to be
44:18
this amazing podcast host you'd be like
44:20
well let me tell you about the series of
44:22
events and like we do that you know it's
44:24
like part of how we make sense of Our
44:26
Lives and the information around us but
44:28
it would probably be a lie you know like
44:30
you you know in the sense that like we
44:31
all kind of twist the evidence we have
44:33
to fit the narrative we want to be true
44:35
because it's simple and lovely and makes
44:36
us happy this is gonna sound self-
44:38
serving but I find I'm the opposite I'm
44:39
like I have no idea how this all came
44:41
about here's here's some things that
44:42
kind of happened and somehow I ended up
44:44
here but maybe I'm being very modest and
44:46
try to not give myself any credit that's
44:49
beautiful thank you for these tropes by
44:51
the way this was fun I didn't know you
44:52
were going to do that so that's a fun
44:54
little collection we've got here um I
44:56
wanted to ask about there's this tweet
44:58
by Patrick hollison that I've brought up
45:00
a couple times on this podcast that I
45:02
think uh is really interesting and his
45:06
tweet is this in my opinion the best
45:08
product will stem from a very strong
45:10
mental model of the domain and user user
45:12
research can help you get such a model
45:15
and validate it along the way but it's
45:17
important to view the syllogism of uxr
45:19
as model as of user research to
45:22
improving your mental model of the user
45:24
to what product you should build
45:26
versus user research tells you what
45:29
product to build does that resonate in
45:32
any way thoughts on that way of thinking
45:34
about user research yeah I mean there's
45:37
there's a double-edged sword we talk
45:38
about a lot in in in the research
45:41
Community which is about making um
45:43
recommendations recommendations for
45:45
design right so the the best research
45:48
doesn't like leave it at that right it
45:51
it tells you and and and it's like uh
45:53
the what the so what and then the then
45:55
what but the problem with that is some
45:59
researchers go too far in the other
46:01
direction where they're like we ran the
46:03
study it yielded these insights and
46:05
therefore this is what we should build
46:07
and everyone else on the team is like
46:09
whoa whoa whoa right glad to hear your
46:12
thoughts on the matter but like there's
46:14
a lot going on here maybe we should talk
46:16
about it like so and that makes perfect
46:18
sense that's like a failure of
46:19
communication and I think that it like
46:21
speaks to the the thing that Patrick is
46:23
saying is like good research can
46:25
sometimes tell us exactly what the
46:27
problem is and exactly how to fix it an
46:29
example that is the the multi-million
46:31
dollar button I told you about but in a
46:34
lot of the bigger picture questions
46:35
especially the macro ones and maybe also
46:37
the really pointed middle range ones the
46:40
point isn't really this is exactly what
46:42
we should do and this is exactly what we
46:44
should build it is let us develop a
46:46
framework which is based on actual
46:48
evidence and then together as a team
46:50
figure out how we want to experiment our
46:52
way to a successful product to close the
46:54
loop on this specific
46:56
thread what is your advice to teams
47:00
researchers to help move out of this
47:03
kind of Reckoning and to move forward
47:05
and help the field both from a user
47:07
researcher perspective and also from
47:08
just like a company that maybe laid off
47:11
a bunch of user researchers or is trying
47:12
to decide what to do with the
47:13
researchers thank you for asking um I I
47:16
think I said to you earlier like you
47:18
know and I feel some pressure as uh
47:20
maybe the first conversation that you've
47:22
had specifically about research on this
47:23
podcast yeah I think so and I want to
47:25
help
47:27
I I believe so much in this discipline
47:29
of research and insights and I think
47:30
there's that you know when I said the
47:32
user uh ex the the ux research
47:35
discipline of the last 15 years is dying
47:37
I didn't mean that I think research is
47:39
dying far from it I think that there's a
47:42
version of it which we're now moving
47:43
past and into a new version we're going
47:45
through an evolution as many do and so
47:46
the question for me is like how can
47:48
researchers and the companies and and
47:51
the other people with with whom they
47:52
work create a new version a different
47:55
version and evolution which is hugely
47:56
impactful for the business and so the
47:58
advice i' give to researchers about that
48:01
is develop diverse research skills
48:04
remembering like the five or five and a
48:07
half uh tool list that I that I
48:09
mentioned earlier really go deep on that
48:12
business knowledge So speaking the
48:14
language of product and business and
48:16
metrics and understanding exactly how to
48:18
use your insights like a scalpel
48:21
building those strong relationships
48:22
which is not a thing that researchers
48:24
can do by themselves it requires two
48:26
engagements and and also in a way which
48:29
allows researchers to do fewer things
48:30
better so most researchers that I know
48:32
are working on teams where they're like
48:34
I'm the only researcher and there's like
48:36
I have seven PMs and 20 designers and
48:39
and like I'm trying to do 10 projects
48:41
and no one's going to do a job good job
48:43
that way so researchers have to learn
48:45
with their Partners about how to say no
48:46
and focus on the most important things
48:49
but that's like only half of it right
48:50
that's the research side I have like two
48:53
thoughts about about what companies
48:55
should be doing
48:57
the first one is it's a little bit of a
48:59
side but not really like one thing I
49:00
learned by through the responses to to
49:04
the article was everybody came out of
49:05
the Woodworks from the variety of
49:07
insights disciplines that are out there
49:09
because I come from a tradition of user
49:11
experience research or user research but
49:14
there are many insights disciplines in
49:15
many Industries and they all wanted to
49:18
claim one type of research or another
49:20
and say like oh well we over here in
49:23
consumer insights or market research
49:26
have been doing that well for years and
49:28
you know there are many insights
49:29
disciplines and generally I think
49:31
creating silos is stupid actually I'm
49:34
curious what you think because here's
49:35
the number one thing I heard when I
49:36
joined Airbnb and you were there is I I
49:39
did like a quick listening tour where I
49:41
talked to a bunch of product people and
49:43
um and they all kind of said the same
49:44
thing they were like listen we have all
49:46
these different people throwing insights
49:48
over the transom and it's great we want
49:51
to hear from the data scientists from
49:53
the product Specialists from the
49:54
customer service people and the voice of
49:56
the customer reset whatever all that
49:58
stuff but they're all coming over the
50:00
side and we don't know what to make of
50:01
it right it's like too much and that as
50:04
much as anything is an argument for
50:05
companies to stop siloing research
50:07
disciplines so when I joined Airbnb I
50:10
like set out to create an integrated
50:12
insights function where it's like let's
50:13
do ux research let's talk about the
50:16
market and competitors when we have to
50:18
Let's integrate smartly with data
50:20
science functions let's integrate all
50:22
the stuff we're getting from customer
50:24
service feedback you know we we brought
50:26
over what was then The NPS program and
50:29
and sort of said like hey like if we're
50:30
getting customer feedback there let's
50:31
all just like use it all to fuel this
50:34
one insights machine so that's the first
50:36
piece of advice I give companies and the
50:38
second one without being a broken record
50:40
is to think differently about the broken
50:42
cycle so integrate researchers into a
50:45
unified lean process so if the
50:48
researcher is not there from beginning
50:50
to end if there are not strong
50:52
relationships between product people and
50:54
design people at every level engineering
50:56
people at every level and somebody who's
50:58
their partner their insight's partner
51:02
we're going to fall back into this
51:03
problem where we're just a service
51:04
discipline we're not extracting the
51:06
maximum value it comes too late we don't
51:08
know what questions to ask we're
51:09
ignorant about what research can do and
51:12
so creating that integrated lean process
51:14
where a researcher is arm inarm from the
51:16
beginning is the most important advice
51:18
I'd
51:19
give that last piece may be the answer
51:21
to this next question but the question
51:23
is how can product managers be better
51:25
Partners to user researchers slash take
51:28
more get more leverage out of user
51:30
researchers I think that is in many ways
51:33
the answer making sure that they are
51:36
creating a process for the product for
51:38
their products that integrates user
51:39
researchers and insights from beginning
51:41
to end also being willing to partner
51:44
with the research on the ruthless
51:46
prioritization um I used to say that a
51:49
full plate for a researcher was probably
51:52
three things two big projects and a
51:54
small project right like a side project
51:57
more than that your researcher is
51:58
probably not doing a very good job and a
52:00
project may take 48 hours that's okay
52:02
but so they need your help to prioritize
52:05
they need you to
52:06
participate great PMS will take the time
52:09
to be with researchers to go into the
52:12
field even to travel did you ever do
52:14
that Lenny I did I went with uh Louise
52:18
who uh introduced who like came up with
52:20
this basically told me to chat with you
52:22
about this topic Lise thanks Louise
52:26
we did a whole uh tour to Paris our
52:29
whole team or the the leads of our team
52:31
went to Paris to do a bunch of focus
52:32
groups and a bunch of us research behind
52:34
like actual mirrors I've never done that
52:35
before that trip and it was amazing we
52:37
learned a t can I tell you a quick story
52:39
about behind the mirror so this is back
52:41
from when I was at Facebook um and it
52:45
was the High Times there was like 12
52:47
2012 13 um and Newsfeed is like really
52:51
taking off ads are going into Newsfeed
52:53
and I was working I was a leader of a
52:55
team that was working among other things
52:57
on how to address uh post quality right
53:00
like how do we think about what's a good
53:02
post and how do we get feedback about it
53:04
and so there was kind of a team of
53:05
Engineers that thought one thing that
53:07
you can do on Facebook is hide a post
53:11
right so they were like this is easy
53:13
let's look at the posts that are hidden
53:15
the most and use that as the signal of
53:19
what's a good post on on Facebook seems
53:21
kind of reasonable right and like
53:23
something tripped me on this one so I
53:25
was like so I did two things so the
53:27
first thing I did is I looked at the
53:29
distribution of hiding by user and found
53:33
out that it's power law distributed like
53:35
everything on the internet there are a
53:38
few people on Facebook Who hide a ton
53:42
and then most people don't hide at
53:44
all and so then what we did was we said
53:48
let's F we called these super hiders we
53:49
called them super hiders and so we said
53:51
let's find super hiders uh around the
53:53
office and we'll get a superher in and
53:56
we'll do like a really traditional user
53:58
interview I just want we just want to
53:59
see so literally the first person who
54:02
walked in I remember because this is a
54:04
person who had those fingernails that
54:06
are so long you don't know how they can
54:07
do c touch screens um but they did they
54:10
were amazing at it and it was one of
54:12
those rooms uh where with the glass and
54:15
I insisted that the end directors the
54:18
product people and they were willing
54:20
whatever so everybody's behind the glass
54:22
and I'm there with them and the re
54:24
excellent researchers in the room and
54:25
they come in and we're just doing a
54:27
traditional thinkaloud study and so they
54:29
hey can you open up your Facebook app we
54:30
would just love to see you you know what
54:32
your experience is like and so they they
54:34
open up Facebook and we're looking and
54:35
they they look at the first story and
54:37
they hide
54:38
it and they go to the Second Story and
54:41
they hide it and this went on for a
54:43
while and like this she's definitely
54:45
using Facebook but every time she'd read
54:47
finish with a story she'd hide
54:49
it and the people in the back room were
54:51
starting to chatter and they're like
54:53
wait what what is happening right now
54:55
and like good research that the good
54:57
researcher that this person was like
54:59
they let it continue and they're like oh
55:01
can you tell me what you're thinking
55:02
right now come to find out that she was
55:05
like um well I hid that story because
55:07
I'd seen it
55:09
already the model she was going for was
55:11
inbox zero which was like sad because it
55:14
was infinitely scrolling she would never
55:16
get there and the reason I like that
55:18
story is um
55:20
because the people in the back room had
55:23
their minds blown it was not that we
55:26
assumed that was common Behavior like
55:29
this person could have been unique but
55:31
it was enough because those people were
55:34
there experiencing the research the N of
55:38
one allowed them to burst their own
55:40
bubble and realize okay we can't think
55:42
so naively about hides as a signal
55:45
anymore and we came up with a better
55:46
solution that is an awesome story and
55:48
such a good example of you don't need
55:51
statistical significance to get massive
55:53
insights like one example just gives you
55:56
a wow this might be exactly what's
55:58
happening let's go validate that versus
56:01
like we are confident 100% this is what
56:03
happen I love that it reminds me
56:05
actually in the mirror in the mirror
56:07
study that I was talking about in Paris
56:09
there's a Facebook element to it too we
56:10
were trying to convince hosts how to
56:12
feel more comfortable accepting guests
56:14
who are booking instantly and one of our
56:16
theories was if they were connected on
56:18
Facebook they would be more comfortable
56:20
letting someone book instantly yeah and
56:22
we're just like hey why don't what if
56:23
you were to connect Facebook and see if
56:24
they're friends and
56:26
everybody in Paris was very afraid of
56:29
connecting and giving Facebook any data
56:32
way ahead of like what the US host were
56:34
feeling yeah so it just made us made it
56:36
very clear nobody wants to actually give
56:38
Facebook any data so it was very anti-
56:39
Facebook at that point yeah that's so
56:41
interesting Germany Germany and and and
56:43
France were always our like bellwethers
56:45
for for what the rest of the world would
56:47
be thinking with private data privacy
56:49
concerns oh man okay a couple more
56:52
things so a lot of this started with a
56:56
lot of layoffs within user research and
56:58
I think between the lines there's this
57:00
sense of teams don't need as many
57:02
researchers as they hired during the Zer
57:04
era I think a question in everyone's
57:06
mind is just like how many researchers
57:08
do we need what is a good ratio I
57:10
imagine there's not a simple answer here
57:11
but just what's your general advice to
57:13
companies of how many researchers is
57:15
right so this is a thing I I've thought
57:17
a lot about especially in my role as the
57:19
head of the design studio that was like
57:21
my fundamental question it's like you
57:22
have all these writers designers
57:23
researchers like how do you structure
57:25
that how many and where and who works on
57:27
what and the organizing principle for me
57:29
was always
57:30
relationships you know you have enough
57:33
when the people who need to have a
57:34
constant research partner have them and
57:37
I would much rather create pain in that
57:39
situation than spread someone too thinly
57:41
so my advice was always like don't try
57:43
to create a researcher to cover this
57:45
entire product space pair a researcher
57:47
up with you know somebody who's going to
57:50
involve them in a consistent engaged
57:52
process and let them go to work and and
57:55
see the impact they're going to have but
57:57
protect their time and then other people
57:58
are like wait a second that person's
58:01
doing great work I want some of that and
58:03
creating that pain for them like because
58:05
it's it's the pain of loss like is the
58:07
number one way to grow headcount that's
58:09
how I always approached getting more
58:11
headcount was not arguing abstractly for
58:13
why research is important but by asking
58:15
Partners who wish they had it to do the
58:18
arguing for me and so there isn't a
58:21
you're right there isn't a clean answer
58:22
for like hey this is how this is the
58:24
right ratio because it depends on the
58:25
nature of the product like is it a is it
58:27
a early stage product is it a late stage
58:29
product are we talking about a startup
58:32
or a late Stage Company but you know I
58:34
would argue there's always room for a
58:36
researcher Lenny I'll tell you and I use
58:38
this in a keynote talk I gave lately you
58:40
published recently uh a list of I think
58:44
it was about 20 B2B companies and their
58:48
first 10
58:49
employees do you remember doing that
58:52
absolutely do you remember how many
58:53
researchers are in anywhere on that list
58:55
list I'll give you a hint not too it's
58:58
between zero and two it's one there is
59:01
one researcher on that list anywhere
59:04
anywhere and that's messed up to me now
59:06
Look it's just these 20 companies and
59:09
like they're you know each is in their
59:10
own space so I'm not going to
59:12
overgeneralize but there is an in a
59:15
researcher can drive incredible value no
59:17
matter what stage a company is at
59:19
because a good researcher makes you go
59:21
faster not slower and they drive impact
59:24
because they answer questions which are
59:26
impossible to answer in any other way
59:28
that's true if you're a startup it's
59:30
true if you're a late Stage Company now
59:31
if it's your first 10 employees you know
59:33
like one one researcher is going to go a
59:35
long way you know as you grow making
59:38
sure that you're matching up researchers
59:40
so that they have strong Partners in the
59:41
key parts of the business is the best
59:42
way to figure out if you have enough
59:45
interesting so your advice is as you're
59:47
starting a company your pitch is that
59:50
you will have a lot more leverage and
59:52
move faster hiring researcher versus
59:54
generally engineer is what you'd be
59:56
trading off essentially that's what most
59:57
of the hir is end up being I'm reluctant
1:00:00
to overgeneralize but I would say um and
1:00:03
and I would say I know in when many com
1:00:05
many Founders who are in startup mode
1:00:07
are like I know what I need to build the
1:00:09
problem is that I need people who can
1:00:11
help me execute and I think that's right
1:00:14
and so everything's a trade-off but
1:00:16
remember imagine that you could have
1:00:18
that Swiss army knife at your disposal
1:00:20
as you maybe maybe you've got um like an
1:00:22
MVP out the door and you're you're
1:00:25
looking to make your first major
1:00:27
iteration or like many startups you need
1:00:29
to Pivot like this is where uh like it's
1:00:31
like hey you don't have to do that alone
1:00:33
like we we we deify startup Founders who
1:00:36
pivot appropriately but I think that is
1:00:38
what we would might call moral luck
1:00:39
where like we we deify the ones who got
1:00:41
it right and and even though they made
1:00:43
exactly the same decisions as the one
1:00:45
who got them wrong and the the fact the
1:00:47
matter is like if you have an insights
1:00:49
person with you but has that Swiss army
1:00:50
knife of tools like you're not in it
1:00:53
alone you don't have to guess Like It
1:00:55
ultimately it will still come down to a
1:00:57
tough decision that you and Founders
1:00:58
have to make but you can have evidence
1:01:01
to that that that bears on that decision
1:01:03
which you wouldn't be able to get any
1:01:04
other
1:01:05
way to close out on this and have just a
1:01:07
couple more questions on this thread I
1:01:10
think one of your big messages to
1:01:12
researchers is you can be empowered like
1:01:15
it's up to you to do the right sort of
1:01:18
research and to move the your career in
1:01:21
the right direction not become a
1:01:22
researcher people in need and there's
1:01:23
this quote that you have at the top of
1:01:25
your post where a lot of the reaction or
1:01:27
I guess the way you put it is I know
1:01:29
what you're thinking they just don't get
1:01:30
it we're so misunderstood our plight is
1:01:33
to deliver insights that users used to
1:01:34
drive business value while we're
1:01:35
forgotten never driving the road map no
1:01:38
seat at the table consistently miscast
1:01:40
only to be laid off in the end and what
1:01:42
I'm hearing from you is like you can
1:01:44
change that like you can push back on
1:01:46
doing research that isn't actually
1:01:48
contributing but let me ask you what's
1:01:50
what's your kind of leave lasting I
1:01:52
don't know advice you would leave
1:01:53
researchers with to
1:01:55
yeah to be successful yeah it's tough to
1:01:57
be operating in a broken system and so I
1:02:00
feel that response where you're like you
1:02:01
feel kind of powerless but I think
1:02:04
that's not likely to lead us past this
1:02:06
moment to the next evolution of research
1:02:08
so that's kind of where it's like I
1:02:10
don't blame any researcher at all for
1:02:13
being in the spot they're in it's a t
1:02:15
it's been a tough go right however
1:02:18
crying about our lot you know is like
1:02:20
not going to get us anywhere so like I I
1:02:22
think the point of the the the article
1:02:24
for me and and this is advice I give
1:02:26
companies all the time when I do
1:02:27
Consulting with them it's like hey we
1:02:30
can set this up in a different way which
1:02:32
responds to the current environment in a
1:02:35
way which will drive a huge amount of
1:02:36
impact now that takes companies making
1:02:38
the right choices it also takes
1:02:40
researchers owning up and developing
1:02:42
skills right pushing back understanding
1:02:45
what research can have the most value
1:02:47
developing the skills and the knowledge
1:02:49
and and language around the business
1:02:52
like becoming more influential being
1:02:53
excellent communic ators like it's one
1:02:56
of the things I would evaluate the most
1:02:57
in hiring especially research leaders
1:02:59
because I needed them to show and teach
1:03:01
by example is like can you if like it
1:03:04
isn't just rigorous research it's like a
1:03:07
if a tree fell in the forest and no one
1:03:08
was there to hear it you know you need
1:03:10
to communicate it effectively and you
1:03:11
need to do it in a way that's
1:03:12
appropriate to the audience because if
1:03:14
I'm I'm if I'm talking to you Lenny it's
1:03:15
different than I'm talking to Brian
1:03:17
chesky you know at at Airbnb and so I
1:03:19
got to be able to give that presentation
1:03:21
effectively and get right to the heart
1:03:23
of it and speak the right language and
1:03:25
so if you're a researcher it's not
1:03:27
hopeless actually the the discipline the
1:03:29
future is so bright and we can help it
1:03:32
along by continuing to develop these
1:03:34
different skills as companies build a a
1:03:37
model that's more
1:03:38
inclusive awesome okay I have one just
1:03:41
random tangential question about NPS you
1:03:45
have strong opinions about NPS and I
1:03:47
just wanted to hear your perspective on
1:03:49
the value of NPS your experience with
1:03:50
NPS yeah I do have a strong opinion
1:03:53
about NPS um I I like to say NPS is the
1:03:57
best example of the marketing industry
1:03:59
marketing itself and I think I'm going
1:04:00
to a the problem is this threatens many
1:04:02
people's livelihoods because there's an
1:04:04
entire industry of consultants and
1:04:06
software providers that want you to
1:04:08
believe NPS is a useful and accurate
1:04:11
metric the problem is so the consensus
1:04:14
in the survey science Community is that
1:04:16
NPS makes all the mistakes so it it's
1:04:19
it's a garbage in garbage out problem so
1:04:21
the likelihood to recommend question is
1:04:24
bad for whole variety of reasons so it's
1:04:26
it's bad because it's a 0 to1 scale it's
1:04:29
bad because it's usually unlabeled so we
1:04:32
label the polls but the that's not the
1:04:34
gold standard for research it's bad
1:04:36
because it's 11 items right and there's
1:04:38
a couple problems with that number one
1:04:40
we find that Precision goes down after
1:04:42
five items on average maybe seven number
1:04:45
two especially on mobile if you're
1:04:47
taking this survey what percentage of
1:04:49
those options are below the fold right
1:04:51
we are not going to get accurate survey
1:04:53
data right and so from a survey
1:04:55
perspective it's really bad there's also
1:04:57
this intuition which is like are you a
1:04:59
person who will let Rec you know like
1:05:01
are you are you like How likely are you
1:05:02
to recommend Windows 11 to your friends
1:05:04
and family and not a person who goes
1:05:07
around recommending operating systems
1:05:09
the question is fundamentally flawed the
1:05:11
argument is that's the that that
1:05:13
question is a good indicator of loyalty
1:05:15
but there's a really simple solution
1:05:17
Lenny customer satisfaction a simple
1:05:20
seat metric is better it has better data
1:05:24
properties it is more precise it is more
1:05:26
correlated to business outcomes I wanted
1:05:29
to prove this this is something that
1:05:30
survey scientists know and marketers
1:05:32
don't want you to know and so I did we
1:05:34
did the work with Mike morami who who
1:05:36
led survey science at at Airbnb and he's
1:05:39
still there great researcher and we
1:05:41
basically redid All that work to find
1:05:43
out if all that stuff was true just for
1:05:44
Airbnb and it is it's simple don't ask
1:05:47
NPS ask customer
1:05:50
satisfaction and the cut customer
1:05:52
satisfaction question what's the actual
1:05:54
question for people overall how
1:05:55
satisfied are you with your experience
1:05:57
with Airbnb or it could be some version
1:05:59
of that which is like overall how
1:06:01
satisfied are you with your experience
1:06:02
with customer service uh when you had a
1:06:05
problem you know so there could be a
1:06:06
more specific version of that question
1:06:08
but those questions have better
1:06:10
properties and you know a lot of people
1:06:11
say well hey everybody's using NPS right
1:06:14
so at least it gives me a benchmark
1:06:15
because I can compare my NPS to Industry
1:06:17
NPS the problem with that is the
1:06:20
research shows that NPS is
1:06:22
idiosyncratic so it it goes up and downs
1:06:25
in in ways that we don't understand and
1:06:27
there's a lot of inconsistency in how
1:06:28
it's asked that creates variations in
1:06:31
the data which means it's not apples to
1:06:32
Apple so you can't even compare your MPS
1:06:35
meaningfully to somebody else's I love
1:06:38
these hot takes I'm curious to see who
1:06:40
comes out of the woodwork to people are
1:06:41
gonna be so mad Lenny love that I think
1:06:44
yeah people like I've heard this many
1:06:46
times and people don't talk about it
1:06:48
okay is there anything else you want to
1:06:49
share or leave people with before we get
1:06:52
to our very exciting lightning round
1:06:55
can I yeah I want to I want to add one
1:06:56
thing if I could because this has come
1:06:58
up on your podcast a few times recently
1:07:01
which is about the idea of people doing
1:07:04
their own product walkthroughs like so
1:07:07
should a PM just rely on their own um
1:07:11
dog fooding of the product and their own
1:07:14
walkthr to figure out how to fix it um
1:07:17
and a couple of times recently this has
1:07:19
come up like this and and I think the
1:07:20
consensus seems to be sort of yes this
1:07:23
is a good thing I have a contrarian
1:07:25
opinion there too which is that I I
1:07:28
think it is really important for
1:07:29
everyone to dog food their own products
1:07:31
the problem is related to relying on
1:07:34
your intuition about those products
1:07:35
which is the thing most PMS um have
1:07:39
trouble with is realizing you are
1:07:41
nothing like the user you are nothing Li
1:07:44
them in ways that will bias the way you
1:07:46
think about what's good and bad in your
1:07:48
product in ways that you can't
1:07:49
necessarily recognize like some things
1:07:52
with a product some problems with a the
1:07:54
product you need a pulse to recognize
1:07:57
and like most good PMS that I know have
1:08:00
a pulse and so cool but a lot of them
1:08:04
require like context of use priorities
1:08:07
constraints that you just don't have and
1:08:10
you can't
1:08:12
imagine purely on the basis of your own
1:08:14
usage so what I think that means is that
1:08:17
you should definitely dog food your own
1:08:18
product doing product walkthroughs to
1:08:21
identify lists of potential issues is a
1:08:23
great thing to do
1:08:25
prioritizing that list figuring out
1:08:28
which ones are more or less a problem
1:08:31
and for whom is an area where you should
1:08:33
be extremely wary of relying on your own
1:08:36
sort of opinion expertise or intuition
1:08:38
when you are dog fooding your own
1:08:39
product thank you for sharing that it's
1:08:41
definitely come up a bunch on this
1:08:42
podcast so I think that's an important
1:08:44
lesson for people to take away anything
1:08:47
else before we get to our very exciting
1:08:48
lightning room I appreciate you Lenny
1:08:50
thanks for having me on I appreciate you
1:08:52
Jed well with that we've reached our
1:08:53
very excting in lightning round are you
1:08:55
ready I am ready what are two or three
1:08:59
books that you've recommended most to
1:09:01
other people I recently read a book by
1:09:04
uh a business book by Barbara Kellerman
1:09:06
called bad leadership and what I love
1:09:09
about it is that we spend a lot of time
1:09:11
talking about good leaders and she
1:09:12
really dives into like the worst leaders
1:09:14
and what makes them bad leaders in a way
1:09:17
that I think is really valuable for
1:09:18
everybody I'd also recommend I read a
1:09:21
lot of fiction so two recommendations
1:09:23
there one U recent poster Prize winner
1:09:25
demon uh Copperhead by a barb king
1:09:27
solver it's like an outstanding read
1:09:29
that also um is like really sad and
1:09:32
moving and illustrative especially when
1:09:34
you if you want to understand rural
1:09:35
poverty and then completely other side
1:09:38
of the fiction Spectrum if you if you're
1:09:39
interested in science fiction which I am
1:09:42
uh read the murderbot Diaries it's like
1:09:44
about a sarcastic robot uh a sarcastic
1:09:46
killer robot and who doesn't love them I
1:09:49
love these fiction recommendations I
1:09:50
feel like we need more of these on the
1:09:51
podcast so thank yeah everybody goes to
1:09:53
business books yeah absolutely uh what
1:09:56
is a favorite recent movie or TV show
1:09:57
that you really enjoyed uh we recently
1:10:00
watched the last of us and it blew our
1:10:02
mind I watched it after I played the
1:10:04
video game after long last if you are a
1:10:06
person who plays video games and you
1:10:07
haven't played The Last of Us play it if
1:10:10
you don't know the the show is based on
1:10:12
the video game not the other way around
1:10:14
do you have a favorite interview
1:10:16
question you like to ask candidates that
1:10:18
you're interviewing think of a topic
1:10:20
that you had to explain lately that was
1:10:23
the most complex and then explain it to
1:10:25
me like I'm
1:10:26
five um and there are a lot of ways to
1:10:28
vary that question but the reason I like
1:10:31
it is because I think and I've asked
1:10:33
this question to you know VP and seite
1:10:36
candidates in like multiple disciplines
1:10:39
and sometimes it's related to a
1:10:41
conversation like I might ask them to
1:10:43
explain something complicated about
1:10:45
Quantum Computing or music theory or it
1:10:47
could be a complex business decision but
1:10:49
I want to see if somebody can break a
1:10:52
complex problem down in a really simple
1:10:55
way and give me an intuition for it in a
1:10:57
short amount of time I think that is a
1:10:59
differentiator between good and great
1:11:01
for many people do you have a favorite
1:11:03
product you recently discovered that you
1:11:05
really like yeah this is a really weird
1:11:07
one but my whole family started uh
1:11:09
indoor rock climbing recently and
1:11:12
there's a challenge you have when you
1:11:13
top rope which is that you're looking up
1:11:15
all the time so they make these glasses
1:11:19
which are called Bay glasses and they
1:11:21
have an angled mirror embedded in the
1:11:22
lens so that you can look straight ahead
1:11:26
and the view you see is up towards the
1:11:28
person um who you're belaying and I just
1:11:30
thought that product is like so perfect
1:11:32
for that like that that's a niche
1:11:35
problem and there isn't a better way to
1:11:37
solve it do you have a favorite motto
1:11:39
that you often come back to that you
1:11:41
share with friends either in work or in
1:11:43
life that you find useful yeah so this
1:11:45
is going to seem like pandering money
1:11:47
but um I don't know if you remember a
1:11:50
conversation that you and I had it must
1:11:51
have been eight years ago I remember
1:11:53
where we were sitting
1:11:54
and it was about
1:11:56
stoicism do you remember this anyway we
1:11:58
had this I don't but I was into stoicism
1:12:01
for a while I know you were because we
1:12:03
talked about it and so the the motto is
1:12:05
comes from stoicism which is basically
1:12:09
focus on the things you can control and
1:12:11
ignore the rest and you know a lot of
1:12:13
people think of this you know a lot of
1:12:14
people think of this as the the Serenity
1:12:16
Prayer or the Serenity saying that that
1:12:18
was a 20th century invention but epicus
1:12:21
was writing about this you know BC and
1:12:24
um I think about it all the time so much
1:12:26
of the stress and pain and uh worry that
1:12:29
we have in life comes from things we
1:12:31
can't control so I try to let those
1:12:33
things go amazing I I learned that
1:12:35
lesson from Seven Habits of Highly
1:12:37
Effective People and just the importance
1:12:40
of thinking about these Circles of you
1:12:42
can control you can influence and you
1:12:43
have no control over and there's no
1:12:45
reason to think about those other things
1:12:47
absolutely Jud this was everything I
1:12:49
hoped it would be we got into some
1:12:50
really good stuff I'm excited to hear
1:12:52
how people react two final question
1:12:53
questions where can folks find you if
1:12:55
they want to learn about what you're up
1:12:56
to actually share what you're up to
1:12:57
these days and how people can find you
1:12:59
uh and then also how can listeners be
1:13:01
useful to you yeah thanks for asking
1:13:03
those questions so uh people can find me
1:13:05
at uh Jud anon.com that's the best way
1:13:07
to find out what I'm up to uh these days
1:13:09
I'm a consultant um I help people with
1:13:11
ux strategy uh org design and crisis
1:13:14
management somehow I love dealing with
1:13:16
other people's dumpster fires and I
1:13:18
found that I'm constitutionally good at
1:13:20
it somehow so um Jud anon.com is the
1:13:23
place to find I also write I write a um
1:13:26
medium post that you can find at onebig
1:13:28
thought.com um and you'll find a lot of
1:13:31
the topics we talked about today
1:13:32
including the original post that started
1:13:34
this at onebig thought.com um if there's
1:13:38
one thing I could ask uh your listeners
1:13:41
to do is to get next to your researcher
1:13:44
you know like I just I just think if you
1:13:46
build those relationships and involve a
1:13:47
researcher an insights person early and
1:13:49
often beautiful things will happen for
1:13:52
you and for the business so that's my
1:13:54
that's the thing everyone can do for me
1:13:56
I love that I've always done that I
1:13:59
loved my researchers that I worked with
1:14:00
many of them reporting to you and so
1:14:03
beautiful takeaway Jud thank you so much
1:14:06
for being here Lenny thank you it's been
1:14:08
a pleasure bye
1:14:10
everyone thank you so much for listening
1:14:12
if you found this valuable you can
1:14:14
subscribe to the show on Apple podcast
1:14:16
Spotify or your favorite podcast app
1:14:19
also please consider giving us a rating
1:14:21
or leaving a review as that really helps
1:14:23
other listeners find the podcast you can
1:14:25
find all past episodes or learn more
1:14:27
about the show at Lenny podcast.com see
1:14:31
you in the next
1:14:33
episode
Start a conversation
Ask anything about The UX Research Reckoning is Here
100/100 free messages remaining
AI responses may be inaccurate. Verify important information.
The UX Research Reckoning is Here
Lenny's Podcast