04:29How one A/B test helped Bing increase revenue by 12%
09:00What data says about opening new tabs
10:34Small effort, huge gains vs. incremental improvements
13:16Typical fail rates
15:28UI resources
16:53Institutional learning and the importance of documentation and sharing results
20:44Testing incrementally and acting on high-risk, high-reward ideas
22:38A failed experiment at Bing on integration with social apps
24:47When not to A/B test something
27:59Overall evaluation criterion (OEC)
32:41Long-term experimentation vs. models
36:29The problem with redesigns
39:31How Ronny implemented testing at Microsoft
42:54The stats on redesigns
45:38Testing at Airbnb
48:06Covid’s impact and why testing is more important during times of upheaval
50:06Ronny’s book, Trustworthy Online Controlled Experiments: A Practical Guide to A/
51:45The importance of trust
55:25Sample ratio mismatch and other signs your experiment is flawed
1:00:44Twyman’s law
1:02:14P-value
1:06:27Getting started running experiments
1:07:43How to shift the culture in an org to push for more testing
1:10:18Building platforms
1:12:25How to improve speed when running experiments
1:14:09Lightning round
07-23The 10 traits of great PMs, how AI will impact your product, and Slack’s product development process | Noah Weiss (Slack, Foursquare, Google)86 分钟原站 ↗
章节 · 22 段展开章节收起章节
00:00Noah’s background
04:22Noah’s advice on new parenthood
07:23Lessons learned from leading product at Foursquare
11:33Advice for working with strongly opinionated founders
14:14Thinking of involvement on a U-shaped curve
16:53Principles at Slack
19:32Implementing ML, AI, and LLMs in meaningful ways
25:11How Slack structures AI teams
26:59Complaint storms and how they help foster empathy
30:01Slack’s approach to prioritization
32:26How delight is baked into the DNA of Slack
34:41How Slack thinks about competition
38:04Building a culture that takes big bets
41:40Rituals at Slack
44:51How Slack unlocked new levers of growth and revived their self-serve business
52:01Slack’s early success and the factors that made them successful
58:08Slack’s pilot programs for testing new features
1:02:03Noah’s famous blog post: “The 10 Traits of Great Product Managers”
1:10:15Book recommendations to improve your writing
1:12:30Managing up and the importance of data fluency
1:14:54The most important skills to improve as an early-career PM and as a senior PM
1:17:16Lightning round
07-16LinkedIn’s product evolution and the art of building complex systems | Hari Srinivasan (LinkedIn)65 分钟原站 ↗
章节 · 23 段展开章节收起章节
00:00Hari’s background
05:04How Twitter brought Lenny and Hari together
06:32LinkedIn’s positive evolution, and what they did right
10:14Specific changes that made LinkedIn’s feed more interesting
11:12Understanding the algorithm and what kinds of content perform best
12:21The talent solution product
15:46The shift to skills-first hiring, and how LinkedIn changed their approach
20:24The open-to-work signal, and the newly released open-to-internal-work signal
22:13The PM talent landscape, and tips for landing a PM role
24:55How to optimize your LinkedIn profile to get noticed by recruiters
28:38Hari’s first product review at LinkedIn
30:38LinkedIn’s North Star, and how to operationalize the North Star at any company
33:24LinkedIn’s members-first value
35:32Building and maintaining complex systems
38:09The RAPID framework and the Five-Day Alignment framework
39:51What LinkedIn looks for in new hires
40:51The latest innovations at LinkedIn
43:16LinkedIn Learning
45:00Hari’s product management course
48:19Advice for people hoping to get into product management
50:40How to level up your PM skills
51:57Hari’s creative side projects
55:02Lightning round
07-13M&A, competition, pricing, and investing | Julia Schottenstein (dbt Labs)61 分钟原站 ↗
章节 · 22 段展开章节收起章节
00:00Julia’s background
04:15How Julia went from VC to working in product at dbt Labs
08:24Four things Julia uses to evaluate a company’s potential
11:10How to identify whether or not you have product-market fit
12:05Distribution strategies
13:11M&A strategies
15:54Lessons from the Transform acquisition
18:01Competitive values at dbt
20:25Keys to dbt’s success
26:35An offsite exercise Julia used to help her team internalize upcoming changes
29:32Determining what features are included in open source
31:56Pricing and willingness to pay
33:34Lessons from dbt Labs’s first pricing change
36:33Whether or not to be public about selling your startup
40:08How to utilize connections during acquisitions
44:57How to communicate selling your company
46:33M&A market forecast
47:28Values at dbt Labs
50:14Lessons from working with strongly opinionated users
52:02The importance of shipping, learning, and iterating
54:08How VC skills translate into product
57:03Lightning round
07-09How to ask the right questions, project confidence, and win over skeptics | Paige Costello (Asana, Intercom, Intuit)63 分钟原站 ↗
章节 · 19 段展开章节收起章节
00:00Paige’s background
04:38What Paige is responsible for at Asana
06:04The evolution of Asana’s product development process
09:10Planning frequency
11:26Examples of areas and metric tracking at Asana
12:46The double-diamond process and how it’s applied at Asana
16:53Asana’s office-centric hybrid work culture and the future of WFH
21:45How to garner trust and win over skeptics
24:45Why you should befriend researchers
26:17How to exude confidence
29:03The 3Es framework
33:43Advice for early-career PMs
38:43Paige’s latest pillars strategy
40:05AI at Asana
41:50Lessons from Paige’s time at Intuit
45:53Challenges new PMs face
48:55Challenges Paige has faced in her career
52:39Paige’s skill-focused career philosophy
55:43Lightning round
07-02Building minimum lovable products, stories from WeWork and Airbnb, and thriving as a PM | Jiaona Zhang (Webflow, WeWork, Airbnb, Dropbox)68 分钟原站 ↗
章节 · 17 段展开章节收起章节
00:00JZ’s background
04:22Common mistakes new PMs make
06:44Why Airbnb Plus didn’t work out, and takeaways from that experience
10:51Executing big dreams step-by-step
13:45The right way to push back against founders
16:54Minimum lovable product vs. minimum viable product
20:53What makes a product lovable
22:20Advice on roadmapping and prioritization
28:04Tips for new PMs to accelerate their career
29:16JZ’s top skills and how they have evolved over her career
31:37Designing crisp OKRs
36:09Lessons from WeWork
43:01Winning the first 90 days at a new company
48:34Why trust is crucial
51:48High-level lessons from Dropbox, Airbnb, WeWork, and Webflow
56:38The one piece of advice that transformed JZ’s career