Career growth, experimentation, and selling to developers with Laura Schaffer

Executive overview

Most people grow their careers by performing well within their role and hoping their manager advocates for them. This caps your trajectory at what one person can see and do for you. A more powerful approach: become the person who surfaces customer insights that executives can't access themselves.

The career accelerant is staying close to customers and sharing what you learn — this builds cross-company trust that opens doors no manager can.

Experimentation is harder than it looks. Around 80% of hypotheses fail even at companies like Netflix and Microsoft. The response isn't to run fewer experiments — it's to validate faster, embrace lower confidence intervals strategically, and measure growth teams over longer timeframes than monthly metrics allow.

When selling to developers, standard growth playbooks break. Developers skip your marketing site, avoid sales, and will not commit until they've proven your product works themselves. Self-serve has to do the selling.

Career growth framework

• Most career growth advice puts all power in one person: your manager. Their ability to advocate, promote, and notice you is the ceiling.
• A stronger move: become the person at your company who knows customers best and shares those insights broadly.
• Executives lose touch with customers as companies scale — the person closest to customer pain holds leverage executives actually want.
• Build a "voice of the customer" digest and circulate it. People will ask to be added. Senior leaders will attend.
• This builds brand recognition across the company, not just within your reporting line.
• It creates "pre-meetings" — by the time you pitch a new idea, trust is already there.
• Customer knowledge is valuable to every role: product, marketing, sales, engineering. It works from any position.

How Laura built the Twilio growth team

• Joined Twilio in product marketing in 2014 — no growth team existed.
• Started sharing a written voice-of-the-customer report shortly after joining.
• Within months, senior leaders were asking to be on the distribution list. The CEO started attending quarterly sessions.
• Discovered a gap: Twilio had high conviction that developers found it easy, but customers were saying it was difficult as product complexity grew.
• Used that credibility to pitch a growth team in annual planning, with a trusted early employee co-sponsoring the idea.
• The pitch was approved with minimal friction because the "meeting before the meeting" had already happened.

Onboarding and user psychology

• Adding four qualification questions to Twilio's signup flow (language, use case, product) increased conversion by 5% — despite adding friction.
• Why it worked: new users arrive anxious. They're scanning for evidence the product will work for them. Questions that confirm "yes, we support your language" are reassuring, not annoying.
• Good friction addresses user anxiety. Bad friction creates it. The distinction matters more than the quantity of friction.
• Later experiment: a structured onboarding checklist with "Get a phone number" as step one hurt conversion because telecom is the bogeyman for developers — unfamiliar and intimidating.
• Fix: kick users out of the console and into docs, burying the phone number step amid familiar code samples. Conversion improved.
• The "pill in the hot dog" principle: embed the scary, unfamiliar thing inside something comfortable and familiar.
• Always ask: what is the user's psychological state at this exact point in the journey? Logic alone doesn't predict conversion.

Running experiments well

• Roughly 80–90% of experiment hypotheses fail at mature companies. This is empirically documented across Netflix, Microsoft, and others.
• Failing is not a wall — it's a compass. The goal is to fail fast and cheaply, not avoid failure.
• A/B tests are among the most expensive validation methods — they require design, engineering, PM time, and runtime.
• Before an A/B test: use painted door tests, mock-ups, or qualitative interviews to invalidate weak hypotheses first.
• Only the ideas that survive cheaper validation should graduate to full A/B testing.
• "If it's not embarrassing, you've gone too far" — the first iteration should be intentionally rough.

On confidence intervals and risk

• The 95% confidence interval comes from academic research where false positives carry serious consequences (pharmaceuticals, education research).
• For product growth teams, the risk calculus is different. A false positive from a conversion experiment rarely causes serious harm.
• Using lower confidence intervals can double or triple experiment velocity across a year, and net more genuine wins.
• This must be decided before running the experiment — not retroactively to salvage a failing result.
• When accepting more statistical risk, increase qualitative validation: look for corroborating signals from user research and session recordings.
• Teams under pressure to show monthly wins are prone to massaging data to fit. This is the real danger, not the p-value.

Growth team expectations and timelines

• Growth teams beholden to weekly or monthly wins will default to vanity metrics and data-massaging.
• The right unit of measurement for a growth team is a year, not a sprint.
• Commit to a portfolio: low/medium/high bets. Make the range of outcomes explicit, not a single number.
• Some wins are lumpy — one initiative at Twilio generated tens of millions in pipeline but took significant time to validate.
• Educating stakeholders on the 80% fail rate using data is a legitimate and important part of the growth lead's job.

Selling to developers

• Developers skip the marketing website. They go directly to signup, context-free. Never assume they've read your positioning.
• They are deeply averse to sales — not just casually. One Twilio customer ran a POC, went to production, and operated for months without engaging sales. A large retailer used personal email addresses to avoid sales contact.
• Why: developers are responsible for the outcome. If the product fails, it falls on them — their reputation, their team's trust, potentially their job. They cannot take someone else's word for it.
• To commit, developers must prove the product works themselves. That requires building something.
• This means self-serve must function as your sales channel for developer audiences.
• Invest in self-serve experiences at a level comparable to your sales investment. The self-serve motion is doing the same job.
• For the non-developer buyer (the person with budget but no coding ability), self-serve still matters: give them a way to experience value without writing code (e.g., Twilio's quick-deploy demo experience).

Moving from sales-led to product-led growth

• A common mistake: take your enterprise product, cut some features, set a price, and call it PLG.
• This skips the essential step: understanding what problems self-serve users actually have, which differ from enterprise buyers.
• PLG is sales via product. Apply the same discipline a good sales rep uses — understand the specific problem of the person in front of you.
• For Amplitude's SMB and startup audience: most don't have dedicated analysts. The CEO builds board dashboards. PMs run their own reports.
• The real product problem to solve: help these users feel competent and informed, not just feature-complete.
• Start with the customer problem, not the product. The gaps will be different than what you expect.

Experimentation strategy and idea sources

• The best growth ideas often come from qualitative signal — user interviews, support tickets, sales calls — supplemented by quantitative data.
• Iterative experimentation consistently outperforms big-bang redesigns. Redesigns are almost always net negative.
• Quantitative data tells you what changed. Qualitative data tells you why. Both are needed to make reliable decisions.
• Tools like Hotjar, session recordings, and quick surveys are underused as pre-validation methods.
• Connecting tools (e.g., Amplitude event → Segment identifier → Hotjar session recording) gives specific, actionable hypotheses without needing to engage customers directly.