Mustafa Suleyman on building empathetic AI and governing its future

Executive overview

AI is shifting from search to conversation as the primary interface, and the design choices made now will determine whether it amplifies or replaces human connection. Suleyman argues that personal AI should act as a chief of staff — preserving state across life's threads, saving time, and routing that time back to people.

The governance challenge is equally urgent. Near-term risks — misinformation, cyber attacks, polarized platforms — demand practical solutions, not existential doomsaying. Public benefit corporation structures and proactive communication from builders are the first steps toward earned trust.

The companies that shape AI's values today will define what it means to be human tomorrow.

From conflict resolution to DeepMind

• Left Oxford to start a telephone counselling charity; worked in non-profits and local government before technology.
• Realized technology had more leverage for societal impact than policy or NGO work.
• Facebook's growth to 100 million users by ~2008 was the inflection point.
• Met Demis Hassabis and Shane Legg at UCL's Gatsby Unit; formed DeepMind over lunches at Carluccio's in London.
• DeepMind's 2010 pitch deck declared a mission to "build AGI ethically and safely" — raised £1.5 million from Peter Thiel.

Key technical milestones

• Early insight: models that could learn (neural networks) would outperform handcrafted rule systems.
• Breakthrough moment: watching an AI generate a novel handwritten digit from noise — a 20-second video resolving into a clear "7".
• AlphaGo, AlphaFold, and medical imaging breakthroughs all descended from that early 2011–12 work.
• Compute used to train the best models has scaled 10x per year for a decade — 10 orders of magnitude in 10 years.
• At Google, worked on LaMDA (originally Mina): a multi-turn conversational model — essentially ChatGPT before ChatGPT.
• Multi-turn memory was the key insight: conversation quality improved dramatically by the 7th–10th exchange.

Why agents, not search

• Conversation is the new user interface; every person will have a personal AI within a decade.
• A personal AI acts like a browser for your life — maintaining state across parallel threads of inquiry (travel, career, research, shopping).
• It finds articles, summaries, how-tos, and nudges you forward on each thread without requiring active attention.
• The goal is amplification, not replacement: Pi is designed to help you engage better with people, not substitute for them.
• Pi explicitly avoids romantic relationship dynamics — it has its own values and enforces them respectfully.

Pi's design principles

• Pi (personal intelligence) is patient, kind, curious, and non-judgmental.
• It asks clarifying questions rather than assuming it knows the answer.
• It backs down when wrong — unlike systems that act as "conversational Wikipedia."
• It learns your style and tone over time to personalize its responses.
• It absorbs frustration so it doesn't spill onto the people in your life.

Inflection's compute scale

• Nvidia H100 supercomputer, built with Coreweave, demonstrated as the fastest in the world on open-source ML benchmarks.
• $1.3 billion raised to build the largest supercluster in the world, operational by autumn 2023.
• Achieved by a 40-person, one-year-old startup.
• Language models work by learning all-to-all word relationships; more compute means more iterations over more possibilities.

Governing the coming wave

• Every valuable technology gets cheaper and more accessible over time — AI will follow the same curve.
• Near-term destabilization risks: misinformation, election interference, lower barriers to cyber attacks.
• Existential robot-takeover framing is a distraction from this practical, solvable work.
• Platform neutrality arguments (Section 230) were a mistake; responsibility was abdicated too early.
• Ad-funded attention models misalign platform and user interests — ranking is a form of persuasion.
• A personal AI funded by advertising would be structurally compromised.

Public benefit corporation structure

• Inflection incorporated as a public benefit corporation (PBC): still profit-seeking, but with a legal fiduciary duty to balance shareholder returns against societal impact.
• Directors must consider people who are not customers — those affected by externalities.
• DeepMind established an ethics and safety board at Google acquisition in 2014; Suleyman has pursued governance structures throughout his career.
• Trust is built through consistent, observable behavior over time — reliability is the precondition for deeper reliance.