How Exa is building semantic search for the AI agent era

Executive overview

Traditional search engines match keywords — they miss documents that don't use the exact terms you searched for. Exa uses transformer models to understand meaning, returning results that match intent rather than text.

The founding insight: GPT-3's deep language understanding could be applied to search. Agents everywhere need search; Exa is built to serve them.

If you train a transformer on enough web data, search can achieve the same quality gains that LLMs did from scaling.

Origin and founding insight

• Founder was writing a history book and found Google inadequate for deep research
• GPT-3 emerged at the same time, showing that transformers could understand meaning at depth
• Question became: what if a search engine understood queries the way GPT-3 does?
• First 18 months: pure research into applying transformer models to search — no prior blueprint

How Exa differs from traditional search

• Traditional engines use keyword matching — results must contain the literal search terms
• Exa understands document meaning — a rocket company in SF matches "futuristic hardware startup in Bay Area"
• Goal: perfect search — whatever information you want, you get exactly that
• Built as an API for AI applications that need external knowledge at query time

Finding product-market fit

• Launched November 2022; ChatGPT followed weeks later
• Inbound API requests started arriving from developers building AI apps
• Initially turned them down — didn't have an API product
• Repeated demand from multiple companies revealed the real market: AI agents needing search
• Lesson: listen to what customers keep asking for, not just what you planned to sell

Planning in a fast-moving AI market

• One-year plans make sense; three-year plans are hard; five-year plans are impossible
• Think from first principles about what the market will still need in a year
• Short-term market needs can be made obsolete by next month's model release
• Mental model: picture how good AI will be in a year, then build for that world

Predicting where AI goes next

• LLMs improve at any task for which training data can be generated
• Web navigation: easy to generate data, so AI will get very good at it soon
• Robotics: harder to gather physical training data, so progress will be slower
• Agents navigating the web are coming — all of them will need search