How 12 Labs built AI video understanding at scale

Executive overview

Video represents 80% of the world's data, yet developers lack tools to search, classify, and analyze it effectively. 12 Labs built foundation models that map human language to video content, enabling semantic search and classification via APIs. The founders—drawn together during military service—stayed focused on infrastructure rather than becoming an application company, a discipline that attracted major investors like Index Ventures, Radical Ventures, and NVIDIA.

Core insight: Staying grounded in your original thesis beats chasing every piece of mentor advice.

Why video foundation models are infrastructure, not apps

The team initially considered building TikTok 2.0 or sales call analysis tools. Mentors pushed these application-layer ideas. Rejection of those directions proved critical—12 Labs' strength is building the infrastructure layer, letting developers build applications on top. 20,000+ developers now use the search API; customers include major creators, media companies, and law enforcement.

Founding during military service and early scrappiness

Three co-founders met in Korean Cyber Command. Discharge dates staggered across three years, so they operated from a bagel shop outside the military base for months, later a cramped office the size of a dressing room. Bootstrapped with $2,000, they competed in ICCB (International Conference on Computer Vision) to gain exposure. Winning the competition—which required betting $200,000 in compute—led to inbound interest from Index Ventures and Radical Ventures.

Choosing the right customers early

Early mistake: aggressively closing a customer who wasn't actually using the product. Learning: optimize for saying no, not yes. True innovators—whether in content creation, law enforcement, or e-learning—don't need persuasion; they recognize the opportunity instantly. Demo use cases with indexed video proved more effective than ROI projections. Top customers now pitch 4-5 applications they want to build simultaneously.

Raising capital and partnership momentum

First pitch deck built at 3:30am Seoul time before meeting Index Ventures. Problem statement was simple: 80% of data is video; no adequate tools exist. VCs asked hard questions—e.g., indexing a billion hours of content takes 10 years at current speed. This exposed massive technical scaling challenges. Company raised ~$30 million in seed funding. NVIDIA partnership emerged naturally after 12 Labs was featured at GTC 2023; NVIDIA sees synergy between their edge chips and 12 Labs' video foundation models for smart cities and real-time streaming.

Founder discipline: saying no to mentors

Mentors offered conflicting advice about product direction. Founder Jay Lee learned the cost of trying to accommodate all of it: your company becomes something different than intended. Key lesson: have foundational conviction about your company's role and politely decline advice that pulls you away. This discipline to say no—even to respected mentors—drove significant growth and clarity.

Hiring for perspective and core values

12 Labs is now focused on hiring. Best engineers apply core values and perspective to problems, not just coding skill. Jay meets candidates for hours and evaluates whether he can learn from their values. Strong candidates articulate how the product should evolve and their vision for impact.

Two-year vision: horizontal infrastructure

12 Labs aims to become the visual cortex for all video applications. Near-term focus: horizontal video understanding infrastructure for businesses and developers working with video. Entry into streaming (real-time video data) and expanding beyond batch processing to live analysis.