AI-Augmented Workers Outperform Solo Workers on Every Metric

Executive overview

A BCG experiment — the largest pre-registered study on AI in professional work — found consultants using GPT-4 outperformed non-AI peers on every measured dimension, with a 40% quality increase. The core insight is that AI plus human beats AI alone or human alone. Pure AI automation produces frequent errors requiring costly correction; human-in-the-loop workflows avoid this. Short-term, people overestimate AI's autonomous impact; long-term, its capabilities are vastly underestimated.

BCG case study findings

• Consultants with GPT-4 access beat non-AI peers on every performance dimension
• Quality scores were 40% higher — less than a year after ChatGPT launched
• The study was the largest pre-registered experiment on the future of professional work
• AI without human oversight produced high error rates in the majority of cases

The short-run vs long-run framing

• Short run: AI cannot fully automate workflows — requires heavy human intervention
• ChatGPT inaccuracy rate cited at roughly 20%, making autonomous reliance risky
• Long run: capabilities are underestimated — potential as transformative as personal robotics
• Mirrors the Bill Gates principle: overestimate one year, underestimate one decade

Three types of AI in use today

• Artificial intelligence — true AI; most current tools don't qualify
• Augmented intelligence — AI enhancing human decision-making (most current use cases)
• Automated intelligence — rules-based automation layered with AI

Practical use cases worth prioritising

• Use AI to analyse ad spend data and flag waste — don't let it make changes autonomously
• Google alone sees ~$160B/year in ad spend; savings from AI flagging dwarf blog-writing savings
• Programmatic SEO at scale: topical mapping + keyword research + GPT-4 templated pages + human spot-check loop
• Programmatic SEO works best for high-domain-authority sites with high-permutation content (e.g. travel, directories)