The Prototype Is the Spec

The Core Insight

In the mid-1980s Marty Cagan spent more than a year of nights and weekends building AI software at HP. It ran on a low-cost workstation and replaced a setup that cost over 100,000 dollars per user. The team won patents, localized the product, and collected great reviews. Nobody bought it.

Inspired, his 2008 book, turns that failure into a method. Cagan ran product at Netscape and then at eBay. He left convinced that the state of the art sat far ahead of the state of the practice. The method compresses to two sentences. The product manager must discover a product that is valuable, usable, and feasible. The proof is a high-fidelity prototype tested on real target users, finished before engineering builds anything.

Most companies believe they discover their products inside the release cycle, with a requirements document up front and customer feedback after launch. Cagan argues those companies do run discovery, with the full engineering team across full releases, on live customers serving as unwitting test subjects. That path takes three or more releases over one to two years before the product makes money. Startups run the same play once and die of it, which he compresses to ready, fire, aim.

The first page names the waste. A good engineering team does not matter if nobody gives it something worthwhile to build.

The Framework

Software projects have two stages. Discovery figures out what to build, which Cagan calls building the right product. Execution builds it, building the product right. The mindsets differ, and the product manager owns the switch between them.

Discovery swaps the document stack for things a user can touch. A ten-question opportunity assessment replaces the market requirements document, and no question in it touches the solution, on purpose. A high-fidelity prototype replaces the PRD, because words and pictures cannot represent software behavior and a prototype can be tested. The backend gets faked. Business logic and release requirements live on a wiki, and the prototype covers every screen and major use case. Design work runs before implementation, since designers need dozens of ideas in days and prototype code must change in hours.

The spec is ready for engineering when it passes three tests.

• Feasibility testing asks engineers whether it can be built with the technology, time, and funds available.
• Usability testing asks whether target users can figure out how to do the key tasks.
• Value testing asks whether users care about those tasks and want to buy the product.

Oversight belongs to a product council of ten or fewer executives, with four go or no-go gates from opportunity selection to launch. Estimates come in two stages: small, medium, or large at the gate, and a committed detailed estimate once the prototype exists. Cagan calls anything more precise before that point unfair to engineering.

Key Ideas

Nine of Ten Releases Fail on the Role

Industry pundits claim as many as nine of ten product releases fail to meet their objectives. Cagan traces the waste to one root cause: poor definition of the product manager role, and weak training of the people chosen for it.

Three broken patterns cover most companies. A marketing-titled owner hands market requirements straight to engineering, skipping detailed requirements and usually design. Two people split the role into a business half and a technical half, neither feels responsible, and product managers decay into a spec-generation service. One person carries product management and project management together, a model that does not scale.

The repaired role is single and full time, with exact ratios. One product manager covers five to ten engineers. One interaction designer supports two product managers, one visual designer supports four interaction designers, and projects past five engineers get a dedicated project manager. In agile terms the product manager is the product owner. Product marketing keeps positioning, messaging, pricing, and launch, and each function is one input to the other.

Discovery Cannot Be Scheduled

The four-week discovery phase collapses the same way every time. Time runs out, the engineers are ready, and you hand over whatever you have. Testing with users is worthless unless you adjust course on what you learn, and course changes refuse a calendar. His comparison is pharmaceutical research, where market discovery is easy, solution discovery is uncertain, and the industry prices that risk in.

Two mechanics keep discovery ahead of the build. Start discovery for the next release the day engineering starts the current one, so new executive requirements land there instead of churning this release. And seat a lead engineer in discovery from the first day, so relative-cost reads arrive with the ideas.

Market research does not substitute for discovery. Cagan knows of no winning product created by market research, and names Google, eBay, the iPod, and Facebook as the record. Customers know neither what is possible nor what they want, which is why focus groups fail. Winning products come from deep knowledge of user need combined with equal knowledge of what is only now possible.

Recruit Ten Charter Customers to End With Six

The charter user program opens at project start. Recruit eight to ten prospective customers from the target market, so that at least six are happy, live, and referenceable at release. They get early input, early access, and a reduced price, if any. You get ongoing dialog, access to their offices and users, prompt deployment of test versions, and a public reference once they are live.

Two rules protect the signal. Take no money in advance, and accept no more than about ten, because a bigger roster pulls the product custom. State up front that you are building a general product rather than a custom solution. And treat failed recruitment as the verdict: if nobody signs up, you are chasing a problem that does not matter enough.

Consumer products run the program with ten to fifteen users. Platform products swap the six customers for six reference applications. And when a company forbids its product managers from talking to customers, his advice is to dust off the resume.

Fix After Two Users, Stop at Six Straight

Cagan calls testing ideas on real users the single most important activity in the job, and gives it the longest chapter. Outside firms charge 10,000 to 20,000 dollars for a round of about ten users. He runs it in-house instead, at a Starbucks table when needed.

Recruiting has numbers. No-shows run as high as 30 percent and fall to 5 to 10 percent when you phone each subject the day before. A standing session every other Friday brings ten to twenty users into the office for a couple of hours each. Consumer subjects get a thank-you or 50 dollars of credit on your site.

The session grades the prototype, never the person. Tell subjects three things up front: the prototype is not real, they cannot hurt your feelings, and only the prototype can fail. When five minutes pass without the user inside the prototype, you are talking too much. Then go quiet, act like a parrot, and play back their own actions and questions without judgment. Watch what they do over what they say, and hunt for spots where the software's model fights the user's model of the problem.

The stop rules cap the spend. Never freeze the prototype for a full round of six to eight users. Fix a problem as soon as it shows, even after only two or three subjects. Six consecutive users who get through the key tasks and see the value means the spec is done. Shelving a loser here is a win, priced against the cost of building and shipping it.

Pay Engineering 20 Percent Off the Top

When engineering demands a stop-everything rewrite, Cagan puts the fault on product management. Prevention is a standing tax: give engineering 20 percent of capacity off the top to rewrite, re-architect, and improve performance. A code base already in bad shape takes 30 percent or more, and he gets nervous below 20.

Teams already in the hole get a three-step exit. Set a realistic schedule, because rewrite estimates come in far too low. Chunk the work so user-visible improvements continue on 25 to 50 percent of capacity, even when the rewrite stretches from nine months to two years. Then pick few features and define them right.

The casualty list carries the argument. eBay came far closer to collapse in 1999 than most people knew. Friendster opened the door for MySpace, Netscape lost the browser war mid-rewrite, and most companies in that position never recover. eBay later rebuilt pre-emptively, twice, including a multi-million-line translation of the entire site that users never noticed.

Emotion Sits at the Top of the Stack

The Apple chapter states a three-layer rule. The hardware serves the software, the software serves the user experience, and the user experience serves the emotion. Multi-touch and the accelerometer exist because the software needed them. Consumers then paid 400 dollars for a phone without comparing it to other phones. The market held over a hundred handsets, and few owners loved any of them.

The lens generalizes. Enterprise buyers run on fear and greed: beaten to market, hacked, deserted, or richer. Consumer emotions get personal: loneliness, lust, greed, and pride. Jeff Bonforte's interview relabels the adoption curve by emotional intensity. His irrationals love the environment enough to pay 22,000 dollars past its benefit for a Prius. Startups fall into the chasm by mistaking technology lovers for irrationals.

Bonforte's sharpest claim is that angry people dictate the future of technology. Skype tapped decades of latent anger at telephony. The webcam serves a need with no anger behind it, so adding video to Skype changed almost nothing. His hunting grounds for latent frustration: telcos, banks, airlines, health care, the music industry.

Practical Applications

Write the opportunity assessment before any solution work. Ten questions run from the exact problem and the target market through the go or no-go recommendation, and none of them touches the solution. The hardest question is the first one. The classic failure fuses problem with solution, then abandons a good opportunity because one solution stumbled.

Staff discovery as a threesome: product manager, interaction designer, and a prototyper or lead engineer. A startup founder runs this for weeks or months, through dozens of prototype versions. The alternative spends 500,000 dollars of seed on stealth engineering. The default startup path, engineers first and an alpha after six months, is the failure model.

Define the minimal product inside the prototype, then defend it. Get committed estimates on the surviving features, and strip the P1, P2, P3 labels from the spec, because it now describes an entire product. After that, pressure gets answered with a schedule slip rather than a feature cut. A product that survives later cuts was never minimal.

Refuse specials. A big check contingent on building exactly what one customer dictates converts customer requirements into product requirements, and that confusion derails product companies. Tease out the need behind the demanded features and keep the product general.

Plan the week after launch as a phase. Rapid response starts at launch, lasts a few days to a week, and the team meets at least daily. Cagan rates it the best return on investment in the whole process. From then on, improve by metrics rather than requests. His worked example: nine of every 100 people finish the subscription flow they start, and at eighteen revenue doubles.

Who This Is For

Founders about to hire engineers get the most. The startup failure model is exact. Seed money, engineers hired first, six months of stealth, then a first viewing that goes badly. The burn outruns the learning. Product managers inside feature factories get their job description back, and their managers get the ratios to staff it.

Skip the middle if you only want the method: two chapters on managing up and surviving large companies read as generic corporate advice. Skip the whole book if your work is delivery against a fixed spec, because every page assumes you decide what gets built.

Date-stamp everything. This is the 2008 first edition, written from Netscape and eBay, and the 2018 second edition reorganized the book. The mechanics aged well: the prototype as spec, the stop rules, the charter program, the headroom tax, discovery split from execution. The context aged badly: the Agile-versus-Waterfall fights, Craigslist recruiting, Friendster as a cautionary tale, and the shipped-software-versus-internet-services divide. The evidence is one practitioner's cases throughout, and the nine-of-ten failure rate arrives credited only to unnamed industry pundits.

The Decision

The audit takes one week and no budget. Take the item engineering starts next, and write down the evidence that it is valuable, usable, and feasible. Evidence that lives only in a document has met no users. So build the cheapest fake version, recruit a round of six to eight target users, and phone each one the day before. Fix what breaks after the second or third user.

Two outcomes end the test. Six users in a row get through the key tasks and want the product, and engineering starts from a spec that earned it. Or they do not, and you shelved a loser for the price of a prototype instead of a release. Every line of production code written before that point is the expensive prototype.