A landing page, a real signup, a measured intent — run against a threshold agreed before the data arrives. I did this for two AI products at Chime Labs, which turned ‘should we build this?’ into a question with an answer rather than an argument.
There's a feature on the roadmap that will take a quarter and nobody can say who asked for it. A new market or segment looks promising in the abstract. Or an idea has strong internal support and no external evidence, and the disagreement in the room is really a disagreement about demand.
The point is to buy information before you buy a build. A fake-door test, a landing page with a real signup, a concierge version run manually behind the scenes — anything that produces a signal from someone outside the building.
What matters is defining, in advance, what result would change the decision. A test with no threshold produces a number everyone interprets to suit the position they already held. So we agree the threshold first, then run it. I ran this for two AI products at Chime Labs, which turned “should we build this?” into a question with an answer rather than an argument.
Ethics come into this. A fake door that wastes people's time or misleads them costs you trust for a data point you could have got another way. There are honest versions of every one of these tests.
One to three weeks per test. The design takes longer than the running, which is the point.
Isn't a fake door dishonest?
It can be. It doesn't have to be — “not available yet, want to be told when it is” is a real signal and an honest message.
How much traffic do we need?
Less than people assume for a directional read, more than people assume for a confident one. We set expectations against your actual volumes before running anything.