Effectiveness · Field guide
The weakest argument in a meeting rarely sounds weak. It sounds like common sense, said with confidence. This guide covers 21 traps that come up often at work. Each card gives what the trap sounds like and the question that counters it. Try to name three of them first.
Guess first
A director says: We have already spent $400,000 on this platform, so we cannot stop now.
Which trap is this?
A product lead says: We interviewed our ten longest-standing customers, and they all love the product. Churn cannot be a product problem.
Which trap is this?
You propose a short list of approved AI tools. A colleague replies: So you are saying we should let anyone install any app they like?
Which trap is this?
The guide
The guide is showing all 21 traps.
These biases shape a choice before anyone argues about it. They show up in budgets, estimates and go or no-go calls.
Sounds like“The vendor opened at $2 million, so $1.4 million feels like a good deal.”
What is going onThe first number you hear pulls every later estimate toward it, even when that number is arbitrary (Tversky and Kahneman, 1974).
AskWhat would I estimate if I had never heard that first number?
Sounds like“We have already spent $400,000 on this platform, so we cannot stop now.”
What is going onMoney and time already spent cannot be recovered whatever you decide next, yet people keep investing to justify what they have spent (Arkes and Blumer, 1985).
AskIf we were starting today with nothing spent, would we fund this?
Sounds like“I asked around, and everyone I spoke to agrees with the plan.”
What is going onPeople look for, notice and remember the evidence that supports what they already believe (Nickerson, 1998).
AskWhat evidence would change my mind, and have I looked for it?
Sounds like“I am certain the migration will be finished by March.”
What is going onConfidence in an estimate tends to run ahead of its accuracy. Ranges that people describe as 98% certain miss the true value far more often than 2% of the time (Alpert and Raiffa, 1982).
AskWhat range would I bet my own money on, and how often have my past ranges held?
Practise it: The Calibration Game →Sounds like“This project is different. We have learned from the last three.”
What is going onPeople estimate their own project from its best-case plan and ignore how long similar projects took (Kahneman and Tversky, 1979).
AskHow long did the last three projects like this take, from start to finish?
Practise it: The Forecast Cone →Sounds like“The current system works well enough. Why take the risk?”
What is going onPeople prefer the current state of affairs and weigh the risks of change more heavily than the risks of standing still (Samuelson and Zeckhauser, 1988).
AskIf we already had the alternative, would we switch back to what we have now?
Sounds like“Did you see the breach at that bank last week? We need to move everything off the cloud.”
What is going onPeople judge how likely an event is by how easily an example comes to mind, so vivid and recent events seem more common than they are (Tversky and Kahneman, 1973).
AskHow often does this happen, as a rate rather than a story?
Practise it: The Base Rate Machine →These fallacies are moves in a discussion. Each one makes a weak argument sound stronger than the evidence behind it.
Sounds like“So you are saying we should let anyone install any app they like?”
What is going onThe speaker replaces the other person’s position with a weaker or more extreme version, then argues against that version.
AskCan I restate their position in a way they would agree with before I disagree?
Sounds like“She has never run a project this size, so her concerns about the timeline do not count.”
What is going onThe argument attacks the person making a point rather than the point.
AskIf someone I respect said the same thing, would the argument be any weaker?
Sounds like“An analyst firm says every company will have an AI strategy by next year, so we need one too.”
What is going onA claim is treated as true because of who said it, whether or not the source has the evidence or the expertise that applies.
AskWhat evidence does the source have, and does that evidence apply to our situation?
Sounds like“Either we replace the ERP this year or we accept falling behind for good.”
What is going onThe speaker presents two options as the only ones when others exist, often including a smaller first step.
AskWhat are three other options, including a small one we could try first?
Practise it: The Thin Vertical Slice →Sounds like“If we let one team work from home on Fridays, soon nobody will come in at all.”
What is going onThe speaker claims a small step will lead to an extreme outcome without showing how each step leads to the next.
AskWhat would have to happen at each step for that outcome to follow, and how likely is each step?
Sounds like“Every one of our competitors has moved to this platform, so the platform must be the right choice.”
What is going onA claim is treated as true or wise because many people believe it or do it.
AskWhat do they know that we do not, and do their reasons apply to us?
Sounds like“Yes, the pilot cut processing time in half, but now I need to see it cut costs as well.”
What is going onWhen the evidence someone asked for arrives, they raise the bar instead of changing their view.
AskWhat result would convince us, agreed before we look at the data?
These traps sit inside the numbers. The chart can be accurate and the conclusion still wrong.
Sounds like“We interviewed our ten longest-standing customers, and they all love the product. Churn cannot be a product problem.”
What is going onLooking only at the cases that made it through a filter hides the cases that did not, and the missing cases often hold the answer (Wald, 1943).
AskWho is missing from this data, and what would they say?
Practise it: The Survivorship Bias Lens →Sounds like“The fraud model is 99% accurate, so every account it flags must be fraudulent.”
What is going onPeople judge a case by how well it matches the evidence and ignore how common the thing is in the first place (Kahneman and Tversky, 1973).
AskOut of 1,000 cases, how many are truly positive, and how many would the test flag?
Practise it: The Base Rate Machine →Sounds like“Sales went up the month we launched the new website, so the website worked.”
What is going onTwo things that move together, or one after the other, are taken as cause and effect when a third factor or chance can explain both.
AskWhat else changed at the same time, and what would we see if the website had no effect?
Practise it: Correlation ≠ Causation →Sounds like“We put the worst branch on an improvement plan, and its numbers recovered the next quarter.”
What is going onAn extreme result is often followed by a more ordinary one by chance alone, so whatever happened in between gets the credit or the blame (Galton, 1886).
AskWould the numbers have moved back toward average with no intervention at all?
Practise it: The Process Behaviour Chart →Sounds like“Look at these three customer quotes. The new feature is a hit.”
What is going onThe speaker shows the data points that support the claim and leaves out the ones that do not.
AskWhat does the full set look like, including the cases that were left out?
Sounds like“Two of the five people who tried the new process found it slower, so the process does not work.”
What is going onSmall samples swing much more than large ones, yet people treat a small sample as if it were as representative as a large one (Tversky and Kahneman, 1971).
AskHow much would this number move if we ran the test again with five different people?
Sounds like“Our approval rate fell overall, so every team must be approving less.”
What is going onA trend that holds inside every group can reverse when the groups are combined, because the mix of groups changed.
AskDoes the pattern hold within each group, or only in the total?
Practise it: Simpson’s Paradox Splitter →No trap matches that search. Try a broader word, or clear the search.
Use it this week
Pick the trap you recognized fastest, and write its question at the top of your notes. When the conversation reaches a decision, ask the question out loud. In our experience, naming a bias does little on its own. The question, asked at the moment of the decision, is what changes the outcome.
Spotting these traps is one part of thinking well when information is abundant and attention is scarce. The Digital Intelligence course builds the rest of that skill one day at a time.