The key ideas

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

Seth Stephens-Davidowitz, 2017

People lie on questionnaires, to their friends and even to themselves, but not to the search bar. A former Google data scientist studied billions of searches and digital traces and learned things about sex, racism, children and money that nobody admits out loud. This book teaches you to judge people by what they do, not what they say, and shows where big data helps and where it deceives.

Who it is for

For anyone curious about what people are really like when no one is watching. And for everyone who makes decisions based on surveys, reviews and their own gut feeling.

Google searches are the most important dataset ever collected on the human psyche.

What the book is about

If you add up how many condoms Americans say they use in a year, you get far more than are actually sold in the whole country. Judging by the data, almost everyone fibs: to other people, to doctors and even to themselves.

But there is one place where people are often more honest. The search bar, late at night, when no one is watching.

Seth Stephens-Davidowitz is an economist with a Harvard PhD and a former data scientist at Google. He decided to learn about people not by asking the people themselves, but by asking their digital traces: searches, clicks and purchases. That is what big data is. This way he learned things about sex, money, children and prejudice that people won't tell even those closest to them.

Ahead: why intuition is so confidently wrong, how a noble speech can stoke anger, and which boring button label helped Obama's campaign. And at the end: where the data itself starts to deceive.

Idea 1. Intuition is a small sample

Who is more likely to make it to the NBA: a kid from a poor neighborhood who has “nothing to lose”, or a kid from a well-off family? Almost everyone answers: the first one, of course. Hunger, anger, basketball as the only way out. Remember your answer.

The author checked the data on players. For Black kids, the odds turned out to be higher if they grew up in wealthier counties and in two-parent families. Desperation matters less than it seems. Health, height, support and the ability to get along with people matter more.

Where does the mistake come from? Our experience is a few dozen vivid cases. We remember the star who escaped poverty and don't see the thousands of stories nobody ever told. The author isn't telling you to throw your gut feeling away. It's good at suggesting where to dig. The trouble starts when it's mistaken for proof.

Many people feel that “all successful people dropped out of college”: everyone remembers a couple of celebrities. Nobody writes about the thousands who dropped out and never succeeded at anything.

A vivid example is easy to remember, which is why it so often distorts the picture. Your gut asks the question, and the data gives the answer.

Read in Livria

About the author

Seth Stephens-Davidowitz is an American economist with a Harvard PhD, a former data scientist at Google and a columnist for The New York Times. He studies what search queries reveal about people.

All books by this author: Seth Stephens-Davidowitz

Terms from the book

  • natural experiment
  • Big data: Huge collections of digital traces: searches, clicks, purchases. They are analyzed to understand how people actually behave.
  • Digital truth serum: The author's name for search data. Alone with a search engine, people are more honest than in surveys: no one is watching, and they need an answer.
  • Social desirability bias: The tendency to answer surveys the way that is expected and approved of, rather than the way things really are.
  • A/B test: An experiment in which one group of people is shown version A, another group version B, and you compare which one works better.
  • Curse of dimensionality: An analysis trap: when there are a great many variables, some of them will line up with the result by pure chance and look like a real pattern.

All terms

Similar books