When you see two big book reviews and an entire special section of The Wall Street Journal devoted to a topic, a curious person should ask: how does this affect me, my family, my business, the world?
The book is Big Data: A Revolution that Will Transform How We Live, Work and Think and one its authors, Kenneth Cukier, is a former colleague of mine. The book has received excellent reviews and deserves to be read by anyone who wants to understand what’s going on in a big sector of world business.
It’s clear that Big Data – the ability today to process more information about people than ever before – has huge implications for businesses. But its effect on a good investigation? Harder to see.
Good investigators don’t care if 68 percent of people who drive Cadillacs also eat at a steak house once every seven weeks. That may be a statistic worth something to steakhouse owners looking for new customers or Cadillac dealers wondering which kind of restaurant social network to link to.
But for us? Hard to see how it will help. Investigations are done one at a time. If we need to know exactly where someone is going to go on a certain day, we need to follow him.
It’s not good enough to know that many of the people in his congregation have summer homes in a particular part of Pennsylvania: if we are doing an asset search, we need to see if HE has a home there. Then we need to be able to prove it.
That’s where big data stops and investigation takes over. Big data is about probabilities, and investigation is about evidence and proof.
Of course, courts do not reject statistical evidence all of the time. The now-famous series of Shonubi decisions in the Second Circuit are well documented here. But given the choice between a statistical probability and hard proof, why would anybody who needed help with fact finding prefer the former?