February 13th, 2012
Three recent articles reflect both our fetish with data and data analysis, and at the same time reveal that data analysis is, in the end, inconclusive because it is subjective. Data analysis is subjective in the categories that researchers select as relevant and how they manipulate those categories. Another layer of subjectivity arises from researchers’ decisions about when the results from a particular operation are meaningful. Finally, it is too easy to elide the difference between correlation and causation, mistaking the former for the latter. Even when researchers don’t make that mistake, people less trained in the research methodologies—people who encounter the “results” through popular press or media, often confuse correlation for causation.
On the increasing use of data analysis: “Big Data’s Impact in the World”
This debate about the usefulness of data analysis reveals the problem of category selection: “On-Line Dating Sites Don’t Match the Hype.” See also the authors’ academic article: “Online Dating:A Critical Analysis From the Perspective of Psychological Science.”
For a person who mistakes correlation for causation: “How To Choose An Elementary School By The Numbers”
Regrettably, this problem with data and data analysis is not new. For an interesting treatment, see Theodore M. Porter’s Trust in Numbers.