If you kiss your bowling ball, you bowl a strike. If you buy ice cream, you turn into a cold-blooded killer. What other correlations that aren't causations can you chime in with?
The classic example of correlation not equalling causation can be found with ice cream and - murder.
That is, the rates of violent crime and murder have been known to jump when ice cream sales do. But,
presumably, buying ice cream doesn't turn you into a killer (unless they're out of your favourite kind?).
You would think by now that we could say unequivocally what causes what. But the question of cause, which has haunted science and philosophy from their earliest days, still dogs our heels for numerous reasons. Humans are evolutionarily predisposed to see patterns and psychologically inclined to gather information that supports pre-existing views, a trait known as confirmation bias. We confuse coincidence with correlation and correlation with causality.
For A to cause B, we tend to say that, at a minimum, A must precede B, the two must covary (vary together), and no competing explanation can better explain the covariance of A and B. Taken alone, however, these three requirements cannot prove cause; they are, as philosophers say, necessary but not sufficient. In any case, not everyone agrees with them.
Speaking of philosophers, David Hume argued that causation doesn't exist in any provable sense. Karl Popper and the Falsificationists maintained that we cannot prove a relationship, only disprove it, which explains why statistical analyses do not try to prove a correlation; instead, they pull a double negative and disprove that the data are uncorrelated, a process known as rejecting the null hypothesis.
With such considerations in mind, scientists must carefully design and control their experiments to weed out bias, circular reasoning, self-fulfilling prophecies and hidden variables. They must respect the requirements and limitations of the methods used, draw from representative samples where possible, and not overstate their results.
Ready to read about 10 instances where that wasn't so easy?
10. The Trouble With Henry (and Hawthorne)
Researchers investigating worker productivity on the factory floor in the early 20th century discovered
the Hawthorne effect, or the idea that participant knowledge of an experiment can influence its results.
People are a pain to research. They react not only to the stimulus you're studying but also to the experiment itself. Researchers today try to design experiments to control for such factors, but such was not always the case.
Take the Hawthorne Works in Cicero, Illinois. In a series of experiments from 1924-1932, researchers studied the worker productivity effects associated with altering the Illinois factory's environment, including changing light levels, tidying up the place and moving workstations around. Just when they thought they were on to something, they noticed a problem: The observed increases in productivity flagged almost as soon as the researchers left the works, indicating that the workers' knowledge of the experiment, not the researchers' changes, had fuelled the boost. Researchers still call this phenomenon the Hawthorne Effect.
A related concept, the John Henry effect, occurs when members of a control group try to beat the experimental group by kicking their efforts into overdrive. They need not know about the experiment; they need only see one group receive new tools or additional instruction. Like the steel-driving man of legend, they want to prove their capabilities and earn respect [sources: Saretsky; Vogt].
9. Always Bet on Black?
If red just came up seven times in a row on the roulette wheel, would you be more likely to bet on red or
black before that eighth turn?
The titular characters of Tom Stoppard's film "Rosencrantz and Guildenstern Are Dead" begin the film baffled, confused and finally frightened as each of 157 consecutive flips of a coin comes up heads. Guildenstern's explanations of this phenomenon range from time loops to "a spectacular vindication of the principle that each individual coin, spun individually, is as likely to come down heads as tails..."
Evolution wired humans to see patterns, and our ability to properly process that urge seems to short-circuit the longer we spend gambling. We can rationally accept that independent events like coin flips keep the same odds no matter how many times you perform them. But we also view those events, less rationally, as streaks, making false mental correlations between randomized events. Viewing the past as prelude, we keep thinking the next flip ought to be tails.
Statisticians call this the gambler's fallacy, aka the Monte Carlo fallacy, after a particularly illustrative example occurring in that famed Monaco resort town. During the summer of 1913, bettors watched in increasing amazement as a casino's roulette wheel landed on black 26 times in a row. Inflamed by the certainty that red was "due," the punters kept plunking down their chips. The casino made a mint [sources: Lehrer; Oppenheimer and Monin; Vogt].
8. The Hot Hand and the Monkey's Paw
Superstitions take all forms in sports, like smooching your bowling ball.
No discussion of streaks, magical thinking or false causation would be complete without a flip through the sports pages. Stellar sports seasons arise from such a mysterious interplay of factors - natural ability, training, confidence, the occasional X factor - that we imagine patterns in performance, even though studies repeatedly reject streak shooting and "successful" superstitions as anything more than imaginary.
The belief in streaks or slumps implies that success "causes" success and failure "causes" failure or, perhaps more reasonably, that variation in some common factor, such as confidence, causes both. But study after study fails to bear this out [sources: Gilovich et al.; Tversky and Gilovich]. The same holds true for superstitions, although that did not stop the Cleveland Indians' Kevin Rhomberg from refusing to make right turns while on the field, or prevent Ottawa Senators centre Bruce Gardiner from dunking his hockey stick in the toilet to break the occasional slump [source: Trex].
The sophomore slump, too, typically arises from a too-good first year. Performance swings tend to even out in the long run, a phenomenon statisticians call regression toward the mean. In sports, this averaging out is aided by the opposition, which adjusts to counter the new player's successful skill set.
7. Hormonal Imbalance
The story of hormone replacement therapy, once widely used to treat symptoms of menopause, turned
out not to be so straightforward after all.
Randomized controlled trials are the gold standard in statistics, but sometimes - in epidemiology, for example - ethical and practical considerations force researchers to analyze available cases. Unfortunately, such observational studies risk bias, hidden variables and, worst of all, a study group that might not reflect the population as a whole. Studying a representative sample is vital; it allows researchers to apply results to people outside of the study, like the rest of us.
A case in point: hormone replacement therapy (HRT). Beyond treating symptoms associated with menopause, it was once hailed for potentially reducing coronary heart disease (CHD) risk, thanks to a much-ballyhooed 1991 observational study [source: Stampfer and Colditz]. But later randomized controlled studies, including the large-scale Women's Health Initiative, revealed either a negative relationship, or a statistically insignificant one, between HRT and CHD [sources: Lawlor et al.; New York Times].
Why the difference? For one thing, women who use HRT tend to come from higher socioeconomic strata and receive better quality of diet and exercise - a hidden explanatory relationship for which the observational study failed to fully account [source: Lawlor et al.].
6. Super Bowl Stock Market Shuffle
When John Elway and his fellow Broncos won the Super Bowl two years running in 1998 and 1999, the
Super Bowl-stock market connection fell apart.
In 1978, sports reporter and columnist Leonard Koppett mocked the causation-correlation confusion by wryly suggesting that Super Bowl outcomes could predict the stock market. It backfired: Not only did people believe him, but it worked - with frightful frequency.
The proposal went as follows: If one of the 16 original National Football League teams - those in existence before the NFL's 1966 merger with the American Football League - won the Super Bowl, the stock market would close higher that following year than it did the preceding Dec. 31. If a former AFL team won, it would go down [sources: Koppett; Koppett; Koppett; Koppett; Zweig].
From 1967 to 1978, Koppett's system went 12 for 12; up through 1997, it boasted a 95 percent success rate. It stumbled in 1998 and 1999, when AFL alums the Denver Broncos won and the market went up [sources: Koppett; Koppett; Koppett; Koppett].
Some have argued that the pattern exists, driven by belief; it works, they say, because investors believe it does, or because they believe that other investors believe it. This notion, though clever in a regressive sort of way, hardly explains the 12 years of successful correlations predating Koppett's article. Others argue that a more relevant pattern lies in the stock market's large-scale upward trend, barring some short-term major and minor fluctuations, and the fact that an original NFL team won every Super Bowl from 1984 to 1998 [source: Norris].
5. Big Data, Little Clarity
Target is just one of many companies combing through big data to boost sales.
Big data - the process of looking for patterns in data sets so large they resist traditional methods of analysis - rates big buzz in the boardroom these days [source: Arthur]. But is bigger always better?
It's a rule that's drummed into most researchers in their first stats class: When encountering a sea of data, resist the urge to go on a fishing expedition. Given enough data, patience and methodological leeway, correlations are almost inevitable, if unethical and largely useless.
After all, the mere correlation between two variables does not imply causation; nor does it, in many cases, point to much of a relationship. For one thing, researchers cannot use statistical measures of correlation willy-nilly; each contains certain assumptions and limitations that fishing expeditions too often ignore, to say nothing of the hidden variables, sampling problems and flaws in interpretation that can gum up a poorly designed study.
Granted, big data has its uses. Inventory control thrives on discovering purchasing patterns, however mysterious their underlying causes. To take a somewhat creepy example, Target has used purchasing patterns to identify pregnant customers and then send them targeted coupons [sources: Duhigg; Hill; Taylor]. So enjoy that rewards card - and 10 percent off your prenatal vitamins - but don't expect too much out of big data in the causality department.
4. Minimum Wage Equals Maximum Unemployment
Striking fast-food workers are joined by supporters, union members and activists at a rally in New York
City's Foley Square to demand an increase in the minimum wage to US$15 an hour.
Any issue dealing with money is bound to be deeply divisive and highly politicized, and minimum wage increases are no exception. The arguments are varied and complex, but essentially one side contends that a higher minimum wage hurts businesses, which drives down job availability, which hurts the poor. The other side responds that there's little evidence for this claim, and that the 3.6 million Americans working at or below minimum wage, which some argue is not a living wage, would benefit from such an increase. They argue that, adjusted for inflation, the federal minimum wage (US$7.25 per hour in December 2013) has tobogganed downhill for the past 40 years [sources: Bureau of Labor Statistics; Irwin].
As George Bernard Shaw reportedly quipped, "If all the economists were laid end to end, they'd never reach a conclusion," and the minimum-wage debate seems to bear that out [source: Ridgers LINK TO LMI]. For every analyst who says minimum wage increases drive jobs away there is another who argues against such a correlation [sources: Baskaya and Rubinstein; Card and Krueger].
In the end, both sides share a fundamental problem, namely, the abundance of anecdotal evidence many of their talking heads rely on for support. Secondhand stories and cherry-picked data make for weak tea in any party, even when presented in pretty bar charts.
3. Breakfast Beats Obesity, Dinner Denies Drugs
The family that eats dinner together stays off drugs together. Er, not quite.
Between books, drugs and surgeries, weight loss in the United States is a US$20-billion-per-year industry, with 108 million Americans bellying up to the weight-loss bar each year [source: ABC News]. Not surprisingly, weight loss studies - good, bad or ugly - get a lot of press in the U.S.
Take the popular idea that eating breakfast beats obesity, a sugar-frosted nugget derived from two main studies: One, a 1992 Vanderbilt University randomized controlled study, showed that reversing normal breakfast habits, whether by eating or not eating, correlated with weight loss; the other, a 2002 observational study by the National Weight Control Registry, correlated breakfast-eating with successful weight-losers - which is not the same as correlating it with weight loss [sources: Brown et al.; O'Connor; Schlundt et al.; Wyatt et al.].
Unfortunately, the NWCR study failed to control for other factors - or, indeed, establish any causal connection from its correlation. For example, a person who wants to lose weight might work out more, or eat breakfast, or go whole-hog protein, but without an experimental design capable of dialling in causal links, such behaviours amount to nothing more than commonly co-occurring characteristics [sources: Brown et al.; O'Connor].
A similar problem plagues the numerous studies linking family dinners with a decreased risk of drug addiction for teens. Although attractive for their simple, appealing strategy, these studies frequently fail to control for related factors, such as strong family connections or deep parental involvement in a child's life [source: Bialik].
2. The Suicidal Sex
Researchers studying suicide occurring across genders have to be aware that suicidal men and women
may reach for different weapons, thus affecting their outcomes.
We often hear it bandied about that men, especially young men, are more likely to commit suicide than are women. In truth, such statements partake of empirical generalization - the act of making a broad statement about a common pattern without attempting to explain it - and mask a number of known and potential confounding factors.
Take, for example, the fact that women make three times as many suicide attempts as men. How then can a higher correlation exist between the opposite sex and suicide? The answer lies in success rate, influenced by differences in methodology: Women resort to pills, while men tend to favour guns [source: O'Connell].
Even if we could dispose of such confounding factors, the fact would remain that maleness, per se, is not a cause. To explain the trend, we need to instead identify factors common to men, or at least suicidal ones. The same point applies to the comparatively high rates of suicide reported among divorced men. Divorce doesn't cause men to commit suicide; if anything, the causal variable hides among related factors, such as isolation, depression, a sense of powerlessness, financial stress or custody loss [sources: Kposowa; Kposowa; Reuters].
1. Vaccination Vexation
Pediatrician with a vial of MMR vaccine and a 15-month-old boy. In the U.S., the Advisory Committee
on Immunization Practices recommends a two-dose vaccine schedule for measles, mumps, rubella
vaccines for children, with the first dose at age 12-15 months and the second at age 4-6 years.
No correlation/causation list would be complete without discussing parental concerns over vaccination safety, rooted in the idea, popularized by celebrities like Jenny McCarthy, that measles, mumps and rubella (MMR) vaccinations are causally linked to autism spectrum disorders. Despite the medical community debunking the 1998 Andrew Wakefield paper that inspired the idea, and despite subsequent studies showing no causal link, even with multiple vaccinations, some parents remain fearful of an autism connection or other vaccine-related dangers [sources: The Lancet; Park; Sifferlin; Szabo].
While it's true that no vaccine is 100 percent harmless, the belief in this causal link arises mainly from natural parental concern, burdened by confusion, fuelled by anecdotal evidence and influenced by confirmation bias, or "if I hadn't believed it I wouldn't have seen it." Further fuelling the confusion is the fact that parents and doctors tend to recognize autism symptoms late, around the ages that children receive many vaccinations. In actuality, autism onset is quite complex and follows more than one pattern. Indeed, studies now show that onset can begin as early as 6-12 months [sources: CDC; Johnson and Schultz; Mandell et al.; NIH; Ozonoff et al.].
It's no harmless misunderstanding. In 2011, Time magazine reported that 13 percent of parents skipped, delayed or split up their children's vaccinations; in some rural areas, that number shot up to between 20 and 50 percent. Meanwhile, 15 years after this panic began, medical centres reported outbreaks of whooping cough and measles. Whether that correspondence is coincidental, correlative or causal is well worth considering [sources: O'Connor; Park; Park].
Author's Note: As much as I abhor poor experimental design, blind reliance on statistics and sensationalistic science reporting, it's worth mentioning that strong correlations, while not alone sufficient to prove cause, often point to areas worth investigating. Clearly, by "correlations" I don't mean auto-correlations, confounding variables or other artefacts of bad design or poorly understood methodological requirements and constraints; nevertheless, maybe the Internet can lay off the "correlation does not imply causation" slogan for a bit, or at least grow a bit more selective in its application.
Related Articles:
Article Sources:
1. ABC News. "100 Million Dieters, $20 Billion: The Weight-Loss Industry by the Numbers." May 8, 2012. (Dec. 18, 2013)
2. Arthur, Lisa. "What is Big Data?" Forbes. Aug. 15, 2013. (Dec. 18, 2013)
3. Baskaya, Yusuf Soner, and Yona Rubinstein. "Using Federal Minimum Wages to Identify the Impact of Minimum Wages on Employment and Earnings Across the US States." London School of Economics and Political Science. 2011. (Dec. 18, 2013)
4. Bialik, Carl. "What Family Dinners Can and Can't Do for Teens." The Wall Street Journal. Nov. 29, 2013. (Dec. 18, 2013)
5. Brown Andrew W., Michelle M, Bohan Brown and David B. Allison. "Belief Beyond the Evidence: Using the Proposed Effect of Breakfast on Obesity to Show 2 Practices that Distort Scientific Evidence." American Journal of Clinical Nutrition. Vol. 98, no. 5. Page 1298. (Dec. 18, 2013)
6. Bureau of Labour Statistics. "Minimum Wage Workers Account for 4.7 Percent of Hourly Paid Workers in 2012." U. S. Department of Labour. March 25, 2013. (Dec. 18, 2013)
7. Card, David and Alan B. Krueger. "Myth and Measurement: The New Economics of the Minimum Wage." Princeton University Press. 1997. (Dec. 18, 2013)
8. Centres for Disease Control and Prevention. "Recommended Immunization Schedule for Persons Age 0 through 18 Years: United States, 2013." (Dec. 18, 2013)
9. Duhigg, Charles. "How Companies Learn Your Secrets." The New York Times. Feb. 16, 2012. (Dec. 18, 2013)
10. ESPN. "Michael Jordan Stats." (Dec. 18, 2013)
11. Gilovich, Thomas, Robert Vallone and Amos Tversky. "The Hot Hand in Basketball: On the Misperception of Random Sequences." Cognitive Psychology. Vol. 17. Page 295. (Dec. 18, 2013)
12. Hill, Kashmir. "How Target Figured Out a Teen Girl was Pregnant Before Her Father Did." Forbes. Feb. 16, 2012. (Dec. 18, 2013)
13. HoopData. "League Average Stats." (Dec. 18, 2013)
14. Irwin, Neil. "The Typical American Family Makes Less Than it Did in 1989." Washington Post. Sept. 17, 2013. (Dec. 18, 2013)
15. Johnson, Eric O. and Lonni Schultz. "Forward Telescoping Bias in Reported Age of Onset: An Example from Cigarette Smoking." International Journal of Methods in Psychiatric Research. Vol. 14, no.3. Page 119. (Dec. 18, 2013)
16. Koppett, Leonard. "Carrying Statistics to Extremes." Sporting News. Feb. 11, 1978. (Dec. 18, 2013)
17. Koppett, Leonard. "If The Bulls and the Bears Have You Buffaloed, Try Our Foxy Formulas." Sports Illustrated. April 23, 1979. (Dec. 18, 2013)
18. Koppett, Leonard. "Super Bowl Stock Theory Making a Comeback?" Dartmouth College. (Dec. 18, 2013)
19. Koppett, Leonard. "Super Bowl Stocks." The Wall Street Journal. (Dec. 18, 2013)
20. Kposowa, Augustine J. "Divorce and Suicide Risk." Journal of Epidemiology & Community Health. Vol. 57. Page 993. (Dec. 18, 2013)
21. Kposowa, Augustine J. "Marital Status and Suicide in the National Longitudinal Mortality Study." Journal of Epidemiology & Community Health. Vol. 54. Page 254. (Dec. 18, 2013)
22. The Lancet. "Retraction - Ileal-Lymphoid-Nodular Hyperplasia, Non-Specific Colitis, and Pervasive Developmental Disorder in Children." Feb. 2, 2010. (Dec. 18, 2013)
23. Lawlor, Debbie A., George Davey Smith and Shah Ebrahim. "Commentary: The Hormone Replacement-Coronary Heart Disease Conundrum: Is this the Death of Observational Epidemiology?" International Journal of Epidemiology. Vol. 33, no. 3. Page 464. (Dec. 18, 2013)
24. Lehrer, Jonah. "How We Decide." Houghton Mifflin Harcourt. 2009.
25. Mandell, David S., Maytali M. Novak and Cynthia D. Zubritsky. "Factors Associated With Age of Diagnosis Among Children With Autism Spectrum Disorders." Paediatrics. Vol. 116, no. 6. Page 1480. (Dec. 18, 2013)
26. Manson, JoAnn, et al. "Menopausal Hormone Therapy and Health Outcomes During the Intervention and Extended Poststopping Phases of the Women's Health Initiative Randomized Trials." Journal of the American Medical Association. Vol. 310, no. 13. Page 1353. (Dec. 18, 2013)
27. National Institutes of Health. "What Is Autism Spectrum Disorder?" (Dec. 18, 2013)
28. The New York Times. "Hormone Replacement Therapy." (Dec. 18, 2013)
29. Norris, Floyd. "The Super Bowl Predicts the Market, and Vice Versa." The New York Times. Jan. 24, 1997. (Dec. 18, 2013)
30. O'Connell, Jeff. "Male Suicide a Growing Concern in Tough Times." Men's Health/NBC News. Dec. 4, 2008. (Dec. 18, 2013)
31. O'Connor, Anahad. "More Parents Skip Childhood Vaccines." The New York Times. Nov. 29, 2011. (Dec. 18, 2013)
32. O'Connor, Anahad. "Myths Surround Breakfast and Weight." The New York Times. Sept. 10, 2013. (Dec. 18, 2013)
33. Oppenheimer, Daniel and Benoît Monin. "The Retrospective Gambler's Fallacy: Unlikely Events, Constructing the Past, and Multiple Universes." Judgment and Decision Making. Vol. 4, no. 5. Page 326. (Dec. 18, 2013)
34. Ozonoff, Sally, Kelly Heung, Robert Byrd, Robin Hansen and Irva Hertz-Picciotto. “The Onset of Autism: Patterns of Symptom Emergence in the First Years of Life.” Autism Research. Vol. 1, no. 6. Page 320.
35. Park, Alice. "1 in 10 Parents Skip or Delay Vaccines." Time Magazine. Oct. 3, 2011. (Dec. 18, 2013)
36. Park, Alice. "Study Linking Vaccines to Autism is 'Fraudulent.'" Time Magazine. Jan. 06, 2011. (Dec. 18, 2013)
37. Park, Madison. "Where Vaccine Doubt Persists." CNN. Oct. 20, 2010. (Dec. 18, 2013).
38. Ridgers, Bill. "The Economist Book of Business Quotations." John Wiley & Sons. 2012.
39. Saretsky, Gary. "The OEO P.C. Experiment and the John Henry Effect." The Phi Delta Kappan. Vol. 53, no. 9. Page 579. (Dec. 18, 2013)
40. Schlundt, David G., James O. Hill, Tracy Sbrocco, Jamie Pope-Cordle and Teresa Sharp. "The Role of Breakfast in the Treatment of Obesity: A Randomized Clinical Trial." American Journal of Clinical Nutrition. Vol. 55, no. 3. Page 645. (Dec. 18, 2013)
41. Shute, Nancy. "The Last Word on Hormone Therapy from the Women's Health Initiative." NPR. Oct. 4, 2013. (Dec. 18, 2013)
42. Sifferlin, Alexandra. "Multiple Vaccinations on Same Day Does Not Raise Autism Risk." Time Magazine. March 29, 2013. (Dec. 18, 2013)
43. Sports Reference. "NBA & ABA Year-by-Year Leaders and Records for Points per Game." (Dec. 18, 2013)
44. Stampfer Meir J. and Graham A. Colditz. "Estrogen Replacement Therapy and Coronary Heart Disease: A Quantitative Assessment of the Epidemiologic Evidence." Preventative Medicine. Vol. 20, no. 1. Page 47. (Dec. 18, 2013)
45. Szabo, Liz. "Full Vaccine Schedule Safe for Kids, No Link to Autism." USA Today. March 29, 2013. (Dec. 18, 2013)
46. Taylor, Chris. "Big Data's Slippery Issue of Causation vs. Correlation." Wired. July 15, 2013. (Dec. 18, 2013)
47. Toporek, Bryan. "Kobe Bryant vs. Michael Jordan: Comparing the G.O.A.T. to the Birthday Boy." Bleacher Report. Aug. 23, 2013. (Dec. 18, 2013)
48. Trex, Ethan. "The 10 Most Bizarre Athlete Superstitions." The Wall Street Journal. July 14, 2009. (Dec. 18, 2013)
48. Tversky, Amos and Thomas Gilovich. "The Cold Facts about the 'Hot Hand' in Basketball." In Preference, Belief, and Similarity: Selected Writings by Amos Tversky and Eldar Shafir. (Dec. 18, 2013)
50. Vogt, W. Paul. "Dictionary of Statistics and Methodology." Sage. 1993.
51. Wyatt, Holly R., Gary K. Grunwald, Cecilia L. Mosca, Mary L. Klem, Rena R. Wing and James O. Hill. "Long-Term Weight Loss and Breakfast in Subjects in the National Weight Control Registry." Obesity Research Vol. 10, no. 2. Page 78. (Dec. 18, 2013)
52. Zweig, Jason. "Super Bowl Indicator: The Secret History." The Wall Street Journal. Jan. 28, 2011. (Dec. 18, 2013)
No comments:
Post a Comment
Please adhere to proper blog etiquette when posting your comments. This blog owner will exercise his absolution discretion in allowing or rejecting any comments that are deemed seditious, defamatory, libelous, racist, vulgar, insulting, and other remarks that exhibit similar characteristics. If you insist on using anonymous comments, please write your name or other IDs at the end of your message.