Is That a Fact? Three Brief Reports on the Credibility of Facts

Americans are bombarded by a constant stream of competing factoids and causal claims about families. Politicians, advocacy groups, pundits, and instant internet “experts” claim that social science “proves” this or that is the impact of divorce, “surveys show” what people think about marriage, or “the facts are clear” about the benefits of one family form or another.

Are some facts more trustworthy than others, and if so, how can we tell the trustworthy from the untrustworthy? What is the difference between a cause, a correlation and a coincidence?

Three brief reports prepared for the Council on Contemporary Families help journalists, students, and general audiences interpret claims of fact and causation about such controversial topics as divorce, marriage, and domestic violence. Written by award-winning researchers at the top of their respective fields, the papers advise when to take research claims with a grain of salt and how to be confident that a study is particularly well done, clear, and reliable. The summary below provides links to each paper in its entirety.

Paper One: The Impact of Divorce and the Benefits of Marriage – Beware of Selective Facts 

Andrew Cherlin, Griswold Professor of Public Policy at Johns Hopkins University, highlights how advocacy groups use facts selectively and thereby over-simplify public discussion of important issues. Professor Cherlin offers examples from both the left and the right to illustrate how this selective use of facts impoverishes public debates.

Paper Two: Does Your Height “Cause” Your Weight? Does Marriage “Cause” Good Health? The Thorny Question of Correlation vs. Cause 

Philip Cowan, Professor of Psychology, Emeritus, UC-Berkeley, and Carolyn Pape Cowan, Professor of Psychology, Emerita, UC-Berkeley, note that many studies show that children whose parents are married to each other are doing better than children whose parents are not married. This is what researchers call a correlation — and a correlation, they explain, is often confused with a cause.

Such confusion of cause and correlation has deformed debates over child policies issues. One school of thought believes that the only effective way to protect children is to promote marriage. They emphasize that having married parents is correlated with better child health outcomes than having divorced or single parents and conclude we must promote marriage and restrict divorce. But that correlation doesn’t mean that marriage causes health. People’s height is correlated with their weight, yet no one claims that height causes weight. We all know that in the case of the association between height and weight, there are other forces in common—like nutrition and genetics—that influence how tall and how heavy a person becomes.

The same thing goes for marriage and child health. It is equally possible that adults in some single-parent families have traits that make them less likely to get or stay married and ALSO to be more likely to raise children who exhibit behavior problems. If so, the problems seen in children raised by these individuals might still develop even if the parents were able – or were forced – to stay together.

Another school of thought holds that ending poverty is the only way to promote child well-being. They cite research showing that poverty is associated with behavioral problems in children. But here too, the Cowans point out, the cause and context of the poverty, rather than poverty itself, may create the problem behaviors. Alternatively, poverty may increase the risk of poor child outcomes through the way that the parents react to it rather than through the mere absence of income. It may thus be possible to teach parents to handle their economic stresses better.

Yet that leads to still another complication. Many programs claim to save marriages or improve parenting, but these programs have seldom been rigorously evaluated. The Cowans urge journalists and policy-makers to look for systematic studies that include randomized clinical trials with control groups when they are approached with claims that a program is causing a change in people’s lives.

Paper Three: When Interviews, Surveys, and Focus Groups Don’t Tell the Full Story — the Example of Domestic Violence in the Lives of Impoverished Women

Linda Burton, James B. Duke Professor of Sociology at Duke University, explains what gets lost when researchers or journalists do one-shot surveys or focus groups rather than in-depth interviews that take place over a prolonged period. Burton’s four-year study examined the impact of welfare reform in the lives of low-income African American, Latino, Hispanic, and non-Hispanic White families in three different cities.

Had Dr. Burton’s team only done a few interviews, they would have overlooked the centrality of domestic violence and sexual abuse in the experience of economically vulnerable women. Few women volunteered the information that they had experienced significant abuse, yet eventually, researchers found that such abuse was a prominent feature in the lives of more than two-thirds of these women. In most cases, it took more than 6 months of interviews for this information to come out, and in almost 20 percent of the cases, the information emerged only after 10 to 24 months.

The extended interview process enabled researchers to rule out the idea that tales of domestic violence were simply a bid for the interviewer’s sympathy. Astonishingly, domestic violence was so “normalized” in these women’s lives that in the majority of cases the information came out accidentally or incidentally, as an aside, or in some cases when the researcher actually walked in during the aftermath of an abusive episode.

One woman revealed her history of abuse only when the researcher asked how she had met her husband. “Liza” stated that this was a “funny story,” because she had met her husband just after ending a relationship with a man who had broken her nose. In another instance, the researcher only discovered the abuse when she arrived at the house while the police were still responding to an incident. Burton cautions people to recognize that “absence of evidence” is not the same thing as “evidence of absence.”

FOR MORE INFORMATION, PLEASE CONTACT:

Stephanie Coontz, Director of Research and Public Education, Council on Contemporary Families