Magic Statistics

The US evangelistic team of Ira Sankey and Dwight L. Moody had a sizeable impact in 19th-century Britain.  Sankey’s gospel songs used to warm up the crowds, and then Moody’s sermons brought ‘em forward by the thousands to repent and accept Jesus.  Even today in south-west England, nonconformist (Britspeak for non-Anglican Protestants) Christians gather for Sankey nights, singing from Sankey’s Victorian songbook Sacred Songs and Solos and listening to a revival sermon.

Jeremy Clarke of the [UK] Spectator recently drove out to one of those meetings.

Two thirds of the elderly congregation were women. Outdoor faces and nut-brown pates predominated among the men. The pews bristled with walking sticks: some plain, others — belonging to those for whom a degree of comfort was not necessarily felt to be a sign of rampant worldliness — with impressive rubber tips.

The vicar, the Reverend Cyril Short, came gliding genially across to extend the hand of friendship and find out who we were. ‘Are we in for a treat tonight!’ he said. For leading tonight’s sing-a-long, he confided, was none other than the great Dr Appleby.

Now this was a surprise, for Dr Appleby is reputed in local nonconformist Protestant circles to be probably the godliest man alive in the south-west of England.

Dr Appleby’s opening prayer was so moving that Mr Clarke became concerned that he might actually get up and walk to the front at the altar call.

[I]f, later on, Dr Appleby made the traditional appeal for convicted sinners to make their way forward for prayer, and if I wasn’t careful, I might find myself blundering down to the front blinded by tears. And that would be a disaster.

He doesn’t elaborate on that last comment.

But first, a few rousing gospel choruses, beginning with “Standing on The Promises of God”, which our anthropologist reporter found exhilarating.

The song is an absolute belter. The chorus — ‘Standing, standing, standing on the promises of God my Saviour!’ — has the singability of a football chant; you can really let rip.

Several more old gospel songs and then a brief sermon.  When Dr Appleby encouraged repentant sinners to come forward, Mr Clarke managed to hold himself back.  All in all, an enjoyable evening.

And driving home afterwards, more elevated by the singing than I have been in the past by an evening of crack pipes, it was easy to see why Sankey nights have endured for so long. I heartily recommend them.

Careful there, Mr Clarke, one of these nights "disaster" may strike: The Holy Spirit could show up and impel you to repent and believe.

That was my initial reaction when my friend Tom Gilson told me that the latest issue of Skeptic magazine features an article extolling Gregory S. Paul’s paper on social problems and religious belief.  Mr Paul’s study, published last year in the Journal of Religion and Society, is entitled "Cross-National Correlations of Quantifiable Societal Health with Popular Religiosity and Secularism in the Prosperous Democracies”; the html version is posted here and the pdf version here.

Mr Paul’s paper presents itself as a statistical investigation of social problems—including homicide, prevalence of sexually transmitted diseases, abortion, pregnancy among young teenagers—and religious belief.  He collected data from eighteen democracies and claimed to have found positive correlations between social dysfunction and religious belief, thus purportedly showing that religiosity is not necessarily beneficial and secularism is not inherently harmful to social mores.

One year ago next week, I wrote one of my most popular blog posts critically assessing Mr Paul’s paper and showing it to be statistically invalid.  I concluded that, in my professional judgment as a statistician, the study should not be taken seriously.

Now comes the Skeptic article, by one Matthew Provonsha, who merely reiterates—in the most uncritical fashion imaginable—the errors and shortcomings in Mr Paul’s original work.  For Provonsha, evidently, Paul can say no wrong.  (Why was this published in a magazine called Skeptic?)

Thus, Mr Provonsha does not wonder why Mr Paul fails precisely to document his data sources or provide sound reasons for his selection of countries to be included.

The study focuses on the prosperous democracies, because “levels of religious and nonreligious belief and practice, and indicators of societal health and dysfunction, have been most extensively and reliably surveyed” in them. Also, “The cultural and economic similarity of the developed democracies minimizes the variability of factors outside those being examined.” With a database of 800 million people, this study is far more reliable than results based on smaller sample sizes used in other such studies.  The data are also current and extensive, collected in the middle and latter half of the 1990s and early 2000s from the International Social Survey Programme, the UN Development Programme, the World Health Organization, Gallup, and other well-documented sources.

From a statistical perspective, the criteria Paul used to include his eighteen chosen countries and omit all others are arbitrary and statistically unsupported, but Provonsha falls for it.  Also, Paul does not document his data sources, in that we don’t know from which particular source he obtained each of the various measures of social health.

The claim that Paul utilised “a database of 800 million people” is laughable.  The sample size of Paul’s study was precisely eighteen: one data point from each country for each data series.  To claim this represents the combined population of all the countries is like a market research firm conducting a survey of 1200 Canadians and then claiming they accessed a database of over 32.5 million people.

I do want to thank Mr Provonsha for re-drawing some of Mr Paul’s charts.  They are now larger and much easier to read.  Their statistical value, however, is still negligible.  Paul seemed to hold the belief that plotting points on charts constitutes statistical analysis.  Not in my world.  Paul and Provonsha should both read Darrell Huff’s classic, How To Lie With Statistics, which devotes an entire chapter to the perils of graphs.

Visual representations of data, e.g., plotting data points on a chart, can be valuable as aids to understanding, but they are no substitute for calculating correlation coefficients and other analytical statistics.  Paul did not calculate any analytical statistics—not one—and Provonsha doesn’t call him on it.  Paul’s paper has many descriptive statistics, but no actual statistical analysis, and Provonsha apparently didn’t even notice.  That says it all.

h/t: Tom Gilson, Thinking Christian

Other recommended commentary on Gregory Paul’s study:

• “Dogma Bites Man”, by George H. Gallup, Jr. at Touchstone Magazine
• “Debunking Gregory S. Paul”, by my friends John and Scott at Verum Serum

Previous related posts:

• From our bulging “How not to do statistics” file
• Mystery man
• A primer on crime statistics