Theology, ethics, and church growth: or, how to prove anything with statistics

There’s an old story about a journalist, a normal person, and an academic, together on a train crossing the border into Scotland and seeing a black sheep. ‘Oh,’ says the journalist, ‘sheep in Scotland are black!’

‘Come on now,’ says the normal person, ‘all we can say is one sheep in Scotland is black.’

‘Actually,’ says the academic, ‘all we can say with any certainty is that one side of one sheep in Scotland is black.’

 

It is an important part of the role of the academic, particularly the academic who chooses to comment in non-specialist arenas, to be very clear about precisely what is, and what is not, shown by a given piece of evidence. I picked up on a minor Twitter storm yesterday concerning claims and counter-claims about the linkage of church decline with a progressive/accepting stance on issues of sexuality. It began with various people tweeting an article from The Federalist which argued that (a) every American denomination which had adopted a progressive sexual ethic is in serious numerical decline and (b) that a step-change in the rate of decline at the point of the adoption of the policy was visible; at the same time, (c) denominations which had remained conservative on the issue were either still growing or only declining very slowly. Others responded, in particular pointing to the Church of England’s report From Anecdote to Evidence (2013; here as pdf), which found no significant statistical correlation between ‘theological tradition’ and church growth or decline. This led some to claim, in the words of one tweeter, that the Federalist article was ‘false’.

What can we say about this?

1. The Federalist would not be my go-to source for good academic evidence (or for much else, to be honest), but Griswold’s article linked transparently to the various denominations’ own published statistics; and there is no doubt that they do show the movements he claims they do. This is good data. That said, Griswold’s interpretation of it, whilst factually correct, invites misunderstanding. It is the case – as, to be fair, he highlights – that a change of policy on the issue of sexuality has been a trigger for conservative congregations (and even dioceses) to leave denominations in the USA (and in the UK, indeed); the step-change effect he describes can, I suspect (the data is not there to do the analysis), be entirely explained by this dynamic, which is rather uninteresting (in that a congregation/diocese leaving a denomination but continuing to exist is not church decline in any meaningful sense). The broader trend data seems more telling: conservative denominations are growing, or shrinking slowly; progressive/affirming denominations shrinking rapidly – but this pattern predates the decision to become progressive/affirming, which suggests that this decision itself is not decisive, but rather is indicative of a broader attitude which is the real correlate.

 
2. From Anecdote to Evidence unfortunately gives us no data; it states (quoting the section in full)

Finally there are some factors which do not appear in research terms to make a significant difference to growth or decline. These include:

> Theological tradition

> The gender, ethnicity or marital status of leader 

This is, it is fair to say, singularly uninformative; interestingly it also looks to be wrong: whilst the dataset is not published, some of the analysis is (pdf here), which indicates (p. 92) that three questions were asked about ‘theological orientation’, all offering a scale of 1-7: ‘Catholic – Evangelical’; ‘Liberal – Conservative’; and ‘Charismatic – non-charismatic’. On the third there, was a small but statistically significant correlation between objective growth and being (or self-denominating as) charismatic; the data suggests that the more charismatic a church believes itself to be, the higher its growth. There is a significant correlation between theological tradition and church growth on this question, at least.

A couple of things are worth saying here: first, the dataset looks large (n=1703), representative, and robust. Second, the theological categories are rather crude (and in the case of the first possibly meaningless; it is at least not clear that ‘evangelical’ and ‘catholic’ represent two poles of a spectrum, although I do accept that within the peculiar context of the Church of England there is a fairly recent historical tradition of thinking that they do). Third, even if the data suggests that more charismatic churches grow more rapidly than less charismatic churches, that says nothing to us about sexuality.

What of the comparison of the two sets of data? It is, essentially, meaningless. Partly this is because it is comparing US with UK data; more pertinently though, Griswold’s piece is about denominations; the Anecdote to Evidence data concerns parishes. The data is just not interestingly comparable. (And the Anecdote to Evidence dataset has no data at all about attitudes to sexuality; at best cautious extrapolations might be made.)

There is other data, however, that perhaps overlaps both these sets, notably the Christian Research analysis of the Church Census data between 1979 and 2005 (some datasets are online here; analysis is in published volumes). This has consistently suggested that local church self-denomination as ‘evangelical’ and ‘charismatic’ both correlate with growth (or with slower rates of decline, which in terms of correlation is the same thing). It may be that trends have changed since 2005; it may be that a general church pattern is not visible in the Church of England (although this seems unlikely, given relative numbers); it may be that the fact that the measures of growth used are different – Anecdote to Evidence is more sophisticated – offers an explanation; it may be that one of the studies just got a biased sample; it happens. One way or another, though, the data is conflicted.

If we move beyond the (Anglican) parish, although (as far as I know) no statistical analysis has been done, Griswold’s broader analysis would seem to apply fairly straightforwardly to the Protestant churches in the UK: Black majority churches are growing astonishingly fast; Pentecostal denominations are generally growing; new charismatic denominations are generally growing; Baptists and non-charismatic conservative Protestant churches are growing slightly, holding their own, or shrinking slightly; Methodism, mainline Scottish Presbyterianism, and the URC are all shrinking at varying degrees of ‘rapidly’. Plotting this against a conservative – liberal axis looks fairly simple and fairly telling at first sight. Where the Church of England, which is endlessly odd, fits into this sort of analysis, I don’t pretend to understand; but even if it is a (moderately large) anomaly, the pattern seems secure.

I find most such analyses essentially uninteresting, or even positively distracting: our call is primarily to faithfulness, not to numerical growth. If I became convinced through sound moral reasoning that we should adopt an accepting/progressive position on same-sex relationships, I would push for that, even if I knew it would lead to a loss in membership; whilst I remain convinced that the best moral reasoning does not lead us to this position, I will hold the ‘conservative’ line, even if it leads to a loss in numbers. That said, on the back of my analysis above, my plea to both sides is simple: if we have to play these statistical games, could we please at least play them well?

10 Comments

  1. Alan Mokineaux
    Aug 28, 2014

    Good analysis Steve. Very helpful.

    You may recall I gave a very short paper at Chester Uni on how evangelicalism tens to measure its ‘success’ quantitatively.

    If you are doing any further analysis on this I would be keen to be involved. Al

    • steve
      Aug 31, 2014

      Thanks, Alan. I don’t have the statistical ability to do anything serious in this area; I know enough to understand and evaluate a statistical argument, generally, but not enough to be confident of my own constructions.

  2. Eddie
    Aug 29, 2014

    Thanks for this, Steve. I saw the exchange on Twitter and was more than a little surprised at the way that apples and oranges were being compared.

    Your last paragraph is excellent and puts the whole debate in a sensible context.

  3. Philip Fellows
    Aug 29, 2014

    Hi Steve, over the last couple of years at Spurgeon’s I’ve started to wonder whether it is helpful to have ‘Baptists’ as a category of denomination in these types of surveys. Surely it is misleading to group ‘baptists’ with ‘conservative protestant denominations, given that baptist churches are defined in part by a commitment to the supremacy of the local church (and can therefore vary massively on the liberal-conservative; charismatic-noncharismatic spectrum). It seems a bit odd to compare groups of churches defined by a doctrine (in this case charismatic or pentecostal worship) with a group who in part define themselves by a refusal to require common confessions. Wouldn’t it be better (and more ‘Baptist’) to distribute individual Baptist churches into other groups based upon the theological commitments of the individual churches? I realise that this is a counsel of perfection and that it is far easier for data-gatherers to operate with ‘Baptist’ as a meaningful category than to engage with the implications of congregationalism but it would make for better data and clearer conclusions.

    • steve
      Aug 31, 2014

      Interesting. Two comments, perhaps: 1. Congregational government and an aversion to confessions do not really belong together; British Baptists wrote confessions with energy in the C17th; American Baptists still do; and in the UK the FIEC (say) is congregational but seriously confessional. 2. UK Baptists presently seem to fit fairly comfortably into an ‘evangelical’ tab; there are exceptions, but broadly this works currently; no doubt the day will come when it doesn’t…

      • Philip Fellows
        Aug 31, 2014

        Thanks for the thoughts Steve. As to 1: Absolutely – baptists do make confessions. But the useful data point is those churches agreeing to the confession, not all ‘Baptist’ churches. So, for example, the BUGB comprises churches which historically have been either general or particular baptists. Both are confessional (in theory, although I am not aware of a general baptist confession in practice) but do not necessarily belong together in an analysis of the type you describe. Similarly, the BUGB comprises churches which are complementarion and some which are egalitarian, some which are charismatic and some cessationist, some liberal evangelical and some conservative evangelical. By simply counting ‘Baptists’ these nuances are missed or distorted and the data is similarly distorted. As to 2: I agree that most Baptist churches are probably loosely evangelical (although on a recent visit to Didcot there didn’t seem to be anyone who knew what the rough makeup of the Union was beyond this). But several are (I imagine) liberal evangelicals rather than conservative or open evangelicals which, when discussing the effect of social politics on church growth, is an important distinction to capture.

  4. simon Russell
    Aug 30, 2014

    Thank you – I started reading the sited article then gave up as it seemed a more propaganda piece than a piece of real reflection. I am sure it is picking up on points made by various American conservative bloggers to support there point of view that to change doctrine leads to the demise of that particular group grouping. I am sure statisticians would call it raw data not taking into effect other variables, but the nature of the debate on human sexuality will use any cannon fodder to make a point.
    I think for the u.k. Christian denominations is to ask if there is a reflection in this country with such raw data and to ask if it shows a pointer or trend. For myself there is a gut reaction that a major change in the doctrine on the expression of human sexuality will lead to a re-arrangement to the chairs on the titanic. There is an observation that groupings like the URC and parts of the Methodist seem to be struggling where such a doctrine has changed but I also acknowledge that Baptist ‘denomination’ is struggling in geographical areas and why? I am not sure if this can be proved by statistics or if anybody would be able to devise a methodology for such an investigation as the variables are so many. Do we have do to a Gamaliel? Sorry strayed form the point The whole issue of labelling is a minefield because of the cross- culture al difference and how some may fall into more than one label e.g. congregational, evangelical , Charismatic (in the mornings) and traditional hymn sandwich in the evenings;- it will only work with agreed definitions and that will take a council of the church!!

  5. Matt Frost
    Aug 31, 2014

    The last I was in the numbers on this issue, and reading real social science of religion articles, I recall noting that the decline in “mainline” denominations is on about a 5-year head start down the same exact track that the “conservatives” are following. And while most of our favorite measures for what churches are doing right or wrong have no significant statistical correlation, policy on birth control and desired family planning strategies is strongly correlated with the delay. Which is about what you’d expect, when the question is not theological but numerical.

    Still, both notional groups, which correlate reasonably well, are on the same massive downslope. And the question that should be asked is not what can be done to reverse this, or who is doing “better,” but: from what point? The answer to which is: the artificial peak of Christian-normative social culture in the post-War West, the US in particular. It didn’t exist prior, and it will not exist again in the future. And any statistician will tell you that if you model your ideals on outliers, you are setting yourself up for disappointment, because long-term trends normalize away from extremes.

    • steve
      Aug 31, 2014

      Thanks, Matt. I assume you’re talking about the US stats? In which case, I think this is right if you restrict the argument to historical denominations; new denominations (Vineyard…) and the rise of significant non-aligned churches (Willow Creek; Saddleback; Mars Hill; …) might shift the picture a bit; I don’t know.

      • Matt Frost
        Sep 2, 2014

        The data wasn’t done on any denominational basis, so I really don’t think historical or emergent makes a difference. Those “non-aligned” are clearly right-conservative-Evangelical in orientation, as are the Vineyard groups. And I don’t recall any of them espousing a positive view of birth control; the low-Prot emphasis on having children thrives especially among churches of no particular affiliation.

        And yet it still means nothing, if you look at the larger picture. The whole US picture is on a population normalization downward. (Frankly, I can’t imagine that the UK is doing anything radically different, but I haven’t worked through the numbers for your side of the pond, so that’s just a hunch. But you’re right, of course, to say that a simple comparison of US and UK data is problematic.)

        But as I re-read your post, I note that a clarification is needed that I left out: these numbers I’ve seen are for historically white churches. You don’t make a distinction, and I think there are good demographic reasons to make one. The churches you see experiencing wild growth are “Black” and Pentecostal, for example. When we live in societies that are both stratified and heterogenized by racial/ethnic and cultural differences, it isn’t necessarily meaningful to compare populations on the basis of political ideology.

        Over here, I can tell you that whatever the narrative of “decline,” the fact is that white-normative society is the demographic in which the historically white churches have hit saturation and begun normalizing downward. Growth is statistically stagnant because there aren’t that many more people “like us” to recruit. And historically white churches do not traditionally have significant success incorporating people of other cultures into existing congregations. Birth rates make all of the significant difference in relative position on the downslope.

        Churches in non-white demographic groups face different conditions, however, and likely have not experienced the kind of demographic saturation historically white churches have. Their growth is not therefore comparable to that of historically white churches. You couldn’t model it and “fix” a white church “problem” if you tried.

Leave a Reply to simon Russell Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

get facebook like button