Pew asks Americans about God, then bungles the results

I’ve griped several times about the paucity of data about beliefs specifically related to (a)theism, rather than “religious affiliation” (which, outside of Christianity and a few other religions, is almost a a meaningless term). To my surprise and delight, a new report by Pew Research Center seemed to be the answer to my evil atheist prayers. Except… no.

Being a bit of a data geek, I’ve had a love/hate relationship with the work of Pew Research Center’s Religion & Public Life project for years. When they do good work, they do very good work. When they don’t… well….

Their most recent report is: When Americans Say They Believe in God, What Do They Mean? Pew being Pew, their focus was on Christian beliefs, of course. But there was still some fascinating information in there about American atheists. Unfortunately, it was not easy to suss out because Pew bungled the data grouping in their report.

Before going any further, I have to stress that this is American data, not Canadian. American data tends to be similar to Canadian data… but very skewed toward religiosity. There is basically no-one doing this kind of research in Canada, unfortunately. So we have to make do.

Things start out pretty well. The first question in the survey was simply: “Do you believe in God, or not?” This isn’t perfect; if it were up to me, I would have asked about “gods” in general, not “God”. Asking specifically about “God” creates ambiguity and wiggle room; a non-Christian theist may decide that “God” refers specifically to the Abrahamic god and say “no”, despite believing in Vishnu or Odin or whatever. Or they may not. Ambiguity.

On the other hand, this is a survey of Americans, which means that everyone responding is either a Christian of some flavour (including unaffiliated Christians, and people who have essentially Christian beliefs but refuse the label), a Jew, or an atheist. The number of people in every other religion totalled is roughly the same as the number of Jews. And in the survey, the number of Jews (the only religion other than Christianity they bothered to consider) was only around half the number of explicitly self-identifying atheists (not even counting agnostics or anything else). So we can pretty well hand-wave away anyone who might believe in gods but not answer “yes” to believing in “God”.

Here’s where things get a little weird though.

So Pew asks the question: “Do you believe in God, or not?”, then depending on the answer, you get different options in the follow-up question.

Let’s ignore all the “no answer” options, because all told, they sum up to 1% or less of the total, well below the error threshold.

So if you answer “yes”, that you believe in God, it asks you to choose between:

• I believe in God as described in the Bible
• I DO NOT believe in God as described in the Bible, but I do believe there is some other
  higher power or spiritual force in the universe

If you answer “no”, it asks you to choose between:

• I DO NOT believe in God as described in the Bible, but I do believe there is some other
  higher power or spiritual force in the universe
• I do not believe there is ANY higher power or spiritual force in the universe

Note the common option. We’ll be getting back to that.

Now, if you were interested in atheists – and assuming you knew the definition of “atheist” – the set of respondents you would be interested in is:

That is, everyone who doesn’t answer “yes” to the question “do you believe in God” (remember: assuming “God” refers to gods in general) is an atheist. That includes people who believe in a “higher power” or “spiritual force”. Someone who believes in no gods but does believe in karma – or who considers “nature” to be a force that mindlessly maintains balance in the universe through rational physical laws – is still an atheist.

That’s even reflected in Pew’s own results. When focusing just in the “Nones”, this is what they found:

18% of explicitly self-identified atheists believe in a “higher power” or “spiritual force”. None believe in the god of the Bible, of course.

Also worth noting is that 28% of the religiously unaffiliated that don’t call themselves “atheist” or “agnostic” believe not just in a god… but specifically in the god of the Bible. This is why I say the terms “None” and “religiously unaffiliated” are useless terms.

It would also have been nice if there had been some distinction made between gods other than “the god of the Bible” and non-theistic “higher powers” or “spiritual forces”. Without that distinction, it is impossible to know how many agnostics are actually atheists. 62% believe in something (other than the Biblical god), but how many of that is other gods? Given Pew’s questions, we can’t know.

Anyway, let’s get back to Pew’s main mistake.

So they asked, “do you believe in God?” and got 80% yes, 19% no. So 19% of Americans (±2.3% 19 times out of 20) are atheists. Cool. But Pew was interested in Americans’ beliefs about gods. So the sensible thing to do, then, is to leave the “no, I don’t believe in God” people separate as their own group, and then split the “yes, I believe in God” into “Biblical god” and “other gods”. That way you have three groups: Biblical god believers, other god believers, and no-god “believers” (atheists). You can ask the first two groups what they believe about their own gods and learn a lot about Americans’ conceptions of the gods they believe. And then for completeness you could ask the atheists the same questions, and learn what they think other people believe about their gods. Also interesting, but in a different way.

Instead what Pew did was bizarre. The took the Biblical god believers as described above… but then they lumped the other god believers in with the atheists who believe in a higher power/spiritual force. Those who don’t believe in any gods or higher powers became the third group. You can see this in their own graphic below:

I can’t for the life of me figure out what sense there is in doing this. You’re asking about the characteristics of gods… why would you want to use this grouping:

1. People who believe in the Biblical god.
2. A group of: 2⁄3 people who believe in other gods and 1⁄3 people who believe in no gods but do believe in non-theistic “higher powers” or “spiritual forces”.
3. People who don’t believe in any gods and don’t believe in any “higher power” or “spiritual force” either.

rather than this grouping:

1. People who believe in the Biblical god.
2. People who believe in other gods.
3. People who don’t believe in gods.

And to be clear, this was a completely unnecessary mistake. They have the data for both sets of groupings. I cannot fathom why anyone thought the first grouping makes more sense for asking questions about gods’ characteristics.

Unfortunately, there’s nothing we can really do about it. This kind of thing is a very good illustration of why we need to finance our own surveys and studies. We need more data being collected that recognizes atheists and atheistic belief, rather than yet another study on Christians and Christian belief.

For all its weirdness, the Pew report does give us some fascinating data (on Americans, unfortunately). If we hand-wave the question about “God” away to being a question about gods in general, we learn that around 19% of Americans are atheist. We also learn that at least 27% of “Nones” are atheists – likely much higher (we’d need Pew to separate the two different “higher power” groups to get a better number). We can make a very rough guesstimate that ~45% of “Nones” are atheists.

There’s also some wacky stuff in there, like that only 80% of Christians actually believe in the Biblical god. Catholics are the least likely: not only do only 69% of them believe in the Biblical god, 2% of them are actually atheist.

And then there’s the usual, unsurprising stuff: younger and more educated Americans are more likely to be atheist and so on.

It is an interesting report to read to get an idea of what Americans actually believe about their gods – it asks lots of questions about things like omnipotence, judgment, and how much God meddles in daily affairs. It’s just unfortunate Pew didn’t care about anything but Christian theistic beliefs. There was so much more we could have learned.