Guest post by Sebastian Thaci
Somebody recently pointed me to this “comprehensive and in-depth new public opinion poll” on religion in Canada conducted by the Angus Reid Institute:
I have read all the study carefully, and I have to reject its results based on the selection bias and conceptual ambiguity.
The study was conducted “among a representative randomized sample of 3041 Canadian adults who are members of the Angus Reid Forum.” This means that the results of this study apply to the members of this online forum, which are not representative of the whole Canadian society. Only computer-literate people, with computers, Internet access, free time and interest in the subjects of the forum are members of the forum. Out of these, only some responded to the survey. Out of those, apparently, only some were considered “representative” (why?).
This is a huge selection bias. How did they go from this select group to “Religion and Faith in Canada”? A randomized phone survey would have had more statistical power.
For example, how many 55+ year old first generation immigrants are members of this forum, and how many of them responded to the survey? How many fundamentalist/conservative religious people do you think are on this forum and also took the survey?
There are also multiple conceptual problems or problems with the questions posed. Just a few examples
- The categories of embracing and rejecting religion do not make sense. Some deeply religious people could reject the present religious institutions because of perceived corruption or other reasons. Are they embracing or rejecting religion? Also, an atheist might go through the moves of a religion just to keep his minority community and culture alive, like many atheist Jewish people are indeed doing.
- The congregation attendance evaluation is subjective, and people can be wrong about this, or they can lie.
- The question “What’s right or wrong is a matter of personal opinion” – many atheists and irreligious persons will say no to this, but there is no alternative except the Ten Commandments. Very wrong. How about the third option of an innate human sense of morality developed during our evolution? Even some religious people who believe in evolution will choose this last option.
- Spiritual or religious – two dictionary definitions do not explain the meaning of these words. What do people really understand by ‘being spiritual’? Asked this question, I myself do not know what to answer. Am I spiritual if I light scented sticks and do Yoga meditation in the morning? Or am I spiritual because I read my horoscope every day?
- There is no ‘conversion’ question – I myself came to Canada an evangelical Christian, now I am an atheist. Interestingly, I rejected institutionalized religion in both cases. When I was a Christian, I thought that the institutionalized corruption of the Church can only be demolished by grass roots, small groups Christianity. Now, I reject any form of religion. According to this poll, I was not embracing religion back then as a Christian, and I am not embracing it now as an atheist. But my mind has completely changed. I take different decisions and vote vastly different now. Also, there are no questions in the study about how people come to be in any category: converted, born into a religion, etc. No question on how free they think they are to move into another category or how likely. We keep ignoring the fluidity of religious social reality. And we also ignore that people are sometimes harassed or even die for leaving a religion or other.
- The immigration factor is not studied enough. There are many variables there, and Canada receives a LOT of religious immigrants, some refugees, some voluntary. Some of them come here to lose their religion, though, or at least to loosen its grip a little. Also: “With greater immigration from Asian countries in particular, the greatest increases have been among Roman Catholics, evangelical Protestants, and other major world faith groups.” Really? I doubt that we receive mostly Roman Catholics and evangelicals from Asia. I may be wrong, but I suspect something is off here.
What is interesting about this study is our reaction to it – the results are already familiar to us. We consider the numbers close to what we knew about religion/irreligion in Canada from previous polls and the last Census of Canada. We can go straight to assuming that the study is well done, and that we can cogently comment on the results. The numbers can be intelligently discussed, as if they are right, just because they sound right, but that does not make the numbers right, unfortunately. Not for the whole of Canada.
How did the science of public opinion survey and statistics become so weak, subjective and muddy?
Will anybody in the Canadian public at large be aware of the shortcomings of this poll? And does the public even care about such details anymore?
Maybe we should do another Angus Reid forum poll to find out. That’s what Angus Reid would do!