Nineteen Sixty-four is a research blog for the Center for Applied Research in the Apostolate (CARA) at Georgetown University edited by Mark M. Gray. CARA is a non-profit research center that conducts social scientific studies about the Catholic Church. Founded in 1964, CARA has three major dimensions to its mission: to increase the Catholic Church's self understanding; to serve the applied research needs of Church decision-makers; and to advance scholarly research on religion, particularly Catholicism. Follow CARA on Twitter at: caracatholic.


Contrasting Portraits?

The Pew Research Center recently released results of a new survey of 1,016 self-identified Catholic adults in the United States. Polls with Catholic samples of this size are not all that common. The survey includes some questions that provide an opportunity to evaluate changes in the Catholic population related to sacraments and marriage. These questions are somewhat parallel to questions used by CARA in surveys conducted in 2007 and 2008. Comparing and contrasting these surveys, two conducted when Pope Benedict XVI was leading the Church and one now with Pope Francis at the helm also offers another possible search for the ever elusive “Francis effects.”

At the same time there is some caution in making these comparisons. First, the question wording does not always match exactly which can make surveys, already blurry portraits of reality a bit more fuzzy. Then there are the methods. Pew surveys by telephone with an interviewer. CARA surveys using Knowledge Networks (now GfK Custom Research) where respondents take interviews on screens (computers, smart phones, tablets, televisions) without a human interviewer. The presence of an interviewer can sometimes lead to social desirability bias where people try to look like the best citizen and say they go to church more often, give to charity, and vote. Pew notes these limitations in their data on page 40 of their report. As you will see this really only appears to affect one question. It is also the case that all the summaries here exclude “don’t know” responses (based on the toplines at the end of the report) and include only on those respondents who self-identified their religion as Catholic at the time of the survey (totals may add to 99 or 101 due to rounding error).

If one literally compared the Mass attendance results of the two surveys it would appear to provide the strongest confirmation yet for a “Francis Effect.” Weekly Mass attendance is up 16 percentage points?! Sorry, no. As Pew notes in footnote 6 of their report, their estimate is likely inflated due to the telephone interview. Sometimes CARA “corrects” telephone survey estimates of Mass attendance by reducing the weekly attenders by 12 percentage points. This correction factor is based on substantial, consistent research at CARA using both telephone and self-administered surveys.

If that is the case, Pew’s weekly estimate might best reflect a weekly attendance rate of about 27%. This is indeed higher than 23% in 2008 (both estimates relatively consistent with other methods of Mass attendance measurement, such as time diaries and head count studies). Yet here again, the blurriness of survey data needs to be addressed. All surveys have margins of sampling error. Once this is considered here we cannot definitively say that Mass attendance is any different now than in 2008. This fits the trend measured in CARA’s national polls since 2000.

Turning to what happens at Mass, only 58% of Mass attending Catholics said they “always” took Communion at Mass in 2008. Pew estimates that even fewer, 46% receive Communion “every time” when they attend Mass in 2015. Does this represent a real decline? Again, a bit hard to know. Pew asked their question to all Catholics who attend Mass at least occasionally. I have restricted the analysis of the CARA data to the most comparable group—those attending at least a few times a year. My hunch is that the Pew respondents include some Catholics who go “occasionally” but that this may be less than once a year. There is certainly no Francis Effect with this question and any possible social desirability bias would be expected to inflate the Pew data rather than make it smaller. It is likely that Catholic Mass attendance has been steady, but slightly fewer are presenting themselves to receive Communion now that in 2008.

Perhaps people are receiving Communion at Mass less often because they feel the need to go to confession first? It is the case that Catholics rarely went to confession in 2008 and apparently still don’t in 2015. Again, margin of error means there is little difference between 3% and 8% saying they go at least once a month.

Yet, 37% of adult Catholics who attend Mass at least occasionally went to confession at least once a year in 2008. Forty-six percent reporting doing so in 2015. That difference is beyond margins of sampling error. There are two ways of looking at this. Either Catholics are sinning more or they sin at the same rate as they used to and are now more likely to go to confession at least once a year. This could be an effect of the greater effort by U.S. dioceses—especially during Lent—to let Catholics know through media ads the “lights are on.” Or perhaps it is the pastoral approach of Pope Francis that encourages some to think that going to confession will not be the judgmental encounter they imagine. One final possibility is again that Pew’s subgroup of “occasional” Mass attenders is different from CARA’s who attend Mass at least once a year.

How important one’s potential last confession is also an important marker of faith. When one is ill they can ask a priest to anoint them. If they are gravely ill they are also offered confession and Communion as “Viaticum (food for the journey) given at the end of life.”

As priests will often tell you, even Catholics (and their families) who have been away from the Church for quite some time will still urgently seek out this sacrament when they are close to death. In 2008, CARA found that 90% of Catholic adults said receiving this sacrament was important to them (at least “a little” to “very”). Pew, with a dichotomous response set, found that 85% of Catholics in 2015 say this is important to them. Again, with margin of error there is no discernible difference here.

No sacrament has been in greater decline than marriage. Looking at the Church’s annual sacrament totals it is clear that fewer marriages are celebrated in the Church in the United States over time. How does this trend represent itself in polling data? The results in the table below are for adult Catholics who were married when surveyed. There is no statistically significant difference between the results in 2007 and 2015. About two-thirds of married Catholics have wed in the Church. An additional one in 20 do not marry in the Church but have their marriage later blessed or convalidated by the Church. About a quarter or more do not marry in the Church nor seek convalidation.

Sometimes marriage ends in divorce. One of the most misunderstood realities is that divorce is not a sin. For example, the Church does not compel Catholics to stay with an abusive spouse and you’ve certainly committed no sin if your spouse leaves you! However, if you marry or partner after divorce (regardless of the circumstances) without seeking and receiving an annulment then the Church would see you as living in a state of sin for as long as those circumstances remain. You would be expected not to present yourself for receiving Communion. Both CARA and Pew asked Catholics who had ever divorced if they had sought an annulment. In 2007, 15% said yes, they had sought this. Pew’s 2015 survey estimates that 26% of Catholics who have ever divorced have sought an annulment at some point.

We do know that the number of annulment cases opened in the United States has been in a long-term decline. An increasing share of ever divorced Catholics reporting an annulment would be puzzling. However, again notice a slight difference in question wording. CARA’s survey asks if the respondent has or is seeking an annulment. The Pew poll asks if the respondent or their spouse has sought an annulment. This is a broader universe and may explain why Pew finds a larger share—even as annulment cases have declined. It is also the case that Catholics are less likely to divorce than others in the United States, which makes the population of “ever divorced” Catholics a smaller share of respondents than one might expect. Margins of error for this sub-group are quite high and even without the question wording difference, there may not be any shift at all between 2007 and 2015.

The more things change, the more they stay the same…

Note: In a previous post we noted some concerns about Pew’s telephone polls coming up with estimates for the size of the U.S. Catholic population that were slightly lower, on average, than other major surveys. It happens here again with the 2015 Pew survey (i.e., 20% of U.S. adults). Our hunch in the past has been that there could be a problem with their sampling and/or response among Hispanic or Latino respondents in Pew surveys. Out of 1,016 interviews with Catholics in the current poll, only 277 were completed with a respondent that self-identified their race or ethnicity as Hispanic. Overall, the Pew survey includes interviews with 621 Hispanic respondents (out of a total of 5,122 interviews). The Pew survey includes too few interviews with Hispanic or Latinos in general and the Catholic affiliation percentage among this group is also lower than what other major surveys would typically estimate. This is not a huge problem when one considers the blurriness of survey research, as is done here. But when so many others assume surveys are accurate to a tenth of a percentage point (when they should not!) it is important to note how estimates can vary. David Gibson does an excellent job pointing this out by reviewing all the Pope Francis approval estimates.

Search This Blog

Blog Archive

© 2009-2021 CARA, Mark M. Gray. Background image courtesy of muohace_dc.