Global Catholicism: Aggregating data to estimate the number of Catholics at the national level on a global scale

In a previous post, Catholic population data reported in the OCD were compared to survey-based estimates at the state-level. This is replicated here at the national-level for 70 countries where survey data are available. Survey estimates are compared to the Catholic Church’s official population reports in the Annuarium Statisticum Ecclesiae (ASE) 2005.

The surveys provide estimates of the percentage of adult respondents in two large international surveys who self-identify themselves as Catholic (see the previous post regarding the use of adult surveys to estimate total population). These surveys include the most recently released wave of the World Values Survey (WVS) for 2005 to 2008, the most recent wave of the Comparative Study of Electoral Systems (CSES) for 2002 to 2006 and the International Social Survey Programme (ISSP) for 2004 to 2006.

The survey-based estimates are then applied to total mid-year population estimates for each country in 2005 from the U.S. Census Bureau’s International Data base (IDB). In cases where there are estimates from all three surveys an average of these are used. Although imperfect, these methods are the best available for providing global comparisons.

Overall, these surveys estimate that there were 769.4 million Catholics residing in the 70 countries in 2005 for which data are available.  In these same 70 countries, the Catholic Church had estimated that there were 883.7 million Catholics (79 percent of the Church’s estimate of all Catholics globally). Thus, the Church’s estimated population for these countries is 15% higher than the aggregated survey estimate.

However, as shown in the tables above, the surveys estimate 0.0% Catholic self-identity in some of the countries where Catholics are known to reside in significant numbers (e.g., there are Catholic parishes and parishioners). These include Egypt, India, Indonesia, Iran, Kyrgyzstan, Morocco, Thailand, and Turkey. The Church reports that a total of 25.3 million Catholics reside in these countries—mostly in India (18.1 million) and Indonesia (6.5 million). Catholics are estimated to make up less than 3% of the population in both of these very large population countries. If one removes these 25.3 million from the Catholic Church’s Catholic population total to reflect the survey estimates the Church’s total Catholic population is 11.6% higher than the survey population estimate (858.4 million compared to 769.4 million).

Based on the survey estimates, the Catholic Church is over-estimating Catholic population in Europe and the Americas to a similar degree (15% in Europe and 16% in the Americas).  The Church is likely under-estimating the Catholic population of Africa and the Middle East combined (-29%). In Brazil, the largest Catholic population country in the world (regardless of the estimation method), the surveys indicate the Church is over-estimating the number of Catholics residing there by 29 million.

Measuring Up: Aggregating data to estimate the number of Catholics at the state level

How many Catholics live in your state?

A recent CARA analysis of data from multiple sources indicates that in most states there may be more Catholics than the Catholic Church is aware of. 

Each year in The Official Catholic Directory (OCD), dioceses report a variety of statistics—including their estimates of the total Catholic population.  The 2009 OCD is intended to be representative of the “status of the Catholic Church as of January 1, 2009.” At CARA, we interpret these data to be reflective of totals in 2008, when these data are collected.

Recently some very large surveys regarding religious identification have been conducted which allow for population estimates at the state level. The first of these was conducted in 2007 by the Pew Forum on Religion & Public Life. The U.S. Religious Landscape Survey included interviews with a representative sample of 35,556 adults in the United States. The second source used here is American Religious Identification Survey (ARIS) conducted by researchers at Trinity College in 2008.  This survey included interviews with a representative sample of 54,461 adults in the United States.  Surveys with such large sample sizes allow for state level estimates of religious identification that are not possible with a typical academic or media poll that may only interview 1,000 respondents nationally.

Both of these large surveys estimate the proportion of the adult population (age 18 and older) that self-identifies their religion as Catholic.  These surveys likely underestimate the Catholic percentage of the total population—including children—as Catholics in the United States are, on average, younger than the non-Catholics (and more likely to be of childbearing age) and Catholics have a higher fertility rate than non-Catholics (both of these demographic factors are related to immigration).  Thus, the percentage of the population under age 18 in the United States that is Catholic is likely greater than the percentage of the U.S. adult population that is Catholic. 

In addition to these surveys, estimates are available from the Association of Statisticians of American Religious Bodies (ASARB).  This organization produces county-level estimates for religious adherents of all ages that coincide with each U.S. Census. Adherents represent any members of the faith, regardless of attendance at services, confirmation, etc.  The most recent estimates from ASARB represent the year 2000. Due to the timing of this data collection, the ASARB estimates, in some cases, misrepresent Catholic population percentages due to changes that have occurred in the last nine years—primarily any significant Catholic mobility (moving from one state to another) and any significant Catholic immigration (Catholics coming to the United States from other countries).

Despite the methodological and timing differences between these four estimates of the Catholic population percentage (i.e. including the OCD), each is strongly correlated to all others at the state level (Pearson’s R >= .915).  

CARA has aggregated these Catholic population estimates by simply averaging them (Note: not all states have estimates from all four sources). This is represented as "Average" in the tables below. We use this average to estimate the total size of the Catholic population in each state based on U.S. Census Bureau estimates of state population size in 2008. This is represented by "Catholic Population Estimate" in the tables below (i.e., multiplying the "Average" by the "Total Population" and creating the estimates in the "Catholic Population Estimate" column. Note: the "Average" includes additional decimal places not shown in the table. Thus, multiplying the table figures for "Total Population" by "Average" will produce slightly different estimates of "Catholic Population Estimate" than what is shown in the table. The complete data are available upon request). 

Using this method, the total estimated size of the self-identified Catholic population (i.e., adults and children) in the United States in 2008 was approximately 70.5 million (by comparison the OCD estimated this to be 65.2 million for the 50 states and Washington D.C.).  This represents 23.3% of the total U.S. population of 303.2 million in that year.

The table below includes states where the Catholic population percentage estimated from the OCD is relatively consistent with estimates made from the surveys and ASARB. 

The next table includes states where the Catholic population percentage estimated from the OCD is either significantly underestimated or overestimated in comparison to the estimates made from the surveys and ASARB.  This table also includes those states where no OCD estimate is available due to diocesan boundaries (e.g., Arizona and New Mexico, Delaware and Maryland).

The outlier states where dioceses have likely underestimated the size of their Catholic population are: Mississippi, Florida, Maine, North Carolina, Tennessee.  In each of these states the size of the Catholic population is estimated to be more than 50% larger than what is reported in the OCD based on the most recent survey estimates. Florida in particular is estimated to have more than 1.4 million more Catholics than what is reported in the OCD (2,255,891 reported in the OCD compared to CARA's estimate of 3,729,817).

The only dramatic outlier state where dioceses have likely overestimated the size of their Catholic population is Nevada where the estimated size of the Catholic population is 25% smaller than what is reported in the OCD based on the most recent survey estimates.

Even the aggregated estimates shown above, for a variety of reasons, likely underestimate the Catholic population in general. As noted, ASARB is based on data collected in 2000 and the figures reported by dioceses in the OCD are often conservative and most consistent with ASARB estimates. The surveys by Pew and ARIS include only adults. Finally, one of the most difficult populations to survey and count in the United States are immigrants. Many recent immigrants to the United States have come from countries in Latin America and Asia that have large Catholic populations. Immigrants in general are known to be less likely to agree to be interviewed in surveys and may also be under-sampled due to the lack of a landline phone (i.e., mobile phone-only households). Researchers using Census Bureau surveys often make adjustments for this "undercount" (see Pew's, A Portrait of Unauthorized Immigrants in the United States). To the degree that Catholic immigrants are missed in surveys, the Catholic population percentages estimated using these data are by definition underestimated.

The Nuances of Accurately Measuring Mass Attendance

How many Catholics are at Mass on a weekly basis? This percentage varies depending on if one is interested in those who attend every week and those who are at Mass on any given weekend.  There are also important differences in how one conducts the poll that generates these numbers.

CARA has conducted 19 CARA Catholic Polls (CCP), national surveys of adult self-identified Catholics, since 2000. Some of these have been by telephone and others have been conducted online using Knowledge Networks national panel. There are important and interesting differences between the results of Mass attendance questions from CARA’s online self-administered polls and CARA's telephone polls that are strongly related to the effects of the presence of an interviewer.

These differences are not limited to Mass attendance and generally are observable for any socially desirable activity from financial giving and frequency of prayer and confession. CARA’s self-administered surveys consistently show lower levels of Mass attendance than what is exhibited typically in CARA’s telephone polls.  Survey researchers have long understood that the personal interaction between interviewer and respondent can create over-reports of certain activities (such as voting or giving to charity) that are considered socially desirable.  Responses to questions regarding attendance at religious services are known to be biased toward estimates higher than actual attendance as measured by head counts.   Below we show varying estimates of Catholic weekly attendance using telephone polls and other methods such as head counts and time diaries.

CARA's self-administered surveys use methods that do not require a human interviewer--respondents are answering questions as they appear on-screen (either through a computer or on their television using MSN TV).  Self-administered surveys, such as this, are known to reduce over-reports for questions that have socially desirable response sets (encompassing attitudes people believe they “should” have or behaviors they feel they “should” do), producing results that more closely reflect actual behavior than estimates derived from telephone polls.  

The estimated percentage of Catholics attending Mass every week using the responses to CARA's self-administered surveys are more consistent with what is found in estimates of the attendance of Catholics derived from other methods, such as sample-based head counts and time diary studies.  As the figure below shows, results from 12 CARA telephone surveys and seven CARA-Knowledge networks surveys (using self-administered methods), produce no statistically significant changes in weekly Mass attendance between 2000 and 2008 by either method of polling.  All variations are within the sampling margin of error.  The difference between the two methods of polling is consistently about 8 to 14 percentage points for those who say they attend weekly or more often.  On average, in CARA's self-administered surveys 22% to 23% percent of adult self-identified Catholics say they attend Mass on a weekly basis (i.e. every week).

Some other surveys, such as those conducted by Gallup, ask about religious service attendance in any given week (e.g., the last seven days).  In the table below, we convert the responses from the CARA question and estimate the percentage of Catholics that attend Mass in any given week rather than every week.  This is estimated to be 31.4%.

By chance one might expect about 2% of those who say they “rarely or never” attend Mass to have attended Mass in any given seven-day period (odds of 1 in 52).  If one takes the 32% of Catholics responding in this manner and multiplies it by 2%, one can estimate that 0.6% of Catholic Mass attendance in any given week is made up of those who say they "rarely or never" attend Mass.  This same calculation can be done for each category of responses that indicate less than weekly attendance. 

Gallup estimates that 45% of Catholics attend Mass in any given week. CARA estimates this to be about 13 percentage points lower.  This is consistent with expectations as Gallup polls use an interviewer over the phone and are thus influenced by social desirability bias.

Note: Knowledge Networks panel has been shown to be representative to well within 1 percentage point to the U.S. Census Current Population Survey (CPS) demographics for gender, age, race and ethnicity, education, and region.  See: Baker et al. (2003), “Validity of the Survey of Health and Internet and Knowledge Networks Panel and Sampling,” Stanford University and Krosnick and Chiat Chang (2001), “A Comparison of Random Digit Dialing Telephone Survey Methodology with Internet Survey Methodology as Implemented by Knowledge Networks and Harris Interactive,” Ohio State University. The panel is updated on a quarterly basis and those persons who are sampled and asked to join the Knowledge Networks panel receive subsidized Internet access and other incentives.  For those who do not own computers, Knowledge Networks provides a television-based Internet system (MSN TV) for free.  These steps ensure that the Knowledge Networks panel is as reflective as possible of the national population and that it is not biased towards those who have pre-existing access to the Internet.

For references or more information on this topic see: 

Mark Chaves and James C. Cavendish. 1994. "More Evidence on U.S. Catholic Church Attendance." Journal for the Scientific Study of Religion. 33: pp. 376-381.
Stanley Presser and Linda Stinson. 1998. "Data Collection Mode and Social Desirability Bias in Self-Reported Religious Attendance." American Sociological Review. 63: pp. 137-145.
C. Kirk Hadaway and Penny Long Marler. 2005. "How Many Americans Attend Worship Each Week? An Alternative Approach to Measurement." Journal for the Scientific Study of Religion. 44: pp. 307-322.