3. Population coverage error

3.1 Sources

Although census data collection and processing have to meet high quality standards, it is impossible to eliminate all errors. There are two kinds of population coverage error. Population undercoverage refers to the exclusion of persons who should have been enumerated, and population overcoverage refers to the inclusion of persons who were enumerated more than once (generally twice).

Undercoverage can occur in the first stage of the census if the list of dwellings used for the dwelling universe is incomplete. This risk is higher, for example, if a dwelling is under construction. Conversely, overcoverage can occur if a dwelling is listed twice or if a dwelling under construction that is not yet habitable is included.

Coverage error can also occur during the field data collection stage. Respondent error is responsible for coverage error when the person completing the census form either excludes or omits someone whose usual place of residence, according to census rules, is the dwelling concerned; this is undercoverage. The person may also include someone whose usual place of residence is not the dwelling concerned; there is overcoverage if this person has already been enumerated at his/her usual place of residence or somewhere else. In most cases, it is easy to determine a person's usual place of residence. However, as stated in the previous section, the process is sometimes more complex, and special rules have been developed for determining an individual's usual place of residence. The rules are spelled out in the census questionnaire, but the list is long, and there can be comprehension difficulties. Coverage error may result when the rules are not consulted or are incorrectly applied. The idea of using Census Day as the reference date for determining usual residence may also be misunderstood, which can lead to coverage error.

Coverage errors may also be committed during the processing stage at any point where records for persons or households are added to or removed from the census database. Records can be deleted by mistake. Questionnaires may be linked to the wrong record or returned too late to be included.

Even though efforts are made to enumerate the homeless population, the risk of undercoverage is high. Some other living arrangements are also susceptible to coverage error. For example, young adults newly away from home may be either undercovered, because neither their roommates nor their parents include them in the census questionnaire, or overcovered, because they are included in both census questionnaires. Persons who maintain a second residence because of their employment can also cause coverage error.

Users should also be aware of the extent to which Indian reserves and Indian settlements participated in the 2011 Census. In some cases, enumeration was not permitted by the community or was interrupted before it could be completed. In other cases, the quality of the enumeration was considered inadequate. In addition, natural disasters prevented enumeration in other communities at the time of the census (for northern Ontario communities hit by natural disasters, however, enumeration was carried out later, and the results were made available to the communities concerned). These geographic areas (31 in all) are considered incompletely enumerated Indian reserves and settlements. There are no 2011 data for incompletely enumerated Indian reserves and settlements, and those areas are not included in the totals. Similar problems have occurred in previous censuses. For example, 30 Indian reserves and settlements were declared incompletely enumerated in the 2001 Census, and 22 in the 2006 Census. Nine of those reserves and settlements participated in the 2011 Census. For more information, see Section 12.2.

The population estimates for the 31 incompletely enumerated Indian reserves and settlements are based on a model. However, since no reliable source is available to verify the assumptions in the model, the estimates must be used with caution. For more information, see Section 12.2.

Top of Page

3.2 Control

Potential sources of coverage error were recognized during the planning stage of the 2011 Census, and the following measures were taken to minimize the associated risks:

• Collection unit (CU) boundaries were carefully defined and mapped to ensure that no geographic areas were left out or included twice.
• List/leave areas: The enumerator's manual contained instructions on how to canvass a CU so as to minimize the risk of missing dwellings. The total number of dwellings from the 2006 Census was provided to the field operations manager to help him/her identify significant changes. In addition, when the listing operation resulted in a substantial difference in the number of dwellings relative to the 2006 Census, the listing was checked. Lastly, specific quality control procedures were applied to the CU to evaluate and correct any changes made in the listing. Section 4.2 provides a definition of list/leave and mail-out areas.
• Mail-out areas: Mail-out was based on a list of addresses from Statistics Canada's Address Register. This list was updated regularly and listing activities were carried out mainly in the fastest-growing areas. These listing activities were carried out continuously, but more intensively in the two years preceding the census. The work of enumerators was closely monitored. Some collective dwellings had to be checked by field staff to verify their occupancy status before the collection stage; if they were occupied then they were identified and included in the census.
• Special procedures were developed for the enumeration of persons who have difficulty responding (e.g., people who are fluent in neither English nor French or are illiterate) and persons located in specific parts of large cities where response or coverage was poor in the past.
• Special procedures were developed for enumerating the population on Indian reserves.
• Advertisements informed Canadians about the census and indicated what to do if they did not receive a questionnaire.
• The Census Help Line (CHL) was available to answer any questions about the census, including questions about coverage.
• There was a 'Whom to include' section in the questionnaire to tell respondents which persons should be included.
• The questionnaire had a question on whether there was anyone the respondent was not sure should be listed. A telephone follow-up was subsequently carried out with the respondent to determine if the person in question should or should not be listed in the questionnaire.
• Telephone follow-up was carried out after questionnaires were reviewed for coverage inconsistencies or to verify household status, including questionnaires containing only foreign or temporary residents.
• Non-response follow-up included a dwelling coverage check.

These procedures, along with appropriate training, supervisory checks and quality control systems during the collection and processing stages, helped to reduce the number of coverage errors.

Top of Page

3.3 Definitions

Algebraic definitions of coverage errors are presented in this section. Let $T$ denote the total or the 'actual' number of persons in the target population. Let $C$ denote the published census count of persons in the target population. The error associated with using $C$ instead of $T$ is as follows:

$N=T-C$

This error, denoted as $N$, is the net population coverage error.

Let $U$ denote population undercoverage, the number of persons not included in $C$ who should have been.

Let $O$ denote population overcoverage, the number of excess enumerations included in $C$ that should not have been. $O$ has two components. One is the excess enumerations of persons enumerated more than once. Coverage studies focus on these excess enumerations that should not have been included in $C$. The other component of $O$ is persons included in $C$ who are not members of the census target population. For example, foreign residents visiting Canada who are listed on a census questionnaire as usual residents of a dwelling should not be included in $C$. Fictitious persons are another example. According to previous studies, the number of persons who are enumerated but are not in the census target population is generally very small and can be ignored. Consequently, the 2011 Census coverage studies did not measure this component of coverage error.

Since $U$ refers to persons who were not enumerated but should be included in $C$ and since $O$ denotes enumerations that should not be included in $C$, the difference between $T$ and $C$ is $U$ less $O$. That is,

$N=U-O$

The actual number of persons in the census target population is therefore

$T=C+N=C+U-O$

An estimate of $T$ is given by $\widehat{T}$, where

$\widehat{T}=C+\widehat{N}=C+\widehat{U}-\widehat{O}$

$\widehat{U}$ is an estimate of the number of persons not included in $C$ who should have been, and $\widehat{O}$ is an estimate of the number of persons included in $C$ who should not have been. We can assume that overcoverage from persons included in $C$ who are not in the census target population is zero, since it is negligible. Consequently, $\widehat{O}$ is simply an estimate of the number of duplicate enumerations. The purpose of census coverage studies is to determine the values of $\widehat{U}$ and $\widehat{O}$.

Census population coverage errors can generally be expressed as rates relative to the actual population. The undercoverage rate $R_U$ is $U$ as a percentage of $T$. The overcoverage rate $R_O$ is $O$ as a percentage of $T$. The net undercoverage rate $R_N$ is the difference between $U$ and $O$ as a percentage of the census target population. These three rates can be estimated by ${\widehat{R}}_U$, ${\widehat{R}}_O$ and ${\widehat{R}}_N$, as follows:

${\widehat{R}}_U=100*\frac{\widehat{U}}{\widehat{T}}=100*\frac{\widehat{U}}{C+\widehat{N}}$

${\widehat{R}}_O=100*\frac{\widehat{O}}{\widehat{T}}=100*\frac{\widehat{O}}{C+\widehat{N}}$

${\widehat{R}}_N=100*\frac{\widehat{N}}{\widehat{T}}=100*\left(\frac{\widehat{U}-\widehat{O}}{C+\widehat{N}}\right)$

A positive net undercoverage rate indicates that the undercoverage rate is higher than the overcoverage rate. That is, there are more people not included in the published census count $C$ than excess enumerations. That is generally the case for all Canadian censuses. For some domains of interest, however, negative net undercoverage has recently been observed.

As indicated above, $U$ denotes the number of persons not included in $C$ who should have been. The census count $C$ is composed of two elements:

$C=E+I$

where

$E=$ the number of enumerations. This is the number of people who were listed on a census questionnaire.

$I=$ the number of imputed persons. This is an estimate of the number of persons missed in non-response dwellings and in occupied dwellings misclassified as unoccupied. For more information on whole household imputation (WHI), see Section 5.7.

Hence, undercoverage is a subset of all persons who were not listed on a census questionnaire but should have been. It does not include persons who were not enumerated either because no completed census questionnaire was returned for the dwelling (non-response dwelling) or because the dwelling was misclassified as unoccupied (classification error) and did not receive a questionnaire.

In summary, the actual population $T$ is composed of the census count $C$ and the net undercoverage $N$. $C$ consists of $E$ plus the number of persons added in WHI $I$, and imputation is done for persons living in non-response dwellings or in occupied dwellings misclassified as unoccupied. $N$ is undercoverage $U$ less overcoverage $O$.

Top of Page

3.4 Measurement

Two postcensal studies were carried out to estimate the 2011 Census population coverage error. The Reverse Record Check (RRC) provided estimates for population undercoverage, while the Census Overcoverage Study (COS) estimated population overcoverage.

The RRC and the COS were conducted after field collection and processing were complete. Preliminary estimates of 2011 Census population coverage error were released on March 28, 2013. Following a lengthy and detailed validation exercise with the Demography Division and the provincial and territorial statistical focal points, final estimates were released on September 26, 2013. The data were released at the same time as the new official population estimates reflecting the update of the base population to the 2011 Census. Census population counts adjusted for net population undercoverage constituted the updated estimates of the base population.

A brief description of the methodology used in the two census coverage studies is presented below:

Reverse Record Check (RRC)

In the RRC, a random sample of individuals representing the 2011 Census target population was selected from frames independent of the census, such as the list of persons enumerated in the 2006 Census and a list of intercensal births according to provincial birth registries. The 2011 RRC sample consisted of 67,840 persons in the provinces and 1,926 persons in the territories. The 2011 Census database was searched to determine whether the persons selected in the sample had indeed been enumerated.

Where necessary, interviews were conducted, mostly via computer-assisted telephone interviewing (CATI) from the regional offices (ROs), to collect further information for use in additional searches of the 2011 Census database. An interview was completed for 79.5% of the 16,955 cases sent to the ROs. The sampling weight was adjusted for non-response. The total sampling weight of non-respondents was divided among a group of respondents most like the non-respondents in their tendency to respond.

The estimate of population undercoverage is based on the number of persons in the RRC sample who were classified as 'missed.' These persons were in scope for the 2011 Census, but no evidence of enumeration could be found in the 2011 Census Response Database. Nationally, 4,745 persons in the RRC sample were classified as missed in the provinces and 901 in the territories.

Census Overcoverage Study (COS)

Overcoverage was measured by matching the final 2011 Census database to a partial list of persons who should have been enumerated according to administrative data sources, and then matching the persons not matched in the 2011 Census database to the database itself. In other words, the COS carried out one probabilistic linkage with administrative sources and another with the census database. Probabilistic linkage identifies matches that are close but not exact. A sample of pairs of potential duplicates was selected for each linkage, and demographic characteristics and names were examined to identify cases of overcoverage.