Sampling and Weighting Technical Report, Census of Population, 2021
5. Evaluation of the weighting procedures

As described in Chapter 4, the first step in weighting the long-form sample was to assign design weights to households. Weights were assigned differently depending on the collection method of the area where the household was located. Private households attached to a collective dwelling or part of a First Nations community, Métis settlement, Inuit region, or other remote area have a design weight of 1. The final weight for these private households corresponds to the design weight and remains at the initial value of 1. All the other private households have a design weight greater than 1. All of the results presented in this chapter were calculated for the subset of households with a design weight greater than 1.

In short, each household was assigned a design weight that was determined by the long-form sample design. Some adjustments were then necessary to address coverage and total non‑response. Non‑linear calibration was performed during this adjustment to estimate the parameters for non‑response models. After being adjusted for coverage and total non‑response, the weights were adjusted further in the final calibration process to produce the final weights. The final weights enabled generally better agreement between the census counts and the long-form estimates.

The next few sections examine the distribution of the weights and, for various characteristics, the discrepancies between the census counts and the sample.

5.1 Distribution of the weights

Chart 5.1.1 and tables 5.1.1 and 5.1.2 illustrate the distribution of the design weights, the weights adjusted for coverage and non‑response, and the weights adjusted in the final calibration. The weights are grouped by 0.5 length intervals, apart from the first and last intervals. The chart shows the percentage of times the weights appear in each interval. All the design weights ranged from 3.75 to 4.25. This is because of the long-form sample design, in which approximately one in four households received a long‑form questionnaire in most areas. The impact on the weights of the coverage and non‑response adjustments and the final calibration adjustments can also be seen. A very noticeable difference is shown in the 3.75 to 4.25 interval. In fact, 55% of households had their coverage and non‑response adjusted weights between 3.75 and 4.25 compared with 100% of design weights in this interval. After the final calibration adjustment only 30% of households had their final weights in this interval. The final weights were more evenly distributed across all categories compared with the design and coverage and non‑response adjusted weights.

Logically, the non‑response adjustment process should tend to increase the weights to compensate for the non‑responding units. This did occur for most cases. The changes between the design weights and the coverage and non‑response adjusted weights can be observed in Table 5.1.1. This table shows that most of the units that left the [3.75, 4.25) range moved to the [4.25, 4.75) or [4.75, 5.25) ranges. However, the coverage and non‑response adjustment process also moved some weights from the [3.75, 4.25) range to the [1.00, 2.75), [2.75, 3.25) or [3.25, 3.75) ranges for some units. The main reason is that the procedure included an adjustment for overcoverage and undercoverage. To the extent that a few population groups may have experienced overcoverage, the weights would have been reduced in those areas.

An important element to notice in Table 5.1.1 is that the non‑responding units originally had positive weights, since they were selected for the sample. The non‑response adjustment process assigned them a weight of 0 and redistributed their original weights among responding units. The non‑responding units correspond to the line labelled “0 (non‑respondents)” in Table 5.1.1 and were removed from Table 5.1.2, since they were not used in the calibration process. Table 5.1.2 presents the changes between the coverage and non‑response adjusted weights and the calibrated weights.

According to Table 5.1.2, most weights experienced only a small modification during the calibration process. In fact, 78.1% of cases either stayed in the same range or moved only one range up or down. The most stable range was 1.00 to 2.75, where 64% of the households with a coverage and non‑response adjusted weight between 1.00 and 2.75 stayed in that category after calibration. The second most stable category was 5.75 to 12.25 where 61.9% of households with a coverage and non‑response adjusted weight between 5.75 and 12.25 stayed in that category (although the calibration range goes up to 20.00 rather than 12.25).

Finally, whereas the coverage and non‑response adjusted weights varied from 1.00 to 12.25, the range of the final weights was from 1.00 to 20.00.

5.2 Discrepancies between census counts and long-form estimates, Canada

Chapter 4 describes the methods used to calculate the final household weights, and Section 5.1 shows some of the relationships between design weights, coverage and non‑response adjusted weights and final weights. The coverage and non‑response adjustment reduced the discrepancies between the census counts and the corresponding long-form estimates for the constraints considered (see Appendix C). Following those adjustments, calibration further reduced or eliminated those discrepancies for certain variables (constraints). However, some discrepancies remain, since constraints are sometimes excluded. The relative difference between census counts and long-form estimates, called the discrepancy, is defined as:

This ratio represents the percentage that the characteristic was overestimated (a positive value) or underestimated (a negative value). For comparison reasons, it is also useful to look at the absolute values of the discrepancy and difference, hereafter referred to as the absolute discrepancy and absolute difference, respectively.

Table 5.2.1 shows the 2021 Canada-level differences between census counts and long-form estimates for the constraints considered for the design weights, the coverage and non‑response adjusted weights and the final weights.

Table 5.2.1 also shows the discrepancy for estimates based on final weights. Looking at these discrepancies sheds more light on the differences. Over 94% of the cases in Table 5.2.1 had a discrepancy from -1% to 1%, and over 99.5% of them had discrepancies ranging from -5% to 5%.

Chart 5.2.2 shows, for all the constraints, the difference between the census counts and each of the three estimates: design weights (blue), coverage and non‑response adjusted weights (orange), and final weights (green). The x-axis represents the population size of the constraint, in thousands, and the three series of dots show for each constraint:

• the difference between the sum of the design weights and the census count
• the difference between the sum of the coverage and non‑response adjusted weights and the census count
• the difference between the sum of the final weights and the census count.

The constraints are sorted, from left to right, by increasing population size.

Chart 5.2.3 shows the percentage discrepancies between census counts and final estimates for all the constraints by population size. For the medium-sized and large-sized constraints, the discrepancies are all small. Only certain small-sized constraints have relatively large discrepancies.

The most important observation from Chart 5.2.2 is that the coverage and non‑response adjustment carries a big improvement over the design-weighted estimates, in terms of reducing the discrepancy. Although it is not apparent in the chart, the coverage adjustment does most of the job. The difference between census counts and long-form estimates for design weights tended to be (much) greater than the difference between census counts and long-form estimates for the coverage and non‑response adjusted weights. This, in turn, tended to be greater than the corresponding difference using the final weights. This shows the importance of the non‑response adjustment and calibration processes. A difference between the census count and long-form estimate could occur in a SADA or ADA for a characteristic if its constraint is excluded during calibration. In other words, the process did not control on the excluded constraint for a given area. If the constraint is excluded in many areas, these differences could partially cancel each other out, or they could cumulate to create a large difference at the Canada level. Total persons (TOTPERS) and total households (TOTHHLD) were the only mandatory constraints for which agreement between census counts and long-form estimates had to be guaranteed for all ADAs. As a result, the final weight difference and discrepancy for these characteristics were 0. However, all other constraints had to be excluded in some areas.

Appendix C along with Table 5.2.1 illustrates that constraints that were excluded frequently tended to exhibit high differences or discrepancies. Looking at the constraints defined only at the SADA level, there was a positive relationship between the number of times a constraint was excluded and the absolute difference between census counts and long-form estimates. The “Persons in a couple (married or common-law)” (COUPLE) constraint exhibited the largest absolute difference of 8,642 while being the fourth most excluded constraint having been excluded 627 times between the coverage and non‑response adjustment and final calibration. Lastly, among the SADA-only constraints with the top 10 largest relative differences, 7 constraints were also a part of the top 10 most excluded.

Across all possible constraints, the top 10 largest absolute differences were between 2,466 and 8,642. However, because the census counts were so high, the discrepancies for these constraints were small (ranging from -0.45% to 0.28%). The largest discrepancies were observed for some of the place of birth categories. Many place of birth categories are uncommon in Canada and are therefore frequently excluded during calibration. This resulted in some large absolute differences and particularly large absolute discrepancies.

• Previous
• Next