One of the most important features of EMSI’s Canadian dataset is that it doesn’t come from a single source. Instead, it’s pulled together from nine different sources, so that we can use the different strengths of different data sources to help cancel out their different weaknesses. It’s the same process that’s made our data the gold standard for labour market information in the US.
But what happens if one of those sources starts to seem less than trustworthy? That’s been the story over the last few years in Canada, as both statistics experts and others have expressed reservations about the results of the 2011 National Household Survey.
What’s the Story with the Census?
A quick primer on the controversy: Statistics Canada, the federal agency responsible for collecting and distributing data about Canada’s economy, has traditionally collected a comprehensive set of census data every five years, most recently in 2011. Until the most recent census, this involved a short-form questionnaire, distributed to all Canadians, and a long-form questionnaire sent to a smaller, but still representative portion of the population. In 2006, for example, every fifth home received the long-form census.
In 2011, Statistics Canada kept the long-form/short-form distinction. But unlike 2006, when both types were mandatory, in 2011 the long-form questionnaire (the “National Household Survey”) was made voluntary. Obviously this made the data it returned significantly less reliable, and numerous experts immediately spoke out against the change. Statistics Canada itself predicted that the response rate would drop from 94% to 50%, and that there was a “substantial risk of non-response bias.”
Until the news about the National Household Survey broke, census data was widely considered the foundational source for Canadian labour market information. In the wake of the controversy, we’ve started hearing a lot of questions from Canadians about how EMSI is dealing with the presumably flawed data the NHS is providing. So how big of a problem is the NHS for EMSI? What is EMSI doing to deal with the problem? And does this mean EMSI’s Canada data isn’t accurate?
What Does This Mean for EMSI’s Data?
Even before the NHS became a widely known issue, EMSI’s data experts had made the decision not to rely heavily on data from the census (2006 numbers at the time). After all, while census data is very detailed, covering every geography in the nation down to census subdivisions, it’s only released every five years—and even when it is released it’s already two years behind. The economy moves too quickly for data released every five years, two years behind reality, to serve as a reliable guide.
Instead, our data experts made the decision to rely primarily on the Labour Force Survey and the Survey Of Employment, Payroll, and Hours to create reliable labour market data. The LFS and SEPH are released far more often and in a more timely manner than the census, which enables us to update Canada data on a biannual basis.
The SEPH and LFS, however, only offer us a limited amount of data at any level more detailed than province-by-province. To create a model that goes as deep as the census subdivision, we use province level data as a guideline, from which we can hang more detailed census data, as well as regularly updated figures from the Canadian Business Patterns. Since the census offers a picture of the general breakdown of industries in the detailed geographies of an area, we can extend that pattern across five years, adjusting it against the SEPH and LFS while using detailed data from the CBP to guide how we model the data.
In other words, every dataset has weaknesses, and every dataset has strengths. By comparing multiple overlapping sources, EMSI can eliminate the weaknesses and use the strengths to create a comprehensive dataset modeling the Canadian economy. We believe that our data offers the most reliable picture of the Canadian economy available, one that’s both easier to access and more accurate than the piecemeal data available from official sources. And it’s one that can help you ride out fluctuations in the source data like these changes in the census.
Analyst for Canada is EMSI’s online labour market research tool. For more information about Analyst for Canada, or to see data for your region, contact Josh Wright. Follow EMSI on Twitter @desktopecon