As is normal, interest.co.nz kindly reproduced the last Raving in which I aired my concern about the quality of some official statistics.
Statistics NZ (SNZ) provided a thoughtful response to interest.co.nz (appended) and a senior manager from SNZ contacted me for a friendly discussion that I appreciated.
However, the discussion with the SNZ official left me concerned about judgements made at SNZ.
The review of the relationship between reported GDP and employment growth in this Raving reinforces my suspicions about the accuracy of especially the labour market data but possibly also the GDP data.
It is possible the source or sources of inaccuracy are outside the control of SNZ while a budget constraint may be part of the problem. Preparing official statistics in a small country with budget constraints isn't an easy task. But whatever the cause(s) I recommend that great caution is used in interpreting reported employment growth.
This casts a shadow over the accuracy of the reported unemployment rate while I am somewhat wary of the accuracy of reported GDP growth and I am inclined to conclude that something needs to be done to improve the quality of decision making at SNZ.
What is wrong with the picture painted by this chart?
If I were to tell you that economic activity grew by 2.7% and that employment grew by 5.8% as SNZ has reported for 2016 or that GDP grew 2.8% while employment fell 1.7% as reported by SNZ for 2012 you would be forgiven for thinking that I was telling porkies. But according to the official numbers released by SNZ that is what happened as shown in the chart.
Allowing for sampling errors that create random variation in the reported numbers there used to be a reasonably sensible relationship between the reported GDP and employment growth prior to 2012 as shown in the chart. The highest correlation between the two at 0.76 - akin to a 76% mark in an exam - was with GDP growth leading employment growth by one quarter. Reflecting this, the red GDP growth line in the chart has been advanced or shifted to the right by one quarter. This fits with what I believe should be expected (i.e. stronger economic growth being followed roughly one quarter later by higher employment growth and weaker GDP growth being followed roughly one quarter later by lower employment growth).
However, since 2012 the peak correlation, still with GDP growth leading employment growth by one quarter, is only 0.46 that can be roughly interpreted as a 46% mark in an exam (i.e. a fail mark). The green boxed area in the chart highlights the period since 2012 when reported employment growth has at times been radically out of line with what should have been expected given GDP growth. In my opinion this raises the most doubt over the accuracy of reported employment growth but my gut feeling, aided by my analysis of the key drivers of economic growth that is contained in our monthly economic reports, is that reported GDP growth may also be off the mark.
Between 1992 and 2011 reported GDP growth averaged 3% per annum while reported employment growth averaged 1.9%. It makes sense that GDP growth runs around one percentage point above employment growth reflecting the contribution of productivity growth to GDP growth. Over this period there were rare exceptions in which reported employment growth was stronger than reported GDP growth that may reflect sampling errors as much as anything. By contrast, in the last three years reported GDP growth has averaged 3% per annum while reported employment growth has averaged 3.5%. To me as an economist with 30 years' experience this just doesn't make sense. Last year this was the result of SNZ including two lost groups of employees for the first time and not making an adjustment to include them on the historical numbers meaning reported employment growth significantly overstated actual growth as discussed in the last Raving - more on this below. Maybe part of the problem is underreporting of actual GDP growth.
In my assessment reported employment growth for 2013 and 2014 should be completely ignored. I severely doubt that employment fell 1.7% in 2013 just to rebound 4.8% in 2014 while over this period GDP growth remained relatively stable. It seems to be pretty clear that the problem is mainly with reported employment growth that may taint the reported unemployment rate but it is still possible there are problems with the accuracy of reported GDP growth.
I see no defence for knowingly reporting inaccurate employment growth
In my opinion SNZ made a major error in judgement in reporting 5.8% employment growth for 2016 when it knew actual growth was weaker. The reported super-charged growth was the result of including two groups of previously overlooked employees in the official numbers starting in the 2016 June quarter and not revising the earlier data to incorporate estimates for these two groups.
Based on what I was told by the SNZ official who kindly rang the quarterly labour force survey didn’t provide the information needed to estimate the impact of the inclusion of the two groups of previously overlooked employees. Consequently, SNZ didn't have a sound basis for grossing up the earlier employment numbers to incorporate estimates for the lost employees. Maybe this was a good judgement but to me there is no defence for knowingly reporting hugely inaccurate employment growth.
On reflection after the discussion with the SNZ official it seems to me the sensible solution was to run the old survey alongside the new survey for at least one quarter. If this had been done it would have been possible to estimate the impact of the inclusion of the previously overlooked employees. This obviously would involve more money and comes back to my point that one of the issues over the quality of some of New Zealand's official economic data may be budget constraints.
As I suggested in the last Raving, maybe SNZ should be a beneficiary of the increase in government spending to be announced in the May Budget; subject to it being targeted at and contingent on increased quality in at least the major economic data (e.g. GDP, employment growth, unemployment rate). But budget constraint or not, as a long time user of official economic data I believe it was a major error of judgement on the part of SNZ for knowingly reporting inaccurate employment growth for 2016. Equally, I struggle to understand why red flags weren't raised at SNZ over the disparities between reported employment and GDP growth in 2013 and 2014 (i.e. something should have been done to rectify the inaccuracies). Consequently, I am inclined to conclude that something needs to be done to improve the quality of decision making at SNZ.
---
Statistics New Zealand's response to the last Raving:
Statistics New Zealand agrees with Interest.co columnist Rodney Dickens that it is important official figures are robust - they must also be impartial and represent our best understanding at the time.
Dickens’ column (Opinion February 28, 2017) discussed the accuracy of official numbers, and how revisions and errors can affect the confidence of our user community.
Stats NZ welcomes feedback of this nature, and the opportunity it provides for us to improve.
We agree that official statistics are vital for decision-makers from the Reserve Bank and government policymakers, to investors of all kinds, whether they are putting money in the bank or setting up a corner dairy.
As Dickens points out there have been errors with some figures recently. Stats NZ is transparent about errors, often found by ourselves - we own up to these errors and make changes as quickly as possible.
In the case of revisions, these are usually driven by new or improved data that affect historical results, or changes to the underlying measure – in both cases we aim to better represent the real life situation of a changing society, economy and environment.
Where possible, we try to give our users as much notice as possible about the planned revisions. We explain the changes and even in some cases publish a revised back-series before new revised data comes out.
In both the case of error corrections and revisions, our goal is to get the most accurate data out to everyone.
Stats NZ welcomes close scrutiny of official figures - it is in everyone’s interest to make sure they are accurate. We welcome questions from economists, media or anyone else to help them understand the data.
Rachael Milicich
General Manager Products, Services & Insights | Ngā Hanga, Ratonga, Kitenga
Customer Strategy & Delivery | Rautaki Kiritaki me te Whakaratonga
Statistics New Zealand | Tatauranga Aotearoa (www.stats.govt.nz)
*Rodney Dickens is the managing director and chief research officer of Strategic Risk Analysis Limited.