Who’s worst?

October 12, 2012

The last several days have seen people in politics raise questions about the validity, or more generously the accuracy and consistency, of recent federal statistics on the unemployment rate, the number of people filing first claims for unemployment compensation, and related data.  Now what will happen, almost certainly, is that this most recent monthly data will be “restated” next month or the month thereafter, a continuous process with flash macroeconomic data in the United States.

Does this mean you cannot rely on US government data?  Almost certainly, yes.  But the US is not unique.  The US is probably just the best of a bad lot.  Consider The Economist’s recent discussions of data in China:

“With China so engaged in the global economy, there is a never-ending stream of data, often unreliable, to feed the appetites of economic-research firms, investment banks, hedge funds, short-sellers, political risk advisors, think tanks, consultancies and financial and military newsletters – not to mention legions of academics, journalists, diplomats and spies.”  Banyan, The Leader Vanishes, September 15, 2012.

“[N]o other important country is as murky [in terms of providing accurate, credible data] as China.” Schumpeter, The summer Davos Blues, September 15, 2012

That China is murky, with respect to data both at the government and company levels, does not excuse the way in which United States collects and processes econometric data.  However, for politicians, businesses, and others, to make decisions based on the movement of 1/10 or 1/100 of some monthly measure from US government statistics is also foolish.

There may be many iron rules about data, but for competitive intelligence, I would propose the following:

Data from only one source is not data.  It is conjecture – until it can be confirmed.

Data based on telephone surveys should be increasingly subject to question.  Our brethren in marketing have already come to this conclusion, given the demographics of the populations that have shifted from landlines to cell phone only service, especially when cell phone users are notoriously difficult to survey.

The smaller the sample is and the more quickly the data is collected, the more likely it is to be inaccurate, and inaccurate in an unpredictable manner.

Combining data sets that are individually unreliable does not necessarily make the conclusion more reliable.  I realize that there are those in the statistics world that would disagree with this, but I do not believe that such aggregations always contain the necessary mutually offsetting mistakes to generate a reliable whole.

Any data that has to be restated should not be relied on at all in the first place, or at least not until it is eventually restated.

Using short-term data to determine the presence and direction of a long-term trend is not forecasting; it is at best guessing and at worst irresponsible.

What is truly ironic that all of this is that the US government releases such statistics, upon which so many rely with so little reason, while it would never allow a firm going public,  such as Facebook[1], to get away with using similarly dubious data.

[1] Linda Sandler, Brian Womack and Douglas MacMillan, “Facebook Fought SEC to Keep Mobile Risks Hidden Before IPO”, Bloomberg, Oct. 10, 2012)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s