Common sense required when analyzing any data
One of the many negative side effects of the intensely partisan environment that exists at all levels in our country is our increasing inability to solve problems based on statistical evidence.
An important way to diagnose problems, at least when it comes to local government, is to compare relative statistics from other jurisdictions and determine how your organization stacks up against the group.
In general, the larger the group or sample from which the data is drawn increases the reliability of the conclusions that can be made.
It’s not much different than seeing your doctor for a check-up. Physicians made the great leap from using leeches and telling their patients to smoke a couple of Chesterfields after each meal, in part, through the same process. They learned, from collecting and analyzing enough data, that the temperature of a healthy human being is 98.6 degrees. A patient with a significantly higher or lower temperature is cause for concern and prompts a process that looks at many other factors until a diagnosis is made.
Unfortunately, the same process that has solved so many scientific problems is being abandoned when it comes to solving the kind of problems faced by governments. When statistics are released that point to problems or even successes, the first reaction, by many groups that disagree with the conclusion, is to accuse the particular researcher of bias.
In defense of this behavior, a host of so-called think tanks have emerged on both ends of the political spectrum. Virtually all of the studies they conduct, or least publicize, confirm the group’s long-held ideologies. Political polling probably represents the high-water mark or, perhaps more accurately the low point, of where the issue stands today.
If proper statistical methods are being applied, it really shouldn’t matter whether Rush Limbaugh or Rachel Maddaw conducted the poll; yet it does. The first question most people now ask, after a poll is released showing that Candidate A is ahead of Candidate B, is who conducted the survey. Conservatives dismiss liberal pollsters and vice versa.
An increasing number of people are dismissing the usefulness of statistics altogether. While we can live without knowing who will win the upcoming election (we will find out after the election), we may not be able to live well if we abandon the idea that good public sector decision-making involves giving serious consideration to statistical evidence of what works and doesn’t work in other places.
The usefulness of statistics is responsible, in no small measure, for its demise in public sector decision-making. Applying a scientific framework to governments makes intuitive sense. With thousands of units of government in Wisconsin alone, and tens of thousands across the country, why reinvent the wheel when it comes to developing solutions to problems we will face? By studying the myriad programs already in existence, we can find ones that are the most cost-effective and produce the best results and then simply replicate them.
Proponents of programs and ideas, some of which may produce less than stellar results, learned two things over time: Statistics sell, and few people have the knowledge or time to look at them critically. As a result, while the use of statistics to justify policy decisions has increased over time, the quality of the methodology has declined.
It isn’t reasonable to expect elected officials to become statisticians, but part of their due diligence should be to make sure that staff members, who are trained in statistical analysis, have reviewed the methodology used in any study and can comment on its reliability. Moreover, it is in the power of any layperson to apply his or her own common sense to the data that is being presented to them and determine whether survey questions were fair or whether a reasonable sample size was used.
If you’re interested in seeing how Wisconsin stacks up statistically with the rest of the 50 states, I would encourage you to check out Wisconsin’s Scorecard. Compiled by the nonpartisan Legislative Reference Bureau, the Scorecard is a treasure trove of information for trivia aficionados and policymakers alike. The report compares Wisconsin to the rest of the country in 48 separate dimensions within the broad categories of demographics, health and education, and government finance.
I felt better about our state after reading it. Unemployment here is lower than the national average. While personal income was slightly lower than average, poverty and income inequality (the disparity between high and low wage earners) were significantly lower than the national average. Taxes in Wisconsin, as we often hear, were higher than the national average (16th), but both our high school graduation rates and eighth-grade test scores were higher, as well (12th in both categories).
The data suggest that solving the issue of higher taxes might not necessarily mean cutting public workers. Despite our relatively high tax burden, Wisconsin ranked 40th in terms of the number of workers on state and local payrolls.
If I made one point in this column, however, don’t take my word for any of this. You can find the full report, including the methodology behind these numbers, at the Legislative Reference Bureau's website, http://legis.wisconsin.gov/lrb/.