Council Bluffs can feel good about itself being in the same metro as the 7th best 'big city'.
Sat, Jul. 08, 2006
Pop science surveys are experts' nightmare
SURVEY SAYS: Rankings you read about in the media are nothing but bunk -- and the stuff marketers' dreams are made of.
BY BOB SHAW
ST. PAUL PIONEER PRESS
ST. PAUL - Surveys show that nine out of 10 people don't know what "gullible" means. OK, that's not true. But every year, millions of Americans are bamboozled by surveys -- a growing industry that marketers love but statisticians say is, well, for the gullible.
Surveys say the weather in Rosemount, Minn., is colder than in St. Paul. Surveys say Minneapolis is America's best city for sleeping -- at odds with a finding that it's the most fun city in America.
Surveys say Lakeville, Minn., is America's fourth-best place to raise children; Hopkins, Minn., is the cleanest place in Minnesota; and Madison is the most polite city.
"These surveys are fun. It doesn't matter if they aren't scientifically valid," said Joan Stewart, owner of Publicity Hound, a Wisconsin firm that tells companies how to get free publicity with their do-it-yourself surveys.
"What matters is that the media love them."
The media -- including this newspaper -- have a history of snatching up surveys that rate communities, whether based on science or chicanery. The surveys drive nine out of 10 experts absolutely bonkers.
"I am firmly convinced this is a field gone crazy," said George John, the Pillsbury-Gerot chairman in marketing at the University of Minnesota Carlson School of Management.
Lists abound claiming to have discovered entire cities that are the most romantic, beautiful, sexy, fun, family-oriented, gay-friendly or Hispanic-friendly places in America.
Most surveys have elements of truth. As a rule, they pluck a few strands of data from the U.S. census -- but use it to reach conclusions that experts say verge on ridiculous.
Thomas Guterbock scoffed at a survey saying Minneapolis was America's best place to sleep.
"What? Do you have a lot of long, cold nights there?" said Guterbock, the director of the Center for Survey Research at the University of Virginia.
Experts say the media -- and readers -- end up manipulated by marketing wonks. Stewart, who prides herself on creating surveys that get noticed, said the point of the surveys is not to tell the truth. The goal is to get a client's name published across the country for little or no money.
It just requires a little imagination, Stewart said. Suppose you worked for the North American Vexillological Association, the one-stop shop for all things vexillological. If you wanted America to know you dealt with flags, you might rank flags according to their beauty.
The group did that, and the results were published nationwide -- showing that Minnesota's wretched banner was 67th out of 72 state and territorial flags.
"When will these surveys stop being effective? When the media stops reporting on them," Stewart said.
That won't happen soon, said marketing expert John. The news media gobble up such surveys because the public loves them.
Mammals, including humans, form groups that depend on a status hierarchy -- which makes any list of rankings irresistible.
"There is a fundamental human need to know where we are relative to others," he said.
Yet surveys can leave cities whipsawed by fast-changing results. In 1994, readers of Money magazine might have concluded that the Twin Cities became unlivable in one year, dropping from third to 32nd in its "Best Places to Live" survey. Pity poor Bremerton, Wash., which plummeted from first place to No. 116 in three years.
Guterbock proved in 1996 that rapidly changing economic data -- cost of housing, employment, income -- were over-weighted.
"It would be better to call it 'Best Places to Invest in a Shopping Center' than 'Best Places to Live,' " Guterbock said.
Sometimes, experts said, statistics that make up rankings vary so slightly that they become meaningless. The Money Web site said Hopkins is the cleanest city in Minnesota, in terms of pollution. Can it be true that Stillwater is fifth-cleanest while Woodbury -- right beside it -- is 17th?
The same Web site said the coldest city in Minnesota is -- drum roll, please -- Rosemount. It edged out Duluth by 0.4 degrees for average low temperature in January. Andover, Anoka and Champlin are tied for next coldest.
Only in the fine print is it explained that the survey includes only cities over 14,000 with "above-average income, population growth and real estate appreciation during the past five years."
Users browsing this forum: No registered users and 0 guests