Magazine features that claim to list the nation’s healthiest cities should be viewed with some skepticism

“Ranking studies often attract media and government attention and are taken seriously, regardless of their designs and limitations,” say Sandra A. Ham, M.S., and colleagues in the American Journal of Public Health. “Nevertheless, controversies exist about whether ratings accurately reflect the ‘livability’ of cities and the extent to which such reports can be misleading.”

The ideal way to compute these rankings would be to start with environmental and behavioral measures weighted to account for their roles in producing health or illness, the researchers say.

But this ideal system doesn’t exist. Everything from geography to the interpretation of data may serve to undermine the credibility of the lists, Ham says.

For instance, health authorities gather some information at the state level, some only within cities, and some by metropolitan statistical area — a city and its surrounding counties.

“State-level averages may not adequately represent the health situation in any of the state’s cities,” she says, adding that combining published reports from various sources only obscures data. Rankings also oversimplify the complex interactions among behavior, environment, health conditions and population dynamics.

Exactly when health data was measured or how it is defined can make a difference too, she says. Data gathered on obesity in the early 1990s can’t be compared to material from the end of the decade when obesity prevalence shot up. The scientific meaning of “overweight,” “obesity” and “physical activity” have all changed in recent years, but both old and new statistical definitions are often lumped together.

Finally, while it’s easy to group the best or worst two or three cities using these indicators, she says, the scores of the in-between places are often statistically indistinguishable from each other, leaving the rankings for most towns in a meaningless limbo. Still, there can be some value in city rankings, despite their flaws.

“Ranking cities can play an important role in raising awareness of public health issues,” Ham says. “Such data also can be used to raise awareness among residents … and to hold government and the private sector accountable for doing what is necessary to keep residents healthy.”

However, readers ought to keep in mind the methodological limitations of these rankings, she says.

http://www.hbns.org/