Some 8.3 percent of California teens smoke cigarettes. No, wait, make that 4.5 percent. Or is it really 14.2 percent?
It can be hard for public health researchers, much less the public, to know what numbers to believe when surveys come up with such varying results. Epidemiologists are often faced with this challenge - as are politicians or any group that relies upon polls, questionnaires and surveys about people's behavior or beliefs.
A researcher at the University of California, Berkeley, highlights this problem in a new study assessing two different survey methods designed to estimate the prevalence of teen smoking in California. The paper, to be published later this month by Oxford University Press in the Winter 2004 issue of the journal Public Opinion Quarterly, tested a relatively new telephone survey method and compared it with an existing one. The researcher found the two methods yielded significantly different results.
Both methods involved interviews with adolescents 12-17 years of age in California in a survey conducted by the Gallup Organization between April 5 and July 6, 2000. Telephone interviewers contacted a random sample of households and asked permission to interview any adolescents at home. Half of the adolescents were interview, and the remaining half completed the survey using the standard interviewer-administered method. More than 2,400 interviews were completed. The overall survey response rate was 49 percent.
In the telephone computer-assisted self-interviewing (T-ACASI) method, participants listened to pre-recorded, computer-controlled questions and responded by pressing the keypad on a touch-tone telephone. In the computer-assisted telephone interviewing (CATI) method, interviewers asked the questions and entered responses into a computer. The questions were the same in both surveys.
The automated T-ACASI survey resulted in an estimate of 8.3 percent of teens who reported smoking in the prior 30 days. In comparison, the CATI survey yielded a significantly lower estimate, 4.5 percent, of current teen smokers. Both those figures are lower than the 14.2 percent prevalence found in a school-based survey of California teens conducted in 2000.
School-based surveys are the most common tools used to estimate behavior among adolescents and children, said Joel Moskowitz, director of UC Berkeley's Center for Family and Community Health and author of the study. But school surveys miss many high-risk youth who are often not in class and may be more likely to smoke or do drugs.
Still, the students who are in school for a survey may feel more open about reporting high-risk behavior there than at home because they are in a setting with peers, said the report.
Notably, 59 percent of respondents in the CATI survey reported that a parent could hear all or part of the interview, compared with 42 percent of respondents in the T-ACASI survey. The perception of a lack of privacy could have led to lower estimates of smoking in both telephone surveys, said Moskowitz.
Yet phone interviews have increased in popularity over the years as nearly all households in the United States now have phones, according to the report. Calling homes may reach some of the high-risk youth that school-based surveys do not. Phone surveys also tend to be more cost-efficient.
At the same time, response rates for phone surveys are on the decline with the advent of answering machines and telemarketing as people screen calls to their home phones, said Moskowitz. In addition more and more people have cell phones and are becoming less dependent on their land lines. When people do respond, their answers vary depending upon the survey method used.
"In the phone survey where respondents spoke with a live interviewer, they may have underreported their smoking behavior because of a perceived lack of confidentiality, even though the two survey groups were equally anonymous," said Moskowitz. "The youths may have felt more comfortable revealing their smoking habits to a computer rather than a person."