New report highlights inherent weaknesses of VA's Airborne Hazards and Open Burn Pit Registry

Inherent features of registries that rely on voluntary participation and self-reported information make them fundamentally unsuitable for determining whether emissions from military burn pits in Iraq, Afghanistan, and other locations in Southwest Asia caused health problems in service members who were exposed to them, says a new congressionally mandated report from the National Academies of Sciences, Engineering, and Medicine. While the U.S. Department of Veterans Affairs' Airborne Hazards and Open Burn Pit (AH&OBP) Registry provides a forum for collecting and recording information on those who choose to participate, a more rigorous and appropriate approach is needed to examine the relationship between the exposures and health outcomes, such as a well-designed epidemiologic study. A previous report by the former Institute of Medicine [now part of the National Academies] found inconclusive evidence on the health effects of exposure to military burn pits and contained advice and recommendations on how a study might be conducted.

For some time, the disposal of trash on military bases through open-air burn pits exposed service personnel deployed in Iraq, Afghanistan, and other locations in Southwest Asia to airborne particulate matter and other potential health hazards, which in turn raised concerns about acute and chronic health consequences in these individuals. In 2013, Congress gave the VA one year to create a registry that would acquire exposure and health information on service members and veterans who may have been exposed to airborne hazards during deployment -- such as smoke from burn pits, oil-well fires, dust storms, or pollution. The VA developed an ambitious program to enroll volunteer participants and created the AH&OBP Registry. In response to a congressional mandate, the VA asked the National Academies to analyze the initial months of data collected by the registry and offer recommendations on ways to improve the instrument and the information it collects.

Registries are structured systems for collecting and maintaining data on a group of people characterized by a specific disease, condition, exposure, or event as a means to facilitate research, monitor health, or provide information to registrants. Registries that rely on voluntary involvement and self-reported information on exposures and health outcomes are not suitable for assessing the health effects of exposure due to respondents' selective participation, inaccurate recall, or inadvertent or intentional under- or overestimation. Thus, they are an intrinsically poor source of information on exposures, health outcomes, and possible associations among these events, said the committee that carried out the study and wrote the report. Even under the best of circumstances, there are substantial limits to the accuracy of the data they collect and -- when the respondents make up only a small, unrepresentative fraction of the eligible population -- to the generalizability of analyses made with them. The committee stressed that even a well-designed and executed registry would have little value as a scientific tool for health-effects research and would not be an effective substitute for an epidemiologic study.

The committee concluded that given the inherent weaknesses, the best use of the AH&OBP Registry is as a means for the eligible population to document their concerns of health problems that may have resulted from their service, bring those concerns to the attention of VA and their health care providers, and supply VA with a roster of people who are interested in burn pit exposure issues. The gathered data could also potentially be used to stimulate and inform new research, such as a well-designed epidemiologic study. The committee recommended that VA's messaging be explicit about the capabilities and limitations of the registry to ensure that participants and others do not form unrealistic expectations.

The registry questionnaire collects a number of pieces of information -- including self-reported signs, symptoms, and diseases -- that can alert providers to concerns and problems that may be forgotten or missed during clinical encounters, without regard to whether the information might be relevant to wartime exposures. For example, someone who reported difficulty walking long distances or climbing stairs might be experiencing joint pain, respiratory problems, vascular disease with congestive heart fail¬ure, obesity, or even anxiety. The committee recommended that the VA develop a concise version of a participant's responses that can be downloaded and discussed with a health care provider.

If the VA chooses to use the registry for the purposes for which is it most suitable, the committee recommended several changes to the questionnaire to improve and streamline participation. Beyond the inherent weaknesses of voluntary, self-report registries, the committee identified correctable features in the registry's structure and operation as well as in the questions that are asked and the way they are asked. For example, the questionnaire fails to ask questions that could yield improved information related to relevant exposures, such as non-burn pit trash burning, and requires respondents to complete a sometimes lengthy set of repetitive questions regarding deployments before addressing core issues like health. The cumulative effect of these flaws is evidenced by the high percentage of respondents who initiated but did not complete the instrument and the number of questions that had high non-response rates. The VA should eliminate the questionnaire sections that collect information that cannot be productively used in studies appropriate for the registry's data.

A 2015 VA report indicated that nearly 40 percent of those who began filling out an AH&OBP Registry questionnaire did not complete it. This is an outcome that should be examined more fully, the committee said. It recommended that the VA evaluate whether and how registrants who did not complete the questionnaire differ from those who did; analyze the determinants of non-completion; and use this information to formulate strategies that encourage registrants to finish and submit their responses as well as improve the completion rate for future participants. An examination of the questionnaire data made available to the committee showed that registry participants who reported more exposures of all types also tended to report more health problems of all types. However, the committee's analyses suggested that such results may be a consequence of the population's selection and the limitations of the self-reported exposure and disease data and do not provide useful information for assessing the health effects of exposure. The committee emphasized that it reached this determination based on the nature of the registry and the data it collects and would have drawn the same conclusion if no or weak associations between the exposures and health outcomes had been found.

More generally, the AH&OBP Registry's data collection, administration, and management efforts could be improved by developing a plan to more fully integrate relevant VA and U.S. Department of Defense data sources with the registry's data, as well as offering alternative means of completing the questionnaire, such as a mail-in form or via a computer-assisted phone interview, to ensure that the subset of eligible persons who do not use or have difficulty using the Internet have the opportunity to participate in the registry.