First Real-Time Flu Forecast Successful

Scientists were able to reliably predict the timing of the 2012-2013 influenza season up to nine weeks in advance of its peak. The first large-scale demonstration of the flu forecasting system by scientists at Columbia University’s Mailman School of Public Health was carried out in 108 cities across the United States.

Results are published online in the journal Nature Communications.

The flu forecasting system adapts techniques used in modern weather prediction to turn real-time, Web-based estimates of influenza infection into local forecasts of the seasonal peak by locality. Influenza activity peaked in cities in the southeast as early as December 2012, but crested in most of the country in the first weeks of 2013.

Year to year, the flu season is highly variable. It can happen anywhere from December to April. But when it arrives, cities can go from practically no cases to thousands in a very short time. “Having greater advance warning of the timing and intensity of influenza outbreaks could prevent a portion of these influenza infections by providing actionable information to officials and the general public,” says first author Jeffrey Shaman, PhD, assistant professor of Environmental Health Sciences at Columbia University’s Mailman School of Public Health. 

For the public, the flu forecast could promote greater vaccination, the exercise of care around people sneezing and coughing, and a better awareness of personal health.  For health officials, it could inform decisions on how many vaccines and antiviral drugs to stockpile, and in the case of a virulent outbreak, whether other measures, like closing schools, are necessary.

Study Results

The new study builds on the researchers’ 2012 study that used the system to retrospectively predict the peak of the flu in New York City for the years 2003-2008. That research was limited to one city and performed as a test of the system. The current study is the first to make predictions in actual real-time and for the whole country.

Beginning in late November of 2012, the researchers used the flu forecasting system to perform weekly estimates for 108 cities. They shared the results with the CDC and posted them online in an academic archive. Near the end of 2012, four weeks into the flu season, the system had predicted 63% of cities accurately. As the season progressed, the accuracy increased. By week four, it successfully predicted the seasonal peak in 70% of the country. It was able to give accurate lead-times up to nine weeks in advance of the peak; most lead-times were two to four weeks.

The flu forecasts were also much more reliable than those made using alternate, approaches that rely on historical data. “Our method greatly outperformed these alternate schemes,” says Dr. Shaman.

The researchers saw regional differences in the accuracy of the system, but they were likely within normal variation. “As an example, retrospectively, we’ve been able to predict the flu in Chicago very well; this year we did a terrible job in that city. For other cities, the opposite held.  It averages out. On the whole the system performed very well,” Dr. Shaman says. However, there were hints of geographical differences. “We were able make better predictions in smaller cities.  Population density may also be important. It suggests that in a city like New York, we may need to predict at a finer granularity, perhaps at the borough level. In a big sprawling city like Los Angeles, we may need to predict influenza at the level of individual neighborhoods.”

Google Flu Trends Goes “Off the Rails”

The researchers designed the flu forecasting system to use combined data from 1) Google Flu Trends, which makes estimates of outbreaks based on the number of flu-related search queries, and 2) region-specific reports from the Centers for Disease Control on verified cases of flu. The system approach is analogous to weather forecasting, which employs real-time observational data to reduce model forecasts error. In the last year, the researchers slightly modified the system to be more representative of flu rather than flu and other respiratory problems. Nevertheless, there was an unusual level of “noise” in the data related to problems with Google Flu Trends.

How did this happen? One explanation is the high number of media stories about the flu, including some about the flu forecasting system itself. The result was a spike in people using Google to research the flu, which could have overloaded the Flu Trends algorithm. It’s an irony not lost on Dr. Shaman. “There was a tremendous amount of media attention accorded to the flu last year. I was part of the problem myself,” he says. Another factor may have been the particular strain of flu in circulation. “The flu was very virulent and was making people very sick, more so than previous seasons,” says Dr. Shaman. Again this could have led to a spike in flu-related Google search queries.  (In October, Google announced that it has revised the Flu Trends, which Dr. Shaman hopes will make flu forecasting more accurate.)

The system will be put back in action as soon as the flu season begins again. “Right now there are few cases of the flu, but as soon as the needle starts to move, we will start making predictions,” says Dr. Shaman. This season the forecasts will be more readily available to the public on a website hosted by Columbia’s Mailman School of Public Health expected to launch in the coming weeks.

Worldwide, influenza kills an estimated 250,000 to 500,000 people each year, according to the World Health Organization. In the U.S. 3,000-49,000 die from the flu every year, and about 45% of Americans were vaccinated for the flu, according to the CDC.

Co-authors include Wan Yang and James Tamerius, post-doctoral students of Dr. Shaman (Dr. Tamerius is currently at the University of Iowa); Alicia Karspeck at the National Center for Atmospheric Research; and Marc Lipsitch at the Harvard School of Public Health.

Funding was provided by the National Institutes of Health (GM100467, ES009089) and the Department of Homeland Security. Dr. Lipsitch discloses consulting or honorarium income from the Avian/Pandemic Flu Registry (Outcomes Sciences; funded in part by Roche), AIR Worldwide, Pfizer and Novartis. All other authors declare no competing financial interests.