Analysis Questions Flu Shot Effectiveness

Oct 28, 2011
Originally published on October 28, 2011 12:32 pm
Copyright 2018 NPR. To see more, visit http://www.npr.org/.


This is SCIENCE FRIDAY. I'm John Dankosky, filling in for Ira Flatow. A new analysis out this week says the seasonal flu shot may not be working as well as we'd like to think. Writing in the journal "Lancet Infectious Diseases," a group of researchers says evidence for protection in adults age 65 or over is lacking.

For younger adults aged 18 to 65, there's evidence that the vaccine gives some protection, but it varies from year to year. The researchers say that some seasons' protection is greatly reduced or even absent. So why the big push to get a flu shot every year? The CDC still says that everyone over 6 months needs to get vaccinated.

So is there any harm in getting a vaccine that might not even help you? That's what we'll be talking about in this first hour. If you want to get in on the conversation, you can give me a call. Our number is 1-800-989-8255. That's 1-800-989-TALK. If you're on Twitter, you can tweet us your questions by writing the @ sign followed by scifri.

If you want more information on what we'll be talking about this hour, go to our website at www.sciencefriday.com, where you will find links to our topic.

Let me introduce our guests. Michael Osterholm is the director of the Center for Infectious Disease Research and Policy, and a professor of environmental health sciences, at the University of Minnesota's School of Public Health. He joins us from the studios of Minnesota Public Radio in St. Paul. Michael Osterholm, welcome back to SCIENCE FRIDAY.

MICHAEL OSTERHOLM: Thank you very much.

DANKOSKY: William Schaffner is president of the National Foundation for Infectious Diseases. He's also professor and chair of the Department of Preventive Medicine at Vanderbilt University Medical Center in Nashville. Welcome back to SCIENCE FRIDAY, Dr. Schaffner.

WILLIAM SCHAFFNER: Hi, John, good to be with you.

DANKOSKY: Again, you can join us at 1-800-989-TALK. Michael Osterholm, first to you. Let's talk about this report. You looked at over 5,500 studies on flu vaccine, and of all those studies over all these years, you decided that about 30 studies were useful. Really?

OSTERHOLM: Well, first of all, we didn't look at that many studies. We looked at that many articles to identify the studies. And ultimately, we came up with 176 studies that were published among those journals. And of those 176, 73 were randomized controlled trials, where that kind of study which is considered the gold standard of measuring the effect of an intervention like a vaccine, where it was a double-blind placebo-controlled trial.

Some got vaccines; some got a placebo. We didn't know who it was until the code was broken. And then we looked at 103 observational studies. These are the kinds of studies where we follow what goes on in the clinic and using some very specific criteria to make sure that we don't bias who is vaccinated, who is actually evaluated. We looked at those, too.

And it's from those 176 we came up with 31 that really provided us, we believe, the very best and the most accurate information about what's happening with the flu vaccine.

DANKOSKY: So based on this analysis, what do we know about the vaccine's effectiveness? Let's talk about adults first, not seniors, not kids but the big path of adults.

OSTERHOLM: Well, first of all, the overall effect of the trivalent inactivated vaccine, which is the shot that we think about, it's the one that has been around largely unchanged for a number of decades. And in that case, when we look at that, in eight of 12 vaccine seasons, or we study influenza during influenza season, we found that the vaccine was protective, so in two-thirds of the studies.

And when it was protective, it was protective at about a 59-percent rate across all the different studies. When we looked for live attenuated vaccine, the puff that goes up the nose that has been around more recently, there we could not identify any studies that either from an observational disease - or observational study standpoint or from an actual vaccine randomized control trial standpoint, showed that the vaccine was effective.

DANKOSKY: But in kids, that nasal vaccine did work a little bit better.

OSTERHOLM: In fact we found just the opposite. Really the best news in this entire study was that among those studies of children, all eight studies, the live attenuated vaccine in children under eight years of age actually worked quite well. It was consistent protection. The pool, the average protection level was 83 percent.

However, there were only two studies in children of the inactivated vaccine in that same age group, both conducted by the same group, a year apart. In the first year, they found the vaccine worked 66 percent of the time. And in the second year, they found it worked minus-7 percent of the time or not a measurable effect.

So from that standpoint, we obviously think that the information about the live attenuated vaccine is very important in terms of everyday clinical practice.

DANKOSKY: Now, of course, the part that a lot of people are reading is how this affects seniors. What did you find for those 65 and older?

OSTERHOLM: Well, this is where it's more troubling, and understandably, there's going to be some absence of data since the seniors have been recommended to get the trivalent inactivated vaccine since 1960. And because it's unethical to do studies where you withhold a vaccine from someone once it's been recommended, it's understandable that there'd be a relative absence of randomized control trials because you couldn't allocate someone to a placebo.

However, even looking - trying to look at observational studies, those that are just following clinical practice, we could only find one study that suggested there was protection that - in fact, I say suggested - show that there was protection in that population.

I would urge that the absence of evidence is not evidence of absence, but I think that the amount of the protection is actually difficult to measure. Even in this year's data for the Centers for Disease Control and Prevention, which was reported several days ago for the 2009-2010 influenza season, they were unable, in their observational studies, to demonstrate a significantly protective effect in those 65 years of age and older.

DANKOSKY: So you're not finding studies that show it's effective. We're not finding studies, though, that show it isn't effective, right?

OSTERHOLM: Well, that's actually not true in the sense of how you want to talk about effective. Back in the early part of the last decade, there were a number of studies that came out that suggested that up to half or even 70 percent of the mortality in those 65 years of age and older could be eliminated by using influenza vaccine.

We now know that in retrospect, those studies suffered from basically a bias of who got vaccinated and who didn't. It was basically a health vaccinee effect: Those over 65 who were healthy got vaccinated. Those were end-of-life or frail did not. That was pretty evident quickly when the data with the - despite the fact of seeing a major increase among the number of people over age 65 getting vaccinated, there was not a commiserate change in the mortality.

But in addition, we found in a number of these studies that the highest benefit for preventing death actually occurred during the summer months, when the virus wasn't even around.

Since that time, there's been a series of studies done by five different groups in three different countries, which demonstrate some moderate impact of the vaccine that isn't measure in vaccine efficacy or effectiveness. And in those studies, it's possible that as high as eight percent of hospitalizations are eliminated, but it's much more difficult to determine or to more specifically elucidate the benefit to those over age 65.

DANKOSKY: If you want to join our conversation, 1-800-989-8255. That's 989-TALK. We'll talk your questions about the flu in just a minute. But I want to get to Dr. Schaffner. What's surprising, to you, in this analysis? What stands out to you?

SCHAFFNER: Well, actually, those of us who are involved in influenza vaccine are familiar with these data and would largely agree, with some footnotes, to what Michael is saying. They throw out some studies that we wouldn't if we looked at them, and we've long known that influenza vaccine is not a perfect vaccine. We need a better influenza vaccine.

But I think that Michael gets vaccinated every year. So do I. And we all promote the immunization of people age six months and older in the United States because, as he says, even in most years, there is, at least, moderate benefit in most age groups. Influenza vaccine will if not eliminating the disease completely, modify the illness so that you do prevent some cases of pneumonia, hospitalization and death.

I like to paraphrase Voltaire: While we wait perfection, that can be the greatest enemy of the current good. And we have a good vaccine; we don't have a great vaccine. We need to use the good vaccine.

DANKOSKY: Is there an argument, Dr. Schaffner, though, that by using the good vaccine, we're not spending enough time, enough effort, enough money to try to get a better vaccine?

SCHAFFNER: I think actually that was correct. Also, the vaccine was so incredibly safe that there was not motivation, and there wasn't financial motivation for companies or the government until about five years ago to invest anything really worth talking about in trying to create a better flu vaccine.

But within the last five years, I think that there's been more investigation into trying to create a better influenza vaccine than there has been in the previous 40, and also though they are stepwise increments - just in the last two years, we've had - licensed a high-dose vaccine for seniors and now the intradermal vaccine.

Those aren't miracle drugs by any means. They are modest advances over what we have. But Francis Collins, the director of the NIH, said just this last weekend that he anticipates that maybe by 2016 we'll have at least a candidate universal influenza vaccine because of research.

And what that means is that we all know that the influenza virus changes so very frequently, that's why we have to create a new vaccine virtually every year. If we had a universal vaccine, one that would protect against the vast majority of strains, maybe we all wouldn't have to get vaccinated except maybe every 10 years, the way we get our tetanus shots. And that would be a huge advance.

We're not there yet, but there's a little bit of light at the end of the tunnel.

DANKOSKY: Michael Osterholm, do you think we're that close to a universal vaccine?

OSTERHOLM: Well, I think you have to break this apart, really into two totally different component pieces. The study that we present here is a small part of a much larger effort that our center has undertaken for the last two years to really look at influenza vaccines from cradle to grave, from the very basic research and development and finance funding, all the way to consumer acceptance and how well and effective they work.

And I think that we actually find ourselves in a Catch-22 that is a very, very critical moment for us in terms of really understanding where we're going with this. I would possibly beg to differ a bit with Dr. Schaffner about how we've talked about these vaccines.

He and his colleagues have recently called them maximally effective, excellent vaccines, miracle of modern science, and what we have found is that we really feel very confident the results we present here are the best results. We can't fudge them. You can argue, and I've seen, Bill, your comments about serology, we would still disagree with you on whether or not you include those studies or not.

But what the really important part that we found is that we have worked so hard to get people to get vaccine, we've promoted it, and understandably, we want people to get vaccinated, I'm not here on the show today to say don't get vaccinated, we have left people with the sense that this is a really good vaccine.

Now, we have interviewed a number of venture capitalists. We have interviewed a number of the start-up companies who have new and novel technologies, not for example the Fluzone that Bill talked about, that high-dose vaccine, which is just more of the same old vaccine, and if you think that the vaccine matches the problem, why is more antibody going to make a difference? I don't know.

But the point being is that in fact we have a vaccine today that's holding back new development because why invest a billion dollars if you already have a good vaccine.

DANKOSKY: Let's take a break there. You can join us, 800-989-8255, as we talk about the seasonal flu here on SCIENCE FRIDAY.


DANKOSKY: I'm John Dankosky. This is SCIENCE FRIDAY from NPR.


DANKOSKY: This is SCIENCE FRIDAY. I'm John Dankosky. This hour we're talking about the flu. My guests are Michael Osterholm, director of the Center for Infectious Disease Research and Policy and a professor of environmental health sciences at the University of Minnesota School of Public Health. William Schaffner is with us, as well, president of the National Foundation for Infectious Diseases, professor and chair of the Department of Preventive Medicine at Vanderbilt University Medical Center in Nashville. Let's go to the phones. Wendy(ph) is calling from Patterson, California. Hi, Wendy, go ahead.

WENDY: Hi, I was calling to ask why we had to get a flu vaccine every year because some of the other vaccines only get when we're young, that we just get once. And you guys touched on that, that the virus changes, and that's the reason. I would like to know, like, how it changes every year. And are we causing or driving that change at all by vaccinating?

DANKOSKY: Good question, Michael Osterholm?

OSTERHOLM: Well, first of all, we're not changing it by vaccinating. We're not driving the vaccine. Mother Nature and evolution is doing that just fine. I think one of the other things we learned in our study here is there are certain dogmas in the influenza world that we really need to take a step back and look at.

For example the idea of vaccine match and how well the vaccine does. We looked carefully at that in our studies, and we could find not rhyme nor reason as to how well the match for a given year really matched up with how well the vaccine did.

Look no further than the data we've presented in the paper from the H1N1 pandemic vaccine strain, where there we had virtually a perfect match, and in young, healthy adults in Europe, where they actually did use what we call an adjuvant, a chemical that actually boosts the immune response, the overall protection was only 69 percent.

And if you look in this country, the data from the CDC showed that where we didn't use the adjuvanted vaccines in that same population, the protection was 59 percent so that even there with a very, very close match, it didn't seem to show us the results one would think if you had a close match.

So I think we have a lot more work to do, and we've actually tried in this subsequent publication that will coming out to actually look at that much more carefully. And I think the more we learn about this, the less I think we may know about it.

DANKOSKY: I have another call from Cathy(ph), and a few callers want to ask this question. So let's go to Cathy in Abington, Massachusetts. Hi there, Cathy.

CATHY: Hi, thanks for taking my call. My question is about the myth that you can't catch the flu from the flu vaccine. And yet since the H1N1, I've had my family immunized, and three out of the last four years, three different ones of us caught the flu really badly. And it's kind of making me not want to get it. And I know it's a myth, but if you could just explain why that seems to happen. I hear it so often.

DANKOSKY: Dr. Schaffner, explain the myth for us.

SCHAFFNER: Well, it is a myth. You cannot get flu from the flu vaccine. And any number of things can explain what Cathy is talking about. The first is that, you know, we're immunizing right now, and there are people who have colds, and you can get the vaccine, and if you then get a cold four days later, you say where did I get that cold, and then you are likely to attribute it to the vaccine that you got a few days earlier.

And as Michael just said, the vaccine is not perfect. He cited success rates in young, healthy adults of 59 and 69 percent. I look at that as the jar half-full. He says it's half-empty. And so even though you're vaccinated, you can get illness on occasion. There's no doubt about it.

DANKOSKY: What about other side effects, especially for older people, Dr. Schaffner?

SCHAFFNER: Well, it's an incredibly safe vaccine. We give the vaccine in the literally hundreds of millions of doses, and other than really a bit of a sore arm, there are few serious adverse effects. It's one of the most incredibly safe vaccines there is. And indeed, apropos of Mike's comments, that's one of the disincentives for companies to invest a lot of money in creating a new vaccine because this one is so safe.

DANKOSKY: Go ahead, please.

OSTERHOLM: Can I just follow up on that? Because I think that, first of all, I want to be really clear. I'm not saying the glass is half-full or half-empty. It's a very straightforward point. If we talk about the vaccine doing its very best, when it's closely matched to the circulating strain, the H1N1 pandemic vaccine was as close a match as we've had for almost 40 years, meaning that the strain did not change in any measurable way that should or could impact vaccine effectiveness.

And all we're pointing out is even with that, we still only got 59 to 69 percent. Do I think that's a lot better than zero? Absolutely. And when I tell people to get vaccinated for that benefit, absolutely. But it begs the question about what do we really know about this vaccine. How well is it really, really protecting?

And I think that that's the challenge that we're facing right now is in a sense we've oversold how well this vaccine's working, which doesn't mean that in fact we think we shouldn't get it. A 50-percent advantage is a heck of a lot better than zero.

But it's holding us back from driving us towards better vaccines because who's going to invest a billion dollars trying to bring a new vaccine to market that's very different than this vaccine when you've got the current conditions: universally recommended, everybody says it's very effective, excellent vaccine, maximally effective, it's considered to be safe, and it's readily available.

So I think that what we're trying to say is unless we want to stay with this basic vaccine antigen, the protein that's in here that's causing us to fool the body into thinking we've been infected, which is now over 60 years old, do we want to stick with that? We're going to have to acknowledge that we really do have a lot of questions that haven't been answered and that hopefully this study is starting to draw some of them out.

I've heard people who just several years ago are quoted widely as to how good this vaccine are now saying today oh, we always knew that it wasn't that good. That's not a credibility gap I think that can be easily jumped, and it's not one that's going to get us to that next vaccine.

DANKOSKY: What is it that you measure, exactly, to know whether or not it's effective? And hasn't that changed over the years?

OSTERHOLM: Well, if you're asking about effective from the standpoint of illness outcome, that's what we're really looking at. What you want to know are what are those things that you can say with certainty. For example, one of the problems we have in looking at deaths, deaths are caused by many things that may be influenza-like illnesses.

If you're looking at influenza-like illness, as Dr. Schattner just said, you know, 20 to - five to 20 percent of illnesses may be influenza, but a lot of them are not. And so one of the things that I think is very important in effectively really measuring what the vaccine does is you have to know were you infected or not.

And our study actually looked at those studies, which actually showed that. We had a number of studies done during the 1960s through the 2000 time period that use serology, blood testing. Yet in the 1950s, researchers showed that if you vaccinated someone and then they got infected, you could culture the virus from them, they were actually ill, and about - very rarely did they ever show a four-fold antibody rise or one that would give you reason to think they were infected.

Such studies were recently done that just demonstrated that again. Seventy-five percent of the people basically who were vaccinated with a trivalent inactivated vaccine never showed a four-fold antibody rise.

So there you have such a strong bias against finding the real answer and over-inflating the importance of the vaccine. So we knew to take a step back and say: What really gives us the data to show these vaccines work or don't work?

And then once we have that understanding, we can move forward and say what do we need for vaccine? Flu is a critical disease. We need to have vaccines that are effective. We should use what we have until we get those effective vaccines. But we can't stand in the way of getting new vaccines by telling people what we have right now is good enough.

DANKOSKY: Getting back to this idea of it being unethical to do studies on people over 65 because we believe that everyone had to get the flu vaccine, where does this leave us? Is it time to start doing these studies in older people now?

SCHAFFNER: Well, what will happen is that as new vaccine candidates come up, they will be studied in comparison to the standard vaccine. We won't be able to withhold vaccines from anyone, but we can compare the new with the standard going forward.

And as a matter of fact, I'm a volunteer in just such a trial now. I got my influenza vaccine, but I don't know whether I got the standard one or a new one. We'll find out at the end of the trial.

DANKOSKY: I want to ask you, Dr. Osterholm, a lot of senior may be offered a vaccine called Fluzone. Can you tell us what that is?

OSTERHOLM: Well, Fluzone is basically the same kind of influenza vaccine, the trivalent inactivated vaccine, with four times higher the dose, in terms of the antigen, which in turn should induce more antibody.

I think that again we'll wait for the data to come out. Clearly they make more antibody, but that is a long stretch. We have to understand that today, we have equated making antibody to this specific antigen as also being equal to protection, and we now know that's not true. There is clearly a correlation.

But we know there's a big difference. For example, the fact that we have to get vaccinated every year shows you that somehow the immune system's not picking this up. And when you put that into perspective with what happened with the pandemic, we had a number of 70- and 80-year-olds during the pandemic who surely had some innate protection against that virus based on just looking at the population risk of getting infected, and it turned out that these were people who were exposed 60 years before or more to that same circulating type of virus, and 60 years later, their bodies still recognized the fact that they had seem a similar virus 60 years ago.

It had induced antibody or a number of other parts of their immune system so that they were protected. That's what we need to move towards. The idea that we have to vaccinate every year already says that we're not doing a very good job of taking that immune system we have and turning it on in the right way to protect ourselves.

Best thing we've got, we should do it, but it's surely, surely far from what we can do and must do.

DANKOSKY: Well, I want to thank our guests. Michael Osterholm is director of the Center for Infectious Disease Research and Policy and a professor of environmental health sciences at the University of Minnesota's School of Public Health. Thank you so much for joining us.

OSTERHOLM: Thank you.

DANKOSKY: Thanks also to William Schaffner, president of the National Foundation for Infectious Diseases. He's also professor and chair of the Department of Preventive Medicine at Vanderbilt University Medical Center in Nashville. Thank you, doctor.

SCHAFFNER: A pleasure, John. Transcript provided by NPR, Copyright NPR.