Investigating vaccine linkage in adverse events - can the methodology be improved?

Health officials and investigators follow established protocols to determine this, but methodologies may still have some room for improvements.

The global reference for such methodology is based on a four-step process outlined in the “Causality assessment of an adverse event following immunization (AEFI)” by the World Health Organisation.

The first step is determining eligibility of the case, followed by a checklist in the second step which relies on relevant available information to address possible vaccine linkages of the AEFI. Needless to say, the section of “vaccine product” (referring to all components in a given vaccine and things related to it) in the checklist may be difficult to answer in the absence of, or insufficient peer-reviewed and published data related to the vaccine and the adverse event in question, biological plausibility and specific tests. This could be particularly difficult in novel or new vaccines, and when longer term clinical trials haven’t been completed.

Other sections in the checklist concerning vaccine quality, immunisation error, and immunisation anxiety are valid questions, but it may require a complete, accurate and detailed log of the entire supply chain, cold chains, administration of the vaccines and post-vaccination monitoring.

Other sections of the checklist include the time window of increased risk, availability of strong evidence against causal association, and other qualifying factors for association. In general, these questions may also face a similar problem of insufficient data in the literature as mentioned earlier. For example, it would be hard to determine the entire time window of increased risk, if we only have relatively shorter-term data with a limited group of people. For Covid-19 vaccines, a longer time window could be considered to ensure any potential longer-term AEFIs that could be associated with vaccines are not overlooked.

As for “strong evidence against causal association” and “other qualifying factors for association”, Italian professor of General Pathology at Verona University, Paolo Bellavite appears to have criticised the overall approach which seems to proceed “by exclusion”, in quest of “other causes”. This could be akin to a scientific experimental design “flaw” that could make it skewed towards a certain outcome.

The heart of the issue is when adverse events are not simple singular cause-and-effect cases, but an issue of being a co-factor. This is a probable scenario given the complexity of vaccine interaction in the body and potential interactions with other underlying conditions.

For example, if we take the death cases is Norway, a top physician at the Norwegian Medicines Academy, Sigurd Hortemo pointed that we cannot exclude the possibility of adverse reactions (side effects such as fever, which are relatively more common) undermining the already-deteriorating health of the frail elderlies with severe underlying diseases, which could have further contributed to more severe conditions and even death. In this case, vaccines may not be “the” cause, nor is it “directly causing” it.

Bellavite commented that “proceeding by elimination of one or more con-causes in what could be multi-factorial is incorrect from a scientific perspective”. The vast potential for biological or biochemical reactions of vaccines, intra- and extracellular interactions, all potential contraindications or triggering capabilities of underlying disease or influencing factors on certain genetic predispositions may not have been fully elucidated.

In the case of multi-factorial adverse events, we may be able to exclude relationship with the vaccine if we can decisively say that the underlying conditions of the patients have not interacted or made worse or influenced in any adverse way by vaccines. Bellavite mentioned this as “independence” of the “other cause”. In standard scientific experiments, this can be achieved via test controls. Even then, “direct linkage” can be hard to observe. It is very common to make conclusions based on strength of association (such as through statistical correlations or deductive reasoning) precisely because direct observations are usually not doable in experiments.

A poorly controlled open world scenario of multi-factorial AEFI causality investigation is likely carrying a magnified version of this similar weakness. Hence, this could be why Bellavite alluded to an approach where probability or weightage of contribution of the various co-factors in a multi-factorial adverse event can be scored. This is in contrast to the current approach which appears to be operating mainly through an exclusion principle.

Another qualifying factor in the checklist is the likelihood of the adverse event occurring without vaccination, which is referring to the “background rate”. Again, taking the case in Norway as reported by Bloomberg, head of the Norwegian Institute of Public Health, Camilla Stoltenberg, mentioned the daily death rate in Norway’s nursing homes of 45 people, pointing that it doesn’t necessarily mean a connection with vaccines or that vaccines contributed to an increase in mortality rate.

This is a valid consideration, and although an overlap with a background rate could be fairly said as “not an evidence of direct causality”, it is critically different than saying “evidence of no direct causality”. Also, the use of the word “direct” may not be suitable if the AEFI is a case of multi-interacting factors, and may require time to manifest. The same attention to nuances in language use (and the crucial difference in meaning it conveys) was also mentioned by Bellavite.

The third step is known as the algorithm, which appears to be a way to process information in the previous step in order to come to a conclusion, which would finally classify the AEFI in the fourth step.

Potential issues in the third step have been reviewed by Bellavite and it appears to be revolving around the same issue mentioned earlier such as the reliance of existing data or literature where it would be difficult to assess rare adverse events, uncertain time windows of increase risk, biological plausibility and other matters.

Therefore, the recommended search for “strong evidence” as shown in the WHO guideline may be reasonable, but it is especially difficult when dealing with multi-factorial and time-bound adverse events that have not been reported before.

Impact of any potential weaknesses of prior steps discussed above may be aggregated and carried forward in the last stage of the categorisation of the AEFI, resulting in flawed categorisation where vaccine co-factor role may go unnoticed.

One consideration is probably to consider some form of control groups even in real world vaccination scenario. Certain control measures to keep the control group safe and minimise influence of “other causes” may have to considered.

Comparing the two cases directly may enable the bypassing of some uncertainties in the causality investigation. Of course, it’s understood that this might be operationally challenging and it might be weaker than a tightly controlled double-blinded clinical trial, but its probably better than no control at all.

Although the issue could be a sensitive matter because the outcome of such investigation can disrupt immunisation programmes and can have financial and reputational implications to pharmaceutical companies as well as compensators in vaccine injury compensation scheme, its prudent (if not ethically compulsory) to ensure that the process of investigating causality to AEFI’s are fair, and not skewed in favour of any parties.

With that said, it’s important to note that the WHO guideline has been developed and refined several times by experts, and this is merely a surface commentary for further investigation and review by other relevant independent experts for improvements, if any.

Even if serious cases are found to be causally related to vaccines, so long as it’s statistically very rare, and the benefit to many still far outweighs the risk to the very few, then it’s a matter of effectively communicating the risk profile instead of trying to overly manage the information.

Of course, the argument for careful information management (not accountability management) for the sake of public health is understandable when there is a deficit in public understanding and trust, particularly in the absence of any better choice to fight the pandemic.

---

Ameen Kamal is the Head of Science & Technology at EMIR Research, an independent think tank focused on strategic policy recommendations based on rigorous research.