shire map
Select a

Shire Global

Science & Innovation

Evaluating Research: Five Ways to Separate Science From Hype

Evaluating ResearchInterpreting findings from a study and assessing their potential clinical relevance is no longer the distinct prevue of investigators, key opinion leaders and healthcare providers. Patients, particularly with rare diseases and specialty conditions, are activity seeking and sharing information about studies to help improve and advocate for the health of their community. But unfortunately not all science is high quality or immediately relevant to every day care.  How can one determine the worthwhile from the not useful?  Look at the evidence.

Shire’s Global Medical Communications team suggests looking for five types of evidence to review clinical science studies, whether reported in the news, in a press release or in a peer-review journal. Our expert team works daily to ensure the science behind Shire’s existing and emerging treatments is well-understood and well-communicated to enable people with life-altering conditions to lead better lives. They review and analyze new data about the origins, diagnosis and treatments of a variety of diseases and place those findings in context for the medical community. Their approach can be useful to see through the hype and determine which research is truly of value.

  1. Look for the red flags. Research news headlines use flashy verbs like “promises,” “reverses” or “cures” to catch attention like a neon sign. But the brevity of headlines doesn't permit the words necessary to communicate the nuances essential to understand the study results. So read beyond the eye-catching headline to get to the article's fine print for a more complete take on the study results.
  2. Consider the sources. Was this study published in a peer-reviewed journal? Not only does this mean the study design, conduct and results were reviewed by independent experts in the field before the journal published the findings, it also means registration of the trial prior to first patient enrollment in registries such as, a threshold for consideration for most peer-review journals. Both factors add to the credibility of the study and its results. News might also stem from presentations at medical meetings, which may have peer-review, too. Another study sourcing aspect to consider is what organization employs the scientists who conducted the study, such as a drug company or a respected academic medical center. Who paid for the study and is there a relationship between the scientists and the funder?  What do the companies whose products were studied have to say about the findings? Many companies are very compliant with regulations and strive to be transparent in how they support and discuss research about their medications. So in addition to news articles, read corporate press releases for key points, journal articles for disclosures and corporate websites for compliancy and transparency statements.
  3. Understand the study design. Determining the design and phase of a trial helps interpret the findings. The objective of a study is easily found in the definition of its end points--the predetermined "what success looks like" goals for changing laboratory test values, like a blood count, or reducing incidents of certain outcomes, like a heart attack. Were the primary study endpoints about effectiveness of the medication or of safety?  If safety measures were the study goal, efficacy findings might have been just observations that must be verified in other trials. What comparator was used in the study? A placebo-controlled arm can be unethical for many conditions like cancer or rare diseases, so such studies use the current “standard of care” as a comparator to the investigational medication. Studies without any comparator are less credible. Finally, what phase was the trial?  A first-in-human, pilot or other early phase study may be promising but need verification in other, more advanced trials. In general, the further along in development a medication or formulation is, the more evidence exists for contextual understanding of the findings.
  4. It’s all in the numbers. Because size is relative, always compare the number of patients tested to the known disease population. For a very common illness like cardiovascular disease, trials enroll large numbers of patients. In a rare disease, however, 50 patients might actually be a very large number. Also check the demographics of the patients --age, gender disease status-- to see how they compare to the known disease population.
  5. Put the research in context. Always approach new research findings with a healthy dose of skepticism, while allowing for the exciting potential that new scientific studies hold. Ask if the medicine is a first-in-class medicine and the findings are the first to be reported, or if they verify, expand or contradict previous reports and how. Does the research enhance existing understanding through new insights? If it’s about a new medication, do the benefits and safety profile offer an improvement or just another alternative to existing treatments?  And what are experts who are not involved in the study saying about its implications for future treatments or future areas of research?

Asking these questions can help gather evidence to place a study in context, perhaps changing a "wow" response to reports of findings to "well, that is interesting" or to "wait and see." Regardless, as the amount of clinical research information expands, strong effective science screening is more essential than ever.


Cookie Use Notification

The site uses cookies to provide you with a more responsive and personalized service. By using this site you agree to our use of cookies as set out in our cookie notice. Please read our privacy notice for more information on the cookies we use and how to delete or block the use of cookies. LEARN MORE