How to Evaluate a Supplement Study: 8 Checks

Quick answer

Check who was studied, what was given and in what amount, how many people took part, whether a control group and blinding were used, what was measured, who paid, and whether other studies agree. NIH says a single study rarely gives a final answer, and that results repeated by different investigators are more reliable.^1,2,3

Key facts

NIH says clinical trials in people give the clearest information on whether something works and is safe in humans, and that basic research is the first step.⁴
A controlled trial compares a group that gets the product with one that gets standard care or an inactive substance, and in a double-blind trial neither participants nor researchers know who got which.²
Small studies can be inconclusive and are more likely to reflect chance, while large studies tend to be more reliable.³
In a Cochrane review of 75 methodology papers, manufacturer-sponsored drug and device studies reported favorable efficacy results more often than other studies (risk ratio 1.27).⁵
FDA says sound health advice rests on research over time, not a single study.⁶
Two trials of creatine with caffeine, with 9 and 54 men, reached different conclusions about whether caffeine blunts creatine.^7,8

What are the eight checks for a supplement study?

Reading a study is a set of questions, and most are answered in its methods and conflict-of-interest sections. The table is the short version; the sections below explain each row.^1,2

Eight checks at a glance^{1,2,3,4,5,6,9,10,11}
Check	What to look for	Source
1. Type of study	People or cells and animals; a single trial or a review of many	NCCIH Know the Science
2. Controls and blinding	A comparison group and masking of who got what	NCCIH Know the Science
3. Size	How many people, and whether the result was repeated	NCCIH Know the Science
4. Amount and form	Whether the dose and form match the product you would use	Creatine reviews; FDA melatonin survey
5. What was measured	A change big enough to matter, not only a statistical difference	NCCIH Know the Science
6. Who paid	The sponsor and the authors' disclosures	NCCIH; Cochrane review
7. Other studies	Whether independent teams and reviews agree	NCCIH; FDA 101
8. Who was studied	Age, sex, health and training status of the volunteers	Creatine reviews

What type of study is it?

NIH separates four kinds. Basic research asks questions about theories and tests hypotheses, and is the first step. Translational research builds on it. Clinical trials are done with people and give the clearest information on whether something works and is safe in humans. Systematic reviews and meta-analyses pool different studies on the same topic, and when many studies reach the same conclusion that helps show the results are reliable. A result in cells or animals points to a question for a trial; NIH reserves the clearest answer on what works and is safe in humans for clinical trials.⁴

Did the study use a control group, blinding and random assignment?

In a controlled trial one group receives the product being studied and a control group receives standard care or an inactive substance, as close to identical as possible otherwise. A double-blind design goes further: neither participants nor researchers know who is in which group. NIH explains why: if either side knows who got what, it can color their impression of whether someone improved, however hard they try to avoid it.^2,9

A reporting guideline called CONSORT is used worldwide to make randomized trials easier to read, so a reader can see how a trial was run and what it found. A paper that says nothing about blinding, a control or how people were assigned is missing what you most need.¹²

How many people were in the study?

NIH says small sample sizes may give inconclusive results and are more likely to reflect chance, while large samples tend to be more reliable and better reflect the population. An essay on research reliability adds that a finding is less likely to be true when studies in a field are smaller, when effects are smaller, and when designs and outcomes are more flexible.^3,13

Creatine and caffeine show how this plays out. A 1996 trial of 9 men found caffeine removed a performance gain seen with creatine. A 2016 trial of 54 men found no difference between creatine alone and creatine with caffeine or coffee. A 2022 systematic review of 10 studies found mixed results. No one trial settled it.^7,8,14

Was the amount and form the same as the product you would use?

A result belongs to what was given. Creatine trials that loaded used about 20 g a day for a week, and the same sources list 3 to 5 g a day as the maintenance amount, so a finding at one amount does not automatically carry to the other. A study of one form of a supplement says little about a different form.¹⁵

Other, usually more expensive forms such as creatine ethyl ester and buffered creatine have not been shown to raise muscle creatine better than monohydrate, or to be easier to digest, more stable or safer.¹⁵

The label is also a claim. In an FDA survey of 110 melatonin products, measured melatonin ranged from 0% to 667% of the declared amount, so even a study's own product can differ from its label unless it was analyzed.¹⁰

What did the study measure, and is the result large enough to matter?

NIH separates two ideas. A statistically significant difference means the observed difference between the groups is not likely to be due to chance. A clinically significant one is a measure of how large the effect is. A study can find a statistically significant difference that is so small it does not matter in practice. Look for the size of the change, not only the p-value, and ask whether the thing measured is the thing you care about.¹

NIH asks readers to check whether the sponsor or the investigators had a financial or reputational stake in the outcome, and notes most journal articles disclose financial relationships. In a Cochrane review of 75 methodology papers, drug and device studies sponsored by the manufacturer more often reported favorable efficacy results (risk ratio 1.27) and favorable conclusions (1.34) than studies with other sponsors. That review covers drugs and devices, not supplements, so it shows why to read the disclosure rather than proving anything about a given supplement trial.^1,5

The conflict statements in the creatine papers cited on this site run from none to long lists of advisory roles and donated product. A 2026 trial of creatine gummies, for example, says a company donated the gummies and that two authors advise creatine companies. A disclosure is not a verdict, and it is the thing to read.^16,17

Do other studies agree?

NIH says a single study rarely provides a final, definitive answer, and that repeating a study with different volunteers and investigators helps show whether results are reliable. FDA's consumer advice agrees: sound health advice is generally based on research over time, not a single study. The date matters too, since NIH notes techniques and standards change and newer work can confirm or contradict older results.^1,4,6

Does the study apply to you?

Results come from the volunteers. Look at their age, sex, health and training status, and whether they resemble you. Women are an example: only a small share of creatine studies enrolled women alone.¹¹

Only 58 of 656 creatine studies, about 9 percent, enrolled women alone.¹¹

What should you ask a doctor or pharmacist before acting on a study?

A study describes a group, not you. FDA says some supplements can interact with medicines, interfere with lab tests, or have dangerous effects during surgery, and NCCIH says many supplements have not been tested in pregnant women, nursing mothers or children. Ask a doctor or pharmacist before starting anything a study makes sound promising, especially if you take medicines, are pregnant or nursing, or have a health condition.^6,18

Questions people ask

Is a study in mice or cells enough to take a supplement seriously?

No. NIH describes basic research as the first step, and says clinical trials in people give the clearest information on whether something works and is safe in humans.⁴

What does statistically significant mean in a supplement study?

It means the difference seen between the groups is not likely to be due to chance. A clinically significant result is a separate question: how large the effect is. A difference can be statistically significant and too small to matter.¹

How many people should a supplement study have?

NIH gives no number. It says small studies can be inconclusive and more likely to reflect chance, and that large studies tend to be more reliable.³

Does a supplement study paid for by a company count?

It can, but read the disclosure. A Cochrane review found manufacturer-sponsored drug and device studies reported favorable efficacy results more often (risk ratio 1.27); it did not study supplements.^1,5

Is one study enough?

No. NIH says a single study rarely gives a final answer, and FDA says sound health advice rests on research over time rather than one study.^1,6

See what we make

Every product shows its whole label and what its lab report covers. The store opens soon.

See the Full Measure line

Sources

This page is information, not medical advice. Talk to your doctor or pharmacist about your own situation.