How much data is enough to substantiate structure/function bene- fit claims for dietary supplements? The unwritten rule, voiced by both FDA and the FTC officials, is that two randomized, placebo- controlled clinical trials using state-of-the-art methods, the actual product in question, and statistical analysis yielding a statistically significant difference compared to placebo are required to show effi- cacy. For consumers, the amount of data required to show efficacy is a highly individualized question—some may say that one study is enough whereas more skeptical consumers would seek or require several studies, and perhaps even some long-term (one year or lon- ger) studies before they would offer it to their children. For health professionals, again the quantity of data required to show efficacy would cover a broad range. A medical doctor may demand several hundred-person studies lasting up to two years (similar to that for pre- scription drugs) whereas a naturopath or chiropractor may be satis- fied with one to two RCTs with a duration of only four to eight weeks. My colleagues and I have found most prospective sponsors of clin- ical trials to be confronted with the designing for dollars conundrum: they want a great study, worthy of publication in a high-impact factor journal, with stunning statistics, and yet at a bargain-basement price. When asked about how many subjects they had in mind, for some ob- scure reason the reply most commonly is 60 subjects. For the first foray into sponsored clinical research, my colleagues and I recom- mend focusing upon the budget rather than the sample size.
Power calculations, which provide researchers with a ballpark esti- mate of how many persons would be needed in a study to measure a difference between two or more groups, are nice and desirable to per- form. The calculated sample sizes depend upon (1) the study hypoth- esis or “what are we proposing to test”; (2) the amount of variability in measurement of what is expected to change, e.g., blood antioxidant activity or bone density; and (3) an estimate of a clinically meaning- ful and significant difference compared to presupplementation values and/or a placebo. However, if the sample size estimated via power calculations is prohibitively large, e.g., 84 subjects, does one close the research checkbook? As botanical product marketers do not have billions of dollars in revenue like drug companies, it is an economic impossibility for the majority of them to sponsor RCTs that enroll
hundreds (or even one hundred) subjects. In our experience, a study conducted with two groups of 12 to 15 or more subjects is noteworthy and provides a starting point, a foundation upon which other evidence can be based.
Strategy in product indication and clinical outcome parameter se- lection assume preeminent importance in designing a clinical study. Because the overwhelming majority of sponsored research is con- ducted with a return on investment in mind, if the results are not con-
sumer relevant and consumer compelling why do the trial? For both
ingredient and finished product marketing companies, the real selling target is the end user. Does a drop in interleukin-1ß or altered expres- sion of uncoupling protein II mean anything, compared to a decrease in pain or a reduction in body fatness? Consumers and most health professionals will be compelled to read further and inquire into use of the product if the outcome measure(s) encompass at least one clini- cally meaningful measure, i.e., one that assesses a widely recognized and/or symptomatic or physical feature or attribute. Lowering of dia- stolic blood pressure, reduction of waist size or body weight, in- creased head hair growth, elevation of HDL cholesterol, or a reduc- tion in fasting blood sugar, insulin, or hemoglobin A1C are examples of relevance and import to persons seeking to achieve these out- comes.
If the product or ingredient of interest requires a duration of use greater than 30 to 60 days the likelihood of extracting a robust return on investment is slim or it will require a string of studies, effectively and repetitively communicated through the media and through adver- tising, e.g., vitamin E, garlic, and echinacea.
It would appear prudent to choose investigators who have a dem- onstrated expertise in clinical research, especially in relation to the parameters intended to be assessed, i.e., a family practice physician would unlikely be skilled in assessing a population of osteoarthritic men and women being assessed for responses to a botanical formula purported to modify joint pain and disease progression (unless he or she had done similar research before). In addition, the use of investi- gators that lack material interests in the product or company market- ing the product would be strongly encouraged. Given the policy of many journals and companies to not provide full disclosure of the in- terests of scientific investigators, many such relationships are opaque to consumers, clinicians, and regulators. Researchers and marketers
are earnestly encouraged to offer full disclosure of any interests, from consultancy fees and honoraria to travel and lodging to equity and stock options.
Mitigation of economic risk in research with innovative, “new to the world” compositions or those enjoying only preclinical (in vitro and animal studies) or anecdotal “validation” may involve taking a path that initially diverges from a phase II study approach. Phase II studies are small-scale RCTs done at a single research center where evaluating the efficacy of a product is the primary interest and the dose range is already established. Thus for a product where the dose response is unknown, it may be financially imprudent undertaking a phase II RCT. My associates and I often suggest that manufacturers do an open-label pilot study (six to ten participants) with applications that are much less prone to strong placebo responses, e.g., not chronic pain conditions, weight loss or appetite reduction, or mood disorders. Alternatively, employing a small sample size (three to five partici- pants) with a conservative washout period and a crossover offers more statistical rigor, albeit at the expense of greater than doubling the time to completion of the study. These tactics can bolster confi- dence in going forward with phase II-type studies and still augment the “package” that encloses an ingredient or product invention.
Assuming one has an idea of a dose response or is willing to make the investment in a phase II study, we recommend the sponsored re- search investment to be dictated by both the budgetary restraints and reality—an RCT (adhering to the guidelines discussed) for $10,000 is unrealistic, as is one with 12 participants (that is, it will carry little weight). A safe number to start with is a sample size of at least 20 sub- jects, coupled with a power calculation done prior to the study. Al- though one may not be able to afford a properly powered study in the first round, one can obtain valuable preliminary evidence and, if the product outperforms expectations, achieve both a clinically and sta- tistically significant result. Inevitably subjects drop out, for a myriad of reasons. One does not need to have a larger population to achieve
statistical nirvana, as the magnitude of “effectiveness” dictates what
is or is not statistically significant. For example, if a combination bo- tanical product being tested for its ability to reduce the severity and size of psoriasis-related lesions (represented as a lesional sum score) shows a very large score reduction after four weeks of supplemen- tation, compared to a very small score reduction in the placebo group,
the difference between the two groups (indicative of the effectiveness of the product) will be large. As long as there is not large variability between subjects in both groups, this result will manifest as both a statistically and clinically significant outcome. If the product in ques- tion has moderate to dramatic bioactivity or efficacy in vivo, this can likely override variability within and between groups and any psy- chogenic/placebo effects. Many preliminary drug studies have been published, with an RCT design, “favorable” statistics, and a sample size of only 20 to 30 subjects. This is a start, and a distinctive start in- deed. If the results are promising and they are appropriately trans- lated to the customer, incremental revenues can self-fund additional, larger studies, of even longer duration.