Behavioral science researchers found to routinely exaggerate study results

Friday, September 13, 2013 by: L.J. Devon, Staff Writer
Tags: exaggerated study results, behavioral science, researchers

(NaturalNews) An analysis from the University of Edinburgh, United Kingdom, finds that behavioral science research coming from the United States is often exaggerated and misrepresented.

In comparison to other countries, US behavioral science is misleading; it's often laced with statistics and conclusions that are blown out-of-proportion.

The new investigative analysis reports that American behavioral researchers are often rewarded for quick, impacting findings in their field of study. This is causing professionals to compete with one another, as they report exciting, eye-catching results to stay ahead of their colleagues.

The result is that much psychological research coming from the US has little integrity and is loaded with unconscious bias.

Investigating the merit of behavioral research publishings

The research analysis, published in the Proceedings of the National Academy of Sciences, was conducted by Daniele Fanelli, biologist from the University of Edinburgh, and John Ioannidis, a physician at Stanford University in California. They sought to study the merit of behavioral research published in major journals around the world.

So the two researchers set out and combed through 82 genetics and psychiatry meta-analyses, combining results from 1,174 individual studies.

Comparing physiological measurements of non-behavioral parameters with the progression of conditions like dementia or depression, the researchers took a closer look at meta-analyses and compared them to original hypotheses.

They further examined the strength of a study's observed result with what was actually found in the meta analysis. What they quickly found out was that behavioral studies are far more likely to be exaggerated when compared to non-behavioral studies. Worldwide behavioral studies were reported with "extreme effects" which drastically differed from the report of the scientific metadata. Behavioral science research from the United States was the most misleading, reporting extreme effects that greatly differed from initial hypotheses.

Coauthor Daniele Fanelli states, "We might call this a 'US effect.' Researchers in the United States tend to report, on average, slightly stronger results than researchers based elsewhere."

"Publish or perish" mentality fueling exaggerated results

"Whatever methodological choices are made, those made by researchers in the United States tend to yield subtly stronger supports for whatever hypothesis they test."

Fanellia believes researchers are under pressure from a "publish or perish" mentality that grips researchers whose careers depend on high profile publications.

Critics of Fanelli's investigation believe extreme outcomes are reported in behavioral research, because more diverse conditions are studied in this field. Some critics believe the volume of research studied by Fanelli can only make a correlation but cannot prove that the US is the leader of exaggerated findings.

With the current economic and competitive demands pressuring researchers who publish in high-profile journals, it is easy to see how many findings are sensationalized and exaggerated. Researchers try to enhance their chances of securing research funds or to progress their career by embellishing their findings.

Fanelli explains, "The very idea that you do science to make strong discoveries is natural but it's a problem to science itself. Science should be about doing good, precise studies. Not necessarily about getting exciting new results every time."

Correcting the unconscious bias in behavioral research

"The US itself should re-think the way they are rewarding researchers. They shouldn't reward researchers only because they get a lot of papers in a lot of high-ranking journals. They should reward research that is methodologically highly accurate."

An answer to the unconscious bias problem is a new study method called study pre-registration.
Problems in behavioral research can be eliminated by asking researchers to register their experimental methods and analyses before they run their experiments. "This eliminates much of the wiggle room that allows problems like those highlighted here to arise in the first place."

Research funders and regulators are being called upon to lead research reform. In the current research climate, scientists are rewarded for putting their personal career first and punished for working for the collective good. Gold-standard working practices should be implemented to eliminate this selfish climate. Researchers should hold one another accountable and not be pressured to compete for position and funds.

Sources for this article include:

http://www.nature.com

http://washington.cbslocal.com

http://retractionwatch.wordpress.com

http://www.theguardian.com