×

Announcing: Slashdot Deals - Explore geek apps, games, gadgets and more. (what is this?)

Thank you!

We are sorry to see you leave - Beta is different and we value the time you took to try it out. Before you decide to go, please take a look at some value-adds for Beta and learn more about it. Thank you for reading Slashdot, and for making the site better!

How Did Those STAP Stem Cell Papers Get Accepted In the First Place?

samzenpus posted about 6 months ago | from the don't-press-send dept.

Biotech 109

bmahersciwriter writes The news team at the scientific journal Nature turns its investigative power on the journal itself. The goal: to try and understand how two papers that made extraordinary claims about a new way to create stem cells managed to get published despite some obvious errors and a paucity of solid evidence. The saga behind these so-called STAP cells is engaging, but sadly reminiscent of so many other scientific controversies.

Sorry! There are no comments related to the filter you selected.

How? (-1)

djupedal (584558) | about 6 months ago | (#47380739)

Blind faith in Japanese researchers, that's how. The boys are fine when it comes to collecting data, but it's not wise to trust what they do with it downstream, sorry.

Re:How? (2, Insightful)

erroneus (253617) | about 6 months ago | (#47381549)

Worse than that. It's blind faith in circular belief in truth. You get just enough people saying it and it becomes "a number of people" and suddenly a critical mass of people are making or supporting the claim and it becomes "truth." This is a general understanding of how lies become truth all over. Such common lies are "god" and "global warming." Deny either of those (among others) and you will be attacked politically. Observe as I get modded down because I dared mention god or global warming as lies.

Re:How? (1)

Anonymous Coward | about 6 months ago | (#47381673)

"Observe as I get modded down because I dared mention god or global warming as lies."

Ooh, so daring. What's next, O King of Iconoclasts, a bold swig of non-skim milk?

Re:How? (2)

erroneus (253617) | about 5 months ago | (#47383909)

Coming from AC, I find that to be only a little ironic.

Re:How? (2)

Blaskowicz (634489) | about 6 months ago | (#47381713)

Global warming is something of a "meta lie" then. See, you get just enough people saying it's a lie and it becomes "a number of people" and suddenly a critical mass of people are making or supporting the claim and the lie about it being a lie becomes "truth".

Re:How? (1)

KeensMustard (655606) | about 5 months ago | (#47386435)

Interesting idea but you example of "global warming" being a circular belief is a bit awkward, counter to objective observation and appears grotesquely out of context, like a naked clown.

Perhaps what you meant is that the notion "global warming is a lie" is itself a circular belief? That would make more sense. After all, a key property of a circular belief is that it's exact origins are somewhat obscure. The theory of global warming has well known, logical origins founded in the laws of thermodynamics and provable via repeatable (and oft-repeated) experiments. The origins of the theory can be traced back to those original observations which can't be logically explained otherwise.

Whereas the belief that "global warming is a lie" has all the properties of a circular belief:

(a) The proof of the belief refers to the belief itself

(b) The origins of the belief are obscure and it's proponents are often at considerable effort to avoid discussing the likely origins

(c) The belief makes no reference to observations which justify it and indeed, contradict known observations.

Is it therefore safe to assume that that is what you meant - that the belief "global warming is a lie" is itself a lie?

Re:How? (0)

Anonymous Coward | about 6 months ago | (#47381669)

"Blind faith in Japanese researchers"

Yes. Such things cannot happen when the researchers are of other ethnicities. [heavy rolling of eyes]

Re:How? (1)

ColdWetDog (752185) | about 6 months ago | (#47381747)

Like the American, Vacanti, who started the whole thing.

The same way many global warming papers got pub (1, Insightful)

drsmack1 (698392) | about 6 months ago | (#47380761)

Peer reviewed. Yeah, right. And just who is reviewing the peers?

Re:The same way many global warming papers got pub (-1)

Anonymous Coward | about 6 months ago | (#47380799)

It was backed up by 97% of the other papers on the subject?? Right, you're fucking stupid.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 6 months ago | (#47380947)

If you torque the source data, it's amazing what the rest of the lemmings in the pipeline can do.

Re:The same way many global warming papers got pub (1)

Anonymous Coward | about 6 months ago | (#47380979)

That's a very libelous claim. Any evidence?

Re:The same way many global warming papers got pub (1)

Anonymous Coward | about 6 months ago | (#47381615)

Re:The same way many global warming papers got pub (1)

ArcadeMan (2766669) | about 6 months ago | (#47381057)

This isn't the same as P2P you know...

Re:The same way many global warming papers got pub (5, Insightful)

Ol Olsoc (1175323) | about 6 months ago | (#47380819)

Peer reviewed. Yeah, right. And just who is reviewing the peers?

And found out in short order.

As for what causes researchers to do this sort of thing, well it's the same thing that other humans do. The biggest difference is that science accepts it's lumps and corrects them.

You'll have someone feeling pressure to get results, and they fudge, make hopeful assumptions, or even fake results. This woman appears to have done all three.

Then she manages to get some highly respected researchers to sign on. Laziness on their part. The pee reviewers see the names, and get a little lazy themselves.

Viola. But in all this, the fact remains that others will be fact checking, and trying to reproduce the results claimed in any paper, especially one making these extraordinary claims And that's the strange thing. If you fake the results, you're going to be found out to a high degree of certainty. She toasted her career with a similar degree of certainty.

Re:The same way many global warming papers got pub (4, Insightful)

Jane Q. Public (1010737) | about 6 months ago | (#47381019)

The biggest difference is that science accepts it's lumps and corrects them.

Yes but...

Science has bigger problems correcting them, and takes much longer to do so, when political and financial pressure tempt people to look the other way. Scientists are people too.

Re:The same way many global warming papers got pub (4, Interesting)

Ol Olsoc (1175323) | about 6 months ago | (#47381127)

The biggest difference is that science accepts it's lumps and corrects them.

Yes but... Science has bigger problems correcting them, and takes much longer to do so, when political and financial pressure tempt people to look the other way. Scientists are people too.

Beg to differ. go to retraction watch.com.

As far as I can tell, there is no correction mechanism for politically based incorrect assumptions. Just as example - the job creator myth. It's still cited as gospel by many, while the evidence is clear it doesn't create jobs.

Galileo Galilei finally got forgiven by the Catholic church, when Pope Paul admitted the churches "errors" the church made - almost 400 years later. Regardless of political bias, I don't see either of the big two parties having a website of fact checked "We were wrong about this" stuff - at all - much less in a timely fashion.

Re:The same way many global warming papers got pub (4, Interesting)

pepty (1976012) | about 6 months ago | (#47381369)

retractionwatch.com is great for the most influential journals but it's safe to say it's only catching a small minority of the infractions, and of those really only the ones published in english. One cynical solution: as grant money continues to dry up, scientists will realize that the easiest way to improve their chances to get their grant approved is to thin out the competition: exhaustively check their competitors' publications and old grant applications for plagiarism and fraud, then complain anonymously.

Re:The same way many global warming papers got pub (1)

serviscope_minor (664417) | about 5 months ago | (#47382293)

That seems unlikely. Most scientists do it because they love science, as opposed to doing it for the money and bling, the hot guys and girls and of course the celebrity status.

As the money dries up you'll see more of them burning out and running themselves into the ground through sleepless nights in the lab than you will see them trying to thin the competition.

It doesn't matter how thin the competition gets, if you have no reaults you'll lose funding no matter how many others you take down with you.

Ultimately though the scientists find their science more interesting than the other people so thats where they'll turn.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 5 months ago | (#47382781)

So... just how do you have the science, which you "love", without the funding? Naturally, I can understand the backyard experiments with vinegar, baking soda and a twitching dog, but STEM cell research?

Re:The same way many global warming papers got pub (1)

Jane Q. Public (1010737) | about 5 months ago | (#47384781)

As the money dries up you'll see more of them burning out and running themselves into the ground through sleepless nights in the lab than you will see them trying to thin the competition.

I don't dispute this, but don't forget there are things like... grants. And too much science lately seems to be more interested in chasing those grants than is healthy.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 6 months ago | (#47381683)

> Galileo Galilei finally got forgiven by the Catholic church, when Pope Paul admitted the churches "errors" the church made - almost 400 years later.

He didn't get in trouble for writing about astronomy in the first place, but for writing nasty caricatures of people (including the Pope, who happened to be a friend of his in spite of this...). Also, they knew and expected a stellar parallax. The fact that the nearest star is an insane distance away made stellar parallax too small to measure for quite some time and, once it was measured, heliocentrism became widely accepted.

As you say, political errors take quite some time to work their way out of the system.

Re:The same way many global warming papers got pub (1)

quarterbuck (1268694) | about 5 months ago | (#47382925)

The biggest difference is that science accepts it's lumps and corrects them.

Yes but... Science has bigger problems correcting them, and takes much longer to do so, when political and financial pressure tempt people to look the other way. Scientists are people too.

Beg to differ. go to retraction watch.com.

As far as I can tell, there is no correction mechanism for politically based incorrect assumptions. Just as example - the job creator myth. It's still cited as gospel by many, while the evidence is clear it doesn't create jobs.

You are making the error in assuming that Economics or Finance is a science. It is called a social science , but it doesnt' work like one. People make assumptions and then form theories. The assumptions like "General Equilibrium Model" or "Rational Markets" are normative rather than what actually see to be the practice in the world. Then based on these assumptions theories are formed. These theories cannot be falsified because experiments are notoriously difficult to set up (you need entire countries) and even if a set-up is made there are 10 other possible hypothesis that explains the results.
Economics is like trying to explain ocean currents without using wave mechanics or Newtons laws.

Re:The same way many global warming papers got pub (1)

Jane Q. Public (1010737) | about 5 months ago | (#47384813)

You are making the error in assuming that Economics or Finance is a science.

No, I agree with you to some extent, but my original comment was misunderstood a couple of different ways.

I wasn't suggesting that it had anything to do with political or economic science. Rather, that science can be unduly influenced by economic or political pressure.

I do agree that there are many economic models, some of which claim to be empirical and others not, etc. But one thing we do know is that by and large Adam Smith's "invisible hand" can and does work, if allowed to do so. (Presuming that appropriate antitrust regulation and enforcement exists, of course.)

Re:The same way many global warming papers got pub (1)

Vesvvi (1501135) | about 5 months ago | (#47383999)

I believe you are misreading the previous post. What they are saying is that political and financial pressure has an impact on scientists. It was not a comment on politics/finance vs science.

Re:The same way many global warming papers got pub (1)

Jane Q. Public (1010737) | about 5 months ago | (#47384703)

Beg to differ. go to retraction watch.com.

I'm not sure where the misunderstanding was, but you aren't "differing" with me, you are agreeing with me.

Science works (2)

Layzej (1976930) | about 6 months ago | (#47381405)

There are examples where papers on global warming tried to game the system but were found out in short order. For an example see this article where Dr. Roy Spencer sneaked a botched paper past the peer-review system by submitting to an off-topic journal. Because the reviewers were not familiar with the topic they were not aware that the methodology described in the paper had already been refuted by previous literature. Within days the journal editor had resigned: http://science.slashdot.org/st... [slashdot.org]

“The problem is that comparable studies published by other authors have already been refuted in open discussions and to some extend also in the literature (Trenberth et al. 2010), a fact which was ignored by Spencer and Braswell in their paper and, unfortunately, not picked up by the reviewers.”

Re: Science works (0)

Anonymous Coward | about 5 months ago | (#47382563)

Yeah, but since global warming gets in the way of American Capitalism, you can't talk about it in terms of science. Find an error? Must be a socialist conspiracy. People using slang and inside references in emails to people they know well? Absolute proof that 98% of scientists worldwide are simultaneously in on a conspiracy to get rich off of grant money. Find new evidence? Flip flopper!

That's the anti science American conservative spin machine and their authoritarian follower audience at work, and of course it's massively detrimental to progress and truth but they don't care about that. They only care about their beliefs and lifestyles never being questioned. Even if their behavior (over consumption of resources and under control of pollution for instance) was done with no ill will and no foreknowledge, to admit they are incorrect is to them admitting they have a character flaw and/or that their way of doing things isn't perfect. The authoritarian follower brain melts down in those situations.

The notion of changing one's position based on new information is utterly alien and completely unacceptable to them. So is the notion that a person they might not like could possibly be right about anything.

That's why we have such a misunderstanding of and hostility twoards science in the US these days.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 6 months ago | (#47381029)

Viola.

I do not think that word means what you think it means.

Re:The same way many global warming papers got pub (0)

ArcadeMan (2766669) | about 6 months ago | (#47381063)

Are mistakes on the Internet making you sad? Let me play the world's smallest viola for you.

Re:The same way many global warming papers got pub (0)

Ol Olsoc (1175323) | about 6 months ago | (#47381159)

Viola.

I do not think that word means what you think it means.

Typo. Other than that, "voila" is the correct spelling. Now, contribute to the discussion, lest ye be thought a spelling cop asshat.

Re:The same way many global warming papers got pub (2, Funny)

Anonymous Coward | about 6 months ago | (#47381395)

(different AC)

I was more worried about the pee reviewers ,,,

Re:The same way many global warming papers got pub (1)

Ol Olsoc (1175323) | about 6 months ago | (#47381435)

(different AC)

I was more worried about the pee reviewers ,,,

Good one! Tater Salad approved.

Re:The same way many global warming papers got pub (2)

geogob (569250) | about 5 months ago | (#47382363)

No. The correct spelling is "voilà".

Re:The same way many global warming papers got pub (1)

Megol (3135005) | about 5 months ago | (#47382529)

It is not commonly spelled so in English. It is a bit naïve to think there is only one true spelling all over the world.

Re:The same way many global warming papers got pub (2)

geogob (569250) | about 5 months ago | (#47382551)

Although alternative spelling may be current, there is only one correct spelling. Ignoring the correct spelling, doesn't make the common alternative spelling suddently correct.

Re:The same way many global warming papers got pub (1)

evilbessie (873633) | about 5 months ago | (#47385441)

No, you seem to not understand how English works, there are many correct spellings for things, for the most part ignoring diacritics. Dictionary's are a record of how a language is used not a unchanging tome (French excepted). If that's how people spell the word and others understand it, that IS the correct spelling.

Re:The same way many global warming papers got pub (1)

Ol Olsoc (1175323) | about 5 months ago | (#47385505)

Although alternative spelling may be current, there is only one correct spelling. Ignoring the correct spelling, doesn't make the common alternative spelling suddently correct.

Thnakss fro the vawllooble input. Yu speling fascists crak mi uwp.

Re:The same way many global warming papers got pub (5, Informative)

Anonymous Coward | about 6 months ago | (#47381399)

Perhaps one source of misunderstanding here is that some people assume that peer review is supposed to be some kind of ultimate validation of a work. It's not, it's a basic sanity check and a validation that what the authors *claim* to have done is sufficiently interesting. It's not an endorsement by the publication venue that the work is correct and the authors are honest, because it's impractical to validate something like that without investing a lot more effort than feasible for a peer review. Reproducing a work might take months, even years.

Re:The same way many global warming papers got pub (1)

Ol Olsoc (1175323) | about 6 months ago | (#47381445)

Perhaps one source of misunderstanding here is that some people assume that peer review is supposed to be some kind of ultimate validation of a work. It's not, it's a basic sanity check and a validation that what the authors *claim* to have done is sufficiently interesting. It's not an endorsement by the publication venue that the work is correct and the authors are honest, because it's impractical to validate something like that without investing a lot more effort than feasible for a peer review. Reproducing a work might take months, even years.

And you post AC? Very well said. The peer review process is just setting the stage for others to do more research or replicate the results. It's a sanity check, not the proof of the paper. The proof comes later.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 6 months ago | (#47381429)

" The biggest difference is that science accepts it's lumps and corrects them."

Could you accept that it's means it is and correct that from now on?

You've just been peer-reviewed.

Re:The same way many global warming papers got pub (0)

Ol Olsoc (1175323) | about 6 months ago | (#47381473)

" The biggest difference is that science accepts it's lumps and corrects them."

Could you accept that it's means it is and correct that from now on?

You've just been peer-reviewed.

It is Science's lumps, it"s lumps. possessive.

Re:The same way many global warming papers got pub (1)

DNS-and-BIND (461968) | about 6 months ago | (#47381491)

You miss the point. The paper did pass peer review and was published. In a credible journal. It is a great embarrassment to have to retract the paper. The vaunted peer review - supposed to eliminate problems like this - failed.

Re:The same way many global warming papers got pub (2)

ColdWetDog (752185) | about 6 months ago | (#47381763)

And the end result of this is that Nature, along with other high profile journals, will continue to improve the peer review system. Just like they taught us in Science School. Experiment, look at results, repeat....

Furthermore, peer review isn't all that 'vaunted' - we've known for a long time that bad science gets through peer review. It's just one semi-convenient method of screening. The ultimate screening tool is repeating the experiment. That isn't practical in many cases. Although in this case, it should really have come to mind since Nature had recently asked another researcher to do just that for a less 'extraordinary' result.

Re:The same way many global warming papers got pub (1)

khallow (566160) | about 5 months ago | (#47382971)

And the end result of this is that Nature, along with other high profile journals, will continue to improve the peer review system.

Unless the amount of errors are below their threshold for action. Or they don't act to improve their peer review systems in response.

Re:The same way many global warming papers got pub (4, Informative)

eli pabst (948845) | about 6 months ago | (#47381931)

The vaunted peer review - supposed to eliminate problems like this - failed.

Not really. Peer review is designed to catch holes in their logic or spot errors, such as if the incorrect analysis method was applied or if their scientific evidence doesn't fully support their claim. When it comes to outright fraud, a peer reviewer really has very limited means of spotting it. In exceptionally rare cases they will request that a claim be replicated by an outside researcher, but that is exceedingly rare and I don't think I've ever heard of a reviewer actually attempting to replicate research themselves as part of the peer-review process.

What normally happens is that other people in the field will read the paper and say "I don't really buy this" and attempt to replicate it themselves. If a consensus of groups can't replicate their findings, then the question becomes whether there was fraud involved or if it was just another example of "winnners curse" or maybe something unique about their study that was different from all the rest (like if they were looking at a different cell line or global population than everyone else). In no case is it really feasible for the peer-reviewer to catch outright deceptive fraud, but usually it gets spotted sooner or later. And the bigger the scientific claim, the bigger the bulls-eye becomes on your back.

Re:The same way many global warming papers got pub (1)

dwpro (520418) | about 5 months ago | (#47382783)

I don't think I've ever heard of a reviewer actually attempting to replicate research themselves as part of the peer-review process.

My wife actually had a reviewer go out into the bush and collect data to counter one of her assumptions, despite numerous publications making the same assumption. Some people just can't help themselves.

Re:The same way many global warming papers got pub (1)

Ol Olsoc (1175323) | about 5 months ago | (#47385249)

You miss the point. The paper did pass peer review and was published. In a credible journal. It is a great embarrassment to have to retract the paper. The vaunted peer review - supposed to eliminate problems like this - failed.

No I didn't. Nature has had papers retracted before.Yes, they were embarrassed. Your "vaunted peer review" comment shows you are just itching to disprove science, or otherwise have some sort of hrdon for it.

Peer Review is there to find glaring errors and omissions. It is a group of people who look at a paper and determine it should be okay to publish.

It's what happens afterward where we find out if the bear shit in the buckwheat or not.

Other researchers try to replicate the results, or somoene sees something wrong and calls it out.

I think that your idea of peer review would limit research to things that we already know, because after all, how can the peers determine that every single item in any paper is correct if they don't already know everything in the paper already?

This is not to belittle the fraud that Haruko Obokata pulled on many people, including her very respected co-authors. This was serious, and she should never work in research again. There are lessons to be learned, including not giving any sort of pass based on reputation - very bad, because she did most of the work, her co-authors probably just a cursury overview. Once it got past that point, the peer process wasn't as thorough as it should have been.

Nastyass bad mistakes, based on the fraud of a researcher who now has as much credibility as the asshat who was in cahoots with a lawyer in order to get money off sympathetic juries in the vaccine/autism fraud case. Main difference here is I'm not certain what her specific reasons were.

But while you are going OH, Noes!, I'm saying looks lika another one was caught.

Re:The same way many global warming papers got pub (1)

AmiMoJo (196126) | about 5 months ago | (#47383081)

At this point it isn't even clear if she really did lie. She certainly screwed up, but the experiments are being repeated to determine if the method actually works or not. It's too early to say if this is fraud or not.

Re:The same way many global warming papers got pub (0)

Anonymous Coward | about 6 months ago | (#47380973)

The science on that is settled, so that is not a good example. The scientists have already voted.

Re:The same way many global warming papers got pub (1)

Luckyo (1726890) | about 5 months ago | (#47382199)

It was peer reviewed and found to be incorrect within a month of publication.

How on earth can you compare this to global warming?

Re:The same way many global warming papers got pub (2)

cheesybagel (670288) | about 5 months ago | (#47382641)

This experiment was actually easy to reproduce. Unlike the global warming climate model. Besides even when the model does not hold, as it has for quite some time, people just prefer to ignore reality instead.

http://www.nature.com/news/cli... [nature.com]

As we speak they prefer to clutch at straws rather than consider that the effect was caused by solar activity. Ah well.

Re:The same way many global warming papers got pub (4, Insightful)

hey! (33014) | about 5 months ago | (#47383665)

Peer reviewed. Yeah, right. And just who is reviewing the peers?

Ha! I knew the denialists would come swarming out of the woodwork on this one.

Consider the stem cell paper that we're talking about here. It was published in January and immediately started going down in flames. Here we are six months later, watching scientists gleefully kick the cold corpse of the authors' reputations. And you're still wondering who keeps the reviewers and editors of a scientific journal honest?

Peer review isn't some kind of certification of a paper's truth. It can't reliably weed out misconduct, experimental error, or statistical bad luck. It's just supposed to reduce the frequency of fiascos like this one by examining the reasoning and methods as described in the paper. It doesn't have to be perfect; in fact it's preferable for it to let the occasional clunker through onto the slaughterhouse floor than to squelch dissenting views or innovation.

That's why climate change denialists still get published today, even the ones who disbelieve climate change because it contravenes their view of the Bible. Peer review allows them to keep tugging at the loose threads of the AGW consensus while preventing them from publishing papers making embarrassingly broad claims for which they don't have evidence that has any chance of convincing someone familiar with the past fifty years of furious scientific debate.

Same As It Ever Was (-1)

Anonymous Coward | about 6 months ago | (#47380785)

And you may find yourself living in a shotgun shack
And you may find yourself in another part of the world
And you may find yourself behind the wheel of a large automobile
And you may find yourself in a beautiful house, with a beautiful wife
And you may ask yourself
Well...How did I get here?

Letting the days go by
Let the water hold me down
Letting the days go by
Water flowing underground
Into the blue again
After the money's gone
Once in a lifetime
Water flowing underground ...

And you may ask yourself
How do I work this?
And you may ask yourself
Where is that large automobile?
And you may tell yourself
This is not my beautiful house
And you may tell yourself
This is not my beautiful wife

Letting the days go by
Let the water hold me down
Letting the days go by
Water flowing underground
Into the blue again
After the money's gone
Once in a lifetime
Water flowing underground

Same as it ever was...

We should expect some wingnuts to say... (2)

thieh (3654731) | about 6 months ago | (#47380797)

"See, this is exactly why we oppose stem cell research. They are all frauds."
Seriously though, I would have imagined that the papers should only get published if the results themselves were reproducible. Somehow those are skipped and the whole peer review system is in trouble. At the end, I would think whoever reviewed the papers should also be disciplined.

Re:We should expect some wingnuts to say... (-1)

Anonymous Coward | about 6 months ago | (#47380955)

No, not exactly. Opposition is mainly against abortion harvesting murderers and gene manipulation the third reich never dreamed of. Having general distaste for lying, scheming morons is just normal.

Re:We should expect some wingnuts to say... (0)

Desler (1608317) | about 6 months ago | (#47380983)

Much better that we incinerate them, right?

Re:We should expect some wingnuts to say... (-1)

Anonymous Coward | about 6 months ago | (#47380965)

Wow. What an asshole you are.

Re:We should expect some wingnuts to say... (3, Insightful)

Anonymous Coward | about 6 months ago | (#47380985)

And where would the money to reproduce the results come from? Journals typically do not even pay peer reviewers, so you're essentially expecting them to use their own grant money and time to reproduce results knowing that it will get them precisely fuck-all in terms of career advancement. If that were actually the expectation the simple reality is that it would kill off the peer review system.

Re:We should expect some wingnuts to say... (5, Informative)

Anonymous Coward | about 6 months ago | (#47380989)

Seriously though, I would have imagined that the papers should only get published if the results themselves were reproducible. Somehow those are skipped and the whole peer review system is in trouble.

Actually, the whole peer review system is not in trouble. See, the peer review done by (volunteer) reviewers for the magazine is just the first step. The next step comes when the article is published and the entire world gets to see the paper. The fact that the fraud was exposed in pretty short order after publication shows that, indeed, peer review does work.

Re:We should expect some wingnuts to say... (0)

Anonymous Coward | about 6 months ago | (#47381751)

I would assume the fear is that a less important lie would get through. But I think you're probably correct, since obviously other experts in any particular portion of the science would quickly discover that the discovery was false. It might require something like a conspiracy to establish a real lie as fact if it could be easily disproved by other researchers with the same focus.

Re:We should expect some wingnuts to say... (4, Insightful)

ColdWetDog (752185) | about 6 months ago | (#47381769)

Exactly. This.

The road to scientist's Hell is paved with journal articles that eventually have been shown to be incorrect. Just because it's published doesn't mean it's gospel. It is Science after all, not religion - even if half of Slashdot seems to think they're one in the same.

Re:We should expect some wingnuts to say... (3, Insightful)

pepty (1976012) | about 6 months ago | (#47381371)

Until grants provide funding for reproducing old results, publishing a paper will remain the first step in getting the work reproduced. Typically by someone who needs the technique/result for their own work.

Re:We should expect some wingnuts to say... (2)

evo2 (2022012) | about 6 months ago | (#47381593)

Seriously though, I would have imagined that the papers should only get published if the results themselves were reproducible.

Who is going to reproduce unpublished work? Who is going to even know there is something reproduce if it hasn't been published? Publication of the original study necessarily comes before independent replication.

Somehow those are skipped and the whole peer review system is in trouble. At the end, I would think whoever reviewed the papers should also be disciplined.

I think you don't understand how peer review works. Publication is just the first step.

Um, this is how it's supposed to work. (3)

aussersterne (212916) | about 6 months ago | (#47381853)

Journals aren't arbiters of Truth (capital T), they're just what they say they are: JOURNALS of the ongoing work of science.

Someone records that they have done X in a journal. Because said journal is available to other scientists, other scientists get to try to make use of the same notes/information/processes. If they are able to do so, they journal it as well. Get enough mentions in enough journals that something works, and we can begin to presume that it does.

If only one mention in one journal is ever made, then it is just another record in another journal of another thing that one scientist (or group of scientists) claim to have done.

Peer review is just to keep journals from expanding to the point that there is too much for anyone to keep track of or read. It is emphatically NOT the place at which the factuality or truthfulness of notes/information/processes are established once and for all. That happens AFTER publication as other scientists get ahold of things and put them through their paces.

Seriously, this is all exactly as it is supposed to work. I have no idea why there is such hoopla about this. There is nothing to see here. One group journaled something, other groups couldn't replicate it, they no doubt will reference this failure in future articles, and "what happened" is recorded out in the open for all of science, thereby expanding our pool of knowledge, both about what consistently works in many situations and of what someone claims has worked once in one situation but appears either not to work in the general case or requires more understanding and research.

Again, there is nothing to see here. Let's move on.

Um, this is how it's supposed to work. (0)

Anonymous Coward | about 5 months ago | (#47386801)

Well put; I agree with just about everything you said. The only slight issue I had regards your statement:

"Peer review is just to keep journals from expanding to the point that there is too much for anyone to keep track of or read."

If that were the primary role of peer review it could be considered an abject failure (I think the primary role of peer review is as a first order coherence and relevance check on the work of researchers - that it has a limiting effect on the proliferation of publications is more of a useful by product). As a matter of fact the number of journals articles have expanded to the point that even ultra specialists have an extremely difficult time keeping up with the literature in their field and I think this will develop into a serious problem. Scientific development requires a degree of coherence between practitioners that I think is currently in danger of being lost. Sure the "coherence time" may a grow a little without causing too much trouble but it can't get arbitrarily large without doing irreparable damage. But maybe this problem will be "solved" by the drying up of funding (though hopefully it can be addressed in a less disastourous manner).

Re:We should expect some wingnuts to say... (1)

jmv (93421) | about 5 months ago | (#47386703)

So you're saying that reviewers should have to reproduce the results (using their own funds) of the authors before accepting the papers or risk being disciplined? Aside from ending up with zero reviewers, I don't see what this could possibly accomplish. Peer review is designed to catch mistakes, not fraud.

I can't imagine... (1)

Arkh89 (2870391) | about 6 months ago | (#47380907)

I can't imagine this was done on purpose... I can't imagine how a scientist would, knowingly, publish wrong results (and perceived as revolutionary/important by their peers). Because this would be nothing more than willingly putting a sword of Damocles over your head / committing professional suicide on the spot. I mean, how is that possible that rational people (scientific minds) would accept to do such thing while being sure it will compromise their entire career (and life) after that?

Re:I can't imagine... (1)

Anonymous Coward | about 6 months ago | (#47380951)

Hmm...well, without some people over time DOING EXACTLY THAT (putting out unorthodox ideas, throwing their careers or lives away because the orthodoxy wouldn't put up with it at the time) we'd still be believing/being told that the earth was flat, the center of the universe, etc.

Re:I can't imagine... (0)

Anonymous Coward | about 6 months ago | (#47381569)

> how a scientist would, knowingly, publish wrong results
Learn to read.

Re:I can't imagine... (5, Insightful)

Beck_Neard (3612467) | about 6 months ago | (#47380967)

It's probably not the case that she wrote the paper cackling to herself madly and proclaiming "Those suckers will never find out!"

It's probably the case that through self-delusion and carelessness she managed to partially convince herself that the results were true, and this, coupled with pressure to produce results, caused her to take a few shortcuts to get it published. What she did was wrong, and her career is over. It's not something a rational mind would have done. But scientists are just human and sometimes prone to making irrational decisions. The great thing is that we have the scientific method to weed out the good from the bad.

Just the scientific method in action (1)

Anonymous Coward | about 6 months ago | (#47381179)

What she did was wrong, and her career is over.

You don't seem to understand how Science works. Theories that later turn out to be wrong are proposed all the time, and doing so is not a blot on your academic record unless it is proved (or strongly believed) that you did it on purpose and hence wasted everybody's time knowingly .

If it wasn't deliberate then making an erroneous proposal is a perfectly normal occurrence in Science, and we rely on other scientists to confirm or invalidate our work. Obviously we try our best to get it right, but scientists are human and that's why one person's results are merely the start of the process and are meaningless without multiple confirmations by others.

The real story here is how come experienced peer reviewers accepted the paper if the substance was as poor as alleged.

Re:Just the scientific method in action (1)

Michael Woodhams (112247) | about 6 months ago | (#47381257)

I think you've misread the post. In "What she did was wrong", I read "wrong" as "unethical", "unscientific", or at the very best "incompetent". Your criticism assumes it meant "something which eventually turned out not to be how reality works".

Re:Just the scientific method in action (1)

Beck_Neard (3612467) | about 5 months ago | (#47382609)

If you look at the case, there's evidence she did slight but deliberate manipulation and misrepresentation of results. That's what I meant by wrong.

Re:I can't imagine... (1)

vossman77 (300689) | about 6 months ago | (#47381409)

Self-delusion seems to be extreme or unlikely in this case. Based on this blog [blogspot.com] , she plagiarized her entire abstract only changing the cell name from ES to STAP.

A simple plagiarism detector would have detected this fraud. I run turnitin.com service on all my students' papers and I catch plagiarizer every semester. I think this should be routine for high profile article going in to Nature or Science.

Re:I can't imagine... (2)

ColdWetDog (752185) | about 6 months ago | (#47381775)

According to TFA, Nature did run it through a similar database. No obvious plagiarism was found. It turned out that the journal she originally published in was not in the database. There are one hell of a lot of journals out there.

Re:I can't imagine... (1)

techno-vampire (666512) | about 6 months ago | (#47382005)

Two questions: first, do you tell your students that you're going to be doing this? (I ask because it it would say quite a bit about the plagiarizers if they knew in advance that you'd be checking.) Second, how do they try to justify their actions when they're caught?

Re:I can't imagine... (1)

vossman77 (300689) | about 5 months ago | (#47382943)

Yes, they can actually look at the plagiarized sections and fix them. The reaction tends to depend on how I contact them. If I am nice and say they can redo it for most the points, they admit their mistake and fix it. If you say they are getting a zero, then they start lying and say they do it in other classes and it is fine.

Re:I can't imagine... (4, Interesting)

gweihir (88907) | about 5 months ago | (#47382319)

One very common scenario for knowingly faked "results" is this: PhD Student has his/her funding running out and gets set an ultimatum (explicitly or implicitly). PhD student fakes something, sometimes looking pretty good at first glance. Advisor is too stupid, lazy or full of him/herself to notice. Paper gets published because advisor is "respected" in the community.

I have seen this happening quite a few times, including one case where all authors, except the first one (the PhD student), wrote what was basically a retraction a year later. But did anything happen to these people? No. The PhD student still has his PhD, despite his results being essentially worthless. The other authors still have their reputation. The faked publications were not retracted. I did recognize the fake in 10 Minutes by numerous inconsistent things and numbers that did not add up and did not make sense at all. None of the reviewers apparently did. Just when people tried to reproduce the results and failed were some question asked. But as I said, no consequences for blatant scientific misconduct by several people. For me, this nearly cost me my PhD as my advisor was not even capable of understanding the fake after I explained it to him in detail and somehow though they were doing something vastly superior to my work. While the low-point of my scientific work, it made me understand that most so-called "scientists" do not qualify for that distinction.

Re:I can't imagine... (1)

schnell (163007) | about 5 months ago | (#47386433)

I did recognize the fake in 10 Minutes by numerous inconsistent things and numbers that did not add up and did not make sense at all. None of the reviewers apparently did.

This isn't intended to be disrespectful, so please don't take it the wrong way: why were you, as a PhD student, able to find this error when the reviewers (and theoretically other scientists in the field) weren't?

Is yours a small field with few people to review? Were the reviewers of this paper lazy or cowed by celebrity or influence? Was this published in a seldom-read journal? Or what? I'm honestly very curious about how a lapse like this happens.

How did this happen? (-1)

Anonymous Coward | about 6 months ago | (#47380927)

Simple, by accident they followed the process used for review of Climate Change research.

World is Addicted to Hopeium (1)

lazy genes (741633) | about 6 months ago | (#47381141)

Everybody wants to believe in miracles.

Re:World is Addicted to Hopeium (1)

UnknownSoldier (67820) | about 6 months ago | (#47381265)

"The reasonable man adapts himself to the world;
the unreasonable one persists in trying to adapt the world to himself.
Therefore all progress depends on the unreasonable man."
  - George Bernard Shaw

Better to aim for the stars and hit the moon
then to give up on hope and be depressed.

World loves a cute little Obokata (1)

donscarletti (569232) | about 6 months ago | (#47381829)

Everyone also wants to believe pretty little Dr Obokata, who is has adorable cartoon characters on her lab equipment and wears a cooking apron while experimenting managed to cure mortality. I mean, she would be such a fantastic science poster girl.

I mean, I just looked at her pictures and completely forgot about anything scientific. One look and I was: "Forget about the chimeric rats, lets see if I can inject some of my non-pluripotent cells into her and create some embrionic stem cells!"

My strategy for 100% acceptance rate. (1)

Snufu (1049644) | about 6 months ago | (#47381191)

"Nice journal you got here. Shame if it got broke..."

Re:My strategy for 100% acceptance rate. (1)

blue trane (110704) | about 6 months ago | (#47381643)

Doug and Dinsdale Piranha, using a combination of violence and sarcasm!

Because peers aren't magical (1)

paiute (550198) | about 6 months ago | (#47381663)

A reviewer might think that the results are not implausible, so the paper can go ahead. Reviewers don't try to reproduce the results in their own lab.

Re:Because peers aren't magical (1)

Arkh89 (2870391) | about 6 months ago | (#47381753)

Reviewers don't try to reproduce the results in their own lab.

Reviewers don't have human*time and financial resources to reproduce results in their own lab.

FTFY

Re:Because peers aren't magical (4, Interesting)

Foske (144771) | about 6 months ago | (#47382013)

As a former reviewer working for a very renowned research institute in Europe I can say: Peers typically don't get/take the time to do their job right, and often outsource the job to less experienced people. Reproducing results is a very expensive and time consuming job, which means: unless it is it won't happen. You must be lucky if the reviewers have at least read the paper till the end. Quite often the review happens by people who are "no experts" in the field of the paper. For many conferences, papers with a bad rating still pass because there are not sufficient good papers, or if it is easy to guess the institute the authors work for, the paper passes without proper review.

Once our institute had a paper rejected, but my boss -who was in the review team- managed to get the paper accepted anyway. High profile conference in Electronic Engineering.

As a former paper author I can say: If your paper is rejected for one conference, you simply resubmit to another until it is accepted. Publish or perish is the holy grail of research, something many bosses will make very clear to you, and quality is less important. You don't write a paper because you have results, you write a paper because this or that major conference has a deadline in two weeks. I have a few paper on my name I am ashamed of: Omitting the bad results in the measurements, compare with competitors only on the features you know you would win because the comparison doesn't make sense at all, bragging about results which are very bad, but you hide that by not comparing to (avoiding any reference to) competitors which are better.

As you might understand, I quit the job. I left research and never ever want to have anything to do with it anymore.

Re:Because peers aren't magical (3, Insightful)

gweihir (88907) | about 5 months ago | (#47382289)

There are some islands of honest and competent conferences and reviewers, but they are usually in not very well known events. All that is mainstream, "Tier-I" conferences and Journals are fundamentally corrupt. I mostly left research for the same reason, but I occasionally still publish something these days. The difference is that I publish if I have something good and interesting, not when some stupid research administrator thinks I should have more papers. And I publish in a venue where I respect the people running the conference even if that gives a lot less scientific "reputation".

They should have an investigation (1)

Karmashock (2415832) | about 6 months ago | (#47381955)

... by the journal itself... investigating itself... the result will be that they don't find any wrong doing in their own journal.

And then the media will report "journal finds no wrong doing"... and then we can all go back to sleep.

Right guys?

The issue is not this one journal. Its a general lack of scrutiny in science itself. They are not being audited. The data is not being checked. The experiments are not being replicated.

Re:They should have an investigation (4, Informative)

serviscope_minor (664417) | about 5 months ago | (#47382275)

The issue is not this one journal. Its a general lack of scrutiny in science itself. They are not being audited. The data is not being checked. The experiments are not being replicated.

I believe you are mistaken, or alternatively I have a different interpretation of events.

No system is perfect, however the system works more or less as follows.

Scientists do work.

They are under pressure to publish because not unreasonably the people paying for the scientists like to see their money is going somewhere useful even if the metric is far from perfect. Anyway it generates strong pressure since science is up or out an unlike many other careers, if you don't get promoted, you're eventually fired.

Work is submitted to a journal or conference. Note in HEP (high energy physics) people just shove ut up at xxx.lanl.gov and send it to a journal something as an afterthought to satisfy the previous point.

Editors filter the papers. At low journlas almost every submission gets sent out to review. At very competitive journals, the editors strongly filter papers. It helps to know an editor to get your papers past this filter. Some journals (Nature) have paid editors. Most to not.

It gets sent out to other scientist volunteers to review it. The review is really a check of reasonableness. Nobody expects the reviews to find deliberate fraud. That's not the point. They're not there to check for actual correctness, only reasonableness. In other words no one excpects them to replicate the results. The reviewers have not much time. Some less scrupulous ones farm it out to students to review. Some are mad as a sack of badgers. Some will insist you cire them more. Others may be able to check some of the results for correctness if they look suspicious (I've done this before when I believe I've spotted a glaring error I can test). In some very simple cases (e.g. Magnesium Diboride as a semiconductor) people are able to verify the results before publication.

The previous two steps repeat until the paper is accepted or rejected.

Now the important part starts.

There are two main possibilities here:
1. The paper is boring. In this case it willprobably get a few citiations bulking up someone's lit review and nothing more.
2. The paper is interesting in which case people will try to build on the results.

And orthogonally:
A. The paper is right.
B. The paper is wrong (or bad or fraudlent or etc).

(in practice there is a continuum on both)

Basically in the case of 1 it doesn't matter if the paper is right or wrong. It languishes in obscurity and the rightness or wrongness never ends up having any bearing on the state of the art and "scientific knowledge".

In the case of 2, people will try to build on it and take it further and advance the state of the art because that's where progres is happening. And that's where the scrutiny starts. The more interesting the result, the more intense the scrutiny.

For results of the magnitude of cold fusion and high temperature scrutiny, every scietist and his dog tries to take a look. The results are very quickly found out to be bad (cold fusion) or good (High Tc).

This was a big result, and what you're seeing now is the scrutiny.

The journal peer review process is merely there to provide a good place to find new results and stop the scientific world getting flooded in a mound of utter dross. It's the very first part of the filtering process.

Once a result is public is can undergo scrutiny and generally this happens in the public eye with people posting comments, other papers, commenting on the internet and so on.

Scientific results in practice undergo more scrutiny than almost anything else, and you're seeing a little bit of it right here even on slashdot.

Re:They should have an investigation (3, Informative)

Karmashock (2415832) | about 5 months ago | (#47382355)

Well, then whether something makes it through peer review or not should not be used as evidence that it is correct or not... merely interesting and reasonable.

Re:They should have an investigation (1)

serviscope_minor (664417) | about 5 months ago | (#47382599)

Well, then whether something makes it through peer review or not should not be used as evidence that it is correct or not... merely interesting and reasonable.

More or less yes. It's a weak piece of evidence in favour but it's very context dependent. Certainly anything new and contraversial should be treated with care.

Re:They should have an investigation (0)

Anonymous Coward | about 5 months ago | (#47382651)

Competent peer review includes demands for appropriate controls. Control experiments are the main thing that differentiates real science from cargo cult science. As a side bonus, it is much harder to fabricate an entire, self-consistent set of multiple experiments than it is to copy one or two images, photoshop them a little, and recaption. Under the argument of "not enough time," it seems like concern for appropriate controls has diminished of late.

Journals like Science and Nature, by intent and design, are too terse to include extensive controls, and authors generally follow a Science paper with a "real" paper, showing extensive methods and controls, in another journal. This means that Science and Nature will often publish astonishing, ground-breaking research that changes the whole direction of a field. It also means that Science and Nature frequently publish things that are completely wrong.

Simple: Peer review is badly broken (4, Interesting)

gweihir (88907) | about 5 months ago | (#47382277)

Peer review works if the people doing the review are honest and competent. Both aspects have been in sharp decline, not only in the biomedical field. These days, positions for Professors and PhD students are more often than not filled with people that can simulate competence and that have no or little personal ethics whatsoever. They will form groups that accept any and all papers from each other and reject anything from others. Anything original also generally has a high chance of getting rejected, unless the reviewers know and like the authors. The peer-review system is so broken and corrupt that it has just stopped working as the quality of the "researchers" forming it is way too often abysmally bad. (And forget about "anonymous reviews". The in-group has all the Tech-Reports from their friends and can recognize all papers written by them.)

This is not a new phenomenon, it seems to just be getting worse again. But remember that Shannon had trouble publishing his "Theory of Information", because no reviewer understood it or was willing to invest time for something new.

Re:Simple: Peer review is badly broken (1)

jmv (93421) | about 5 months ago | (#47384005)

This is not a new phenomenon, it seems to just be getting worse again. But remember that Shannon had trouble publishing his "Theory of Information", because no reviewer understood it or was willing to invest time for something new.

That's the problem here. Should the review system "accept the paper unless it's provably broken" or "reject the paper unless it's provably correct". The former leads to all these issues of false stuff in medical journals and climate research, while the latter leads to good research (like the Shannon example) not being published. This needs to be more than just binary. Personally I prefer to accept if it looks like it could be a good idea, even if some parts may be broken. Then again I don't work on controversial stuff and nobody dies if the algorithm is wrong. I can understand that people in other fields have different opinions, but I guess what we need is non-binary review. Of course, reviewers are also just one part of the equation. My reviews have been overruled by associate editors more often than not.

Re:Simple: Peer review is badly broken (1)

gweihir (88907) | about 5 months ago | (#47384527)

I think what is missing is that a) more reviewer actually need to be experts and practicing scientists and b) doing good reviews needs to get you scientific reputation rewards. At the moment,investing time in reviewing well is a losing game for those doing it.

I agree that good reviews do not need to be binary. You can also "accept if this is fixed", "rewrite as an 'idea' paper", "publish in a different field", "make it a poster", etc. But all that takes time and real understanding.

Re:Simple: Peer review is badly broken (1)

jmv (93421) | about 5 months ago | (#47386313)

I think what is missing is that a) more reviewer actually need to be experts and practicing scientists and b) doing good reviews needs to get you scientific reputation rewards. At the moment,investing time in reviewing well is a losing game for those doing it.

Well, there's also the thing that one of the most fundamental assumption you have to make while reviewing is that the author's acting in good faith. It's really hard to review anything otherwise (we're scientists, not a sort of police)

I agree that good reviews do not need to be binary. You can also "accept if this is fixed", "rewrite as an 'idea' paper", "publish in a different field", "make it a poster", etc. But all that takes time and real understanding.

It goes beyond just that. I should have said "multi-dimensional" maybe. In many cases, I want to say "publish this article because the idea is good, despite the implementation being flawed". In other cases, you might want to say "this is technically correct, but boring". In the medical field, it may be useful to publish something pointing out that "maybe chemical X could be harmful and it's worth further investigation" without necessarily buying all of the authors' conclusion.

Personally, I prefer reading flawed papers that come from a genuinely good idea rather than rigorous theoretical papers that are both totally correct and totally useless.

Global Warning (0)

Anonymous Coward | about 5 months ago | (#47382825)

In my country we don't get tornados. But a few weeks ago we got 3 of them in one day. They must have been around F1-F2.

So, why did STAP paper get published? Monetary interests as usuall.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?