Flashback on peer review

In 1999, Richard Smith of BMJ wrote an article Opening up BMJ peer review which discussed some of the defects of anonymous peer review.

Although written BEFORE the frauds of Jan-Hendrik Schon and Hwang Woo Suk in the journal Science, Smith observed:

Peer review is at the heart of the scientific process yet was until recently largely unexamined. Now we begin to have a body of evidence on peer review (www.wame.org), and it illustrates many defects. Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless for detecting fraud. Smith of course was right, and was certainly proved right by Schon and Hwang.

Smith's article was written after the classic idea theft from the Chu paper on superconductors, and after the 1995 Law & Order episode "Big Bang," and observed:

Finally, openness should eliminate some of the worst abuses of peer review, where reviewers—under the cloak of anonymity—steal ideas or procrastinate.

As to "peer review" of patent applications, a patent applicant is nominally protected from idea theft (it is a patent application), BUT a competitor-reviewer may have a conflict-of-interest in not wanting the patent granted. Idea theft is one thing, and depriving someone else of proper credit/reward is a different thing.

On the flip side, the reviewer's analysis may be influenced by fear of reprisal. Smith wrote:

The main argument against open peer review—a sad one—is that junior reviewers will be reluctant to criticise the work of senior researchers for fear of reprisals.

*****

One IPBiz reader asked about Chu. Back in 1987, the New York Times reported part of the story:

Ytterbium was indeed the element named in the manuscript that had been submitted by Chu's team, relative unknowns at the University of Houston, to Physical Review Letters, the premier journal for reporting breakthroughs in physics. But when the journal appeared on March 2, the final paper named a different element, yttrium.

Chu had pleaded with the journal for special handling, insisting on secrecy, fearful that the editors would leak. ''Which we now know they did - like a sieve,'' says Arthur J. Freeman, a theoretical physicist at Northwestern University. ''Only they leaked ytterbium instead of yttrium. I had heard for weeks that the material was ytterbium, and now I know where it came from.''

There was little doubt that Chu's "ytterbium" idea was stolen through leaks at PRL. Chu however had placed an incorrect idea in the PRL drafts, so the thieves got worthless chaff, but had the audacity to complain about it.

Also in the NYT article:

As news of the yttrium-ytterbium affair spread through the scientific world, the journal's editors denied vehemently that they had divulged the secret. They privately expressed anger at Chu, suspecting an intentional deception on his part to mislead competing researchers. (Chu's friends share the suspicion. They have been retelling the joke about the king who leaves to his favorite knight the key to his queen's chastity belt, only to hear the knight gallop up behind him, shouting angrily, ''It's the wrong key.'')

(...)

Patent lawyers were chaperoning the research teams like pilot fish surrounding sharks.
(...)

Then Bednorz came across a paper by some French chemists describing an oxide of copper mixed with two other elements, barium and lanthanum. The French had never cooled the substance to look for superconductivity. Bednorz and Muller did -and, on Jan. 27, 1986, they struck gold. As Bednorz cooled his sample, measuring a current passing through tiny wires attached to it, he discovered a sharp drop in resistance. By April, the two men had raised the record for a superconductor from 23 kelvins to 35 - still 397 degrees below zero Fahrenheit. That was not warm enough for new practical applications, but it was warm enough to rekindle interest in superconductivity's future.

Muller and Bednorz made no announcement. They did not even tell scientists at other I.B.M. laboratories. They submitted a modest paper, not to Physical Review Letters, but to the German journal, which they knew would remain unread by most physicists. One reason for the two scientists' caution was that the history of superconductivity had been littered with false alarms. Another was that Muller and Bednorz wanted to continue their work in peace. ''We expected we would have a delay time of one or two years,'' Bednorz says. ''Even at I.B.M. we didn't spread the preprint. We wanted to measure as much as possible without being pressed by competitors.''

CHU'S GROUP HAD A routine: the researchers divided up the journals and were responsible for catching any news of even the remotest significance. ''I told them you don't have to understand what you read, but come back and tell us what you think is exciting,'' Chu says. Zeitschrift fur Physik is not an obscure journal, but at places like Bell Laboratories it went unnoticed. ''It's not in our tea room library,'' said Bertram Batlogg, who rushed into the fray at Bell a month later. Chu, calling his staff together that morning last November, had a headstart. [IPBiz note: years later Batlogg would become enmeshed in the Schon science fraud.]

Racing to confirm the Zurich discovery and advance it to practical temperatures, Chu called on the intuition that his mentor Matthias had tried to foster. Even with the Zurich recipe at hand, Chu was operating almost blind: no one knew exactly what the desirable crystal structure was, because four elements mixed together can produce dramatically different substances depending on how they are baked or how they are cooled. Muller and Bednorz had stumbled upon a particular crystal by an accident of preparation - a different structure from the one discovered by the French. ''That was a kick of luck,'' Bednorz says. Chu found that he could duplicate the accident, but his first samples of the material were unstable. One day they would prove to be superconductors; four days later, after reacting with water vapor and carbon dioxide in the air, these porous ceramics would once again be worthless. In the middle of this work, at a scientific meeting in Boston last Dec. 4, Chu gave a long-scheduled talk on an earlier oxide superconductor and, at the end, described his latest results with the new materials. Koichi Kitazawa, a physicist at the University of Tokyo, was in the audience. His group, too, had read the I.B.M. paper and begun a race to pursue its promise. After telephoning Tokyo for the latest data in his lab there, he told the Boston meeting about his results so far.








*****
Note a californiastemcellreport post

Steven Kessler, a scientific director at Advanced Cell Technology of Los Angeles, was not happy with the response he received from CIRM staff on a letter he wrote concerning what he said was a conflict of interest on the part of a unnamed reviewer.

According to the transcript of the meeting, here's how Kessler summarized his position for CIRM directors:

"If a grant reviewer has a financial relationship with company "X"...that is, he's receiving funding from that organization or he's expecting royalty income from some company by virtue of having licensed technology to that company and that reviewer is sitting in on reviews from other for-profit organizations...and doesn't recommend those for funding, to us, from a business perspective, that's a conflict of interest."

Kessler said he had cited "numerous instances" of conflicts on the part of the reviewer, where there would be "every incentive to help impede the competition for the company that he has a relationship with.".

Kessler said,
"I was told that the way CIRM interprets its own conflict of interest policy, the example I gave you was not a conflict of interest."