NYTimes on frontier science

Nicholas Wade, Lowering Expectations at Science's Frontier, NYT 15 Jan 06:

How then can the fraudulent claims by Dr. Hwang Woo Suk have been accepted by Science, a leading journal that rejects most papers submitted to it? How can the community of stem-cell scientists have allowed a very visible claim to have stood unchallenged in their field for 20 months? Little wonder that Richard Doerflinger, an official of the United States Conference of Catholic Bishops, ridiculed the dreams of therapeutic cloning in a statement last week, scoffing that scientists were chasing miracle cures "in pursuit of this mirage."

The contrast between the fallibility of Dr. Hwang's claims and the general solidity of scientific knowledge arises from the existence of two kinds of science - a distinction that is often blurred when new advances are reported first by scientific journals and then by the news media. There is textbook science and frontier science, and the two types carry quite different expiration dates.

Textbook science is material that has stood the test of time and can be largely relied upon. It may include findings made just a few years ago, but which have been reasonably well confirmed by other laboratories.

Science from the frontiers of knowledge, on the other hand, is wild, untamed and often either wrong or irrelevant to future research. A few years after they are published, most scientific papers are never cited again.

Scientific journals try to impose order on the turbulent flow of new claims by having expert reviewers assess their merit. But even at the best journals, reviewers provide only a rough screen. Many papers slip through that later turn out to be innocently wrong. A few, like Dr. Hwang's, are found to be fraudulent.

This rough screening serves a purpose. Tightening it up, in a vain attempt to produce instant textbook science, could retard the pace of scientific advance.

But the roughness of the proceedings is not prominently advertised by journal editors, except when cases of blatant fraud are detected, whereupon they proclaim that peer review cannot reasonably be expected to detect fraud. They do not protest so much when newspapers report their journals' claims as if they were certifiably true. Because of Science's authority, Dr. Hwang's claims to have cloned human embryonic cells were prominently reported and presented to the public as if they were important breakthroughs.

But any new advance belongs to frontier science, which is inherently fallible, and a journal's imprimatur, though worth something, is no guarantee of truth. An advance only becomes solid when other laboratories have confirmed it, by which time it is no longer news. This presents a serious problem for journalists: many scientific claims, including those in leading journals, turn out to be overstated or wrong, and science reporting that presents these journals' products as gospel is likely to be misleading.

Scientists and journal editors are, of course, well aware of the tentative nature of frontier science. As Donald Kennedy, the editor of Science, observed when the Hwang case first broke, journals often publish work that is innocently wrong. "The public needs to understand that the journals and peer review are not perfect," he said.

But last week Dr. Kennedy announced he was considering revising the journal's publication procedures, though not with any great hope of preventing future cases of fraud. He suggested that authors would be required to state in writing their specific contributions to a report, a reform perhaps aimed at Dr. Gerald Schatten of the University of Pittsburgh. Dr. Schatten accepted senior authorship of - and thus responsibility for - one of Dr. Hwang's papers, even though Dr. Schatten had performed none of the experiments and was not in a position to vouch for them. All the work was done in Seoul.

A second proposed change is to have all authors state that they agree with an article's conclusions. [On IPBiz, we have noted that all authors should at least acknowledge reading the manuscript, and understanding it has been submitted. This did not happen with the Hwang 2005 paper at Science.]

Both procedures may seem to include a certain potential for generating strife. Each author could overstate his or her contribution, arousing the wrath of all the others. Some authors may think a conclusion too timid, while others consider it an overstatement.

But some medical journals, like The Journal of the American Medical Association, already require authors to state who did what. The system works very well, said Drummond Rennie, the journal's deputy editor and the instigator of the idea. Requiring authors to specify that they agree with the conclusions leads to conservative statements, a result that is also beneficial, in Dr. Rennie's view.

Tightening up the reviewing system may remove some faults but will not erase the inescapable gap between textbook science and frontier science. A more effective protection against being surprised by the likes of Dr. Hwang might be for journalists to recognize that journals like Science and Nature do not, and cannot, publish scientific truths. They publish roughly screened scientific claims, which may or may not turn out to be true.