Referees of science papers check whether a report's conclusions follow from data
A question both Science and Nature have considered is that of whether their editors and reviewers should have caught the errors in Dr. Hwang's papers before publication. But as in past cases of fraud, the journals' editors and other scientists assert that their system depends basically on trust and that reviewers can check only whether a report's conclusions follow from the data presented.
from NICHOLAS WADE, NYTimes, Dec. 25, 2005 as reproduced in the Lakeland, FL Ledger.
As I pointed out in Intellectual Property Today in June 2005, some patent reformers, such as Rebecca Eisenberg in 19 Berkeley Law Tech J. 885 (2004) have advocated the use of referees in the review of patent applications. In 2004, Eisenberg did not mention referee flaws in the Jan-Hendrik Schon incident two years earlier. Now, in 2005, we have an even bigger issue with refereeing by Science and Nature in the Hwang matter. As is evident by the superficial review of the June 17 Hwang paper [the absence of photographs to document all 11 cell lines, the suspicious nuclear DNA profiles, the absence of mitochondrial DNA profiles, the issues with oocyte donors], the reviewers basically accepted on faith what was presented and did not look for inconsistencies. Further, after more explicit identification of problems developed, Science denied problems. For example, on December 6, Science referred to a mere mix-up of photos of the 11 cell lines, even though about 10 days later, even Hwang admitted he had only 8 cell lines at the time of submission to Science (March 15, 2005). If Science thought on March 15 that there was evidence for 11 cell lines, when in fact only 8 existed, there are some major problems. Contrary to what Eisenberg suggests, the review procedure at Science is not one that the USPTO ought to be emulating.
Other problems with Eisenberg's proposed plan were pointed out in the June 2005 article, Patent Reform 2005: Can You Hear Me, Major Tom? The brief of the 24 law professors in KSR v. Teleflex is flawed.
***
Other points from the Wade article:
In addition, the main premise of Dr. Hwang's article, that he could perform human cloning very efficiently, with just a few human eggs, was untrue. The committee said he used far more eggs than he reported.
The panel is also reviewing Dr. Hwang's 2004 claim in which he was apparently the first to clone human cells. If that also proves false, the goal of therapeutic cloning - repairing patients' cells with their own tissues - may be considerably further off than it seemed a few months ago.
How did Dr. Hwang manage to rise so quickly in the scientific firmament and convince so many leading experts that his work was sound?
Three ingredients of his ascent were attracting generous support from the South Korean government, compartmentalizing his laboratory so that few others had any overall view of what was going on and reporting plausible advances that scientists abroad felt they, too, might have achieved if they had access to as many human eggs as Dr. Hwang obtained.
In addition, Dr. Hwang invited well-known American researchers to be co-authors on his articles, which he may have hoped would make his findings more acceptable to leading journals like Science and Nature. He even invited Dr. Gerald Schatten, a stem cell expert at the University of Pittsburgh, to be the lead author on the June 2005 report although Dr. Schatten had done none of the experiments. But Dr. Donald Kennedy, the editor of Science, said the inclusion of American co-authors "certainly did not affect us."
The starting point of Dr. Hwang's rise to fame was his skill in making the South Korean system work for him. The government had invested about $65 million in his research before the collapse came, and the Ministry of Science and Technology had acclaimed him as an "Outstanding Korean Scientist."
The Health and Welfare Ministry promised to provide $15 million next year to set up a World Stem Cell Hub in which Dr. Hwang's technicians would have cloned human cells for scientific customers abroad.
An indication of Dr. Hwang's good connections to the government was the inclusion of Dr. Park Ky Young as a co-author of his 2004 report on human cloning. A botanist by training, Dr. Park may not have contributed much scientifically to the task of cloning of human cells. She is, however, the science adviser to Roh Moo Hyun, the president of South Korea.
The frequent American visitors to Dr. Hwang's growing operation were impressed at the scale and skill of his operation and how he divided his scientists into task forces that specialized in each step of the cloning process. But this compartmentalization may have meant that not all of his co-workers knew what was going on. Few seem to have seen the colonies of embryonic cells Dr. Hwang said he had cloned from patients.
Outside of South Korea, the debacle has left a bitter taste with scientists who trusted Dr. Hwang's work. "It's a sad business," said Dr. Kennedy, the Science editor. "We don't feel like it's our best day."
Science and Nature, two leading journals that compete with each other in publishing striking scientific advances, accepted reports from Dr. Hwang. But Science is now reviewing the accuracy of articles of 2004 and 2005 in which Dr. Hwang said he had cloned human cells, and Nature is re-examining his claim to have cloned a dog, which he named Snuppy.
"We are investigating the Snuppy paper and will review whether we and referees acted appropriately, and whether standards should change," said Dr. Philip Campbell, the editor of Nature.
A question both journals have considered is that of whether their editors and reviewers should have caught the errors in Dr. Hwang's papers before publication. But as in past cases of fraud, the journals' editors and other scientists assert that their system depends basically on trust and that reviewers can check only whether a report's conclusions follow from the data presented.
"Peer review is not set up to test for fraud," Dr. Campbell said. "It is set up to provide expert assessment of the scientific credibility and reliability of what scientists report, taking the report itself in good faith."
Dr. Kennedy noted that journals often published articles that were later shown to be innocently in error. "The public needs to understand that the journals and peer review are not perfect," he said.
Reviewers can, however, recommend that the author provide more data if they are dissatisfied on any point. Science's reviewers asked Dr. Hwang to furnish DNA fingerprints showing that each of his embryonic cell colonies had the same DNA fingerprint as the patient from whom they were said to be derived. Dr. Hwang sent in pairs of fingerprints, some of which had the identical background noise, suggesting the same print was being presented twice. But this anomaly was noticed only later by Dr. Hwang's critics.
Nature's reviewers did not ask Dr. Hwang to provide evidence that would have proved Snuppy was cloned from another dog. Dr. Campbell said that Nature, as part of its investigation of the article, would consider whether its standards of proof should be changed in the future.
Dr. Hwang's failure may have repercussions on the standing of South Korean science.
"Clearly the scientific credibility of Korean investigators has been compromised," said Dr. John Gearhart, a stem cell expert at Johns Hopkins University and a member of Science's board of reviewers. He referred to the fact that duplicate and misidentified photos had turned up in articles by other South Korean authors besides Dr. Hwang.
Dr. Kennedy said, "You cannot avoid a sense of taint from an experience like this." He added, however, that many leading American universities had had at least one case of scientific fraud.
It was also South Koreans who took the lead in detecting Dr. Hwang's falsifications. Dr. Zach Hall, president of the California Institute of Regenerative Medicine, noted that young South Korean scientists had brought to light many problems with Dr. Hwang's papers in Web site postings, and that Seoul National University seemed to be conducting a vigorous inquiry.
The ultimate test of a scientific claim is whether other laboratories independently confirm it. Some scientists have argued that even if Dr. Hwang's errors had remained undetected by the scientific journals and their readers, his work would have fallen under suspicion if no one could repeat it. However, if other scientists had succeeded in cloning human cells before any challenge had emerged to Dr. Hwang's work, it is not so clear that he would have been exposed.
"If the procedure works indeed and other labs would have repeated it, the credit would have gone to Hwang," said Dr. Rudolf Jaenisch of the Whitehead Institute in Cambridge, Mass.
The people best situated to detect scientific problems are those inside the laboratory who see the raw data being generated and have some practical reason for suspicion. As in many other cases of scientific fraud this was true of Dr. Hwang's, too.
It was a whistle-blower in Dr. Hwang's lab who informed the South Korean television network MBC of problems in his work, and that led South Korean journalists to begin to investigate.
But for the whistle-blower, Dr. Hwang might well be continuing his meteoric career on the wings of his reports in Science and Nature.
The fraud passed right through the hands of the referees.
from NICHOLAS WADE, NYTimes, Dec. 25, 2005 as reproduced in the Lakeland, FL Ledger.
As I pointed out in Intellectual Property Today in June 2005, some patent reformers, such as Rebecca Eisenberg in 19 Berkeley Law Tech J. 885 (2004) have advocated the use of referees in the review of patent applications. In 2004, Eisenberg did not mention referee flaws in the Jan-Hendrik Schon incident two years earlier. Now, in 2005, we have an even bigger issue with refereeing by Science and Nature in the Hwang matter. As is evident by the superficial review of the June 17 Hwang paper [the absence of photographs to document all 11 cell lines, the suspicious nuclear DNA profiles, the absence of mitochondrial DNA profiles, the issues with oocyte donors], the reviewers basically accepted on faith what was presented and did not look for inconsistencies. Further, after more explicit identification of problems developed, Science denied problems. For example, on December 6, Science referred to a mere mix-up of photos of the 11 cell lines, even though about 10 days later, even Hwang admitted he had only 8 cell lines at the time of submission to Science (March 15, 2005). If Science thought on March 15 that there was evidence for 11 cell lines, when in fact only 8 existed, there are some major problems. Contrary to what Eisenberg suggests, the review procedure at Science is not one that the USPTO ought to be emulating.
Other problems with Eisenberg's proposed plan were pointed out in the June 2005 article, Patent Reform 2005: Can You Hear Me, Major Tom? The brief of the 24 law professors in KSR v. Teleflex is flawed.
***
Other points from the Wade article:
In addition, the main premise of Dr. Hwang's article, that he could perform human cloning very efficiently, with just a few human eggs, was untrue. The committee said he used far more eggs than he reported.
The panel is also reviewing Dr. Hwang's 2004 claim in which he was apparently the first to clone human cells. If that also proves false, the goal of therapeutic cloning - repairing patients' cells with their own tissues - may be considerably further off than it seemed a few months ago.
How did Dr. Hwang manage to rise so quickly in the scientific firmament and convince so many leading experts that his work was sound?
Three ingredients of his ascent were attracting generous support from the South Korean government, compartmentalizing his laboratory so that few others had any overall view of what was going on and reporting plausible advances that scientists abroad felt they, too, might have achieved if they had access to as many human eggs as Dr. Hwang obtained.
In addition, Dr. Hwang invited well-known American researchers to be co-authors on his articles, which he may have hoped would make his findings more acceptable to leading journals like Science and Nature. He even invited Dr. Gerald Schatten, a stem cell expert at the University of Pittsburgh, to be the lead author on the June 2005 report although Dr. Schatten had done none of the experiments. But Dr. Donald Kennedy, the editor of Science, said the inclusion of American co-authors "certainly did not affect us."
The starting point of Dr. Hwang's rise to fame was his skill in making the South Korean system work for him. The government had invested about $65 million in his research before the collapse came, and the Ministry of Science and Technology had acclaimed him as an "Outstanding Korean Scientist."
The Health and Welfare Ministry promised to provide $15 million next year to set up a World Stem Cell Hub in which Dr. Hwang's technicians would have cloned human cells for scientific customers abroad.
An indication of Dr. Hwang's good connections to the government was the inclusion of Dr. Park Ky Young as a co-author of his 2004 report on human cloning. A botanist by training, Dr. Park may not have contributed much scientifically to the task of cloning of human cells. She is, however, the science adviser to Roh Moo Hyun, the president of South Korea.
The frequent American visitors to Dr. Hwang's growing operation were impressed at the scale and skill of his operation and how he divided his scientists into task forces that specialized in each step of the cloning process. But this compartmentalization may have meant that not all of his co-workers knew what was going on. Few seem to have seen the colonies of embryonic cells Dr. Hwang said he had cloned from patients.
Outside of South Korea, the debacle has left a bitter taste with scientists who trusted Dr. Hwang's work. "It's a sad business," said Dr. Kennedy, the Science editor. "We don't feel like it's our best day."
Science and Nature, two leading journals that compete with each other in publishing striking scientific advances, accepted reports from Dr. Hwang. But Science is now reviewing the accuracy of articles of 2004 and 2005 in which Dr. Hwang said he had cloned human cells, and Nature is re-examining his claim to have cloned a dog, which he named Snuppy.
"We are investigating the Snuppy paper and will review whether we and referees acted appropriately, and whether standards should change," said Dr. Philip Campbell, the editor of Nature.
A question both journals have considered is that of whether their editors and reviewers should have caught the errors in Dr. Hwang's papers before publication. But as in past cases of fraud, the journals' editors and other scientists assert that their system depends basically on trust and that reviewers can check only whether a report's conclusions follow from the data presented.
"Peer review is not set up to test for fraud," Dr. Campbell said. "It is set up to provide expert assessment of the scientific credibility and reliability of what scientists report, taking the report itself in good faith."
Dr. Kennedy noted that journals often published articles that were later shown to be innocently in error. "The public needs to understand that the journals and peer review are not perfect," he said.
Reviewers can, however, recommend that the author provide more data if they are dissatisfied on any point. Science's reviewers asked Dr. Hwang to furnish DNA fingerprints showing that each of his embryonic cell colonies had the same DNA fingerprint as the patient from whom they were said to be derived. Dr. Hwang sent in pairs of fingerprints, some of which had the identical background noise, suggesting the same print was being presented twice. But this anomaly was noticed only later by Dr. Hwang's critics.
Nature's reviewers did not ask Dr. Hwang to provide evidence that would have proved Snuppy was cloned from another dog. Dr. Campbell said that Nature, as part of its investigation of the article, would consider whether its standards of proof should be changed in the future.
Dr. Hwang's failure may have repercussions on the standing of South Korean science.
"Clearly the scientific credibility of Korean investigators has been compromised," said Dr. John Gearhart, a stem cell expert at Johns Hopkins University and a member of Science's board of reviewers. He referred to the fact that duplicate and misidentified photos had turned up in articles by other South Korean authors besides Dr. Hwang.
Dr. Kennedy said, "You cannot avoid a sense of taint from an experience like this." He added, however, that many leading American universities had had at least one case of scientific fraud.
It was also South Koreans who took the lead in detecting Dr. Hwang's falsifications. Dr. Zach Hall, president of the California Institute of Regenerative Medicine, noted that young South Korean scientists had brought to light many problems with Dr. Hwang's papers in Web site postings, and that Seoul National University seemed to be conducting a vigorous inquiry.
The ultimate test of a scientific claim is whether other laboratories independently confirm it. Some scientists have argued that even if Dr. Hwang's errors had remained undetected by the scientific journals and their readers, his work would have fallen under suspicion if no one could repeat it. However, if other scientists had succeeded in cloning human cells before any challenge had emerged to Dr. Hwang's work, it is not so clear that he would have been exposed.
"If the procedure works indeed and other labs would have repeated it, the credit would have gone to Hwang," said Dr. Rudolf Jaenisch of the Whitehead Institute in Cambridge, Mass.
The people best situated to detect scientific problems are those inside the laboratory who see the raw data being generated and have some practical reason for suspicion. As in many other cases of scientific fraud this was true of Dr. Hwang's, too.
It was a whistle-blower in Dr. Hwang's lab who informed the South Korean television network MBC of problems in his work, and that led South Korean journalists to begin to investigate.
But for the whistle-blower, Dr. Hwang might well be continuing his meteoric career on the wings of his reports in Science and Nature.
The fraud passed right through the hands of the referees.
posted by Lawrence B. Ebert at 8:05 AM
0 Comments:
Post a Comment
<< Home