Peer review of patent applications: a bad idea

An IP Blog on May 3, 2006 featured an article by Beth Noveck, Peer to Patent: Collective Intelligence and Intellectual Property Reform, advocating the creation of a peer review online system for patent examination. I could not help but notice the irony in timing, as the New York Times had published on May 2, 2006 an article by Dr. Lawrence Altman strongly criticizing peer review, including the text:

Many nonscientists perceive reviewers to be impartial. But the reviewers, called independent experts, in fact are often competitors of the authors of the papers they scrutinize, raising potential conflicts of interest. Except when gaffes are publicized, there is little scrutiny of the quality of what journals publish. Journals have rejected calls to make the process scientific by conducting random audits like those used to monitor quality control in medicine. The costs and the potential for creating distrust are the most commonly cited reasons for not auditing. In defending themselves, journal editors often shift blame to the authors and excuse themselves and their peer reviewers. Journals seldom investigate frauds that they have published, contending that they are not investigative bodies and that they could not afford the costs. Instead, the journals say that the investigations are up to the accused authors' employers and agencies that financed the research. Editors also insist that science corrects its errors. But corrections often require whistle-blowers or prodding by lawyers. Editors at The New England Journal of Medicine said they would not have learned about a problem that led them to publish two letters of concern about omission of data concerning the arthritis drug Vioxx unless lawyers for the drug's manufacturer, Merck, had asked them questions in depositions. Fraud has also slipped through in part because editors have long been loath to question the authors. ''A request from an editor for primary data to support the honesty of an author's findings in a manuscript under review would probably poison the air and make civil discourse between authors and editors even more difficult than it is now,'' Dr. Arnold S. Relman wrote in 1983. At the time, he was editor of The New England Journal of Medicine, and it had published a fraudulent paper. Fraud is a substantial problem, and the attitude toward it has changed little over the years, other editors say. Some journals fail to retract known cases of fraud for fear of lawsuits. Journals have no widely accepted way to retract papers, said Donald Kennedy, editor in chief of Science, after the it retracted two papers by the South Korean researcher Dr. Hwang Woo Suk, who fabricated evidence that he had cloned human cells.

I myself had criticized peer review as to patents in 88 JPTOS 239 (March 2006):

The frauds of Hwang and Schon illustrate how bad work can be published in high-impact journals in spite of peer review. The problems of confidentiality and conflict-of-interest may well outweigh the benefit to the USPTO of gaining outside expertise, especially if that advice is limited to the plausibility of the arguments.

(...)

The failure of editors and referees of the journal Science to detect the fraud in manuscripts of Woo Suk Hwang prior to publication, and the widespread acceptance of the work after publication, illustrates some difficulties in relying on peer review to authenticate the validity of scientific work. Neither journals nor the USPTO have the resources to perform experiments or even to rigorously review the authenticity of data. Under existing review protocols, fraudulent science or even bad science can pass review. An increase in the amount of time spent in review could enable the discovery of errors such as duplicated figures or graphs, but would likely be ineffective against a dedicated advocate of fraud.