Oullier and Michel-Kerjan (2001) have misinterpreted several key concepts needed to understand how a consensus journal functions. Therefore, they are advised to review more recent publications and reconsider their critique.
2. Oullier and Michel-Kerjan (2001) state, "This article dates back to the early 90's when, compared to today, the [I]nternet and electronic exchange in general were at their embryonic stage for most people. We have to admit that Stodolsky's views were fortunately pretty innovative for the time when they had been proposed. This enables us to discuss his ideas almost ten years later." Unfortunately, Oullier and Michel-Kerjan (2001), and others (Brent, 1995; Stodolsky,1991) have not appreciated just how innovative this model is, and therefore have failed to understand its operation and implications. This can not be a result of unclear explication, since some have grasped the objectives and tried to implement the model, in a somewhat modified form (Cambridge, 1999). Thus, it probably is most appropriate at this time to offer corrections for the most obvious errors in Oullier and Michel-Kerjan (2001) and suggest a strategy which they can use to reassert their critique.
3. Oullier and Michel-Kerjan (2001) state, "2. In paragraph 1 the author suggests that 'the intensity of evaluation is much greater with electronic journals'. According to Stodolsky, quality of judgement will thus be the result of a transition from paper to electronic publication, i.e. only by changing the medium of communication." The point here is that many judgements, combined correctly, yield superior evaluation as compared to few evaluations combined correctly. The logistics of paper-based peer review make combining large numbers of evaluations problematic.
4. Oullier and Michel-Kerjan (2001) later state, "The main weakness of a two or three referee evaluation is its main strength. Randomness can play a role in publication and give an unpopular theory a chance to be diffused." This contrasts sharply with another readers conclusion, "While providing an index of quality through peer review, the system also makes every contribution to the database and discussion forums accessible, so that unconventional approaches which may be powerful in a minority of local contexts can still be located and put into use (Cambridge, 1999)."
5. Oullier and Michel-Kerjan (2001) continue, "Editors send articles to reviewers, this would not be the case if reviewers have to withdraw articles from a database." It is not reviewers that withdraw articles from the database, but authors who have reached the conclusion that their article is indefensible.
6. Oullier and Michel-Kerjan (2001) comment, "This proposal to send a short summary of the article before submitting it as a whole seems incomplete. If you have an experiment that you want to publish, both method and data analysis can evolve in time while writing or previewing results. The reviewer will only have a slight idea of what the article could bring to the research domain." The editor is not expected to determine from the summary what the article could bring to the research domain, he is only expected to classify the content, that is a task similar to identifying key terms.
7. Oullier and Michel-Kerjan (2001) conclude, "Finally we would like to focus on the attempt to find a way to measure performance through statistics in order to achieve the 'fully automatic invitational journal'. We consider this concept as very odd if not dangerous for science. Such statistical methods to decide about publication lead to uniformity and don't seem to be the best way to test methods and concepts." An advantage of the statistical model advocated is that multiple consensus positions can be identified and supporting statements can be invited from those most likely to accurately state those different positions.
8. Oullier and Michel-Kerjan (2001) would be wise to review the most recent and most readable presentation of the consensus journal model (Stodolsky, 1995), since they have misinterpreted several of the key concepts necessary to understand how that model functions. Failure to understand the role of withdrawal of articles, and of the statistical consensus model makes meaningful criticism impossible. Furthermore, a clear distinction should be maintained between the consensus journal model, which depends upon the statistical consensus of referees (Stodolsky, 1995) and the invitational journal based upon editorial consensus, which uses a very different procedure (Stodolsky, 1992).
Brent, D. (1995). Stevan Harnad's "Subversive Proposal": Kick-Starting Electronic Scholarship. EJournal, 5(1). URL http://www.ucalgary.ca/ejournal/archive/rachel/v5n1/article.html
Cambridge, D. (1999). Supporting the Development of a National Constellation of Communities of Practice in the Scholarship of Teaching and Learning Through the Use of Intelligent Agents. In C. Hoadley & J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference, Dec. 12-15, Stanford University, Palo Alto, CA. Mahwah, NJ: Lawrence Erlbaum Associates. URL http://kn.cilt.org/cscl99/A09/A09.HTM
Oullier, O. & Michel-Kerjan, E. (2001). Does Scientific Publication Need a Peer Consensus? Psycoloquy, 12(022). URL http://cogprints.ecs.soton.ac.uk/psyc-bin/newpsy?consensus-journals.2
Stodolsky, D. S. (1990). Consensus Journals: Invitational journals based upon peer consensus. Psycoloquy, 1(15). (Available by anonymous ftp from PRINCETON.EDU in directory /pub/harnad.) URL http://www.ai.univie.ac.at/archives/Psycoloquy/1990.V1/0008.html
Stodolsky, D. S. (1991). Reply to "Response to D. S. Stodolsky's 'Consensus Journals'" [Letter]. Psycoloquy, 2(1). (Available by anonymous ftp from PRINCETON.EDU in directory /pub/harnad) URL http://www.ai.univie.ac.at/archives/Psycoloquy/1991.V2/0000.html
Stodolsky, D. S. (1992). Invitational Journals based upon Editorial Consensus: A New Editorial Role in Electronic Journal Publication. Psycoloquy, 3(67). (Available by anonymous ftp from PRINCETON.EDU in directory /pub/harnad.) URL http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?3.67 http://www.ai.univie.ac.at/archives/Psycoloquy/1992.V3/0066.html
Stodolsky, D. S. (1995). Consensus journals: Invitational journals based upon peer review. The Information Society, 11(4). [1994 version in N. P. Gleditsch, P. H. Enckell, & J. Burchardt (Eds.), Det videnskabelige tidsskrift (The scientific journal) (pp. 151-160). Copenhagen: Nordic Council of Ministers. (Tema NORD 1994: 574)]