Law reviews work hard to prevent and correct errors. They exert prodigious cite-checking, editing, and proofreading efforts to make sure their articles are as error-free as possible. They also try to prevent errors by readers: they publish articles aimed at correcting existing errors, and they edit articles with an eye toward eliminating misleading statements that might unintentionally lead readers into error. Yet new technologies can let law reviews do more to prevent and correct errors, without a vast amount of extra work.

I. Posting Unpublished Sources

Many articles cite court decisions, agency rulings, pleadings, briefs, corporate policies, or datasets that haven’t been published (in print or online). Let me suggest a new professional norm, or even a Bluebook rule: law reviews should generally place such unpublished materials online, and point readers to the online version.

Such a policy would save other scholars effort. If a source was useful for one article, it might be useful again. Why put others to the trouble, time, and expense of unearthing it again in the future?

And placing such sources online can also avoid reader errors. Say that I come across a source mentioned in an article and want to use it in my own future work (an article, a brief, a class discussion, or what have you). The responsible thing for me to do is to read the source myself, because even if the article’s characterization (or quotation) of the source is accurate and fair, it necessarily omits much important context. Maybe something else in that source will undermine the purpose for which I want to use the source; I can’t tell unless I read the original.

Yet busy users of an article are naturally tempted to cut corners by relying on indirect accounts of original sources. That way lies error—error that the law review could have helped prevent by simply putting the source online (or insisting that the author do the same).

Placing the source online also helps pressure the author who first cites the source into being totally candid in characterizing it. Very few authors, of course, would deliberately lie about a source, and cite-checkers will likely catch the most blatant inadvertent errors. Still, there’s often a subtle subconscious temptation to slant one’s characterization of the source in a way that favors one’s thesis. That temptation diminishes if the author knows that any reader can easily check the sources and cry foul if the author’s summary of the source is biased or sloppy.

Finally, a norm of placing sources online can also import into legal empirical scholarship the norm of sharing data. Such data-sharing helps other scholars check the initial analysis, or check whether the original author has made some coding errors or controversial coding judgment calls.

Naturally, there’ll need to be some exceptions for sources that pose potential privacy or copyright problems (e.g., interviews with sources who were promised anonymity, or drafts of unpublished articles). But most sources should be put online, and easily can be—every law school has a scanner that can easily scan in even a long document.

One can imagine extra refinements. For instance, we might worry that individual law reviews might not properly back up the sources, or might change the URLs at which the sources are located, thus breaking previously published links. To solve that problem, law reviews might put together a consortium that would store all the items using URLs that are sure not to change (e.g., http://lawreview.org/yalelj/107/2431/sourcename.pdf).

But that’s a frill. Start with a simple norm: unless there’s a good reason, the law review or the author should put online all the unpublished material on which the article relies and should link to the material in the article.

II. Making It Easier To Find Critiques of an Article

You’d never rely on a case without Shepardizing it (or Instaciting it) to make sure that it hasn’t been overruled. You’d probably also want to check whether it’s been seriously questioned.

Likewise, if you’re using a law review article as an authority—or even as a source of argument, especially in a field that you don’t know well yourself—you should want to know whether the article’s reasoning has been cast into doubt. If the article contains errors, you don’t want its errors to become your own. If someone has raised serious counterarguments to the article’s thesis, you don’t want to make the error of ignoring those arguments.

Unfortunately, finding such criticisms is often hard. Shepard’s, Instacite, or a simple Westlaw or Lexis search will tell you who has cited the article. But (unlike with cases) Shepard’s and Instacite won’t tell you whether the citation is a serious disagreement or just one item in a “See generally” string cite. And if you find several dozen articles citing the one you’re referring to, you may often not have the time to carefully look at each of them.

Yet the Internet can solve this problem, albeit with a modest amount of work on the part of law reviews. Say that law reviews set up a central Web-accessible database that tracks articles which criticize other articles. It might look something like this:

For each new article, the law review will probably need to add only a few entries. The entries’ contents will be clear to the editors when the criticizing article is being edited. Creating the entries will thus involve only a modest amount of work on the journal’s part, a tiny addition to the existing work of cite-checking, proofreading, and editing.

Of course, it would be better if the database provided more details on articles that don’t criticize an earlier work, but elaborate on the work, apply it, and so on, just as Shepard’s provides many such details. It would be better still if the database listed articles that don’t cite the earlier work but still reach a result contrary to that of the earlier work. But that would be much harder, and the better shouldn’t be allowed to be the enemy of the good here. The modest system that I suggest would be useful and fairly easy to implement; it should be implemented even if we can think of still more useful but also more work-intensive projects.

Once this database is created, Lexis, Westlaw, HeinOnline, and the Social Science Research Network (SSRN) should be quite willing to automatically check it, and report the data, whenever someone looks up an article—after all, reporting the information will mean that people (including, as to all but SSRN, paying customers) will access more articles. When a reader pulls up an article, the electronic service can show a list of all the criticizing sources, plus the short descriptions of the criticisms. When the reader reaches a page number, the service can show a list of those criticizing sources that relate to that page.

Criticisms of the article will be quickly accessible, alongside the article itself. Academic debates can proceed more effectively, because people will easily notice both sides. And remarks like, “Huh, Volokh, you rely on this article, but I’d expected you to at least deal with Smith’s counterarguments to that piece,” would become a thing of the past.

Such a project would of course require some coordination among law reviews. But law reviews already engage in cooperative projects—consider the Bluebook, which is jointly written and updated by four leading law reviews, and then used by hundreds more.

Moreover, it will be in each journal’s interest to start contributing to the database, since having the journal’s articles listed as “criticizing articles” will lead more people to read and cite those articles. Once a few journals start participating, authors will pressure others to do the same, since each author whose work criticizes others’ work will want to be in the database. And the database will be useful even if at the beginning it includes only a few dozen journals (so long as LEXIS, Westlaw, HeinOnline, or SSRN are persuaded to automatically check it, something that they can do with little technical effort).

III. Allowing Corrections or Updates of Articles

This journal is called The Yale Law Journal Pocket Part, but it doesn’t really aim to do what traditional pocket parts do. Traditional pocket parts essentially correct material that was accurate when printed, but that has become inaccurate. This “correction” is usually a way to deal with material’s becoming no longer up-to-date, but in principle the pocket part could also correct errors in the original (if the publisher is more worried about serving its readers than about avoiding embarrassment for itself). Yet there’s no such mechanism for The Yale Law Journal or any other law review.

At best, one can search through the Pocket Part archives for articles responding to articles in the Journal itself; perhaps that can be adapted to let authors correct or update their own past articles, but this is not officially part of The Pocket Part’s charter. Moreover, I suspect that few people will get in the habit of automatically running such searches through The Pocket Part whenever they’ve read (or are planning to read) an article from The Yale Law Journal. One of the advantages of traditional pocket parts is that their physical presence (usually) reminds you to check them.

This lack of a true pocket part analog means that an otherwise very useful article can become entirely or partly inaccurate when a new case is handed down or a new statute is enacted. If that happens, readers might rely on the old article, not realizing that there have been new developments, and thus get a mistaken picture of the law. Or readers might recognize that the article doesn’t mention the new developments, and avoid error—but they’ll also probably avoid reading the article closely, and discard it as no longer relevant.

Yet often the author can easily address the new development and explain how it changes the article’s thesis (or why, properly understood, it shouldn’t change the article’s thesis). This is in the author’s and the publishing law review’s interest, since it can prolong the article’s influence and citability. But right now this can’t be done effectively.

Likewise, the lack of pocket parts means that substantive errors (large or small) can’t be easily corrected, either. A 1998 N.Y.U. Law Review article of mine, for instance, cited some work by Michael Bellesiles, work that was discredited a few years later (because it was based at least partly on falsified data). Fortunately, the article’s main thesis isn’t materially affected by the error. But I surely don’t want readers to rely on those assertions that were based on Bellesiles’ work (for instance, because they hadn’t heard of the Bellesiles scandal).

And this is just an especially blatant case; we often find smaller errors in our own work and want to correct them. We also sometimes find errors in others’ work, errors that even the author (or at least the law review editors) will acknowledge once they’re pointed out.

Fortunately, the article-Shepardizing database described above can also handle corrections. The journal that published the original article can simply put an author’s correction on a separate web page, and add it to the database:

Lexis, Westlaw, HeinOnline, and SSRN can again link to the correction (and even include its entire text), whether at the start of the article or at the relevant page or footnote. Readers will thus be easily able to see the corrected text, avoid error, and recognize that the article is still up-to-date and useful.

Naturally, none of this will do much for readers who only look at the article in the original issue. Yet how many readers do that these days? If you’re looking for an article, you usually find it through Westlaw or Lexis, or maybe SSRN. If you want to see a neat copy, you usually print it from HeinOnline, or have your library print it from HeinOnline.

The print version sits in libraries, and sometimes gets read—but I suspect rarely, and more rarely each year. If we can link the electronic versions of articles to error corrections, updates, and attempted refutations, that will help a lot of readers.

Conclusion

My friend Steve Cooper speaks of the “second migration.” When you switch technologies, whether upgrading from one computer to another, or going from mostly paper distribution to largely electronic distribution, the first step is to migrate your data so that it’s at least as easily available on the new technology as on the old. Lexis and Westlaw started that process for law review articles; HeinOnline has added to it.

But the second step is to ask: how we can take advantage of features of the new technology that don’t just emulate the old technology, but go beyond it? From which limitations of the old technology can we now free ourselves?

Searching is one obvious answer, which Lexis and Westlaw pioneered decades ago. But there are other answers, answers that can help legal research be more accurate and not just easier, more comprehensive, and cheaper. In this Essay, I’ve tried to offer three such answers—devices that, with modest work on the part of law reviews (plus some help on the part of electronic delivery services), can harness the Internet to avoid reader errors, to deter and correct author errors, and to extend articles’ productive lifespans.

Eugene Volokh is the Gary T. Schwartz Professor of Law, UCLA School of Law (volokh@law.ucla.edu).

Preferred Citation: Eugene Volokh, Law Reviews, the Internet, and Preventing and Correcting Errors, 116 Yale L.J. Pocket Part 4 (2006), http://yalelawjournal.org/forum/law-reviews-the-internet-and-preventing-and-correcting-errors.