|Service Delivery, Resource Allocation, and Access to Justice: Greiner and Pattanayak and the Research Imperative|
|Jeffrey Selbin, Jeanne Charn, Anthony Alfieri & Stephen Wizner, Monday, 30 July 2012 [View as PDF]|
How should we deliver legal services to low-income clients in need?
As legal services lawyers and clinical law professors who have spent the bulk of our careers in neighborhood-based antipoverty programs, we grapple constantly with these individual, institutional, and systemic challenges. Legal needs in low-income communities far outstrip our ability to meet them, so we develop gatekeeping mechanisms to manage client demand and expectations. Among eligible clients, we face seemingly intractable choices about whom to serve and how much to serve them. While we struggle with individual delivery and programmatic allocation decisions, we also strive to expand access to justice systemically.
James Greiner and Cassandra Pattanayak’s provocative article—reporting the results of a randomized controlled trial evaluating legal assistance to low-income clients at the Harvard Legal Aid Bureau (HLAB)—forces us to confront how little we know about these questions.Studying the outcomes of appeals from initial denials of unemployment insurance (UI) benefit claims, they asked: what difference does legal representation make? They find that “an offer of HLAB representation had no statistically significant effect on the probability that a claimant would prevail, but that the offer did delay the adjudicatory process.” That is, not only was an offer of legal assistance immaterial to the case outcome, but it may have harmed clients’ interests.
In Part I of this Essay, we describe the Greiner and Pattanayak study and the skeptical reaction it initially received from many of our colleagues. We then offer a more optimistic reading. Building on this alternative reading, we argue in Part II that empirical research can inform service delivery, resource allocation, and access-to-justice questions. In fact, in light of the growing demand for legal services and shrinking supply, we contend that such research has become imperative. In Part III, we discuss recent developments in law schools, the professions, and policymaking that provide support, infrastructure, and incentives for such research. We conclude with a call for legal services lawyers and clinical law professors to embrace an expansive, empirical research agenda.
I. Reading Greiner and Pattanayak
Legal services lawyers, clinicians, and policymakers should read Greiner and Pattanayak’s work with care. The study cannot easily be dismissed on methodological grounds or because we do not like what it suggests about our work. At the same time, the study does not require us to surrender our belief in the efficacy of legal representation or to abandon the wisdom we have accumulated over decades of practice. Instead, it creates space for new ideas, debate, and action.
In this Part, we summarize the study’s design, findings, and conjectures about causal mechanisms. Next, we offer a typology of some of the initial skeptical responses to the piece. We then turn to our own, more hopeful read of the study’s findings. This view has been largely absent from early reactions and suggests the need for much more work of this kind.
A. The Study and Findings
Greiner and Pattanayak measured the impact of offers of legal representation from the Harvard Legal Aid Bureau on unemployment insurance claimants. Under the supervision of clinical faculty, HLAB students represent claimants in first-level, de novo appeals of UI benefit denials before an administrative law judge. Students conduct intake by telephone, and demand for HLAB’s UI services far exceeds capacity. During the study, after confirming callers’ general eligibility, students submitted potential UI clients to Greiner and Pattanayak for randomization. On instruction from Greiner and Pattanayak within twenty-four hours of the initial conversation, students offered callers either representation or referral to other agencies.
Greiner and Pattanayak designed their study to measure the impact of representation by randomizing the only action within HLAB’s control—the initial offer of representation. HLAB could not control who accepted its offer, who sought help elsewhere, or who decided to do nothing. Over a period of almost two years, Greiner and Pattanayak randomized more than two hundred cases. They then compared case outcomes—specifically, benefits awards and adjudication time—between those claimants who were offered HLAB assistance (the treatment group) and those who were not (the control group). With respect to benefits, the treatment group prevailed no more frequently at hearings than the control group. However, claimants in the treatment group received their benefits decisions on average two weeks later than those in the control group. In other words, claimants who received an offer of assistance from HLAB fared no better, and perhaps worse, than those who were turned away.
Greiner and Pattanayak considered a number of explanations for these findings. They convincingly reject two hypotheses: first, that low-quality (student) lawyering at HLAB was responsible for the results; and second, that turned-away claimants obtained help elsewhere at such a high rate that they were indistinguishable from HLAB-represented claimants. They then identify three plausible explanations: first, that the strong personal characteristics or case merit of everyone who contacted HLAB muted differences between the treatment and control groups; second, that the special efforts of administrative law judges to assist self-represented claimants compensated for the lower rate of representation in the control group; and third, that the relative simplicity of the cases made them especially amenable to self-representation. They conclude, however, “that without further study with different service providers and, perhaps, different study designs, we do not know which of these mechanisms (or something else we have not mentioned) is rendering an offer of representation unlikely to have resulted in a large effect on win/loss.”
B. The Initial Response
As observed more than forty years ago regarding the challenges of mounting an empirical research protocol to study the impact of lawyers in delinquency proceedings, “No professional group easily supports research which may cast a shadow over its fundamental beliefs and practices.” Greiner and Pattanayak’s research findings challenge such deeply held beliefs, and the legal services and law school clinical communities responded swiftly and largely negatively to the study. Despite longstanding critiques of deficiencies in client intake and delivery systems, failings in caseload control and practice routinization, and shortcomings in client decisionmaking and lawyer accountability, most lawyers both inside and outside the academy remain certain that representation of indigent clients in civil matters is fundamental to the pursuit of justice.
Greiner and Pattanayak catalogued the various objections they received, most of which focused on research design, and particularly the extent to which the study removed lawyer judgment and discretion from case selection. We describe here a general set of criticisms that we have heard from colleagues. Our purpose is not to debate the relative merits of each criticism—the work stands on its own, and the authors can fend for themselves—but to map the early reactions. In the subsequent section, we share our affirmative views of the study’s import.
One critique focuses on validity—the study was poorly done, and therefore cannot possibly tell us anything useful about what we do. In measuring the impact of offers of representation instead of actual representation, Greiner and Pattanayak asked the wrong question. We should care only about whether the representation itself made a difference, and Greiner and Pattanayak cannot (or will not) tell us whether it did. Further, in studying the impact of a clinical program where most of the services are provided by law students and not lawyers, Greiner and Pattanayak picked the wrong site of inquiry. The question is whether experienced lawyers, not novices, make a difference. Finally, in pursuing a randomized, controlled trial not grounded in theory and to the exclusion of other empirical methods, Greiner and Pattanayak offer findings that are at best partial and at worst misleading. Had the authors fielded their study after conducting a literature review (instead of before), they would have addressed the more salient issue—namely, the impact of representation by lawyers, not the impact an offer of representation by law students.
The second critique focuses on tradeoffs—even if something useful can be gleaned through research like Greiner and Pattanayak’s, we should not divert any of our service resources to such efforts. From 1976 to 1981, the Legal Services Corporation (LSC) Research Institute conducted similar studies and concluded that staffed field offices were the best way to serve low-income clients. Even after the research arm was eliminated by Congress, LSC has continued to update and improve program evaluation. Of course the system could be better, but it is not broken. The bottom line, according to the critics, is that we should not forego immediate and important services to clients in exchange for the distant and uncertain benefits of research.
The final critique, perhaps underlying all other objections, focuses on vulnerability: irrespective of this study’s validity or the tradeoffs involved, why would we expose legal services for the poor to further political attack by right-wing ideologues committed to its destruction? LSC has been subjected to relentless assault and existential threats by conservative critics almost since its inception. In 1981, President Reagan attempted to eliminate LSC, which resulted in a twenty-five percent cut to its budget. Fifteen years later, Congress enacted further crippling budget cuts (another thirty-three percent) and imposed severe program restrictions. Even today, the Republican Study Committee’s fiscal year 2013 budget proposes eliminating LSC because of its “lobbying, advocacy of leftist causes, and litigation against the federal government.” In such an environment, research—especially anything that does not clearly demonstrate the value of legal services—is a naïve luxury we cannot afford.
C. Our Reading
While we share some of the concerns expressed above, we are neither surprised nor especially troubled by Greiner and Pattanayak’s findings. Given the particular characteristics of unemployment insurance law, adjudication, and claimants, we might guess that lawyers are less important in these cases than they are in more complex settings on behalf of clients in greater need. Maybe legal representation in the unemployment insurance context is unnecessary for some claimants, at least in closely monitored, high-performing venues such as the office studied here. The results might also reflect the success of sustained individual and institutional advocacy by experienced lawyers whose efforts over many decades paved the way for effective self-representation. More fundamentally, perhaps facilitating effective self-representation—rather than providing a lawyer for every indigent civil litigant—should be a primary goal of the access-to-justice movement.
Greiner and Pattanayak acknowledge that they “come to no firm conclusions regarding a use-of-representation effect on the win rate” in the unemployment insurance context. And they make no claim about the impact of offers or actual representation in other legal settings. But their findings are suggestive of important forces at work, including when and where self-representation might be equal or superior to legal representation, and how the role of lawyers might shape such opportunities.
For example, as Greiner and Pattanayak note, initial appeals by claimants in UI cases may be particularly amenable to self-representation. First, outcomes in UI cases are binary. The claimant is either awarded or denied benefits. Second, pursuing an appeal requires little technical legal knowledge. Evidence and procedural rules are relaxed, the agency develops the record prior to the hearing, and information about separation from employment—the point on which most UI cases turn—is often readily available. Third, the agency in general, and some administrative law judges in particular, may be supportive of self-representation. Finally, people claiming UI benefits may be especially well equipped for self-representation, at least relative to many legal services clients. UI claimants have recently engaged in formal employment, which may correlate with personal attributes and resources that enhance their ability to navigate the system with minimal or no help.
The relatively straightforward nature of UI appeals contrasts with many other areas of practice common in legal services offices and law school clinics. For instance, when representing a tenant in an eviction case, outcomes typically involve a series of complex, interrelated choices between retaining or ceding possession, demanding or forgoing specific repairs, and trading off rent due to landlords with damages due to tenants. In fact, Greiner and Pattanayak recently completed a housing study conducted in a Massachusetts District Court, which revealed that the offer of representation to tenants by experienced lawyers had a strong positive impact on their case outcomes.
Greiner and Pattanayak also note that legal advocacy in the Boston UI office may have helped to produce claimant-friendly processes. Over thirty years ago, Gary Bellow suggested that “focused case pressure”—in which lawyers and clients bring targeted streams of individual cases—could positively and broadly impact institutional players and practices. Experienced employment lawyers in Boston represent a relatively high volume of UI claimants, and they also engage in institutional advocacy. Although no one has tested focused case pressure to determine when, where, or how such a strategy is effective, Greiner and Pattanayak’s findings may reflect the impact of advocates’ sustained work over several decades. Additional research may show that a UI hearing office not subject to focused case pressure is less friendly to self-represented claimants, or we may discover that even a well-performing agency requires ongoing monitoring by lawyers to prevent institutional backsliding.
Indeed, for reform-minded lawyers, Greiner and Pattanayak’s findings may point toward new strategies to influence administrative systems that adjudicate the rights of low- and moderate-income people. When these systems are subjected to careful and sustained empirical study, we may learn that representation focused on program norms, rules, and practices is equally or even more effective than impact and test-case litigation. Conceived by civil rights lawyers more than a half century ago and adapted by public interest and cause lawyers across a spectrum of legal rights campaigns, the traditional model of law reform litigation relied largely on anecdote, intuition, and pragmatism in making advocacy judgments to target and restructure core public institutions such as prisons, schools, and welfare offices. Greiner and Pattanayak’s findings compel us to consider more exacting evidentiary standards in making scarcity-driven decisions about when and how to reform institutions.
II. How Research Can Improve Legal Service Delivery, Inform Resource Allocation, and Increase Access to Justice
The controversy generated by Greiner and Pattanayak’s study is due in part to the void into which it was proffered. Legal services programs and law school clinics have served tens of millions of low-income clients since the 1960s, yet we lack basic information, let alone rigorous empirical data, about the impact of our work. Although we are generally confident about the added value of our advocacy, few legal services programs or law school clinics conduct formal quality control or evaluate service outcomes, and fewer still have opened their practices to external scrutiny. In fact, Greiner and Pattanayak exhaustively review decades of research purporting to measure the impact of civil representation. Even as they emphasize that identifying methodological research flaws is different from saying the findings are wrong, they conclude that almost all prior studies “suffer from methodological problems so severe as to render their conclusions untrustworthy.”
In this Part, we argue that a robust research program would generate more reliable and actionable knowledge relevant to the overlapping dimensions of individual representation (service delivery), program design (resource allocation), and systemic issues (broader access to justice). In terms of individual representation, research can improve service delivery by measuring the tangible and intangible benefits that we provide to our clients. For individual programs, research can strengthen our understanding of both client needs and provider resource-allocation decisions. And at the systemic level, research can increase access to justice by strengthening the argument for appointed counsel in some civil cases and developing effective nonlawyer or less-than-full-representation approaches in others.
A. Improving Service Delivery to Clients: The Representational Imperative
Like many professionals, we lawyers are confident about our ability to help clients. That confidence may turn out to be well placed, but how would we know? We select our clients, and we generally make the strategic and tactical decisions in their cases. We manage client demand and expectations, triage access to our services, and counsel clients about what is possible and desirable. Very few lay clients have the knowledge or experience to challenge our advice, and even fewer clients have the ability to reject it or to go elsewhere. In such a closed system, it is easy to misjudge our impact. Empirical research can help us better understand what effect, both material and nonmaterial, we actually have on our clients’ lives.
For decades, LSC field offices have reported the numbers and types of cases they handle as well as some process data, such as whether a case involves only counsel and advice or whether it is resolved by settlement with or without a hearing. However, LSC does not require field offices to report substantive outcomes, and non-LSC programs and law school clinics infrequently collect such data. Nor do we obtain meaningful feedback from clients. To the extent that programs solicit such input, it is typically through “client satisfaction surveys” distributed at the end of representation. These surveys are rarely developed in consultation with social scientists or other experts and are not generally representative, reliable, or valid indicators of service quality. Without well-designed research to provide objective and credible evaluation of our efforts, we have no basis to compare different full-representation models with each other or with less-than-full representation or nonrepresentation alternatives.
Beyond scorekeeping and client satisfaction surveys, research can help us learn more about the socioeconomic benefits we deliver to clients. If the goal of an income-support collaborative is to help clients who are unable to work or obtain disability benefits, we should track how many of our clients moved from welfare to Supplemental Security Income benefits—with the resulting increase in household income—as a result of our advocacy. If the goal of a foreclosure prevention project is to maintain home ownership for moderate-income families, we should collect data on how many of our clients remained in their homes at the end of our assistance. If the goal of a community reentry program is to help people with criminal records overcome barriers to employment, we should follow up to learn whether our clients’ earnings increased after our intervention.
We can also learn more about our effectiveness over time. Some of our work is analogous to emergency room or trauma care—e.g., trying to stop an imminent eviction or helping someone access emergency food stamps—while some of it is more preventive or capacity-building in nature. In the former case, we might not expect the benefits of our services to extend much beyond the initial intervention. In the latter case, longitudinal studies would be the only way to determine whether we are achieving our service goals. If, after preventing foreclosure in the first instance, we learn that many of our clients lose their homes to foreclosure within twelve to twenty-four months, we might rethink our service delivery model (or at least our aspirations for it).
And beyond material benefits, what else do we provide clients? Pecuniary success at a hearing, as noted by Greiner and Pattanayak, may not be the only goal of representation. Dignity interests, such as the experience of being heard or taking a stand, can be defined and measured. Some of these less tangible benefits may be inextricably linked with material benefits. For example, expunging a client’s conviction may reduce stigma and increase her confidence in job interviews, which together with a clean record would improve her employment prospects.
Research can also reveal hidden costs of self-representation, such as anxiety or stress imposed on successful as well as unsuccessful claimants. Those whose claims are denied may be less likely to accept the adverse result if they did not have legal assistance. On the other hand, claimants who self-represent may gain confidence in themselves, in claimant-initiated institutional reform, and in the legal system. They may be more likely to assert claims in the future and may be better informed and better able to seek the legal advice they need. These examples are not meant to be exhaustive, nor do we mean to suggest that measuring such costs and benefits will be a simple matter. Studying material outcomes alone may miss important psychosocial benefits of various forms of representation, and nonmaterial goals and outcomes typically turn on normative judgments, not empirical ones.
B. Informing Resource Allocation Decisions: The Programmatic Imperative
Resource-allocation decisions in legal services run the gamut from societal choices, such as annual LSC appropriations, to individual decisions, such as which case a lawyer prioritizes on any given day. At the office level, programs make two related distributive decisions about gatekeeping, or who gets in the door, and about scope, or how much service we offer to people we screen in. Many of these decisions are based on assumptions about client needs. Empirical research can better inform us both about client needs (demand side) and about the range of services we should offer (supply side).
On the demand side, based largely on studies conducted decades ago, the conventional wisdom is that at least four out of five of the legal needs of the poor go unmet. Recent LSC reports, relying on a mix of nonscientific methods, reach roughly the same conclusion. Such studies are undertaken for the admirable purpose of highlighting the justice gap—the difference between the legal needs of the poor and our ability to meet them. But as a programmatic matter, they tell us almost nothing about the range of client needs nor how to allocate existing resources among legal assistance, nonlegal assistance, and other forms of advice and dispute resolution.
Pioneering research in the United Kingdom suggests that many consumers prefer alternatives to lawyer-centric services. Similarly, in the United States, consumer surveys and anecdotal evidence from attorneys who offer limited-scope assistance indicate that even clients who could afford traditional lawyer services sometimes prefer a more active role in their cases and experience higher satisfaction with more flexible and limited arrangements with lawyers. The point is that more robust and sophisticated research can tell us not only about clients’ needs, but also about their preferences for how and from whom they might get the most help.
On the supply side, every legal services office and law school clinical program applies filters to allocate scarce service resources, such as geography, income, case type, health status, age, gender, ethnicity, sexual orientation, and other eligibility criteria. Once clients get through the door, we may provide a range of services along a continuum from advice and counsel to limited scope legal assistance (unbundling) to full lawyer representation. In the last fifteen years, innovative service-delivery models have proliferated, such as hotlines for information and advice, court-assistance programs to aid self-represented parties, and web-based services. Though the variety of service methods has grown substantially, we have done little to figure out what type of help best meets each client’s particular needs. When we refer clients for limited assistance, we do so often because representation is not available, not because we know with any certainty that it is a reliably effective option.
Greiner and Pattanayak’s work suggests—even if we do not know exactly why—that some claimants can successfully represent themselves. Moreover, it raises the very real possibility that with further study we could identify who is likely to succeed on their own or with limited advice and who needs more extensive assistance. More nuanced research can provide valuable information at the intersection of supply and demand. Are there subsets of clients—e.g., language minorities, the disabled, or the elderly—who need a different level of assistance than other clients? Are there characteristics of claims—novel issues or ambiguities in existing laws or routines—that make self-representation less effective? Are there certain dispute-processing venues—administrative tribunals, jury trials, appellate proceedings—that are more or less friendly to claimants who self-represent or have limited assistance from lawyers? Such research could fuel a more efficient and effective use of resources, a more consumer-driven delivery system, and greater transparency and accountability in the crucial gatekeeping and scope decisions that all providers must make.
C. Increasing Access to Justice: The Systemic Imperative
At the systemic level, access-to-justice advocates have sought an entitlement to court-appointed legal representation in civil matters mirroring the right to assistance in criminal matters enunciated by the Supreme Court in Gideon v. Wainwright. In its strongest form, so-called civil Gideon would require states to provide lawyers to virtually all civil litigants who cannot afford counsel. In a more modest form, indigent litigants would be entitled to lawyers at government expense for a subset of cases with core rights at stake. As a constitutional matter, the Supreme Court has made clear that any mandated right to counsel in civil cases requires evidence that nothing less than appointed legal representation will safeguard procedural rights and prevent error.
Alongside the normatively focused right-to-counsel movement, as noted above, we have seen the rapid expansion of localized, pragmatic, bottom-up innovations in nonlawyer and less-than-full-service assistance. The self-help or pro se movement aims to support litigants to represent themselves. Court simplification efforts are designed to reduce barriers to unrepresented parties. Where court procedures cannot be simplified and self-help is inadequate, lay advocacy or limited-scope representation by lawyers is prescribed. While the impetus for many of these developments was to provide a stopgap until lawyer services could be expanded, they have become significant features of the access-to-justice landscape in their own right.
The Supreme Court’s recent decision in Turner v. Rogers may significantly alter this landscape. Turner is the first civil right-to-counsel case to reach the Court in thirty years. The question in Turner was whether a low-income father facing jail time for failure to pay child support in a civil contempt proceeding was entitled to a court-appointed lawyer. Because the risk of incarceration has been a deciding factor in criminal and juvenile right-to-counsel cases, civil Gideon advocates believed they had a strong claim. While a five-to-four majority of the Supreme Court agreed that the defendant’s due process rights had been violated, not a single Justice found a constitutional entitlement to counsel. Instead, the Court held that court-aided self-representation would have been sufficient to ensure petitioner Michael Turner’s due process rights.
Civil Gideon advocates reacted with a combination of dismay and cautious optimism. Some scholars and clinicians involved in the case applauded the outcome and called for increased pro se reform. Yet, regardless of one’s normative views, after Turner, the route to court-ordered civil Gideon will require research that compares lawyer representation to less expensive alternatives. In fact, during oral argument, three of the five Justices in the Turner majority specifically requested data as they sought to weigh due process considerations. As one leading commentator recently observed about Turner and the challenge of determining when lawyers are constitutionally required, “[w]hile one can state the equation, one cannot do the math because the data are missing.” Thus, for national right-to-counsel advocates and local innovators, developing a sizable, credible, and independent research capacity has become not just a good idea but an urgent necessity.
Indeed, an empirical study such as Greiner and Pattanayak’s invites a more fundamental reassessment of the place of self-representation in the access-to-justice movement. Self-help can mitigate concerns that professionals in general and legal services lawyers in particular create unnecessary and unwarranted dependency. Lawyers should be available when our expertise makes a difference, and we should play a more limited role, if any, when other forms of representation are equally or more effective. From this perspective, self-help is not a second-best alternative until everyone can have a lawyer. Rather, we should aspire to a justice system in which people with “the legal problems of everyday life” can obtain fair and prompt dispute resolution on their own or with limited assistance.
III. The Time Is Right for a Robust Research Agenda
Greiner and Pattanayak’s work comes at an opportune time to develop and pursue a more robust legal services research agenda. First, there are important changes afoot in law schools and the service professions that provide fertile ground for empirical inquiry. Second, we have unprecedented infrastructure to support the growth of such research, including more than a thousand law school clinics, which can serve as laboratories to ask and answer important delivery, allocation, and efficacy questions. Finally, the policy marketplace is increasingly evidence-driven—at the most basic level, providers, policymakers, and funders want to know what works. In fact, it is difficult to imagine that we can garner significantly more funding without much better data.
In recent years, law school faculties have become more methodologically diverse, with a proliferation of scholarly approaches that complement the traditional doctrinal core, including interdisciplinary studies, policy-focused research, and empirical legal studies. Of these developments, the most salient for our purposes is the rise of empiricism. This new empiricism comes in at least two distinct but related forms, both of which offer lenses through which we can better understand the service delivery, resource allocation, and access-to-justice dimensions of our work. Likewise, other service professions have increasingly applied the use of best evidence to inform daily practice, especially evidence-based medicine, which has been adopted by the public and private sectors and is reflected in professional education. From these sister professions we can learn some of the potential and limits of evidence-based approaches.
While these trends in law schools and the service professions provide the expertise and examples to develop an evidence-based legal services delivery system, law school clinics can serve as primary sites of such inquiry. Established in many cases as marginal, oppositional practices with an explicit social-justice mission, clinics have become institutionalized in the nation’s law schools to a degree almost unimaginable to their founders. During the last forty years, clinics have both contributed to the hands-on educational needs of law students and mitigated the persistent justice gap for low-income clients and community groups. Clinicians have pioneered a wide array of service-delivery methods, and clinics are well positioned to serve as research sites. Clinics have access to clients, courts, and other institutional actors. They operate within law schools full of potential research collaborators—it is no coincidence that Greiner and Pattanayak’s first study was conducted in partnership with a law school clinic—and they can leverage the additional labor, expertise, and requisite detachment of social scientists within larger research universities.
Multiple nodes of complementary legal services research activity have emerged in recent years. Under new leadership, the Legal Services Corporation has begun to commission studies to get a better handle on current needs, activities, and expenditures. In 2010, the Department of Justice established an Access to Justice Initiative, and one of its primary goals is to “[e]xpand research on innovative strategies to close the gap between the need for, and the availability of, quality legal assistance.” Outside the federal government, the American Bar Foundation hired a prominent socio-legal scholar to direct a new access-to-justice research initiative. In 2011, faculty from leading law schools—in collaboration with the Department of Justice and the American Bar Foundation—established a consortium to support and produce high-quality research on issues of importance to the legal services and access-to-justice fields.
These initiatives will take on even greater salience in tight budget times. Policymakers are increasingly demanding evidence of efficacy to fund new or existing social programs. The Government Accounting Office and the Office of Management and Budget have examined a wide range of evaluation methods. Those offices have called for greater investment in program evaluation and recommend prioritizing programs backed by strong evidence. Federal funding will be directed toward programs for which success can be measured and rigorously evaluated—where practical and ethical—by random assignment of clients to service and control groups. State legislatures, foundations, and other funders have likewise begun to insist upon such demonstrable outcomes. To maintain and expand upon our current resources, we have little choice but to facilitate and participate in more rigorous scrutiny of our work.
So what does it mean to engage in such inquiry, and how might we begin to work more closely with researchers? We suggest several related and mutually reinforcing activities. First, we should acquire basic literacy in empirical methods so that we can communicate more effectively with researchers and interpret and apply research more wisely. Second, we should explicitly articulate our service goals to determine which processes and outcomes can be measured—and thus inform delivery and allocation decisions—and which judgments can only be made through clinical experience. Finally, and foundationally, we should commit ourselves “to a continuing effort to generate alternative methods, to put into operation whatever recommends itself to our objective appraisal, and to evaluate remorselessly our fondest pet notions.” Such literacy, transparency, and open-mindedness are keys to advancing knowledge in the field.
There are many obvious challenges associated with the pursuit of a robust research agenda. As noted above, stakeholder resistance to research is inevitable. In addition, it will be no small task to identify new resources and to map research demands onto existing obligations of time, money, and expertise in legal services and clinical programs. And partnering with research colleagues from inside the academy and clients from outside the gates will not be easy, especially if we want to collaborate in the planning, implementation, and dissemination of research.
The Greiner and Pattanayak study coincides with a moment of crisis in American law and society, as exhibited by rising rates of poverty and inequality. This widening crisis is exacerbated by deteriorating conditions in public access to courts and legal representation. High-quality research offers a valuable opportunity to understand and improve local and institutional responses to this growing crisis. Greiner and Pattanayak should prompt us to reflect on our representational models, distributive choices, and systemic-access efforts. Indeed, in these lean times, evidence of efficacy may be our best hope of attracting more human and financial capital to our vital work. Legal services lawyers and clinical law teachers should seize this opportunity with the same passion, ingenuity, and integrity that we have devoted to the cause of justice since the inception of our modern movements more than four decades ago.
Jeffrey Selbin is a clinical professor of law at the University of California, Berkeley, School of Law and the faculty director of the East Bay Community Law Center. Jeanne Charn is a senior lecturer on law at Harvard Law School and the director of the Bellow-Sacks Access to Civil Legal Services Project. Anthony Alfieri is a Dean’s Distinguished Scholar, a professor of law, and the director of the Center for Ethics and Public Service at the University of Miami School of Law. Stephen Wizner is the William O. Douglas Clinical Professor Emeritus of Law at Yale Law School. The authors thank Tony Lo Presti for impeccable research assistance and Catherine Albiston, Rebecca Sandefur, Richard Zorza, and The Yale Law Journal’s editors for generous input.
Preferred citation: Jeffrey Selbin, Jeanne Charn, Anthony Alfieri & Stephen Wizner, Service Delivery, Resource Allocation, and Access to Justice: Greiner and Pattanayak and the Research Imperative, 122 Yale L.J. Online 45 (2012), http://yalelawjournal.org/2012/07/30/selbin-charn-alfieri&wizner.html.
See Gary Bellow & Jeanne Kettleson, From Ethics to Politics: Confronting Scarcity and Fairness in Public Interest Practice, 58 B.U. L. Rev. 337 (1978).
See Paul R. Tremblay, Toward a Community-Based Ethic for Legal Services Practice, 37 UCLA L. Rev. 1101 (1990).
D. James Greiner & Cassandra Wolos Pattanayak, Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?, 121 Yale L.J. 2118 (2012).
See id. at 2121.
Id. at 2124.
Greiner and Pattanayak recommend the same caution. See id. at 2197.
HLAB is a highly respected teaching clinic housed in one of the nation’s elite law schools. It was founded by law students in 1913 and was one of the first law school clinics of any kind. See Harry Sandick & John A. Freedman, A History of the Harvard Legal Aid Bureau 2-4 (1996); Greiner & Pattanayak, supra note 3, at 2140-42.
Greiner and Pattanayak imply that excessive client demand for services ethically justifies the use of randomization. See Greiner & Pattanayak, supra note 3, at 2122, 2141; see also D. James Greiner, Cassandra Wolos Pattanayak & Jonathan Hennessy, The Limits of Unbundled Legal Assistance: A Randomized Study in a Massachusetts District Court and Prospects for the Future, 126 Harv. L. Rev. (forthcoming 2012) (manuscript at 47), http://ssrn.com/abstract=1948286 (explicitly stating that “oversubscription makes randomization easily permissible”).
For details on the study’s methodology, see Greiner & Pattanayak, supra note 3, at 2143-44. The research protocol, which contained client consent provisions, was approved by the Harvard University Institutional Review Board. See id. at 2143 n.95.
Greiner and Pattanayak devote an entire second introduction of the paper to defending this methodological choice. See id. at 2127-32.
Id. at 2144.
See id. at 2153-54.
See id. at 2172-73.
See id. at 2173-74.
Id. at 2174.
Richard Schwartz, Foreword to W. Vaughn Stapleton & Lee E. Teitelbaum, In Defense of Youth: A Study of the Role of Counsel in American Juvenile Courts, at ix, xii (1972). On the “deep resistance to the demystification of expertise,” see Daniel Kahneman, Thinking, Fast and Slow 224 (2011).
See, e.g., David Udell, Exec. Dir., Nat’l Ctr. for Access to Justice, Remarks at Panel on Empirical Research of Ira C. Rothgerber, Jr., Conference on Constitutional Law (Nov. 5, 2011), available at http://lawweb.colorado.edu/events/details.jsp?id=3462; Bob Sable, What Difference Representation—A Response, Concurring Opinions (Mar. 28, 2011, 12:10 AM), http://www.concurringopinions.com/archives/2011/03/what-difference-representation-a-response.html; David Udell, What Difference Presentation?, Concurring Opinions (Mar. 28, 2011, 8:04 AM), http://www.concurringopinions.com/archives/2011/03/what-difference-presentation.html.
See, e.g., Anthony V. Alfieri, Reconstructive Poverty Law Practice: Learning Lessons of Client Narrative, 100 Yale L.J. 2107 (1991).
See, e.g., Gary Bellow, Turning Solutions into Problems: The Legal Aid Experience, 34 Nat’l Legal Aid Ass’n Briefcase 106 (1977); Jeanne Kettleson, Caseload Control, 34 Nat’l Legal Aid Ass’n Briefcase 111 (1977).
See, e.g., Raymond H. Brescia, Robin Golden & Robert A. Solomon, Who’s in Charge, Anyway? A Proposal for Community-Based Legal Services, 25 Fordham Urb. L.J. 831 (1998); Ann Southworth, Lawyer-Client Decisionmaking in Civil Rights and Poverty Practice: An Empirical Study of Lawyers’ Norms, 9 Geo. J. Legal Ethics 1101 (1996).
See Russell Engler, Toward a Context-Based Civil Right to Counsel Through “Access to Justice” Initiatives, 40 Clearinghouse Rev. 196 (2006); Debra Gardner & John Pollock, Civil Right to Counsel’s Relationship to Antipoverty Advocacy: Further Reflections, 45 Clearinghouse Rev. 150 (2011); Alan Houseman, The Justice Gap: Civil Legal Assistance Today and Tomorrow, Ctr. for Am. Progress (June 2011), http://www.americanprogress.org/issues/2011/06/pdf/justice.pdf.
Greiner & Pattanayak, supra note 3, at 2201-04.
Although we heard much of what we describe here directly and discreetly from colleagues, some of these criticisms were expressed publicly. See supra note 17 and accompanying text.
See Greiner & Pattanayak, supra note 3, at 2127-28; see also Pascoe Pleasence, Trials and Tribulations: Conducting Randomized Experiments in a Socio-Legal Setting, 35 J.L. & Soc’y 8, 17-18 (2008) (making the same methodological choice).
See Greiner & Pattanayak, supra note 3, at 2172. One might argue that a low-stakes venue is a good one for training law students, but the clinical setting is also suspected of slowing down the adjudication process for educational purposes. The study, however, found that representation from any source delayed claimants’ receipt of benefits. See id. at 2169-70.
See Rebecca Sandefur, What Was the Question? Or, Scholarly Conventions and How They Matter, Concurring Opinions (Mar. 28, 2011, 10:41 AM), http://www.concurringopinions.com/archives/2011/03/what-was-the-question-or-scholarly-conventions-and-how-they-matter.html.
See Legal Servs. Corp., Delivery Systems Study: A Policy Report to the Congress and the President of the United States (1980).
Houseman, supra note 21, at 15; Performance Criteria Referenced to the ABA Standards for the Provision of Legal Aid, Legal Servs. Corp. (2007), http://www.legalaidnc.org/public/participate/legal_services_community/LSCPerformanceCriteriaReferencingABAStandards_2007edition.pdf.
See Additional Details on Budget Savings: Fiscal Year 1982 Budget Revisions, H.R. Doc. No. 97-41, at 362 (1981); Cong. Budget Office, An Analysis of President Reagan’s Budget Revisions for Fiscal Year 1982, at A-79 (1981).
Omnibus Consolidated Rescissions and Appropriations Act of 1996, Pub. L. No. 104-134, tit. V, § 504, 110 Stat. 1321, 1352-53 (1996). For details on the restrictions, see Michael J. Belaen, Change We Need: Why Enacting the Civil Access to Justice Act of 2009 Is Necessary To Expand Legal Aid for the Poor, 31 Hamline J. Pub. L. & Pol’y 329, 366 (2009); and Liza Q. Wirtz, The Ethical Bar and the LSC: Wrestling with Restrictions on Federally Funded Legal Services, 59 Vand. L. Rev. 971, 983-84 (2006).
Cut, Cap, and Balance: A Budget for Fiscal Year 2013, Republican Study Comm. 15 (2012), http://rsc.jordan.house.gov/UploadedFiles/RSC_FY_2013_Budget_Cut_Cap_and_Balance_Summary--FINAL.pdf.
Id. at 2123.
A metastudy of impact-of-counsel research found that the lawyers make a greater difference in cases with more procedural complexity. See Rebecca L. Sandefur, The Impact of Counsel: An Analysis of Empirical Evidence, 9 Seattle J. Soc. Just. 51, 52 (2010).
Greiner, Pattanayak & Hennessy, supra note 8. But see D. James Greiner, Cassandra Wolos Pattanayak & Jonathan Philip Hennessy, How Effective Are Limited Legal Assistance Programs? A Randomized Experiment in a Massachusetts Housing Court 1-2 (Oct. 23, 2011), http://ssrn.com/abstract=1880078 (finding “no statistically significant evidence that the service provider’s offer of full, as opposed to limited, representation had a large (or any) effect on the likelihood that the occupant would retain possession, on the financial consequences of the case, on judicial involvement in or attention to cases, or on any other litigation-related outcome of substantive import”).
Greiner & Pattanayak, supra note 3, at 2198-99.
See Bellow, supra note 19, at 121-22; see also Gary Bellow, Steady Work: A Practitioner’s Reflections on Political Lawyering, 31 Harv. C.R.-C.L. L. Rev. 297, 297-300 (1996) (discussing several successful examples of the strategy of focused case pressure).
See Greiner & Pattanayak, supra note 3, at 2198-99.
See Scott L. Cummings, The Pursuit of Legal Rights—and Beyond, 59 UCLA L. Rev. 506 (2012) (describing the growth and diversification of public interest law, including theoretical, organizational, and practical critiques of litigation as a law reform strategy).
Jeanne Charn, Time for a System Wide Quality Agenda, Mgmt. Info. Exchange J., Summer 2004, at 3. Partly in response to concerns about funder-mandated evaluation, the legal services field has embarked on “program-owned evaluation.” Martha Bergmark et al., “Program Owned Evaluation” as a Leadership and Management Tool in Civil Legal Services Programs (Nov. 2003) (unpublished draft), http://www.nlada.org/DMS/Documents/1071075437.57/ProgramOwnedEvaluation-draft.pdf.
See Greiner & Pattanayak, supra note 3, at 2175-84.
Id. at 2125-26. The common problems plaguing all but two prior studies include ill-defined interventions, multiple layers of selection bias, and lack of statistical significance. Id. at 2175-84.
See Jane Goodman-Delahunty et al., Insightful or Wishful: Lawyers’ Ability To Predict Case Outcomes, 16 Psychol. Pub. Pol’y & L. 133 (2010) (finding in a national study that lawyers are overconfident in their litigation-outcome predictions, even in the face of debiasing techniques).
See Model Rules of Prof’l Conduct R. 1.2 (2010) (specifying that clients establish the ends of representation, while lawyers determine the means).
See Carrie Menkel-Meadow & Robert G. Meadow, Resource Allocation in Legal Services: Individual Attorney Decisions in Work Priorities, 5 Law & Pol’y Q. 237 (1983).
See Gary Bellow, Management of Legal Services: Legal Aid in the United States, 14 Clearinghouse Rev. 337, 343 (1980). See generally Kahneman, supra note 16 (describing the cognitive biases that result in overconfidence of thinking and intuitive judgment, especially in irregular, low-validity clinical environments).
See, e.g., Fact Book 2010, Legal Servs. Corp., at 23-28 (2011), http://www.lsc.gov/sites/default/files/LSC/pdfs/LSC_2010_Fact_Book.pdf (reporting data for 2010).
Greiner and Pattanayak’s subsequent study of landlord-tenant cases in a Massachusetts District Court makes such a comparison between limited scope and full representation. Greiner, Pattanayak & Hennessy, supra note 8, at 2-5.
See, e.g., Edward Barnes & Liam Galbreth, E. Bay Cmty. Law Ctr., Lifting Welfare Employment Sanctions in the CalWORKs Program: The Experience of the Family Advocacy and Services Team (FAST), CALegalAdvocates.org (Sept. 13, 2001), http://www.calegaladvocates.org/library/attachment.76054 (evaluating the experience of FAST’s nonprofit work from July 2000 to July 2001).
See Jeanne Charn, Preventing Foreclosure: Do Clients Who Win Their Cases Keep Their Houses? (Jan. 19, 2007) (unpublished manuscript) (on file with authors).
Greiner & Pattanayak, supra note 3, at 2205-06.
Dignity interests have been studied in medicine for the purpose of improving end-of-life care. See, e.g., Harvey Max Cochinov et al., The Patient Dignity Inventory: A Novel Way of Measuring Dignity-Related Distress in Palliative Care, 36 J. Pain & Symptom Mgmt. 559 (2008).
See Monique W. Morris, Michael Sumner & Jessica Z. Borja, A Higher Hurdle: Barriers to Employment for Formerly Incarcerated Women, Thelton E. Henderson Ctr. for Soc. Justice (Dec. 2008), http://www.law.berkeley.edu/files/A_Higher_Hurdle_December_2008%281%29.pdf.
The procedural-justice literature tells us that participants’ experience in the system is determined in part by whether they feel they were treated fairly. See Tom R. Tyler, What Is Procedural Justice?: Criteria Used by Citizens To Assess the Fairness of Legal Procedures, 22 Law & Soc’y Rev. 103 (1988).
See Lee Epstein & Gary King, The Rules of Inference, 69 U. Chi. L. Rev. 1, 82-97 (2002) (noting the difficultly of assessing the validity of measurements, particularly of inherently normative judgments); see also Alon Harel & Ariel Porat, Commensurability and Agency: Two Yet-To-Be-Met Challenges for Law and Economics, 96 Cornell L. Rev. 749 (2011) (discussing the failure of law and economics to explain specific normative decisions). See generally Rethinking Social Inquiry: Diverse Tools, Shared Standards (Henry E. Brady & David Collier eds., 2d ed. 2010) (providing a variety of perspectives on and criticisms of the status quo in social science research).
See Bellow & Kettleson, supra note 1.
Am. Bar Ass’n, Report on the Legal Needs of the Low-Income Public: Findings of the Comprehensive Legal Needs Study (1994); Barbara A. Curran, The Legal Needs of the Public: The Final Report of a National Survey (1977).
Documenting the Justice Gap in America: The Current Unmet Civil Legal Needs of Low-Income Americans, Legal Servs. Corp. (Sept. 2009), available at http://www.legalaidnc.org/public/learn/publications/Documenting%20the%20Justice%20Gap/DocumentingtheJustice%20Gap_Update_FINAL_Sept_30_09.pdf.
See Pascoe Pleasence et al., Legal Servs. Comm’n, Causes of Action: Civil Action and Social Justice (2d ed. 2006); Rebecca L. Sandefur & Jeanne Charn, Class and Advice Seeking: Comparative Insights (Mar. 3, 2009) (unpublished manuscript), http://www.ilagnet.org/conf_2009/papers/Class%20and%20Advice%20Seeking%20%20Comparative%20Insights%20%28RS%20and%20JC%29.pdf; Hazel Genn et al., Understanding Advice Seeking Behaviour: Further Findings from the LSRC Survey of Justiciable Problems, Legal Servs. Res. Centre (2004), http://www.lawcentres.org.uk/uploads/Understanding_Advice_Seeking_Behaviour_LSRC_2004.pdf.
See, e.g., Bruce D. Sales et. al., Is Self-Representation a Reasonable Alternative to Attorney Representation in Divorce Cases?, 37 St. Louis U. L.J. 553, 576-88 (1993) (presenting a study indicating that self-represented family law litigants report a high rate of satisfaction); Liz Pejeau, Limited Scope Representation: Making Representation Affordable . . . and Ethical, Orange County Law., Oct. 2006, at 38 (noting that many self-represented litigants seeking limited attorney assistance are interested in retaining increased control over their cases); John M. Greacen, Self-Represented Litigants and Court and Legal Services Responses to Their Needs: What We Know, Cal. Admin. Office of the Cts. (2003), http://www.courts.ca.gov/partners/documents/SRLwhatweknow.pdf (providing an overview of studies on self-represented litigants).
Research suggests that many Americans do not resolve civil justice problems with lawyers or in tribunals. See Rebecca L. Sandefur, The Fulcrum Point of Equal Access to Justice: Legal and Non-Legal Institutions of Remedy, 42 Loy. L.A. L. Rev. 949, 969 (2009); Rebecca L. Sandefur, The Importance of Doing Nothing: Everyday Problems and Responses of Inaction, in Transforming Lives: Law and Social Process 1, 113 (Pascoe Pleasence et al. eds., 2007).
See Mary Helen McNeal, Report of the Working Group on Limited Legal Assistance, 67 Fordham L. Rev. 1819 (1999).
The idea that some clients can effectively represent themselves in certain settings is not new. See, e.g., Ralph C. Cavanagh & Deborah L. Rhode, Project, The Unauthorized Practice of Law and Pro Se Divorce: An Empirical Analysis, 86 Yale L.J. 104 (1976) (establishing that some pro se litigants can effectively represent themselves with limited assistance).
Three decades ago, the Civil Litigation Research Project, funded by the Department of Justice, studied the relative merits of courts as dispute-processing venues, generating a considerable literature from which a new research agenda can draw both substantive and methodological lessons. See David M. Trubek et al., Civil Litigation Research Project Final Report (1983); see also Herbert M. Kritzer, Bibliography of Publications and Papers of the Civil Litigation Research Project, U. of Wis.-Madison (last updated Dec. 28, 2004), http://users.polisci.wisc.edu/kritzer/research/clrpbib.htm (listing publications produced by the project).
372 U.S. 335 (1963).
For an overview of civil Gideon, see Russell Engler, Shaping a Context-Based Civil Gideon From the Dynamics of Social Change, 15 Temp. Pol. & Civ. Rts. L. Rev. 697 (2006); and Clare Pastore, A Civil Right to Counsel: Closer to Reality?, 42 Loy. L.A. L. Rev. 1065 (2009).
See Russell Engler, Connecting Self-Representation to Civil Gideon: What Existing Data Reveal About When Counsel Is Most Needed, 37 Fordham Urb. L.J. 37 (2010).
See Lassiter v. Dep’t of Soc. Servs., 452 U.S. 18 (1981) (applying the balancing test proffered by Mathews v. Eldridge, 424 U.S. 319 (1976), to hold that the Due Process Clause of the Fourteenth Amendment does not require the appointment of counsel for indigent litigants in parental-termination proceedings); see also Jason Parkin, Adaptable Due Process, 160 U. Pa. L. Rev. 1309 (2012) (arguing that the constitutional adequacy of due process safeguards must evolve with the facts and circumstances of administrative adjudication).
See Comm’n on Nonlawyer Practice, Am. Bar Ass’n, Nonlawyer Activity in Law-Related Situations: A Report with Recommendations (1995); Deborah L. Rhode, Access to Justice 89-91 (2004).
See, e.g., Richard Zorza, Access to Justice: The Emerging Consensus and Some Questions and Implications, 94 Judicature 156, 156-57 (2011) (describing the four key access-to-justice elements as “court simplification and services, bar flexibility, legal aid efficiency and availability, and system of triage and assignment”).
131 S. Ct. 2507 (2011).
Both parties referenced the Greiner and Pattanayak study in their briefs. The respondents’ brief stated: “Lawyers are unlikely to make a substantial difference in these simple proceedings. A recent randomized, controlled Harvard study of simple, nonjury litigation found no significant difference in success rates between litigants who were offered legal representation and those who were not.” Brief of Respondents at 54-55, Turner, 131 S. Ct. 2507 (No. 10-10), 2011 WL 481836. The petitioner’s brief replied: “Respondents cite a study for the proposition that lawyers are unlikely to make a difference, but as its authors concede, the study is ‘useless’ for determining the value of legal representation.” Reply Brief for Petitioner at 16 n.11, Turner, 131 S. Ct. 2507 (No. 10-10), 2011 WL 805230 (citation omitted).
In Lassiter, the Court—in holding that there is no right to appointed counsel in cases involving termination of parental rights—noted a “presumption that there is a right to appointed counsel only where the indigent, if he is unsuccessful, may lose his personal freedom.” Lassiter, 452 U.S. at 27.
Turner, 131 U.S. at 2520.
Id. The Justices themselves seem concerned with the existence of data or empirical evidence. See Transcript of Oral Argument at 23-24, Turner, 131 U.S. 2507 (No. 10-10) (question of Justice Kennedy) (requesting data about how often counsel appears in child-support contempt cases); id. at 30 (question of Justice Ginsburg) (asking for a jurisdiction where a “less than counsel” procedure has been instituted); id. at 50 (question of Justice Breyer) (asking for data on gender disparity in lawyers and stating that “[t]here must be some organization that’s studied that”); id. at 53-54 (question of Justice Ginsburg) (asking whether anyone has computed how expensive counsel would be as opposed to confinement costs).
See Symposium (Turner v. Rogers), Concurring Opinions, http://www.concurringopinions.com/archives/category/symposium-turner-v-rogers (last visited June 24, 2012).
Benjamin H. Barton & Stephanos Bibas, Triaging Appointed-Counsel Funding and Pro Se Access to Justice, 160 U. Pa. L. Rev. 967, 970-71 (2012).
The case will also present new challenges and opportunities for an over-taxed judiciary. See Richard Zorza, The Implications of Turner v. Rogers: A New Day for Judges and the Self-Represented, 50 Judges’ J. 16 (2011).
See supra note 75.
Judith Resnik, Fairness in Numbers: A Comment on AT&T v. Concepcion, Wal-Mart v. Dukes, and Turner v. Rogers, 125 Harv. L. Rev. 78, 158 (2011).
Thoughtful commentators are already imagining what an access-maximizing system might look like in the post-Turner world. See, e.g., Richard Zorza, The Access to Justice “Sorting Hat”: Towards a System of Triage and Intake that Maximizes Access and Outcomes (2012) (unpublished manuscript) (on file with authors).
See Stephen Wexler, Practicing Law for Poor People, 79 Yale L.J. 1049, 1053 (1970).
See Richard Moorhead & Pascoe Pleasence, Access to Justice After Universalism: Introduction, 30 J.L. Soc’y 1, 2-3 (2003).
Ab Currie, The Legal Problems of Everyday Life, in 12 Sociology of Crime, Law, and Deviance: Access to Justice 1 (Rebecca Sandefur ed., 2009).
Jeffrey Selbin, Josh Rosenthal & Jeanne Charn, Access to Evidence: How an Evidence-Based Delivery System Can Improve Legal Aid for Low- and Moderate-Income Americans, Ctr. for Am. Progress (2011), http://www.americanprogress.org/issues/2011/06/pdf/evidence.pdf.
Empirical Legal Studies is a positivist-quantitative strand often associated with law and economics; the New Legal Realism is an interpretivist-qualitative strand often associated with socio-legal studies. See Elizabeth Chambliss, When Do Facts Persuade? Some Thoughts on the Market for “Empirical Legal Studies,” Law & Contemp. Probs., Spring 2008, at 17, 23, 31-32; Howard Erlanger et al., Foreword, Is It Time for a New Legal Realism?, 2005 Wis. L. Rev. 335, 336, 340.
See, e.g., Gordon Guyatt et al., Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine, 268 J. Am. Med. Ass’n 2420 (1992); David L. Sackett et al., Evidence Based Medicine: What It Is and What It Isn’t, 312 BMJ 71, 71 (1996) (defining evidence-based medicine as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” through “integrating individual clinical expertise with the best available external clinical evidence from systematic research”).
See, e.g., W.A. Rogers, Evidence-Based Medicine and Justice: A Framework for Looking at the Impact of EBM upon Vulnerable or Disadvantaged Groups, 30 J. Med. Ethics 141 (offering a social justice critique of evidence-based medicine); Mark R. Tonelli, The Limits of Evidence-Based Medicine, 46 Respiratory Care 1435 (2001) (arguing that clinical experience differs in kind, not degree, from empirical evidence and should be equally valued in medical decision-making).
Jeanne Charn & Jeffrey Selbin, The Clinic Lab Office (2010) (unpublished manuscript), http://www.nyls.edu/user_files/1/3/4/15/1009/ClinicLabOffice.pdf. See generally Donald A. Schön, Knowing-in-Action: The New Scholarship Requires a New Epistemology, 2 Change 27, 29 (1995) (arguing that law practice is “a setting not only for the application of knowledge but for its generation”).
Rebecca L. Sandefur & Jeffrey Selbin, The Clinic Effect, 16 Clinical L. Rev. 57 (2009).
See David A. Santacroce & Robert R. Kuehn, The 2010-11 Survey of Applied Legal Education, Ctr. for the Study of Applied Legal Educ. (2012), http://www.csale.org/files/CSALE.Report.on.2010-11.Survey.5.16.12.Revised.pdf.
Jeanne Charn & Jeffrey Selbin, Legal Aid, Law School Clinics and the Opportunity for Joint Gain, Mgmt. Info. Exchange J., Winter 2007, at 28.
See, e.g., Rebecca L. Sandefur & Aaron C. Smyth, Access Across America: First Report of the Civil Justice Infrastructure Mapping Project (2011) (conducting the first-ever national study of the availability of civil legal services).
The Access to Justice Initiative, U.S. Dep’t of Justice, http://www.justice.gov/atj (last visited June 24, 2012); see also Laura K. Abel, Evidence-Based Access to Justice, 13 U. Pa. J.L. & Soc. Change 295 (2010) (discussing the need for increased empirical research to better target various access-to-justice interventions).
For an example of the potential of the new American Bar Foundation initiative, see Sandefur & Smyth, supra note 93.
Deborah Rhode, Access to Justice: An Agenda for Legal Education and Research, 61 J. Legal Educ. (forthcoming 2012), http://ssrn.com/abstract=1903769.
U.S. Gov’t Accountability Office, GAO-10-30, Program Evaluations: A Variety of Rigorous Methods Can Help Identify Effective Interventions (2009); Peter Orszag, Building Rigorous Evidence To Drive Policy, Office of Mgmt. & Budget (June 8, 2009, 8:39 AM), http://www.whitehouse.gov/omb/blog/09/06/08/BuildingRigorousEvidencetoDrivePolicy.
For example, the National Institute of Justice recently solicited research proposals in the area of indigent defense “to examine issues relevant to access to counsel and effective assistance of counsel in a scientifically rigorous way.” Nat’l Inst. of Justice, Social Science Research on Indigent Defense, U.S. Dep’t of Justice 9 (2012), https://www.ncjrs.gov/pdffiles1/nij/sl001006.pdf. The solicitation noted that “[w]ithin applications proposing evaluation research, funding priority will be given to experimental research designs that use random selection and assignment of participants to experimental and control conditions.” Id. at 9.
See, e.g., Sargent Shriver Civil Counsel Act, Cal. Gov. Code § 68651(b)(5) (West 2011); Assembly Bill No. 590, 2009-2010 Sen., Reg. Sess. (Cal. 2009), available at http://leginfo.ca.gov/pub/09-10/bill/asm/ab_0551-0600/ab_590_bill_20091011_chaptered.pdf; Sean Stannard-Stockton, The Rise of Evidence-Based Grantmaking, Tactical Philanthropy (May 11, 2010, 8:29 AM), http://www.tacticalphilanthropy.com/2010/05/the-rise-of-evidence-based-grantmaking; see also Ashley Allen, Investing for Impact: Innovations in Measurement and Evaluation, Philanthropy News Digest (Sept. 28, 2011) http://foundationcenter.org/pnd/impact/impact_item.jhtml?id=355100010 (discussing the development of evidence-based systems to guide grantmaking). But see Garry W. Jenkins, Who’s Afraid of Philanthrocapitalism?, 61 Case W. Res. L. Rev. 753, 786 (2011) (sounding a note of caution about the rising emphasis on metrics-based and results-oriented giving, which can undermine philanthropic values and discourage individual and government giving).
For example, the Center for American Progress has established a “Doing What Works” project to encourage the evaluation of government programs’ effectiveness and the corresponding investment of resources. See Doing What Works, Ctr. for Am. Progress, http://www.americanprogress.org/projects/doing_what_works (last visited June 24, 2012).
Frank Michelman, The Legal Profession and Social Change: The Challenge to the Law Schools, in The Path of the Law from 1967: Proceedings and Papers at the Harvard Law School Convocation Held on the One-Hundred Fiftieth Anniversary of Its Founding 125, 128 (Arthur E. Sutherland ed., 1968) (referring to Harvard Law School’s Community Legal Assistance Office, established in the fall of 1966 as a demonstration project funded by the federal Office of Economic Opportunity).
Michael D. Greenberg & Geoffrey McGovern, An Early Assessment of the Civil Justice System After the Financial Crisis, RAND Inst. for Civ. Justice 44 (2012), http://www.rand.org/content/dam/rand/pubs/occasional_papers/2012/RAND_OP353.pdf (noting that the increased demand and diminished supply for civil legal services will “disproportionately affect the most vulnerable members of society . . . [and] could be a major focus for empirical research in the future”).