Introduction

This year marks the fiftieth anniversary of the Freedom of Information Act (FOIA). In the words of President Obama, this law, “which encourages accountability through transparency, is the most prominent expression of a profound national commitment to ensuring an open government.”1 Since its enactment, the FOIA has become an indispensable mechanism for facilitating the public’s understanding of the operations and activities of the federal government. The statute applies to all federal agencies and functions in a straightforward way: virtually anyone, for any reason, can request access to agency records, and agencies, in turn, are required to provide those records unless they are protected from disclosure by nine, carefully crafted exemptions.2

Since its enactment, the FOIA has brought an unprecedented level of transparency to government operations in the United States. Intervening developments in technology, particularly the invention and use of the Internet, have significantly impacted both how agencies administer the FOIA and how the public interacts with agencies. Where agency records once primarily consisted of “memorandums or letters,”3 in the Internet age there has been an explosion of electronic records, particularly email messages, which by their very volume have sometimes challenged agencies’ capacities to respond to requests. At the same time, agencies have harnessed technology to assist in the processing of requests and, more importantly, have used the Internet as a means to provide even greater access to information, facilitating its dissemination and use by a wide range of interested stakeholders. In short, the Internet has increased the impact and utility of the FOIA in ways that could not have been foreseen when the law was passed fifty years ago.

When it was enacted, the FOIA represented a dramatic departure from previous federal agency disclosure mandates. Congress passed the FOIA in 1966 after many years of debate and hearings to determine the appropriate contours of the new access law.4 Previously, the Administrative Procedure Act (APA) of 1946 contained a provision concerning public access to government records, providing that “matters of official record shall in accordance with published rule be made available to persons properly and directly concerned except information held confidential for good cause found.”5 The FOIA fundamentally altered that statutory scheme. First, it gives any person—rather than only “persons properly and directly concerned”—the right to request access to government records.6 Second, the FOIA articulates nine specific grounds for exempting information from disclosure, instead of the APA’s vague standard of “good cause found.”7 Third, the FOIA provides for judicial review of the government’s responses to FOIA requests. Finally, the FOIA requires agencies to proactively make several specific types of records available to the public, such as statements of agency policy, final opinions, and frequently requested records.8

Congress has amended the FOIA several times to account for the availability of the Internet. Those revisions began in 1996, when the subjects of electronic records and electronic availability were first addressed in the statute. For instance, the 1996 Electronic FOIA (EFOIA) Amendments obligated agencies to begin posting required proactive disclosures online in what were then called “electronic reading rooms.”9 The EFOIA Amendments also made extensive revisions to the FOIA’s annual reporting requirements, requiring agencies to report on, inter alia, the numbers of requests received and processed each year, and, notably, to make their Annual FOIA Reports available electronically.10 The Department of Justice, in turn, was required to establish “a single electronic access point” for those agency Annual FOIA Reports.11 The OPEN Government Act of 2007 required agencies to make the data used to compile their Annual FOIA Reports available electronically to the public upon request.12 Most recently, the FOIA Improvement Act of 2016 required agencies to post the raw data used to compile their Annual FOIA Reports and to publish their FOIA Reference Guides on their websites.13

Over the years, agencies have expanded and improved their FOIA web presence far beyond these statutory mandates, using the Internet to dramatically increase the number of records and data made available proactively, posting more material in open format so it is readily searchable, and utilizing web applications to make information more accessible. Agencies have also leveraged new technologies to improve communication with requesters and improve efficiency in FOIA administration, even as the numbers of requests and agency records have increased over time. The Office of Information Policy (OIP) at the Department of Justice (DOJ) is responsible for encouraging agency compliance with the FOIA and provides guidance to agencies on various aspects of FOIA administration. Going forward, OIP is working to further advance FOIA administration through two initiatives that could not have been imagined prior to the age of the Internet: implementation of a “Release to One is a Release to All” presumption and creation of a consolidated, government-wide FOIA portal.

I. impact of the internet on foia administration

A. More Information Available Online in Usable Formats

The Internet has revolutionized government transparency by allowing agencies to make more information available to the public than ever before. OIP’s Guidance on Proactive Disclosures encourages agencies to systematically post information, make information available in user-friendly and open formats, and post records in locations most helpful to the public.14 Agencies have created online FOIA Libraries that contain operational documents, frequently requested records, FOIA reports, and other records of interest to the public. Online FOIA Libraries make proactive disclosures more widely available to the public than the pre-Internet conventional reading rooms, which required a member of the public to physically visit the agency to view documents.

In addition to the materials contained in FOIA Libraries, agency websites contain significant amounts of proactively disclosed material on program and topic-specific pages. For example, to help consumers make better decisions regarding financial products, the Consumer Financial Protection Bureau regularly publishes consumer information, such as a Consumer Complaint Database (containing anonymized complaint data) and the Home Mortgage Disclosure Act database. The agency also seeks to make financial markets more transparent, through initiatives such as Know Before You Owe (addressing credit cards, mortgages, and student loans).15 The Federal Bureau of Investigation’s (FBI) FOIA Library, “The Vault,” contains over 6,700 documents and other media on subjects such as the Alger Hiss investigation and the FBI’s “Digital Evidence Policy.”16 The National Science Foundation continually updates its public database of documents relating to its awards with “abstracts of awarded proposals, resulting publications, and final reports on research results.”17

Developments in technology have not only allowed agencies to regularly post information on their websites, but also to make that information available in open formats that are easier to locate and analyze. President Obama has stated that “[i]nformation maintained by the Federal Government is a national asset” and should be disclosed “in forms that the public can readily find and use.”18 The United States’ First Open Government National Action Plan included a commitment to harness the power of technology in FOIA administration by working to ensure that information is searchable and readily usable by the public.19 OIP has encouraged agencies to use metadata to make documents more searchable and easier to find, and to lay the foundation for a “virtual government-wide FOIA library.”20

Each year, in Chief FOIA Officer Reports, agencies describe steps taken to make information more usable for the public. For example, the U.S. Agency for International Development continues to make agency data available in standard, nonproprietary, and machine-readable formats, and solicits public feedback on the information that the agency makes available. “Users can interact with [agency] data, see how [other members of the public] are using the data, and leave questions or comments” about the data for the agency.21 The Institute of Museum and Library Services (IMLS) hosted an Open Data Open House, convening data wonks, librarians, curators, researchers, developers, and representatives from a wide range of organizations to explore how IMLS data could be used and how its data catalog could be improved.22 In sum, the Internet allows agencies to easily make datasets and other information available in open formats. This accessibility facilitates oversight of agency operations and promotes innovation by members of the public.

B. Greater Transparency and Accountability of Agencies’ FOIA Administration

The Internet has also facilitated greater accountability for FOIA administration itself through the availability of data on FOIA.gov and through OIP’s summaries and assessments of agency FOIA reports. Each year in their Annual FOIA Reports, agencies provide detailed statistics on the FOIA requests received and processed during the fiscal year, including response times, exemptions used, dispositions, and information about administrative appeals. While agencies publish their reports in human-readable (PDF or HTML) and machine-readable (XML) formats on their websites, the data is also included on FOIA.gov, the government’s one-stop shop for FOIA data and resources. Among many other functions, FOIA.gov takes these detailed statistics contained in agencies’ Annual FOIA Reports and displays them graphically. FOIA.gov brings greater transparency to agencies’ compliance with the FOIA itself, making it easy for the public to obtain detailed statistics over time about backlogs, FOIA processing costs, and use of exemptions. The availability of information on FOIA.gov also gives the public a better idea of what to expect when submitting requests to agencies, for instance, by allowing them to see average processing times for simple versus complex requests.

The Internet therefore allows both private citizens and public sector actors to play a more informed role in monitoring FOIA administration. OIP, through its role in encouraging compliance with the FOIA, assesses agencies each year based on information contained in both Annual FOIA Reports and Chief FOIA Officer reports. Chief FOIA Officer Reports detail each agency’s efforts throughout the year to implement the Department of Justice’s 2009 FOIA Guidelines.23 Each year, OIP extensively analyzes the reports to determine the government’s overall progress in implementing the Department of Justice’s FOIA Guidelines and to identify any areas for improvement. OIP issues a narrative summary and detailed assessment in which OIP scores each agency on several milestones tied directly to the five key areas addressed in the Department’s FOIA Guidelines. These five key areas are: applying the presumption of openness, having an effective system in place for responding to requests, increasing proactive disclosures, utilizing technology, and improving timeliness and reducing any backlogs. To make the assessment progressively more challenging, each year OIP updates the milestones that it scores agencies on, based on the input of civil society and the progress that agencies have already made in implementing the FOIA Guidelines. All agencies’ Annual FOIA Reports, Chief FOIA Officer Reports, and Summaries and Assessments from OIP are posted centrally on OIP’s website to provide easy access for the public.

OIP promotes agency accountability under the FOIA in a number of ways. Agencies’ Annual Report data and the CFO Report assessment results are useful tools that help OIP identify areas where additional policy guidance and training are needed, whether across the government as a whole or targeted towards specific agencies. OIP, as well as civil society organizations, can draw on OIP’s policy guidance to encourage agency compliance with specific aspects of the FOIA. Additionally, through its review of each agency’s reports, OIP can work individually with an agency, in some cases by meeting with its Chief FOIA Officer, to address any discrepancies or areas in need of further improvement.24 OIP may also review inquiries made by the public that raise issues regarding agencies’ compliance with the FOIA and the Department of Justice’s FOIA Guidelines. By overseeing agencies’ FOIA reporting obligations, providing policy guidance and training, and conducting targeted outreach to agencies as needed, OIP works to encourage compliance and promote accountability with the FOIA.

C. Improved Communication Between Requesters and Agencies

Communication between agencies and requesters has also evolved for the better as a result of the Internet. In their FOIA Memoranda issued in 2009, the President and the Attorney General called on agencies to “use modern technology to inform citizens about what is known and done by their Government.”25 OIP has also issued policy guidance “on the importance of good communication” with requesters, encouraging agencies to communicate electronically.26 Agency websites contain FOIA contact information, instructions for submitting requests, and often a portal that allows requesters to submit and track their requests electronically. Nearly all agencies communicate electronically with requesters whenever feasible if that is the requester’s preferred mode of communication.27

D. Leveraging Technology for Greater Efficiency as Records and Requests Increase

Though the Internet has brought substantial benefits to the administration of the FOIA, it has also contributed to new challenges through the creation of more records. New “technologies have continuously allowed federal agencies to create and accumulate more records.”28 Agencies also generate records on new platforms, such as social media.29 And, while email has revolutionized how government employees communicate and conduct business, email has also led to a proliferation of agency records. As a result, agencies must search through and process many more records in response to FOIA requests than in the pre-Internet era. In addition to increased numbers of records, agencies continue to receive record numbers of requests. Whereas Congress originally anticipated that agencies would receive only a few thousand requests each year, the actual numbers of requests have far outpaced those early predictions.30 The government overall received 713,168 requests in fiscal year 2015 alone.31

There are likely several factors contributing to this large and growing number of requests. First, in the age of the Internet and 24-hour news cycles, there is far more dissemination and discussion of news and events. In this information-hungry world, the FOIA serves a valuable role in providing yet another avenue for the public to learn more about government activities and operations. The FOIA provides an easy way for the public to become or remain engaged in public debate over issues of concern. Individuals also request records about themselves or their family members using FOIA. Moreover, as agencies themselves post more information online, the public’s appetite for information seems to increase, with requesters submitting more detailed and complex requests for records related to a posted document. As agencies disclose more information both proactively and in response to requests, the increased volume of requests, coupled with the explosion of records generated by agencies, challenge agencies’ abilities to efficiently keep pace with demand.

The proliferation of records can make locating and processing responsive material incredibly time-consuming. Fundamentally, with more records created, there are more records to search through and to ultimately process in response to requests. Searching email, for example, may require a FOIA processor to sort through hundreds or thousands of pages of potentially responsive emails that result from even a tailored electronic search, while only identifying a small number of responsive documents. While agencies are encouraged to work with requesters throughout the process to ensure that the FOIA specialist is locating the material the requester seeks, if a high volume of responsive records ultimately requires a line-by-line review, it will inevitably take more time than processing a single document. The ever-increasing numbers of requests compound these general processing challenges.

Fortunately, agencies’ use of technology has helped them become more efficient in processing requests each year. While the number of full time FOIA staff has increased approximately twelve percent since 2008, the relative increase in the number of requests processed has been far greater. In fiscal year 2015, the government overall processed 769,903 requests—a nineteen percent increase from the number processed in fiscal year 2014 and a twenty-four percent increase from the number processed in 2008.32 Many agencies are now able to utilize document management software typically used in the litigation context to assist with FOIA processing. Digital tools that offer de-duplication and advanced search capabilities can assist agencies in searching for and processing high volumes of records.

In 2013, OIP and the Civil Division at DOJ conducted a pilot program that demonstrated the significant time and cost savings of using digital tools.33 Since then, several DOJ components began “using tools to search for, sort, and de-duplicate responsive documents.”34 Other agencies, such as the Departments of Commerce, Homeland Security, Defense, Agriculture, and Education, have implemented advanced technologies to share documents, conduct more advanced searches, and de-duplicate documents more efficiently.35 Agencies continue to look for ways to leverage technology across all phases of the FOIA process. Such innovations will remain critical as the volume and types of records and requests continue to increase over time.

II. new foia initiatives

While the government has adapted to challenges in complying with the FOIA as a result of developments in technology, it continually searches for new ways to use the Internet to improve FOIA administration and government transparency overall. OIP is currently working on two initiatives to improve FOIA administration, both of which are directly tied to harnessing the capabilities and benefits of the Internet.36 The first initiative is promoting broader release of records through a “release to one is a release to all” presumption. The second initiative is the launch of a centralized FOIA request portal.

A. “Release to One Is a Release to All” Presumption

In July 2015, OIP launched a six-month pilot program with seven volunteer federal agencies to assess the viability of a policy that would direct agencies to proactively post online the records processed for release under the FOIA. This approach would ensure that all citizens—not just those making a request—have access to information released under the FOIA. The President directed the newly established Chief FOIA Officers (CFO) Council37 to consider the lessons learned from the DOJ pilot program and work to develop a federal government policy establishing a “Release to One Is a Release to All” presumptive standard for federal agencies when releasing records under FOIA.

The CFO Council began by examining issues critical to this policy’s implementation, including the impact on investigative journalism efforts and the best way to address technological and resource challenges, at its first meeting on July 22, 2016.38 The Council invited feedback from agencies, journalists, and other members of the public, which it discussed at its second meeting on September 15, 2016.39 The President directed that guidance on implementing the “release to one is release to all” presumption be issued by January 1, 2017. As agencies begin adopting a “release to one is release to all” presumption for the FOIA, they will be turning a long-standing FOIA maxim into a reality. Rather than merely sending FOIA-processed records to one requester, agencies will begin making processed records available to all. These disclosures will further the President’s and DOJ’s direction to agencies to use modern technology to make information available to the public.

B. Consolidated Request Portal

The Department of Justice has also been working on creating a consolidated FOIA request portal and expanding the services offered on FOIA.gov. These efforts, which began under the Administration’s Second and Third Open Government National Action Plans, will help fulfill the FOIA Improvement Act of 2016’s requirement that the government create a consolidated portal that would allow a requester to make a request to any agency from a single website.40

For requesters unfamiliar with the FOIA process or the organization of agencies and components, a consolidated request portal could provide helpful guidance and convenience. The DOJ’s vision for the consolidated portal, however, extends far beyond those initial considerations. The Department is planning for the portal to guide requesters through the process, assisting them in finding the agency most likely to maintain the records they seek. Standardized forms for each agency will ensure that the requests meet each agency’s requirements. Significantly, the Department’s vision for the portal includes features to direct potential requesters to already-posted material where they may find the information they seek without even having to make a FOIA request. The portal can also assist agencies in compiling their Annual FOIA Reports and can be used in updating their contact information. Finally, the vision for the portal includes ultimately creating a feature to allow requesters to track their submitted requests online.

Through these various functionalities, we are striving to assist both requesters and agencies by facilitating a more efficient FOIA process. For example, if requesters can readily locate already-posted material, they may not need to make a request, or if they do, the request can be more focused than it otherwise would have been. Similarly, if requesters know which agencies are most likely to possess the records they seek, that will reduce the number of misdirected requests, achieving efficiencies all around.

OIP is proud to be working on these two new initiatives, and will continue to encourage all agencies to constantly examine ways technology can help them achieve greater efficiencies in FOIA administration.

Conclusion

Since the FOIA was passed in 1966, FOIA administration across the government looks drastically different in the age of the Internet, and the federal government has become more transparent as a result. Agencies can post most information online, anyone can more easily request access to records, and the public is far better informed about the operations and activities of their government. Government accountability—including for the administration of the FOIA itself—can also be accomplished more efficiently. While the launch of the Internet and the resulting explosion in the number of electronic records has contributed to new challenges in complying with the FOIA, agencies are leveraging technology to meet these challenges. In the coming months and years, FOIA administration will continue to evolve as agencies embrace technology and work to further the spirit of openness that the law embodies.

Melanie Ann Pustay is the Director of the Office of Information Policy of the United States Department of Justice, the lead agency responsible for the implementation of the Freedom of Information Act across the federal government.

Preferred Citation: Melanie Ann Pustay, Memorandums to Messages: The Evolution of FOIA in the Age of the Internet, 126 Yale L.J. F.252 (2016), www.yalelawjournal.com/forum/memorandums-to-messages.