Using Metadata in FOIA Documents Posted Online to Lay the Foundation for Building a Government-Wide FOIA Library

On April 9, 2012, the Department of Justice released version 2.0 of its Open Government Plan which detailed the Department’s “ongoing and anticipated efforts to increase openness” and which announced new Department initiatives concerning the administration of the Freedom of Information Act (FOIA).  A key FOIA initiative in the Department’s Open Government Plan is the development of metadata standards “that will facilitate the ability of interested persons to search for and retrieve documents across websites and disparate record keeping systems.” 

Put simply, metadata is “data about data.”   Metadata provides a description and important facts about material that is posted online and is itself machine-readable and can be searched by other computers.  When related documents, such as FOIA documents that are proactively disclosed in agency FOIA Libraries, are posted with uniform metadata it is far easier to locate them during a search of the Internet.  Because of the value that metadata adds to search capabilities, the Department’s Office of Information Policy (OIP) is introducing a uniform metadata “FOIA tag” for agencies to use when they post FOIA processed data and documents online in agency FOIA Libraries.

Through the consistent use of this standardized “FOIA tag” in posting records in their own FOIA Libraries as outlined below agencies will in effect create a “virtual” government-wide FOIA Library by allowing the public to run simple, topical keyword searches that will efficiently retrieve documents from across the universe of Federal government FOIA Libraries.  Such searches will be able to be conducted naturally through commercial search engines that are already used by millions of people every day, by combining any topical keyword with the term “FOIA.”  OIP will be issuing further guidance to agencies on this topic in the months ahead and plans to customize features on FOIA.gov so that users have the option to access all FOIA-tagged documents across the federal government from one single location. 

Evolution of Proactive Disclosures under the FOIA

Subsection (a)(2) of the FOIA requires agencies to “make available for public inspection and copying” four categories of records.  5 U.S.C. § 552(a)(2) (2006 & Supp. IV. 2010).  The first three categories, which collectively can be referred to as “operational” records, consist of final opinions made in the adjudication of cases, policy statements, and administrative staff manuals and instructions to staff that affect a member of the public.  Id. § 552(a)(2)(A-C).  The fourth category, which is referred to as “FOIA-processed” records, consists of records requested under the FOIA which “have become or are likely to become the subject of subsequent requests.”   Id. § 552(a)(2)(D).   When agencies make these records available it is known as making a “proactive disclosure.”  The Department’s Guide to the Freedom of Information Act provides detailed information about the FOIA’s proactive disclosure requirements. 

In addition to the proactive disclosures that are required by the FOIA statute, the President has directed agencies to “take affirmative steps to make information public,” without “wait[ing] for specific requests from the public,” and the Attorney General, in his FOIA Guidelines, has directed agencies to “readily and systematically post information online in advance of any public request.”   Thus, in compliance with these directives, agencies are engaged in an ongoing process of identifying information of interest to the public that can be made available online without the need to make a FOIA request. 

Historically, agencies made proactive disclosures by placing hard copies of the records in dedicated places in their buildings, known as “Reading Rooms.”  With the passage of the e-FOIA Amendments of 1996, agencies were required to use “computer telecommunications” or “other electronic means” to make proactive disclosures of records created after November 1, 1996.  As a result, while some agencies still maintained physical Reading Rooms, over time, agencies established “electronic Reading Rooms” which were dedicated pages on agency FOIA websites where proactive disclosures were posted.  More recently, even electronic Reading Rooms are being re-imagined as agencies make documents available online in a more organic fashion, posting them throughout their overall agency websites in locations where users are more likely to look for them. 

In March 2010, OIP introduced the concept of the online “FOIA Library,” providing a centralized location for agency FOIA disclosures while allowing greater flexibility for agencies to post records wherever, and in whatever format, best meets their own needs and the needs of the community of individuals who access their websites.  While proactively disclosed documents generally may be posted throughout agency websites, at a minimum, agencies should include FOIA-processed documents (such as “frequently requested” records and any other records that have been disclosed pursuant to a FOIA request that an agency chooses to post online) in their designated FOIA Libraries.  Additional information on the requirements for the electronic availability of proactive disclosures is available in the Department’s Guide to the Freedom of Information Act. 
 
Agency websites are now the primary mechanism for making proactive disclosures under the FOIA, including both required disclosures pursuant to subsection (a)(2) of the FOIA and additional information that is posted online as a matter of agency discretion.  Agencies have overwhelmingly embraced proactive disclosures as a way to provide the public with information about their activities.  The details of those efforts are included each year in agency Chief FOIA Officer Reports and are frequently highlighted in agencies’ Success Stories. 

Yet, as the volume of material posted to websites increases, the importance of establishing uniform methods to locate it increases as well.  Because proactively disclosed FOIA records are posted on websites maintained by 100 federal departments and agencies, and given that information on a given topic often is separately maintained by multiple agencies, it is essential that the public can quickly retrieve records of interest that are posted across these government websites.  Technology has allowed agencies to move beyond the days when records had to be physically stored together and organized with paper indices that needed to be reviewed in order to locate material of interest.   Today the public is accustomed to using commercial search engines to find information online simply by entering key words.  Agencies can ensure that such web searches effectively locate proactive disclosures by consistently utilizing uniform metadata when posting records online, while at the same time, retaining their ability to post their records on their individual agency websites in an organic manner that serves the needs of the frequent visitors to their sites. 

Enhanced Search Capabilities through the use of Metadata

Both the President and the Attorney General have emphasized the importance of proactive disclosures and greater use of technology in creating a more open government.  President Obama has called on agencies to use modern technology and “innovative steps” to make government more collaborative, participatory, and transparent.  In order to achieve improved transparency, agencies must make information accessible and usable, not just available. 

Agencies not only are posting more information online than ever before, they are harnessing the power of social media, creating specially designed topical websites, and providing open format data sets to put a wealth of information into the hands of the public in a variety of different formats.  In the FOIA context, agencies frequently still collect documents that are maintained in hard copy paper format.  Such paper documents are typically then scanned (e.g. into PDF format) in order to be digitized for posting online.  However, the scanning process itself does not create reliable metadata -- for example, the date that a particular document is scanned may automatically “attach” to the document instead of the date that the document was actually created.  This can inhibit the locating of particular documents when searched for by the public. 

Federal agencies should already have processes in place for the inclusion of metadata when posting documents online.  Agency FOIA professionals are encouraged to work with their IT staff on the inclusion of metadata in FOIA proactive disclosures.  In doing so, agencies should be cognizant of what keywords the public are likely to use in searching for documents online.  Relevant fields may include date, document type, subject, author/recipient name, and publishing office, as well as a variety of keywords that would help the public locate pertinent documents.  While the content of these fields will vary by document, by adding accurate and relevant metadata based on a document’s content, agencies will increase the likelihood that a document is returned in keyword searches by members of the public.  Through this initial guidance and as described below, OIP is now advising agencies to include in their standard metadata a “FOIA tag” so that documents posted in an agency’s FOIA Library are specifically designated as FOIA-related in the metadata associated with the document.   Additional information on the incorporation of metadata into posted documents is available on the General Services Administration’s website HowTo.gov. 

Currently, the Office of Management and Budget (OMB) is leading an initiative to address larger issues related to the overall management of government information.   OIP has collaborated with OMB and OIP’s Guidance applies specifically to the use of metadata in documents posted in agency FOIA Libraries.  As further OMB guidance is issued in this area and as agencies and the public gain more experience in utilizing metadata, OIP will provide additional guidance on the implementation and inclusion of metadata standards in FOIA documents posted online.  OIP will welcome feedback on the use of FOIA metadata so that we can make further improvements across the government to facilitate the ability of the public to locate FOIA-related material online.   

Uniform “FOIA Tag” for Documents Posted in FOIA Libraries

With the proliferation of information and documents posted across hundreds of government websites, and the growth of material included in FOIA Libraries in particular, there is a growing interest in creating a centralized and efficient way for the public to view all FOIA-related  documents on a given topic all at once -- i.e. having  access to a government-wide FOIA Library.   There is a way to do just that.  Collectively, all agencies can create a “virtual” government-wide FOIA Library simply by adding “FOIA” as a metadata tag to all documents posted online in their individual FOIA Libraries.  This will allow members of the public to pair individualized search terms on the topic they are interested in, such as “Mars rover” or “world population,” with the term “FOIA” and conduct a search of the Internet using a regular search engine in order to identify records that have been tagged with these terms.  Additionally, OIP plans to update features on FOIA.gov to enable the public to readily locate all agency documents that contain the FOIA tag, allowing the public to quickly and efficiently identify records on specific topics of interest to them that are located in FOIA Libraries.

The addition of this tag should fit within existing procedures for the inclusion of metadata within documents posted by agency web managers.  For example, the Appointment Affidavit of Attorney General Holder is a frequently requested Department of Justice record which is posted in PDF format in OIP’s FOIA Library.  For this document, relevant metadata fields have been “tagged,” including the new “FOIA” tag, as follows:

• Document Title:  Appointment Affidavit for Attorney General Eric Holder
• Author:  Eric Holder
• Author Title:  Attorney General of the United States
• Keywords:  Holder, OPM-61, Standard Form 61, SF-61, Holder Appointment Affidavit, Appointment, Affidavit, Oath of Office, Oath, 2009, 2/3/2009, FOIA

In the coming months OIP will issue further guidance providing specific instructions for agencies to follow on the inclusion of the “FOIA” tag in the metadata for documents posted online in their FOIA Libraries.  That guidance will establish the time frame for implementation of the new tagging system.  To prepare, agency personnel should familiarize themselves with this guidance, as well as any other metadata standards required by their individual agencies.  Additionally, FOIA personnel should begin identifying documents of significant public interest that are already available in their FOIA Libraries and which would benefit from the addition of the “FOIA” tag.

Conclusion

The way that federal agencies make records available to the public, and the way that the public accesses these records, continues to evolve in the Internet era.  As the volume of posted material increases, it is vital that the public have a fast and efficient way to locate particular documents that are of interest to them.  Our goal is to build off of existing agency procedures and standards for the posting of documents online in order to optimize the public’s ability to locate material of interest while leveraging common search tools that utilize plain language and are familiar to the public.  The consistent addition of a FOIA metadata tag to documents posted in agency FOIA Libraries will enable the public to more efficiently locate and retrieve government records that have been made available online pursuant to the FOIA’s proactive disclosure provision.  The above guidance lays the groundwork and the foundation for future guidance from OIP and OMB which -- in conjunction with updates to features on FOIA.gov -- will create the first-ever virtual government-wide FOIA Library.