Finding that Production of Privileged ESI Effected Waiver, Court Describes Risks of Privilege Review Using Keyword Searches and Offers Guidance on Proper Assertion of Privilege
Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D. Md. 2008)
In this case, plaintiff sought a ruling that 165 electronic documents produced by defendants were not privileged because their production occurred under circumstances that waived any privilege or protected status. The parties had previously agreed to a joint protocol to search and retrieve relevant ESI responsive to plaintiff’s Rule 34 requests. The protocol contained detailed search and information retrieval instructions, including nearly five pages of keyword/phrase search terms aimed at locating responsive ESI.
In March 2007, defense counsel notified the court that individualized privilege review of the responsive ESI would delay production unnecessarily and cause undue expense. To address this concern, defendants gave their computer forensics expert a list of keywords to be used to search and retrieve privileged and protected documents from the population of documents that were to be produced to plaintiff, and requested that the court approve a “clawback agreement” fashioned to address the concerns noted in Hopson v. Mayor of Baltimore, 232 F.R.D. 228 (D. Md. 2005). Later, when the discovery deadline was extended, defense counsel notified the court that defendants would be able to conduct a document-by-document privilege review, thereby making a clawback agreement unnecessary. Following their privilege review, defendants produced responsive ESI in September 2007.
In reviewing the production, plaintiff’s counsel soon discovered documents that potentially were privileged or work-product protected. They immediately segregated the information and notified defense counsel of its production, following this same procedure each time they identified potentially privileged/protected information. Defense counsel responded by asserting that the production of any privileged or protected information had been inadvertent. Defendants also belatedly provided plaintiff with a series of privilege logs.
The parties disagreed substantially in their characterization of how defendants conducted their privilege review. Defendants contended that after the joint ESI search protocol was implemented and responsive ESI identified, their computer forensics expert conducted a privilege search using approximately 70 keyword search terms. The resulting documents were segregated and provided to defense counsel “for the first phase of the pre-production privilege review.” The privilege keyword search was performed on those ESI files (4.9 gigabytes) that were in text-searchable format. For the other ESI files that were not in text-searchable format (33.7 gigabytes), defense counsel and an individual defendant stated they performed a manual privilege review. This “second phase” of the privilege review was a page-by-page review; however, “due to the compressed schedule and time constraints in reviewing these tens of thousands of documents within the time permitted, this review was undertaken by reviewing the page titles of the documents.”
The court noted it remained unclear whether the 165 documents at issue were produced from the text-searchable or non-text-searchable ESI, but that the implied conclusion that the court was invited to draw, from the limited information provided by defendants, was that the 165 documents were contained within the population of non-text-searchable ESI files, making their production inadvertent. The court, however, found the inference not so easily drawn:
First, the Defendants are regrettably vague in their description of the seventy keywords used for the text-searchable ESI privilege review, how they were developed, how the search was conducted, and what quality controls were employed to assess their reliability and accuracy. While it is known that M. Pappas (a party) and Mohr and Schmid (attorneys) selected the keywords, nothing is known from the affidavits provided to the court regarding their qualifications for designing a search and information retrieval strategy that could be expected to produce an effective and reliable privilege review. As will be discussed, while it is universally acknowledged that keyword searches are useful tools for search and retrieval of ESI, all keyword searches are not created equal; and there is a growing body of literature that highlights the risks associated with conducting an unreliable or inadequate keyword search or relying exclusively on such searches for privilege review. Additionally, the Defendants do not assert that any sampling was done of the text searchable ESI files that were determined not to contain privileged information on the basis of the keyword search to see if the search results were reliable. Common sense suggests that even a properly designed and executed keyword search may prove to be over-inclusive or under-inclusive, resulting in the identification of documents as privileged which are not, and non-privileged which, in fact, are. The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive. There is no evidence on the record that the Defendants did so in this case. Rather, it appears from the information that they provided to the court that they simply turned over to the Plaintiff all the text-searchable ESI files that were identified by the keyword search Turner performed as non-privileged, as well as the non-text searchable files that Monkman and M. Pappas’ limited title page search determined not to be privileged.
Plaintiff disputed defendants’ assertion that the text-searchable ESI was in a format that was difficult to search for privileged or protected materials. Plaintiff contended that it was able to do a keyword search of the text-searchable ESI in about one hour using a “readily-available desktop search tool.” Further, plaintiff contended that the non-text-searchable files consisted primarily of image files, such as photographs, catalogs, and drawings, which were not likely to contain privileged or protected information. Plaintiff pointed out that the 165 documents at issue were all in text and thus were all searchable using standard text search tools.
Thus, plaintiff argued that defendants had waived any privilege because they failed to take reasonable precautions by performing a faulty privilege review of the text-searchable files and by failing to detect the presence of the 165 documents, which were then given to plaintiff as part of defendants’ ESI production.
Finding of Waiver
Magistrate Judge Paul W. Grimm (author of the Hopson decision) concluded that any privilege/protection that attached to the 165 electronic documents was waived by the voluntary production of the documents to plaintiff by defendants. The court found that under either the strict approach to privilege waiver, or the intermediate test, the finding of waiver would be the same. The court observed that the intermediate test requires the court to balance the following factors to determine whether inadvertent production of attorney-client privileged materials waives the privilege: (1) the reasonableness of the precautions taken to prevent inadvertent disclosure; (2) the number of inadvertent disclosures; (3) the extent of the disclosures; (4) any delay in measures taken to rectify the disclosure; and (5) overriding interests in justice.
Reasonableness of Precautions to Prevent Inadvertent Disclosure
The court found that the first factor weighed in favor of finding waiver. The court criticized defendants’ lack of detail, and discussed at length the risks associated with conducting privilege review through keyword searches:
Defendants, who bear the burden of proving that their conduct was reasonable for purposes of assessing whether they waived attorney-client privilege by producing the 165 documents to the Plaintiff, have failed to provide the court with information regarding: the keywords used; the rationale for their selection; the qualifications of M. Pappas and his attorneys to design an effective and reliable search and information retrieval method; whether the search was a simple keyword search, or a more sophisticated one, such as one employing Boolean proximity operators; or whether they analyzed the results of the search to assess its reliability, appropriateness for the task, and the quality of its implementation. While keyword searches have long been recognized as appropriate and helpful for ESI search and retrieval, there are well-known limitations and risks associated with them, and proper selection and implementation obviously involves technical, if not scientific knowledge. See, e.g., United States v. O’Keefe, 537 F. Supp. 2d 14, 24 (D.D.C. 2008) (“Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…. Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread.”); Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 333 (D.D.C. 2008), (“[D]etermining whether a particular search methodology, such as keywords, will or will not be effective certainly requires knowledge beyond the ken of a lay person (and a lay lawyer)….”); In re Seroquel Prods. Liab. Litig., 244 F.R.D. 650, 660 n. 6, 662 (M.D. Fla. 2007) (criticizing defendant’s use of keyword search in selecting ESI for production, noting the failure of the defendant to provide information “as to how it organized its search for relevant material, [or] what steps it took to assure reasonable completeness and quality control” and observing that “while key word searching is a recognized method to winnow relevant documents from large repositories … [c]ommon sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness.”); The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 194-95, 201-02 (“[A]lthough basic keyword searching techniques have been widely accepted both by courts and parties as sufficient to define the scope of their obligation to perform a search for responsive documents, the experience of many litigators is that simple keyword searching alone is inadequate in at least some discovery contexts. This is because simple keyword searches end up being both over- and under-inclusive in light of the inherent malleability and ambiguity of spoken and written English (as well as all other languages).”). Id. at 194-95. To address this known deficiency, the Sedona Conference suggests as best practice points, inter alia:
Practice Point 3. The choice of a specific search and retrieval method will be highly dependent on the specific legal context in which it is to be employed.
Practice Point 4. Parties should perform due diligence in choosing a particular information retrieval product or service from a vendor.
Practice Point 5. The use of search and information retrieval tools does not guarantee that all responsive documents will be identified in large data collections, due to characteristics of human language. Moreover, differing search methods may produce differing results, subject to a measure of statistical variation inherent in the science of information retrieval.
Practice Point 6. Parties should make a good faith attempt to collaborate on the use of particular search and information retrieval methods, tools and protocols (including as to keywords, concepts, and other types of search parameters).
Practice Point 7. Parties should expect that their choice of search methodology will need to be explained, either formally or informally, in subsequent legal contexts (including in depositions, evidentiary proceedings, and trials).
Id.; and Information Inflation: Can the Legal System Adapt, 13 Rich. J.L. & Tech. 10, at *38, 40 (as cited at www.westlaw.com) (“[I]t is not surprising that lawyers and those to whom they delegate search tasks may not be particularly good at ferreting out responsive information through use of simple keyword search terms…. Accordingly, the assumption on the part of lawyers that any form of present-day search methodology will fully find ‘all’ or ‘nearly all’ available documents in a large, heterogeneous collection of data is wrong in the extreme.”).
Use of search and information retrieval methodology, for the purpose of identifying and withholding privileged or work-product protected information from production, requires the utmost care in selecting methodology that is appropriate for the task because the consequence of failing to do so, as in this case, may be the disclosure of privileged/protected information to an adverse party, resulting in a determination by the court that the privilege/protection has been waived. Selection of the appropriate search and information retrieval technique requires careful advance planning by persons qualified to design effective search methodology. The implementation of the methodology selected should be tested for quality assurance; and the party selecting the methodology must be prepared to explain the rationale for the method chosen to the court, demonstrate that it is appropriate for the task, and show that it was properly implemented. In this regard, compliance with the Sedona Conference Best Practices for use of search and information retrieval will go a long way towards convincing the court that the method chosen was reasonable and reliable, which, in jurisdictions that have adopted the intermediate test for assessing privilege waiver based on inadvertent production, may very well prevent a finding that the privilege or work-product protection was waived.
The court found that, in this case, defendants failed to carry their burden of explaining what they had done and why it was sufficient. The court was not persuaded by defendants’ argument that the volume of ESI and time constraints presented them with no other choice. The court noted that defendants were aware of the danger of inadvertent production of privileged/protected information and initially sought the protections of a non-waiver agreement such as that discussed in Hopson. “Had they not voluntarily abandoned their request for a court-approved non-waiver agreement, they would have been protected from waiver.”
Remaining Factors
The court addressed the remaining factors, pointing out that it was not a case where a single document “slipped through the cracks.” It further noted that many of the documents were substantive – including numerous communications between defendants and their counsel. The court stated: “As noted by other district courts within the Fourth Circuit, any order issued now by the court to attempt to redress these disclosures would be the equivalent of closing the barn door after the animals have already run away.”
Finally, the court found it noteworthy that the defendants did not discover the disclosure, but rather the plaintiff made the discovery and notified the defendants that potentially privileged/protected ESI had been produced. Accordingly, even under the intermediate test, the court found that defendants were not insulated from waiver.
Sufficiency of Defendants’ Assertion of Privilege/Protection
Having decided the waiver issue against the defendants, the court stated there was no need to reach the question of whether privilege/protection was properly asserted in the first instance. “However, given the recurring problems associated with resolving disputed privilege/protection claims during discovery, it would be helpful to state the procedures that need to be followed in this process for the benefit of future cases.”
The court discussed the limited utility of privilege logs, and the enormous burden to the court that in camera review of disputed documents entails, something “about which the parties and their attorneys often seem to be blissfully unconcerned.” The court concluded:
Thus, insuring that a privilege or protection claim is properly asserted in the first instance and maintained thereafter involves a several step process. First, pursuant to Fed. R. Civ. P. 26(b)(5), the party asserting privilege/protection must do so with particularity for each document, or category of documents, for which privilege/protection is claimed. At this first stage, it is sufficient to meet the initial burden by a properly prepared privilege log. If, after this has been done, the requesting party challenges the sufficiency of the assertion of privilege/protection, the asserting party may no longer rest on the privilege log, but bears the burden of establishing an evidentiary basis-by affidavit, deposition transcript, or other evidence-for each element of each privilege/protection claimed for each document or category of document. A failure to do so warrants a ruling that the documents must be produced because of the failure of the asserting party to meet its burden. If it makes this showing, and the requesting party still contests the assertion of privilege/protection, then the dispute is ready to submit to the court, which, after looking at the evidentiary support offered by the asserting party, can either rule on the merits of the claim or order that the disputed documents be produced for in camera inspection.
The court observed that, had it not ruled that any privilege/protection already was waived, then the effect of a failure by defendants to comply with the court’s order regarding the proper manner in which to assert privilege/protection would have warranted an order to produce the materials for failure to carry the burden of demonstrating the existence of the privilege/protection claimed.