MEMORANDUM AND ORDER

          PAUL W. GRIMM, United States Magistrate Judge.

          The plaintiff, Victor Stanley, Inc. (" VSI" or " Plaintiff" ) filed a motion seeking a ruling that five categories of electronically stored documents produced by defendants Creative Pipe, Inc. (" CPI" ) and Mark and Stephanie Pappas (" M. Pappas", " S. Pappas" or " The Pappasses" ) (collectively, " Defendants" ) in October, 2007, are not exempt from discovery because they are within the protection of the attorney-client privilege and work-product doctrine, as claimed by the Defendants. VSI argues that the electronic records at issue, which total 165 documents, are not privileged because their production by Defendants occurred under circumstances that waived any privilege or protected status. Alternatively, as for a subset of nine email communications from M. Pappas to a computer forensics expert Defendants retained to assist them with producing electronically stored information (" ESI" ), VSI contends that the attorney-client privilege is inapplicable, and with regard to another two email communications (one draft, the other actually sent) from M. Pappas to one of his attorneys, VSI contends that they are neither privileged nor protected. Finally, as for two email communications from M. Pappas to two of his attorneys, VSI argues that they are beyond the scope of the attorney-client privilege because they fall within the crime/fraud/tort exception. Defendants acknowledge that they produced all 165 electronic documents at issue to VSI during Rule 34 discovery, but argue that the production was inadvertent, and therefore that privilege/protection has not been waived. As to the various email communications, Defendants argue that they are within the scope of the attorney-client privilege and work-product protection, and that the crime/fraud/tort exception is not applicable. The motion has been fully briefed, Paper Nos. 212, 221, 225, and 230, and I find that a hearing is not necessary. Local Rules of the United States District Court for the District of Maryland, Rule 105.6. For the reasons that follow, I find that all 165 electronic documents are beyond the scope of the attorney-client privilege and work-product protection because assuming, arguendo, that they qualified as privileged/protected in the first instance, and assuming further that Defendants properly complied with their obligation to particularize any claims of privilege/protection imposed by Fed.R.Civ.P. 26(b)(5), Local Rules of the United States District Court for the District of Maryland, Appendix B, Discovery Guideline 9.c (" Discovery Guideline" ), and the orders of this court detailing how such assertions must be demonstrated once they were challenged by VSI, the privilege/protection was waived by the voluntary production of the documents to VSI by Defendants.

          Background Facts

          The following facts are not subject to dispute. The Defendants' first Rule 34 response was a " paper production," not ESI, made in May 2007. Pl.'s Supp'l Mem. 3, Paper No. 221. Plaintiff objected to its sufficiency, and following a hearing, the court ordered the parties' computer forensic experts to meet and confer in an effort to identify a joint protocol to search and retrieve relevant ESI responsive to Plaintiff's Rule 34 requests. Id. This was done and the joint protocol prepared. Pl.'s Supp'l Mem. Ex. 9, Paper No. 221. The protocol contained detailed search and information retrieval instructions, including nearly five pages of keyword/phrase search terms. It is noteworthy that these search terms were aimed at locating responsive ESI, rather than identifying privileged or work-product protected documents within the population of responsive ESI. After the protocol was used to retrieve responsive ESI, Defendants reviewed it to locate documents that were beyond the scope of discovery because of privilege or work-product protection. Counsel for Defendants had previously notified the court on March 29, 2007, that individualized privilege review of the responsive documents " would delay production unnecessarily and cause undue expense." Pl.'s Letter of Mar. 29, 2007, Paper No. 79. To address this concern, Defendants gave their computer forensics expert a list of keywords to be used to search and retrieve privileged and protected documents from the population of documents that were to be produced to Plaintiff. Id. However, Defendants' counsel also acknowledged the possibility of inadvertent disclosure of privileged/protected documents, given the volume of documents that were to be produced, and requested that the court approve a " clawback agreement" fashioned to address the concerns noted by this court in Hopson v. Mayor of Baltimore, 232 F.R.D. 228 (D.Md.2005). In response, the court held a telephone conference to discuss the proposed clawback agreement, and thereafter issued a letter order requesting additional briefing by the parties " regarding the burdens associated with conducting a privileged [sic] review of the information to be produced in the time frame required by [the] discovery [schedule] in this case." Letter Order, Apr. 24, 2007, Paper No. 92. However, on April 27, 2007, Defendants' counsel notified the court that because Judge Garbis recently had extended the discovery deadline by four months, Defendants would be able to conduct a document-by-document privilege review, thereby making a clawback agreement unnecessary. Defs.' Letter of Apr. 27, 2007, Paper No. 93. Accordingly, Defendants abandoned their efforts to obtain a clawback agreement and committed to undertaking an individualized document review.

         Following their privilege review, Defendants made their ESI production to Plaintiff in September 2007. Pl.'s Supp'l Mem. 5, Paper No. 221. It is noteworthy that by the time of this production, Defendants had discharged their local attorneys, Messrs. Mohr and Ludwig from Meyer, Klipper & Mohr, and brought in new counsel.

         After receiving Defendants' ESI production in September, 2007, Plaintiff's counsel began their review of the materials. They soon discovered documents that potentially were privileged or work-product protected and immediately segregated this information and notified counsel for Defendants of its production, following this same procedure each time they identified potentially privileged/protected information. Pl.'s Supp'l Mem. Exs. 11-15, Paper No. 221. Defendants' Counsel, Mr. Schmid, responded by asserting that the production of any privileged or protected information had been inadvertent. Pl.'s Supp'l Mem. Ex. 17, Paper No. 221. Defendants also belatedly provided Plaintiff with a series of privilege logs, purportedly identifying the documents that had been withheld from production pursuant to Fed.R.Civ.P. 26(b)(5). Defs.' Opp'n Mem. Exs. 4, 6, and 9, Paper No. 225.

          The parties disagree substantially in their characterization of how Defendants conducted their review for privileged and protected documents before the ESI productions were made to Plaintiff. Defendants contend that after the joint ESI search protocol was implemented and the responsive ESI identified, their computer forensics expert, Ms. Genevive Turner, " conducted a privilege search using approximately seventy different keyword search terms ... [that] had been decided upon previously by Mr. Pappas, his former attorney, Christopher Mohr, and another attorney, F. Stephen Schmid.... All documents which were returned during the keyword search were segregated and provided to one of Mr. Pappas' attorneys, John G. Monkman, Jr. for the first phase of the pre-production privilege review." Defs.' Opp'n Mem. 4, Ex. 1 (Pappas Aff.) and Ex. 3 (Monkman Aff.), Paper No. 225. This characterization, however, is somewhat misleading. In actuality, after the joint retrieval protocol had been executed, Ms. Turner determined that there were some ESI files (4.9 gigabytes) that were in text-searchable format and others (33.7 gigabytes) that were not. Defs.' Opp'n Mem. Ex. 2 (Turner Aff. ¶ . 7), Paper No. 225. Turner conducted a search for privileged material on the text-searchable files using the seventy keywords developed by M. Pappas, Mohr and Schmid. As to the nontext-searchable files, she produced them to Monkman for manual privilege review. Turner Aff. ¶ ¶ 6-7. Monkman reviewed each of the files identified as privileged/protected by Turner based on her keyword searches. Monkman Aff. ¶ 7. Additionally, Monkman and M. Pappas teamed up to begin doing a " page-by-page" manual privilege review of the nontext-searchable ESI files. Id. at ¶ 8. According to Monkman: " [t]he second phase of review consisted of page-by-page review of ... [the non text-searchable ESI files], which was undertaken by Mr. Pappas and me. However, due to the compressed schedule and time constraints in reviewing these tens of thousands of documents within the time permitted, this review was undertaken by reviewing the page titles of the documents. Documents whose page titles indicated that the privilege might be applicable were reviewed in their entirety by Mr. Pappas or me. This was the only way for us to complete the unwieldy review of these documents within the time permitted" . Id.

         The foregoing affidavits create the impression that the keyword search Turner conducted on the text-searchable ESI files, using the seventy keywords developed by M. Pappas and his attorneys, successfully culled out the privileged/protected documents; and this status was confirmed by Monkman's review, and they were withheld from production. Further, the Defendants' characterization of the privilege review suggests that as to the non-text searchable files, Pappas and Monkman did all that could be reasonably expected of them in the time allowed to make the ESI production, which was to review only the title page of the documents and not their entire content. From the affidavits Defendants provided, the court is left to infer that the text-searchable documents that were not flagged by the keyword search Turner conducted were produced to the Plaintiff, as well as the nontext-searchable files that Monkman and M. Pappas determined were not privileged or protected based on their limited title-page review. This is because the Defendants fail to delineate exactly which documents were and were not provided to the Plaintiff, or where the 165 documents at issue were located within the ESI productions made to the Plaintiff.

         The implied conclusion that the court is invited to draw, from the limited information provided by the Defendants, is that the 165 documents that are the subject of the present motion were contained within the population of nontext-searchable ESI files that were produced by the Defendants to the Plaintiff, making their production inadvertent. However, this inference is not so easily drawn.

          First, the Defendants are regrettably vague in their description of the seventy keywords used for the text-searchable ESI privilege review, how they were developed, how the search was conducted, and what quality controls were employed to assess their reliability and accuracy. While it is known that M. Pappas (a party) and Mohr and Schmid (attorneys) selected the keywords, nothing is known from the affidavits provided to the court regarding their qualifications for designing a search and information retrieval strategy that could be expected to produce an effective and reliable privilege review. As will be discussed, while it is universally acknowledged that keyword searches are useful tools for search and retrieval of ESI, all keyword searches are not created equal; and there is a growing body of literature that highlights the risks associated with conducting an unreliable or inadequate keyword search or relying exclusively on such searches for privilege review. Additionally, the Defendants do not assert that any sampling was done of the text searchable ESI files that were determined not to contain privileged information on the basis of the keyword search to see if the search results were reliable. Common sense suggests that even a properly designed and executed keyword search may prove to be over-inclusive or under-inclusive, resulting in the identification of documents as privileged which are not, and non-privileged which, in fact, are. The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive. There is no evidence on the record that the Defendants did so in this case. Rather, it appears from the information that they provided to the court that they simply turned over to the Plaintiff all the text-searchable ESI files that were identified by the keyword search Turner performed as non-privileged, as well as the non-text searchable files that Monkman and M. Pappas' limited title page search determined not to be privileged.

         The Plaintiff paints an entirely different picture of the Defendants' privilege review. VSI vigorously disputes Defendants' assertion that the text-searchable ESI received by Defendants' computer forensic expert, Turner, following the execution of the joint search and retrieval protocol was in a format that was difficult to search for privileged or protected materials. Plaintiff contends that it was able to do a keyword search of the text-searchable ESI produced by Defendants in about one hour using a " readily-available desktop search tool." Pl.'s Reply Mem. 3 and Ex. 1 (Slaughenhoupt Aff. ¶ 6), Paper No. 230. VSI further contends that the nontext-searchable files that Monkman and M. Pappas reviewed by looking at the title pages consisted primarily of image files, such as photographs, catalogs, and drawings, which are not likely to contain privileged or protected information. Id. at ¶ 9. Most importantly, however, the Plaintiff argues that the Defendants' complaint-that they could not effectively conduct a privilege review of the nontext-searchable files because there were so many of them-is a red herring because " the privileged materials [that are the subject of this motion] were all in text and thus were all searchable using standard text search tools. Contrary to Mr. Pappas' assertion, a majority of the .PDF files in the ESI were searchable using readily available search tools. The ESI contained 9008 .PDF files, the majority of which were searchable and the remaining could have been made searchable using readily available OCR software and/or the native OCR Text Recognition tool within Adobe Acrobat." Id. at ¶ 8.

         Thus, according to the Plaintiff, the Defendants have waived any claim to attorney client privilege or work-product protection for the 165 documents at issue because they failed to take reasonable precautions by performing a faulty privilege review of the text-searchable files and by failing to detect the presence of the 165 documents, which were then given to the Plaintiff as part of Defendants' ESI production. As will be seen, under either the Plaintiff's or Defendants' version of the events, the Defendants have waived any privilege or protected status for the 165 documents in question.

          Applicable Law

          As this court discussed in some detail in Hopson, 232 F.R.D. at 235-38, courts have taken three different approaches when deciding whether the inadvertent production to an adversary of attorney client privileged or work-product protected materials constitutes a waiver. Under the most lenient approach there is no waiver because there has not been a knowing and intentional relinquishment of the privilege/protection; under the most strict approach, there is a waiver because once disclosed, there can no longer be any expectation of confidentiality; and under the intermediate one, the court balances a number of factors to determine whether the producing party exercised reasonable care under the circumstances to prevent against disclosure of privileged and protected information, and if so, there is no waiver. Id. As also noted in Hopson, the Fourth Circuit Court of Appeals has yet to decide which approach it will follow, although individual district courts within the circuit have adopted the intermediate balancing approach. Id. at 236 n. 18; see also Cont'l Cas. Co. v. Under Armour, Inc., 537 F.Supp.2d 761, 768 n. 3 (D.Md.2008). As Hopson pointed out, however, a careful reading of the Fourth Circuit's decisions regarding waiver of the attorney-client privilege, albeit in contexts not closely related to the facts of this case, suggest that it is more inclined to adopt the strict approach than the intermediate or lenient one. Hopson, 232 F.R.D. at 236-37. Under the strict approach, there is no legitimate doubt that Defendants' production of the 165 asserted privileged/protected documents waived the attorney-client privilege and work-product protection. Even under the intermediate test, however, the result would be the same.

          The intermediate test requires the court to balance the following factors to determine whether inadvertent production of attorney-client privileged materials waives the privilege: (1) the reasonableness of the precautions taken to prevent inadvertent disclosure; (2) the number of inadvertent disclosures; (3) the extent of the disclosures; (4) any delay in measures taken to rectify the disclosure; and (5) overriding interests in justice. McCafferty's, Inc., v. Bank of Glen Burnie, 179 F.R.D. 163, 167 (D.Md.1998) (citing cases). The first of these factors militates most strongly in favor of a finding that Defendants waived the privilege in this case.

          Assuming that the Plaintiff's version of how Defendants conducted their privilege review is accurate, the Defendants obtained the results of the agreed-upon ESI search protocol and ran a keyword search on the text-searchable files using approximately seventy keywords selected by M. Pappas and two of his attorneys. Defendants, who bear the burden of proving that their conduct was reasonable for purposes of assessing whether they waived attorney-client privilege by producing the 165 documents to the Plaintiff, have failed to provide the court with information regarding: the keywords used; the rationale for their selection; the qualifications of M. Pappas and his attorneys to design an effective and reliable search and information retrieval method; whether the search was a simple keyword search, or a more sophisticated one, such as one employing Boolean proximity operators; or whether they analyzed the results of the search to assess its reliability, appropriateness for the task, and the quality of its implementation. While keyword searches have long been recognized as appropriate and helpful for ESI search and retrieval, there are well-known limitations and risks associated with them, and proper selection and implementation obviously involves technical, if not scientific knowledge. See, e.g., United States v. O'Keefe, 537 F.Supp.2d 14, 24 (D.D.C.2008) (" Whether search terms or ‘ keywords' will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics.... Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread." ); Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 333 (D.D.C.2008), (" [D]etermining whether a particular search methodology, such as keywords, will or will not be effective certainly requires knowledge beyond the ken of a lay person (and a lay lawyer)...." ); In re Seroquel Prods. Liab. Litig., 244 F.R.D. 650, 660 n. 6, 662 (M.D.Fla.2007) (criticizing defendant's use of keyword search in selecting ESI for production, noting the failure of the defendant to provide information " as to how it organized its search for relevant material, [or] what steps it took to assure reasonable completeness and quality control" and observing that " while key word searching is a recognized method to winnow relevant documents from large repositories ... [c]ommon sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness." ); The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 194-95, 201-02 (" [A]lthough basic keyword searching techniques have been widely accepted both by courts and parties as sufficient to define the scope of their obligation to perform a search for responsive documents, the experience of many litigators is that simple keyword searching alone is inadequate in at least some discovery contexts. This is because simple keyword searches end up being both over- and under-inclusive in light of the inherent malleability and ambiguity of spoken and written English (as well as all other languages)." ). Id. at 194-95. To address this known deficiency, the Sedona Conference suggests as best practice points, inter alia: Practice Point 3. The choice of a specific search and retrieval method will be highly dependent on the specific legal context in which it is to be employed.

Practice Point 4. Parties should perform due diligence in choosing a particular information retrieval product or service from a vendor.

Practice Point 5. The use of search and information retrieval tools does not guarantee that all responsive documents will be identified in large data collections, due to characteristics of human language. Moreover, differing search methods may produce differing results, subject to a measure of statistical variation inherent in the science of information retrieval.

Practice Point 6. Parties should make a good faith attempt to collaborate on the use of particular search and information retrieval methods, tools and protocols (including as to keywords, concepts, and other types of search parameters).

Practice Point 7. Parties should expect that their choice of search methodology will need to be explained, either formally or informally, in subsequent legal contexts (including in depositions, evidentiary proceedings, and trials).

          Use of search and information retrieval methodology, for the purpose of identifying and withholding privileged or work-product protected information from production, requires the utmost care in selecting methodology that is appropriate for the task because the consequence of failing to do so, as in this case, may be the disclosure of privileged/protected information to an adverse party, resulting in a determination by the court that the privilege/protection has been waived. Selection of the appropriate search and information retrieval technique requires careful advance planning by persons qualified to design effective search methodology. The implementation of the methodology selected should be tested for quality assurance; and the party selecting the methodology must be prepared to explain the rationale for the method chosen to the court, demonstrate that it is appropriate for the task, and show that it was properly implemented. In this regard, compliance with the Sedona Conference Best Practices for use of search and information retrieval will go a long way towards convincing the court that the method chosen was reasonable and reliable, which, in jurisdictions that have adopted the intermediate test for assessing privilege waiver based on inadvertent production, may very well prevent a finding that the privilege or work-product protection was waived.

         In this case, the Defendants have failed to demonstrate that the keyword search they performed on the text-searchable ESI was reasonable. Defendants neither identified the keywords selected nor the qualifications of the persons who selected them to design a proper search; they failed to demonstrate that there was quality-assurance testing; and when their production was challenged by the Plaintiff, they failed to carry their burden of explaining what they had done and why it was sufficient.

          Further, the Defendants' attempt to justify what was done, by complaining that the volume of ESI needing review and time constraints presented them with no other choice is simply unpersuasive. Defendants were aware of the danger of inadvertent production of privileged/protected information and initially sought the protections of a non-waiver agreement such as that discussed in Hopson, supra. Had they not voluntarily abandoned their request for a court-approved non-waiver agreement, they would have been protected from waiver. Instead, they advised the court that they did not need this protection and elected to do a document-by-document privilege review. According to Defendants version of the facts, when they undertook an individualized review of the nontext-searchable ESI and determined that they could only review the title pages, they neither sought an extension of time from the court to complete an individualized review nor reinstated their request for a court-approved non-waiver agreement, despite their awareness of how it would have provided protection against waiver. In these circumstances, Defendants' protests that they did their best and that their conduct was reasonable rings particularly hollow.

         The remaining factors to be assessed under the intermediate test may be quickly disposed of. The Defendants produced 165 asserted privileged/protected documents to the Plaintiff, so this case does not present an instance of a single document slipping through the cracks. Further, the court's in camera review of the documents reflects that many of them are email and other communications between the Defendants and their various attorneys, as well as draft discovery responses, documents relating to settlements in unrelated litigation, comments from M. Pappas to counsel regarding discovery responses, and email correspondence between M. Pappas and Ms. Turner, the ESI forensic expert retained by Defendants. Thus, the disclosures were substantive-including numerous communications between defendants and their counsel. As noted by other district courts within the Fourth Circuit, any order issued now by the court to attempt to redress these disclosures would be the equivalent of closing the barn door after the animals have already run away. FDIC v. Marine Midland Realty Credit Corporation, 138 F.R.D. 479, 483 (E.D.Va.1991) ( " Any order issued now by the court would have only limited effect; it could not force NBNE to forget what has already been learned" ); Parkway Gallery Furniture, Inc. v. Kittinger/Pennsylvania House Group, 116 F.R.D. 46, 52 (M.D.N.C.1987) (" [W]hen disclosure is complete, a court order cannot restore confidentiality and, at best, can only attempt to restrain further erosion." ). And, while the precise dates of the disclosures of the documents at issue are not clear from the record-since the Defendants made a series of ESI productions over a several week period-it is noteworthy that the Defendants did not discover the disclosure, but rather the Plaintiff made the discovery and notified the Defendants that potentially privileged/protected ESI had been produced. Therefore, this is not an instance in which a party inadvertently produced privileged information to an adversary, discovered the disclosure promptly, and then took immediate steps to inform the adversary that they had received the information inadvertently, thus demanding that it be returned.

         While Defendants' counsel did assert privilege and inadvertent production promptly after being notified by the Plaintiff of the production of possible privileged/protected information, the more important period of delay in this case is the one-week period between production by the Defendants and the time of the discovery by the Plaintiff of the disclosures-a period during which the Defendants failed to discover the disclosure. Finally, the Defendants have pointed to no overriding interests in justice that would excuse them from the consequences of producing privileged/protected materials. The Plaintiff is blameless, but the Defendants are not, having failed to take reasonable precautions to prevent the disclosure of privileged information, including the voluntary abandonment of the non-waiver agreement that the Plaintiff was willing to sign. Every waiver of the attorney-client privilege produces unfortunate consequences for the party that disclosed the information. If that alone were sufficient to constitute an injustice, there would never be a waiver. The only " injustice" in this matter is that done by Defendants to themselves. Marine Midland Realty Credit Corp., 138 F.R.D. at 483 (" It is seldom ‘ fundamentally unfair’ to allow the truth to be made public, and under the circumstances ... the Court finds that it would not be fair to reward Rowe's carelessness [in disclosing privileged materials] with a protective order." ). Accordingly, even under the intermediate test, the Defendants are not insulated from waiver.

          Sufficiency of Defendants' Assertion of Privilege/Protection

          In addition to arguing that the Defendants waived any privileged or protected status that the disclosed documents had, the Plaintiff also appears to contend that the Defendants failed to properly establish the existence of the privilege and protection asserted in the first instance. Specifically, the Plaintiff argues that despite the requirements of Fed.R.Civ.P. 26(b)(5) and Discovery Guideline 5.c of this court that claims of privilege must be particularized, the Defendants failed to meet this obligation, and further, failed to comply with an order of this court regarding the proper way to assert privilege in responding to discovery requests. See Pl.'s Supp'l Mem. 1, 6, 10, Paper No. 221; Pl.'s Reply Mem. 1, Paper No. 230. While Defendants do not deny that they failed to comply with the court's order, they contend that when the parties met to confer regarding the sufficiency of the Defendants' privilege/protection claims, the parties " essentially" came to an agreement that for all but eleven of the 165 documents at issue, the asserted privilege/protection was legitimate. Accordingly, Defendants argue that if the waiver issue is decided in their favor, there are only eleven documents for which the assertion of privilege/protection is challenged by Plaintiff. Defs.' Opp'n Mem. 6-7, Paper No. 225. Having decided the waiver issue against the Defendants, there is no need to reach the question of whether privilege/protection was properly asserted in the first instance. However, given the recurring problems associated with resolving disputed privilege/protection claims during discovery, it would be helpful to state the procedures that need to be followed in this process for the benefit of future cases.

         While the scope of discovery in civil cases broadly encompasses facts relevant to the claims and defenses raised in the pleadings, and, on a showing of good cause, may even be extended to facts relevant to the subject matter of the litigation, Fed.R.Civ.P. 26(b)(1), it does not include privileged information. Id. Similarly, work-product protected information is beyond the reach of discovery unless the requesting party makes a showing of substantial need for the information and inability to obtain its substantial equivalent without undue hardship. Fed.R.Civ.P. 26(b)(3). Because the responding party is entitled to refuse to produce requested discovery if it is privileged or work product protected, the rules require that when doing so, the responding party must " describe the nature of the documents, communications, or tangible things not produced or disclosed-and do so in a manner that, without revealing information itself privileged or protected, will enable other parties to assess the claim." Fed.R.Civ.P. 26(b)(5)(A)(ii). This requirement was added in the 1993 amendments to the rules of civil procedure, and in the words of the advisory committee:

The party [asserting privilege/protection] must also provide sufficient information to enable other parties to evaluate the applicability of the claimed privilege or protection. Although the person from whom the discovery is sought decides whether to claim a privilege or protection, the court ultimately decides whether, if this claim is challenged, the privilege or protection applies. Providing information pertinent to the applicability of the privilege or protection should reduce the need for in camera examination of the documents.

Fed.R.Civ.P. 26 advisory committee's note. Neither the rule nor the advisory committee comment specifies exactly how the party asserting privilege/protection must particularize its claim. The most common way is by using a privilege log, which identifies each document withheld, information regarding the nature of the privilege/protection claimed, the name of the person making/receiving the communication, the date and place of the communication, and the document's general subject matter. See, e.g., Discovery Guideline 9.c.; Paul W. Grimm, Charles S. Fax, & Paul Mark Sandler, Discovery Problems and Their Solutions, 62-64 (2005)(" To properly demonstrate that a privilege exists, the privilege log should contain a brief description or summary of the contents of the document, the date the document was prepared, the person or persons who prepared the document, the person to whom the document was directed, or for whom the document was prepared, the purpose in preparing the document, the privilege or privileges asserted with respect to the document, and how each element of the privilege is met for that document." ). Id. at 62-63.

         In actuality, lawyers infrequently provide all the basic information called for in a privilege log, and if they do, it is usually so cryptic that the log falls far short of its intended goal of providing sufficient information to the reviewing court to enable a determination to be made regarding the appropriateness of the privilege/protection asserted without resorting to extrinsic evidence or in camera review of the documents themselves. Few judges find that the privilege log is ever sufficient to make the discrete fact-findings needed to determine whether a privilege/protection was properly asserted and not waived. Further, because privilege review and preparation of privilege logs is increasingly handled by junior lawyers, or even paralegals, who may be inexperienced and overcautious, there is an almost irresistible tendency to be over-inclusive in asserting privilege/protection. While some of this tendency is understandable given the consequences of mistakenly producing privileged/protected information, the experience of many judges is that when the documents themselves are reviewed, it often turns out that a much smaller percentage of documents actually meet the requirements of the asserted privilege/protection than was claimed by the asserting party. Counsel should be wary of filing a response to a Rule 34 document production request that asserts privilege/protection as a basis for refusing to make requested production without having a factual basis to support each element of each privilege/protection claimed for each document withheld, because doing so is a sanctionable violation of Fed.R.Civ.P. 26(g).

         Requesting parties also know of the limited utility of privilege logs (for they likely have served similar privilege logs in response to their adversary's discovery requests), and thus, when they receive the typical privilege log, they are wont to challenge its sufficiency, demanding more factual information to justify the privilege/protection claimed. This, in turn, is often met with a refusal from the producing party, and it does not take long before a motion is pending, and the court is called upon to rule on the appropriateness of the assertion of privilege/protection, often with the producing party's " magnanimous" offer to produce the documents withheld for in camera review. In camera review, however, can be an enormous burden to the court, about which the parties and their attorneys often seem to be blissfully unconcerned.

          For example, in order for the court to determine whether the attorney-client privilege was properly asserted regarding a particular document, the court must make the following fact determinations:

(1) the asserted holder of the privilege is or sought to become a client; (2) the person to whom the communication was made (a) is a member of the bar of a court, or his subordinate and (b) in connection with this communication is acting as a lawyer; (3) the communication relates to a fact of which the attorney was informed (a) by his client (b) without the presence of strangers (c) for the purpose of securing primarily either (i) an opinion on law or (ii) legal services or (iii) assistance in some legal proceeding, and not (d) for the purpose of committing a crime or tort; and (4) the privilege has been (a) claimed and (b) not waived by the client.

In re Allen, 106 F.3d 582, 600 (4th Cir.1997) (holding also that " a district court's holding that the attorney-client privilege does not protect communications rest[s] essentially on determinations of fact" ). Id. at 601. Sometimes the document itself makes this clear, such as when the lawyer writes to the client to provide an opinion and the correspondence reflects that it is a confidential communication. Often times, however, it is impossible to determine if the privilege applies without extrinsic evidence, which must be provided by affidavit, deposition transcript, or other source. The time it takes the court to review this extrinsic evidence on a document-by-document basis can be extensive, particularly given the tendency of lawyers to be over-inclusive in the assertion of privilege/protection in the first place. It should go without saying that the court should never be required to undertake in camera review unless the parties have first properly asserted privilege/protection, then provided sufficient factual information to justify the privilege/protection claimed for each document, and, finally, met and conferred in a good faith effort to resolve any disputes without court intervention. United States v. Zolin, 491 U.S. 554, 571-72, 109 S.Ct. 2619, 105 L.Ed.2d 469 (1989) (" [W]e cannot ignore the burdens in camera review places upon the district courts, which may well be required to evaluate large evidentiary records without open adversarial guidance by the parties.... Before engaging in in camera review ... ‘ the judge should require a showing of a factual basis adequate to support a good faith belief by a reasonable person’ ... that in camera review of the materials may reveal evidence to establish the claim [of privilege/protection]" .) (internal citations omitted); United States v. Family Practice Assocs. of San Diego, 162 F.R.D. 624, 627 (S.D.Ca.1995) (" Prior to an in camera review there must first be a sufficient evidentiary showing of a legitimate issue as to application of a privilege or other protection. In camera review should not replace effective adversarial testing of the claimed privileges and protection." ); Diamond State Ins. Co. v. Rebel Oil Co., 157 F.R.D. 691, 700 (D.Nev.1994)(" In camera review is generally disfavored. It is not to be used as a substitute for a party's obligation to justify its withholding of documents. In camera review should not replace the effective adversarial testing of the claimed privileges and protections. Resort to in camera review is appropriate only after the burdened party has submitted detailed affidavits and other evidence to the extent possible." ) (internal citations omitted); and Caruso v. Coleman Co., 1995 WL 384602, at *1 (" [R]esort to in camera review is appropriate only after the burdened party has submitted detailed affidavits and other evidence to the extent possible. Unfortunately, this court is put in the untenable position of having to speculate in order to determine which privileges apply to the individual documents. A court has the right to refuse to engage in such speculation, since the burden of proving the attorney-client or work-product privileges rests on the party claiming the privilege." ) (internal citations omitted) (emphasis in original); Weber v. Paduano, No. 02 Civ. 3392, 2003 WL 161340, at *14 (S.D.N.Y. Jan. 22, 2003) (holding in camera review should not be undertaken routinely, but only after the party asserting privilege has submitted an adequate record to support the claim); Bowne of New York City v. AmBase Corp., 150 F.R.D. 465, 475 (S.D.N.Y.1993) (" AmBase's suggestion of in camera review in lieu of an evidentiary presentation is misplaced. Such review ... is not, however, to be routinely undertaken, particularly in a case involving a substantial volume of documents, as a substitute for a party's submission of an adequate record in support of its privilege claims." ); and King v. Conde, 121 F.R.D. 180, 190 (E.D.N.Y.1988) (" After giving defendants an opportunity to respond (with a brief and possible supplemental affidavits or declarations), the court can determine whether the defendants have made the requisite threshold showing to invoke the privilege. If the court finds that the defendant has not satisfied its threshold burdens, direct disclosure is in order. If the threshold burdens are met, the court may then review the materials at issue in camera and decide which, if any to withhold from disclosure." ).

          All of this has led to some fairly strongly worded statements from courts about what a party must do to substantiate its claim of privilege or protection. See Parkway Gallery Furniture, Inc. v. Kittinger/Pennsylvania House Group, 116 F.R.D. 46, 48 (M.D.N.C.1987) (" Disputes over whether the attorney-client privilege has been waived through inadvertent production of the documents or on the basis of the fraud or crime exception to the privilege often involve contested facts necessitating an evidentiary showing. Generally, the proponent or party claiming rights or benefit of an assertion bears the burden of establishing his contention." ); Caruso, 1995 WL 384602, at *1 (" A general allegation of privilege is insufficient. Instead, a clear showing must be made which sets forth the items or categories objected to and the reason for that objection. Accordingly, the proponent must provide the court with enough information to enable the court to determine the privilege, and the proponent must show by affidavit that precise facts exist to support the claim of privilege." ); United States ex rel. Burns v. Family Practice Associates of San Diego, 162 F.R.D. 624, 627-28 (S.D.Cal.1995) (finding Defendant's failure to make out a factual showing by " detailed affidavits or other evidence" waived privilege); Nutmeg Ins. Co. v. Atwell, Vogel & Sterling, 120 F.R.D. 504, 510 (W.D.La.1988) (" In considering whether a proponent of the privilege is entitled to protection, the courts must place the burden of proof squarely upon the party asserting privilege. Accordingly, the proponent must provide the court with enough information to enable the court to determine privilege, and the proponent must show by affidavit that precise fact exist to support the claim of privilege." ); Church of Scientology Intern. v. U.S. Dept. of Justice, 30 F.3d 224, 231 (1st Cir.1994) (" These declarations are written too generally to supplement the index in any meaningful way.... Thus, none of the functions of the index ... are served: the declarations do not demonstrate careful analysis of each document by the government; the court has not been assisted in its duty of ruling on the applicability of an exemption; and the adversary system has not been visibly strengthened." ); United States v. First State Bank, 691 F.2d 332, 335 (7th Cir.1982) (" A taxpayer need not reveal so many facts that the privilege becomes worthless but he must at least identify the general nature of that document, the specific privilege he is claiming for that document, and facts which establish all the elements of the privilege he is claiming. These allegations must be supported by affidavits." ); In re French, 162 B.R. 541, 548 (Bankr.D.S.D.1994) ( " Debtor did, indeed, file an affidavit claiming the privilege, but the timing was off, and, even then, it was in the form of a ‘ blanket’ assertion rather than articulated specific facts giving rise to a privilege." ).

          Thus, insuring that a privilege or protection claim is properly asserted in the first instance and maintained thereafter involves a several step process. First, pursuant to Fed.R.Civ.P. 26(b)(5), the party asserting privilege/protection must do so with particularity for each document, or category of documents, for which privilege/protection is claimed. At this first stage, it is sufficient to meet the initial burden by a properly prepared privilege log. If, after this has been done, the requesting party challenges the sufficiency of the assertion of privilege/protection, the asserting party may no longer rest on the privilege log, but bears the burden of establishing an evidentiary basis-by affidavit, deposition transcript, or other evidence-for each element of each privilege/protection claimed for each document or category of document. A failure to do so warrants a ruling that the documents must be produced because of the failure of the asserting party to meet its burden. If it makes this showing, and the requesting party still contests the assertion of privilege/protection, then the dispute is ready to submit to the court, which, after looking at the evidentiary support offered by the asserting party, can either rule on the merits of the claim or order that the disputed documents be produced for in camera inspection.

         In this case, the court made it clear to the Defendants that they were obligated to follow this procedure. See Letter Order, Dec. 28, 2007, Paper No. 194, and Pl.'s Letter of Feb 20, 2007 at 3, Paper No. 212, (citing to the Transcript of Dec. 21, 2007 Telephone Hearing. pp 16-17, when the court outlined the procedures to be used when submitting a privilege log). The Plaintiff argues that Defendants failed to comply with the court's order, and the Defendants have not demonstrated that they did. Had I not ruled that any privilege/protection already was waived, then the effect of a failure by the Defendants to comply with the court's order regarding the proper manner in which to assert privilege/protection would have warranted an order to produce the materials for failure to carry the burden of demonstrating the existence of the privilege/protection claimed.

          Conclusion

          For the reasons stated, the court finds that the Defendants waived any privilege or work-product protection for the 165 documents at issue by disclosing them to the Plaintiff. Accordingly, the Plaintiff may use these documents as evidence in this case, provided they are otherwise admissible. In this regard, the Plaintiff has only sought use of the documents themselves, and the court has not been asked to rule, and accordingly does not, that there has been any waiver beyond the documents themselves.

[I]t is essential to the success of this approach in avoiding waiver that the production of inadvertently produced privileged electronic data must be at the compulsion of the court, rather than solely by the voluntary act of the producing party, and that the procedures agreed to by the parties and ordered by the court demonstrate that reasonable measures were taken to protect against waiver of privilege and work product protection.

The Committee [on the Rules of Practice and Procedure] has repeatedly been advised that the risk of privilege waiver and the work necessary to avoid it, add to the costs and delay of discovery. When the review is of electronically stored information, the risk of waiver, and the time and effort required to avoid it, can increase substantially because of the volume of electronically stored information and the difficulty in ensuring that all information to be produced has in fact been reviewed.

Moreover, if the court is to be given scientific or technical information to resolve a contested discovery matter, what standards should govern its evaluation? Should the court ignore a purported ESI expert's lack of qualifications if that shortcoming is demonstrated by the party opposing his opinion? Should the court accept opinions shown to be unsupported by sufficient facts or based on demonstrably unreliable methodology? The answer is obviously " No." Viewed in its proper context, all that O'Keefe and Equity Analytics required was that the parties be prepared to back up their positions with respect to a dispute involving the appropriateness of ESI search and information retrieval methodology-obviously an area of science or technology-with reliable information from someone with the qualifications to provide helpful opinions, not conclusory argument by counsel. The goal of Federal Rule of Evidence 702 is to set standards to determine whether information is " helpful" to those who must make factual determinations involving disputed areas of science, technology or other specialized information. The rule is one of common sense, and reason-opinions regarding specialized, scientific or technical matters are not " helpful" unless someone with proper qualifications and adequate supporting facts provided such an opinion after following reliable methodology. That these common sense criteria are found in the rules of evidence does not render them off-limits for consideration during discovery. It is not unusual for pretrial factual determinations in civil cases to look to the Federal Rules of Evidence for assistance in resolving fact disputes. Indeed, in summary judgment practice, Fed.R.Civ.P. 56(e) requires that the parties support their motions with " such facts as would be admissible in evidence." The message to be taken from O'Keefe, Equity Analytics, and this opinion is that when parties decide to use a particular ESI search and retrieval methodology, they need to be aware of literature describing the strengths and weaknesses of various methodologies, such as The Sedona Conference Best Practices, supra, n. 9, and select the one that they believe is most appropriate for its intended task. Should their selection be challenged by their adversary, and the court be called upon to make a ruling, then they should expect to support their position with affidavits or other equivalent information from persons with the requisite qualifications and experience, based on sufficient facts or data and using reliable principles or methodology.

For those understandably concerned about keeping discovery costs within reasonable bounds, it is worth repeating that the cost-benefit balancing factors of Fed.R.Civ.P. 26(b)(2)(C) apply to all aspects of discovery, and parties worried about the cost of employing properly designed search and information retrieval methods have an incentive to keep the costs of this phase of discovery as low as possible, including attempting to confer with their opposing party in an effort to identify a mutually agreeable search and retrieval method. This minimizes cost because if the method is approved, there will be no dispute resolving its sufficiency, and doing it right the first time is always cheaper than doing it over if ordered to do so by the court. Additionally, cost can be minimized by entering into a court-approved agreement that would comply with Hopson, or if enacted, Proposed Evidence Rule 502. In addition, there is room for optimism that as search and information retrieval methodologies are studied and tested, this will result in identifying those that are most effective and least expensive to employ for a variety of ESI discovery tasks. Such a study has been underway since 2006, when the National Institute of Standards and Technology (NIST), an agency within the U.S. Department of Commerce, embarked on a cooperative endeavor with the Department of Defense to evaluate the effectiveness of a variety of search methodologies. This project, known as the Text Retrieval Conference (TREC), evolved into the Trec LegalTrack, a research effort aimed at studying the e-discovery review process to evaluate the effectiveness of a wide array of search methodologies. This evaluative process is open to participation by academics, law firms, corporate counsel and companies providing ESI discovery services. See: http:// trec- legal. umiacs. umd. edu. The next test will occur in the summer of 2008. The goal of the project is to create industry best practices for use in electronic discovery. This project can be expected to identify both cost effective and reliable search and information retrieval methodologies and best practice recommendations, which, if adhered to, certainly would support an argument that the party employing them performed a reasonable ESI search, whether for privilege review or other purposes.