Summary
excluding expert testimony under Daubert where expert did not adhere to generally accepted threshold of two standard deviations
Summary of this case from Thomas v. Deloitte Consulting LPOpinion
Case No. 1:99CV2329
April 25, 2003
MEMORANDUM ORDER
In anticipation of trial, plaintiff Equal Employment Opportunity Commission ("EEOC") filed a motion in limine (docket no. 63), seeking to exclude the testimony of Erich Speckin, an expert designated by defendant Ethan Allen, Inc. The Court GRANTED this motion on the record at a pretrial conference, and promised to explain its ruling in more detail in an opinion to be issued later. Thereafter, the parties settled their dispute, based at least to some degree on the Court's ruling on the motion in limine. Ethan Allen and the EEOC have now filed a joint motion for Order approving and entering a consent decree (docket no. 106). The joint motion is GRANTED.
Furthermore, because of the importance of the issues raised in the motion in limine, and because the Court promised it would explain its reasoning, the Court provides here an explanation of its conclusion regarding the admissibility of Mr. Speckin's expert testimony. The Court granted the motion in limine for the reasons and to the extent stated below.
I. Speckin's Methods and Expert Opinion.
Ethan Allen retained Speckin to offer expert testimony regarding the age of certain ink writings found on a handwritten letter. The letter was written by plaintiff Tracey Mora to Ann Emanuele, a manager at Ethan Allen, and describes sexual harassment that Mora claims she suffered. The only chronological indication in the letter itself is the word "April," written in the upper right hand corner of the first page. Mora asserts she gave the letter to Emanuele in April of 1994, but Emanuele returned it to her, stating no one would believe what she had written. Ethan Allen apparently suspected that Mora did not write the letter in April of 1994 at all, so it hired Speckin to determine whether he could "date" the letter using forensic analysis. In response, the EEOC hired Dr. Albert Lyter as its own ink expert.
Speckin proceeded to test the ink on the letter. The first test he did was an "extraction test," to determine precisely what ink was used to write the letter. This test involves removing from the document several "microplugs" of paper and ink, placing the microplugs in solvent, and then performing a chromatographic separation of the colors in the ink. Speckin then compared the chromatographic separation results of the unknown ink from the Mora letter to a personal library of chromatographic separation results from known inks. Based on his comparisons, Speckin concluded that the unknown ink was actually a blue ballpoint ink formulation known as Formulabs 315, made by Formulabs, Inc. (now known as Sensient Technologies, Inc.). Lyter, who examined all of Speckin's testing procedures and results, does not disagree with this conclusion.
Sometimes, the determination of the type of ink used can, by itself, prove conclusively that the claimed date of writing is false. For example, a given ink may: (1) have been manufactured only during certain years; and/or (2) contain "tags," which are chemical components added to the ink for the precise purpose of providing a date of manufacture. If an expert knows that a given ink was manufactured only after, say, 1996, a claimed date of writing of 1994 must be false. In this case, Speckin's determination that the ink on the Mora letter is Formulabs 315 ink does not, by itself, show that the claimed date of writing is false.
Between about 1969 and 1994, many ink manufacturers in the United States added these tags at the request of the United States Department of Treasury.
In deposition, Speckin testified that he looked for a date-tag in the ink taken from the Mora letter, but he did not find one. Speckin then opined in deposition — but not in his expert opinion — that this absence suggested the ink was manufactured after June of 1994. Depo. at 33. Speckin further asserted he could "get the numbers" and calculate the likelihood that the ink from the Mora letter was manufactured after June of 1994, by researching whether and to what extent licensee manufacturers of Formulabs 315 ink included date-tags. Id. at 31-34. But Speckin had not obtained this data before his deposition and never amended his expert report to include it. Speckin's failure to actually include any date-tag analysis in his expert report suggests he was less than confident that the answer would actually help date the Mora letter.
Speckin then conducted two "relative ink age comparison tests." The common premise underlying these tests is that, when placed into a solvent, fully dried ink will usually leach from the paper to a lesser degree than the same ink that is not fully dried. In the case of ballpoint inks like Formulabs 315, experts generally agree that it takes between three and four years for the ink to dry completely. Thus, a finding that the ink taken from the Mora letter leaches into solvent significantly more than does a sample of the same ink that is known to be completely dried would support the conclusion that the ink from the Mora letter was less than four years old.
As discussed further below, some of the scientific literature suggests that ink can take as long as 6 years to dry, but the Court will accept, for the purposes of this Order, Speckin's contention that the ink in this case normally dries in 3 1/2 to 4 years. Given the Court's other conclusions, acceptance of the 3 1/2 year date is ultimately immaterial.
To obtain a "completely dried" sample of Formulabs 315 ink, Speckin used what he refers to as the "accelerated aging method." Speckin removed several paper-and-ink microplugs from the Mora letter and then heated them in an oven at 100 degrees Celsius for about 30 minutes. This method is supposed to dry out the ink completely, so that the amount of the artificially aged "baked ink" that is leached into a solvent will be the same as if the ink had aged naturally for four or more years. In his own report, Lyter criticizes this method of comparison. Lyter states that it would be more accurate to heat a fresh example of Formulabs 315 ink to obtain the artificially aged comparison sample, rather than heat the ink on the Mora letter, which necessarily has already aged somewhat. But Lyter does not criticize generally the concept of artificially aging ink by baking it, and he has used the method himself.
Speckin contends that his method of artificial aging of ink does control for at least one possible error in ink identification that Lyter's method would not. Speckin asserts that, even if he was wrong in identifying the ink on the Mora letter as Formulabs 315, his relative ink age comparison test still compares leaching rates of two samples of the same ink — one taken from the Mora letter "as-is," and one taken from the Mora letter and baked. Thus, any difference in leaching results, all other things being equal, would be fairly attributable to age difference. Lyter's method would compare an ink sample taken from the Mora letter "as-is" to a fresh sample of ink that Lyter obtained himself, from another source, and then baked. This means that, if Lyter identified the ink formula incorrectly, the fresh sample that Lyter baked to use for comparison would be the wrong ink, so that a difference in leaching results could be attributable to that independent factor. In any event, as explained below, the Court ultimately concludes that Speckin's technique of baking ink to obtain an "aged" sample is not scientifically sound for other, unrelated reasons.
Having obtained two samples of ink — one from the Mora letter "as-is," and one from the Mora letter that he artificially aged — Speckin then ran two comparative tests to measure relative dryness. The first was the "Rate of Extraction test," also known as an "R-ratio test," a simplified description of which is as follows. Speckin placed the two samples ("as-is" and "artificially aged") in separate vials of a weak solvent, and then measured how much of each ink leached into the solvent at certain time intervals — specifically, at 30 seconds, 90 seconds, and 180 seconds. Generally, one would expect that: (1) if the "as-is" sample is not fully dried, then the rate of leaching from the "as-is" sample would be greater than the rate of leaching from the heated sample; and (2) if the "as-is" sample is fully dried, then the rate of leaching from the "as-is" sample would be the same as the rate of leaching from the heated sample.
It is unclear why Speckin chose the 30-, 90-, and 180-second intervals to take his measurements. In a 1998 article that Speckin wrote, canvassing a number of case studies, the measurements were taken at various intervals over a total of ten minutes, not three. Brunelle Speckin, Technical Report with Case Studies on the Accelerated Aging of Ball-Point Inks, 4 Int'l J. of Forensic Doc. Examiners 240, 242 (July/Sept. 1998) ("Brunelle Speckin"). This fact, alone, raises a question of whether Speckin is using a methodology that is agreed-upon by the scientific community.
In this case, the results Speckin obtained from his R-ratio test were quite odd. Speckin performed the R-ratio test three times on an as-is sample, and three times on an artificially aged sample, and then averaged the results. In two of the "as-is" samples, and in two of the artificially aged samples, the amount of ink that had leached into the solvent was less at 90 seconds than at 30 seconds. The amount of ink that leached into the solvent at 180 seconds, however, was highest. Even more strange, in one of the artificially aged samples, the amount of ink that had leached into the solvent was highest at the 30-second interval, and then steadily decreased when measured at the 90- and 180-second intervals.
Thus, for example, Speckin's raw data for the second of his artificially aged samples showed densitometer readings (which are used to measure the ink-leaching rate) as follows:
measurement a 30 seconds 961
measurement b 90 seconds 780
measurement c 180 seconds 982
This shows that the measured amount of ink that leached from the sample actually decreased from 30 seconds to 90 seconds, but then increased over the next 90 seconds to a higher level than at 30 seconds.
See depo. exhibit 2, Bates page 2221.
In deposition, Speckin gave no clear explanation of how or why these odd results occurred. Interestingly, Speckin obtained similarly odd results in another case, In re Estate of Wang The Huei, 2002 WL 1341762, [2002] HKEC 1424 (Hong Kong Special Administrative Region Ct. of First Instance) (Nov. 21, 2002) ("Wang"). When asked to explain the results in Wang, Speckin speculated that the ink had "leach[ed] back into the paper," but he offered no scientific basis for this conjecture. Id. at ¶ 25.22. The Wang court rejected Speckin's speculation, stating: "If the tests were performed correctly, logic dictates that the values of measurements would increase with time, as the concentration of the substance being extracted in the solvent increases with time. It is analogous to one putting a teabag in a glass of hot water." Id. at ¶ 25.3.
In any event, Speckin did not discard his data; rather, he: (1) computed the averages of the three "as-is" samples and the three artificially aged samples; (2) computed the standard deviations from these averages; and (3) concluded there was no statistically significant difference in leaching rates from the "as-is" sample and the artificially aged samples. That there was no significant difference in the leaching rates could reasonably support the proposition that the "as-is" sample was fully dried. Instead, Speckin concluded that the R-ratio test was inconclusive.
The second test Speckin conducted was the "Percent Extraction test," a simplified description of which is as follows. After obtaining his data from the R-ratio test, which used a weak solvent, Speckin took all of the ink samples and placed them in a strong solvent. The strong solvent dissolved virtually all of the ink remaining in the samples. Speckin then computed the "total" amount of ink that leached from each sample by adding: (1) the amount of ink that had dissolved into the weak solvent at the 180 second interval; plus (2) the "remaining" amount of ink that dissolved into the strong solvent. Next, Speckin computed the percentage of ink that had leached from the sample at the 30-, 90-, and 180-second intervals, relative to the "total" amount of ink that had leached from the sample. Generally, one would expect that: (1) if the "as-is" sample is not fully dried, then the percentage of ink that leached from the "as-is" sample would be greater, at each time interval, than the percentage of ink that leached from the artificially aged sample; and (2) if the "as-is" sample is fully dried, then the percentage of ink that leached from the "as-is" sample would be the same, at each time interval, as the percentage of ink that leached from the artificially aged sample.
Put differently, the total amount of ink that leaches from paper in strong solvent will normally be about equal, regardless of ink age, while the amount of ink that leaches from paper in a weak solvent will normally decrease with age. Accordingly, the weak solvent/strong solvent ratio for a given ink ("Percent Extraction") will normally decrease with age.
Once again, the results Speckin obtained from his Percent Extraction test were odd. First of all, as one would expect (given the results from the R-ratio test), the percent extraction was less at 90 seconds than at 30 seconds for two of the "as-is" samples, and for two of the artificially aged samples; and, for one of the artificially aged samples, the percent extraction was even lower at 180 seconds. Even more odd, after averaging the results of all of the "as-is" samples and all of the artificially aged samples, Speckin found that the average percent extraction was higher for the aged samples, at every time interval, than the average percent extraction for the "as-is" samples. As noted, one would expect that, to the contrary, if there is any difference at all, the average percent extractions for the "older," (or "fully dried," or "artificially aged") heated samples would be lower.
Thus, for example, Speckin's raw data for the second of his artificially aged samples showed as follows:
measurement a 30 seconds 961 percent of total (a/e) = 57%
measurement b 90 seconds 780 percent of total (b/e) = 46%
measurement c 180 seconds 982 percent of total (c/e) = 58%
measurement d Strong solvent 703
measurement e Total of c + d 1685
This shows that the measured percentage of ink that leached from the sample actually decreased from 30 seconds to 90 seconds, but then increased over the next 90 seconds to a higher level than at 30 seconds.
See depo. exhibit 2, Bates page 2221; compare footnote 6, above.
Speckin asserts that it is generally known that some inks sometimes show "reverse extraction," meaning that the amount of ink that leaches from paper in a weak solvent will (contrary to normal expectation) increase with age. Speckin states that the ink on the Mora letter, Formulabs 315, is one of those inks that shows reverse extraction. He asserts that this explains why his average percent extraction is higher for the artificially aged samples than the "as-is" samples. In deposition, Speckin also stated that:
(1) he did not know previously that Formulabs 315 showed "reverse extraction;" and (2) whether an ink actually shows reverse extraction when tested (even an ink previously known to do so) can depend on type of paper and other factors. Depo. at 109. Speckin offered no explanation for why reverse extraction occurs.
As Speckin wrote in an article on the subject: "While newer inks usually extract faster and more completely than older inks into organic solvents, this is not always the case. For example, Bic black ball-point ink is frequently, but not always, found to extract older inks faster and more completely than newer inks [sic]." Brunelle Speckin at 242 (emphasis added). Other research suggests that the heating of certain inks may cause them to degrade and make them more susceptible to weak solvents, which would completely undermine the fundamental premise of Speckin's accelerated aging test. See Wang, 2002 WL 1341762 at ¶ 21.18(b).
Despite his odd data results, Speckin believed he could reach a definitive conclusion on the age of the ink on the Mora letter. Speckin asserted that, because the average percentages of ink that leached from the "artificially aged" and "as-is" samples were different — albeit in reverse to their expected comparative relationship — his Percent Extraction test still yielded meaningful data. In particular, Speckin calculated that the average percentage of ink that leached from the "as-is" samples was different, to a statistically significant degree, than the average percentage of ink that leached from the artificially aged samples, at each of the three time intervals. That this difference was because the "as-is" ink consistently leached less than the artificially aged ink is, he contends, immaterial.
Speckin further asserts that, so long as one of his tests — the R-ratio test or the Percent Extraction test — shows a statistically significant difference in leaching measurements, he can form a valid, scientific opinion. Here, he asserts his R-ratio test was inconclusive, but his Percent Extraction test showed a statistically significant result. Accordingly, Speckin opined on December 10, 2001 that, "to a high degree of scientific certainty, the [Mora] letter was not written at or about its purported date of 1994. Rather, the letter was written within the last three and one-half years." Explaining what he meant by "a high degree of scientific certainty," Speckin wrote: "The data from the [the R-ratio test and the Percent Extraction test] were analyzed statistically to determine if the differences were statistically significant or not. The standard that has been set by the Society of Forensic Ink Analysts (SOFIA) as a minimum threshold for a `significant statistical difference' is one standard deviation."
II. EEOC's Arguments.
The EEOC attacks the reliability of Speckin's expert conclusion on a number of grounds. Among others, the EEOC asserts that Speckin's testing procedures and ultimate opinion suffer the following flaws:
(1) Speckin did not ensure that the solvents he used were fresh; (2) Speckin did not ensure that the tinctures he created were not contaminated; (3) the 30-, 90- and 180-second intervals Speckin used were not the optimal intervals; (4) Speckin did not sufficiently control for the volume of ink contained in the various microplugs; (5) Speckin did not calibrate the densitometer he used to conduct his tests; (6) Speckin did not perform a test on blank paper, containing no ink, as a control; (7) Speckin did not adhere to certain ink identification standards published by the American Society for Testing and Materials ("ASTM"); (8) Speckin relied on the work of others in his laboratory, who performed tests without sufficient oversight; (9) Speckin used an excessively lenient statistical analysis; (10) Speckin's education and background do not qualify him as an expert; (11) Speckin's testing methodologies have not been subjected to peer review and are not generally accepted in the scientific community; and (12) Speckin's use of heat to artificially age ink leads to unreliable results in relative ink age comparison tests.
See, e.g., ASTM Designation E 1789-96 ("Standard Guide for Writing Ink Identification").
Notably, as the EEOC is quick to point out, another court has found many of these very grounds well-taken in a recent, high-profile case, in which the plaintiff offered Speckin as an ink-dating expert. See generally Wang, 2002 WL 1341762. In Wang, two of Asia's most wealthy individuals — the father and wife of Teddy Wang, who controlled the Chinachem Investment Company before he was kidnaped for ransom and killed — fought over Wang's estate. The key question was whether the two signatures on a 1990 Will, which purportedly made Mrs. Wang the sole executor of the decedent's multi-billion-dollar empire, were genuine. Id. at ¶ 1.14. The Wang court conducted a trial involving 172 hearing days that lasted over one year, id. at ¶¶ 32.1-32.2, ultimately publishing an incredibly detailed opinion hundreds of pages long. Speckin submitted an expert report on behalf of the plaintiff father, opining that the signatures on the 1990 Will produced by Mrs. Wang "were not written on or near the purported date in 1990 but at a later time `at least in 1996 or later.'" Id. at ¶ 1.9.
The Wang court's analysis of Speckin's ink-dating tests are found under the section titled "Sections III IV — Ink-dating Final Conclusion."
Counsel for Mrs. Wang decided to secretly test Speckin's methods by instructing a third party to have Speckin test 12 documents of known origin — "a sort of `set-up' made by the defence in order to find out whether or not the method used by [Speckin] for ink-dating is valid, reliable and accurate." Id. at ¶ 19.1. Of the 12 documents he tested, Speckin was wrong on 4 of them, concluding incorrectly that: (a) three writings that were actually less than 3 years old were more than 3 1/2 years old; and (b) one writing that was actually 10 years old was less than 3 1/2 years old. The Wang court then undertook an exhaustive review of the methodology Speckin used to test the 1990 Will at issue, citing the standards set forth in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). Among many other relevant observations and conclusions, the Wang court found as follows:
• Other evidence gave "a clear indication that the process of decomposition of ink dye components will continue and does not stop (or in the case of Formulab 587, does not even slow down) after five years, contrary to [Speckin's] claim that this process will stop after three to three and a half years." Wang, 2002 WL 1341762 at ¶ 21.16.
• Speckin's accelerated aging method "should be endorsed by a consensus of practitioners, with widely disparate backgrounds and interests, such as those from private and public laboratories all agreeing on what should be done and not simply (as in this case), members from a small and young association namely, SOFIA." Id. at ¶ 22.6.
• Speckin's methods "have not undergone any validation studies. On the contrary, it appears that SOFIA, consisting of a very limited number of persons including Dr. Lyter, Mr. Robert Kuranz and Mr. Brunelle, and of which [Speckin] is the Vice President, has only just started to validate the other ink-dating procedures namely, the Percent Extraction method and the R-ratio method on comparison between known and questioned ink samples. . . ." Id. at ¶¶ 22.8-22.9.
• Speckin's accelerated aging method has "not been used by a majority of chemists/scientists in private practice and government laboratories," including Bundeskriminalamt of Germany (The German Police Laboratory), the Forensic Science Service of the United Kingdom, the Forensic Science Division of the Zurich Canton Police, the National Forensic Laboratory of the United States Internal Revenue Service, and the United States Secret Service, and "is not even one of the methods endorsed by SOFIA itself." Id. at ¶ 23.2-23.3, 23.8 (emphasis in original).
• There was "no evidence . . . that the methods/techniques upon which the conclusion was drawn by [Speckin] in this case, have been in use for 15 years and routinely accepted by courts. This Court is only presented with a technique which may have been followed in general outline by two or three other people, which may have been accepted by some U.S. Courts and which is certainly not accepted by any law enforcement agencies in the United States and all over the world. I am not satisfied that the Dye-ratio method coupled with the accelerated aging method used by [Speckin] in this case are valid, or reliable and from which a responsible conclusion could be drawn." Id. at ¶ 23.24.
• When asked for an explanation regarding the 12 documents secretly submitted to him by the defendant, Speckin wrote that "it was possible that the heat was unable to induce age in the ink." Id. at ¶ 19.23(f). This is apparently an admission by Speckin that the accelerated aging method he uses is not always reliable.
• Speckin had a "penchant for exaggeration" regarding his own background. Id. at ¶ 29.7. For example, the American Board of Forensic Document Examiners accused Speckin of engaging in a "fraudulent misrepresentation" by stating he was eligible for their certification, and Speckin falsely testified that he was "a member of the Questioned Document Section of the Mid-Western Association of Forensic Scientists," when he was not. Id. at ¶¶ 29.52, 29.29.
Indeed, two document examiners working for the United States Secret Service have noted that there were "many unanswered questions with respect to the impact and effect of applying heat to an ink entry," and concluded that the "accelerated aging technique" was not yet "shown to be solidly based." L. Stewart S. Fortunato, "Distinguishing Between Relative Ink Age Determinations and the Accelerated Aging Technique," International Journal of Forensic Document Examiners, vol. 2 no. 1 at 10, 13-14 (Jan/Mar. 1996).
Although Speckin did not use the "Dye-ratio method" in this case, the Wang court's criticisms and observations remain completely relevant.
Ultimately, the Wang court: (1) found that Speckin was not credible, (2) rejected Speckin's evidence and expert opinion entirely, and (3) was left with the "lingering suspicion" that Speckin "tailored his results to suit his client's requirements." Id. at ¶ 29.31(f).
The Wang court also found flaws in many of Speckin's laboratory techniques. Interestingly, although the Wang court found that Speckin's expert opinion, which supported the position of the plaintiff father, was worthless, the court ultimately ruled in the father's favor. Following the court's ruling, Mrs. Wang was arrested. A thumbnail sketch of this case and its aftermath may be found at D. Lague, "Arrest of Asian Tycoon's Wife Adds Twist in `Battle of Wills,'" 12/13/02 Wall St. J. Eur. at A14, 2002 WL-WSJE 103490334.
Many of the criticisms leveled at Speckin by the Wang court could also serve as bases for this Court's conclusion that, based on the standards imposed by Daubert, Speckin's testimony is inadmissible in this case. Ultimately, however, the Court finds two particular grounds especially compelling and independently sufficient to justify its conclusion: (1) Speckin's statistical analysis is deeply suspect; and (2) Speckin's accelerated aging method to determine the relative age of ink is not based on sound scientific theory.
III. Relevant Standards
Federal Rule of Evidence 702 addresses the admissibility of testimony by experts. The Rule states:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
Fed.R.Evid. 702 (emphasis added). Notably, the Rule sets out three critical conditions for admissibility of expert testimony. In Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), the Supreme Court held that district court judges are to serve a "gatekeeping role," and must assure the conditions set out in Fed.R.Evid. 702 are met.
The Supreme Court also suggested a non-exclusive list of factors for trial courts to consider when deciding whether proposed scientific expert testimony is sufficiently "reliable," as required by the second and third conditions in Rule 702. The specific factors listed by the Daubert Court are: "(1) whether the expert's technique or theory can be or has been tested — that is, whether the expert's theory can be challenged in some objective sense, or whether it is instead simply a subjective, conclusory approach that cannot reasonably be assessed for reliability; (2) whether the technique or theory has been subject to peer review and publication; (3) the known or potential rate of error of the technique or theory when applied; (4) the existence and maintenance of standards and controls; and (5) whether the technique or theory has been generally accepted in the scientific community." Fed.R.Evid. 702 (advisory committee notes, 2000 amendments); Daubert, 509 U.S. at 593-594.
In Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 147 (1999), the Supreme Court explained that the "basic gatekeeping obligation" explained in Daubert applies not just to "scientific" testimony, but to "all expert testimony."
In addition to the non-exhaustive list of factors identified explicitly in Daubert, other courts have recognized at least five other factors relevant to the determination of whether expert testimony is sufficiently reliable to be admitted into evidence. See generally Fed.R.Evid. 702 (advisory committee notes, 2000 amendments) (listing these factors and discussing cases). One such factor is whether the field of expertise claimed by the expert, even if it is "generally accepted," is known generally to reach reliable results at all.
See Kumho Tire Co. v. Carmichael, 119 S.Ct. 1167, 1175 (1999) (Daubert's general acceptance factor does not "help show that an expert's testimony is reliable where the discipline itself lacks reliability, as for example, do theories grounded in any so-called generally accepted principles of astrology or necromancy"); Moore v. Ashland Chemical, Inc., 151 F.3d 269 (5th Cir. 1998) (en banc) (clinical doctor was properly precluded from testifying to the toxicological cause of the plaintiff's respiratory problem, where the opinion was not sufficiently grounded in scientific methodology); Sterling v. Velsicol Chem. Corp., 855 F.2d 1188 (6th Cir. 1988) (rejecting testimony based on "clinical ecology" as unfounded and unreliable). Another critical factor is whether the expert has adequately accounted for obvious alternative explanations. See Claar v. Burlington N.R.R., 29 F.3d 499 (9th Cir. 1994) (testimony excluded where the expert failed to consider other obvious causes for the plaintiff's condition); cf. Ambrosini v. Labarraque, 101 F.3d 129 (D.C. Cir. 1996) (the possibility of some uneliminated causes presents a question of weight, so long as the most obvious causes have been considered and reasonably ruled out by the expert).
Courts should consider also whether the expert proposes "to testify about matters growing naturally and directly out of research they have conducted independent of the litigation, or whether they have developed their opinions expressly for purposes of testifying." Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1317 (9th Cir. 1995). Similarly, it is important to assess whether the expert "is being as careful as he would be in his regular professional work outside his paid litigation consulting." Sheehan v. Daily Racing Form, Inc., 104 F.3d 940, 942 (7th Cir. 1997); Kumho Tire Co. v. Carmichael, 119 S.Ct. 1167, 1176 (1999) (Daubert requires the trial court to assure itself that the expert "employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field"). And the court must not admit expert testimony if the expert "has unjustifiably extrapolated from an accepted premise to an unfounded conclusion." Fed R. Evid 702 (advisory committee notes); see, e.g., General Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997) (noting that in some cases, a trial court "may conclude that there is simply too great an analytical gap between the data and the opinion proffered").
Finally, the trial court "must scrutinize not only the principles and methods used by the expert, but also whether those principles and methods have been properly applied to the facts of the case." Fed.R.Evid. 702 (advisory committee notes, 2000 amendments). As the court noted in In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 745 (3rd Cir. 1994), "any step that renders the analysis unreliable . . . renders the expert's testimony inadmissible. This is true whether the step completely changes a reliable methodology or merely misapplies that methodology."
Applying these standards in this case, it is clear that the Court must close the gate to Speckin's proposed expert testimony, because it is not "the product of reliable principles and methods." Fed.R.Evid. 702.
IV. Analysis. A. Speckin's Technique Does Not Yield A High Degree of Scientific Certainty.
As noted above, Speckin analyzed the data in this case "to determine if the differences [between ink samples] were statistically significant or not. The standard that has been set by the Society of Forensic Ink Analysts (SOFIA) as a minimum threshold for a `significant statistical difference' is one standard deviation." Because he did find differences between ink samples of one standard deviation in his Percent Extraction test, Speckin opined "to a high degree of scientific certainty, [that] the [Mora] letter was not written at or about its purported date of 1994."
That a small number of analysts got together and agreed that statistical significance in ink dating is acceptable at the level of one standard deviation, however, does not make it so. It is an elementary statistical truth that a test using one standard deviation ("1STD") as its measure of statistical significance yields a 68% confidence level in the results. This is the same as saying there is about a one in three chance that the test results are not significant at all. Speckin acknowledged this fact himself. See depo. at 37 ("It would be 68 percent chance of it being right or 32 percent chance of it being just due to chance."). In comparison, a test that uses a 2STD measure of statistical significance yields a 95% confidence level in the results, and a test that uses a 3STD measure of statistical significance yields a 99.7% confidence level in the results. By adopting the 1STD measure, Speckin and his SOFIA cohorts agreed that their ink-dating tests would be only moderately sensitive to error, even assuming completely "logical" data. This weakness is compounded by deeming test results that do not meet the 1STD measure "inconclusive," worthy only to be ignored. It is probably not coincidental that adoption of the 1STD measure allows the SOFIA analysts to more frequently opine on behalf of their clients that their ink-dating test results are "significant," rather than "inconclusive." See Speckin depo. at 89-90 (admitting that using a two standard deviation criteria would result in a lesser number of conclusive determinations).
A good layman's explanation of the concept of standard deviation may be found at http://www.robertniles.com/stats/stdev.shtml.
More precisely, for data which follows a "normal," bell-shaped curve distribution, approximately 68% of all data will fall within one standard deviation of the data mean, 95% of all values will fall within two standard deviations, and 99.7% of all data will fall within three standard deviations. For data falling in "non-normal" distributions, at least 75% of all values fall within two standard deviations, while at least 89% fall within three standard deviations. See http://www.statlets.com/usermanual/glossary2.htm (defining "standard deviation"). The experts in this and other ink-dating cases seem to agree that the data they are generating fall in a normal distribution.
When challenged on this point at deposition, Speckin asserted there was no need to "justify" the use of his 1STD criteria because "[i]t's up to the trier of fact to decide if it's significant enough to influence their decision one way or the other." Id. at 90. But Speckin seeks to offer an "expert" opinion that, in fact, the differences he finds are significant: "Once it was past one standard deviation, it was my conclusion to a reasonable degree of scientific certainty [that the ink's ages] were different." Id. at 38. Even after cross-examination, a jury might not understand that Speckin's expert opinion "was buried beneath the term `1STD' or `to a reasonable degree of scientific certainty' but then the confidence level of the opinion or conclusion is only about [68%]." Wang, 2002 WL 1341762 at ¶ 27.8. Speckin also answered the 1STD criticism by noting that "at least one of [his] measurements" was significant even at a level of 2STD, depo. at 38, but the strength this single measurement adds to his ultimate opinion is, at best, marginal.
Because Speckin has used a 1STD measure of statistical significance, he simply cannot validly opine "to a high degree of scientific certainty" that the Mora letter was written within the last 3 1/2 years, and not in 1994. A high degree of scientific certainty may be attained by tests using a 2STD measure of statistical significance, but the confidence level underlying Speckin's results is only slightly higher than the predicted results of tossing a coin. Unsurprisingly, the Wang court concluded that its own confidence level in Speckin's opinion could only be "weak," and noted that using a test with a sensitivity of only 1STD is a "departure from the accepted norms of analytical chemistry." Wang, 2002 WL 1341762 at ¶¶ 27.7, 27.10.
The Wang court is not alone. Another court that examined Speckin's ink-dating analysis before Wang arrived at essentially the same conclusion:
The Court will not place any reliance on the results of Expert Speckin's "relative-ink-date testing," also known as "accelerated aging." The Court recognizes that the methodology is vouched for by both Experts Speckin and Brunelle, the latter especially preeminent in the field. The problem is that the test tries to draw large conclusions from tiny differences in leach rates and to do so after artificial "accelerated aging" (i.e., heating in an oven) of part of the test sample (so as to provide a "known" old sample for comparison). Each was tested at different durations of leaching to detect differences in the leach rates. In all cases, most of the differences at various durations were inconclusive and, at most, only a few were conclusive.
Aptix Corp. v. Quickturn Design Systems, Inc., 2000 WL 852813 at *19 (N.D.Cal. June 14, 2000), affirmed in pertinent part, 269 F.3d 1369 (Fed. Cir. 2001).
In sum, the statistical analysis used by Speckin to reach his expert opinion does not support that very opinion. Daubert instructs that this court "should consider the known or potential rate of error" in the tests underlying an expert's conclusion. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 594 (1993). The rate of error of Speckin's analysis, given his use of a 1STD measure of statistical significance, is about one out of three. As such, the Court finds that any expert testimony offered by Speckin regarding ink-dating using relative ink age comparison tests cannot properly be admitted as an expert conclusion.
B. Speckin's Accelerated Aging Technique Is Not Based On Sound Scientific Theory.
Moreover, even if Speckin had used a more stringent measure to determine the statistical significance of his test results, the Court would still grant the EEOC's motion in limine. This is because there is a dearth of support for Speckin's use of the accelerated aging method as sound scientific theory. Indeed, to the contrary, there is evidence that determination of the age of ink using Speckin's accelerated aging technique is inherently problematic. Speckin explained that the premises underlying his relative ink age comparison tests are as follows: (1) it takes ballpoint ink about 3 1/2 to 4 years to fully dry; (2) when placed into a solvent, fully dried ink will normally leach from the paper to a lesser degree than the same ink that is not fully dried; and (3) a sample of fully dried ink can be obtained by taking a sample of fresh ink and baking it, thereby "artificially aging" it.
Even accepting the first of these premises, there is evidence that the second premise is not consistently true, and there is no evidence supporting the third premise. Regarding the second premise, it proved untrue in this case. The results of Speckin's Percent Extraction test showed a higher "leach rate" for the artificially aged ink sample than from the Mora letter "as-is." Speckin stated this was due to "reverse extraction," which sometimes occurs with some inks. But Speckin cannot predict when reverse extraction will occur. He does not know whether or in what circumstances a particular ink will "reverse extract."
The Court notes, however, that other ink experts doubt the first premise. See Wang, 2002 WL 1341762 at ¶ 21.16 (noting evidence that "the decomposition of dye components does not stop after six years"). Even Speckin has written that factors affecting ink-dryness include "how long it takes that particular ink to completely dry on the particular paper involved and how it has been stored," so that inks "stored under very age preserving conditions may appear fresher than if stored under `normal' conditions." Brunelle Speckin at 242.
Furthermore, regarding the third premise, Speckin does not even know whether Formulabs 315 ink — the ink at issue in this case — reverse extracts naturally, or only when he uses his accelerated aging technique. More broadly, nowhere does Speckin point to any evidence of test results comparing samples of an ink that has fully dried normally and the same ink that has been dried through "artificial aging." In other words, there is no telling whether Speckin's "accelerated aging technique" reliably yields a sample equivalent to natural, fully-dried ink.
Speckin asserts he can still reach conclusions regarding the relative age of an ink, even when reverse extraction occurs — he asserts that, so long as there is some difference between the two samples, he can conclude the questioned sample is newer than his artificially aged sample, meaning newer than 3 1/2 years old. But there are a variety of other factors that could explain the difference between samples, including the artificial aging process itself. See Wang, 2002 WL 1341762 at ¶ 21.18(b) ("When these samples were heated, the methyl violet components were further decomposed. We should, therefore, expect to receive a statistically significant difference between the heated and the unheated samples. * * * [But] any statistically significant difference between the heated and unheated ink samples would likely be the result of, at least, the thermal instability of the methyl violet components in the ink"). Accord, Learning Curve Toys, L.P. v. PlayWood Toys, Inc., 2000 WL 343205 at *3 (N.D.Ill. March 31, 2000) ("PlayWood rests its argument on the premise that fresher inks extract faster than older inks and thus yield a higher extraction percentage. PlayWood goes so far as to claim that this premise is "undisputed." But in the very articles and case studies that PlayWood submits to the court, even this seemingly fundamental premise is uncertain.") (granting a motion to bar Lyter from giving expert testimony on ink dating).
See also "The Dating of Handwriting Through Ink Analysis — Accelerated Aging," 35 Am. Jur. Proof of Facts 3rd § 20 at 591 (1996) ("at least some ink formulas undergo color changes when heated to 100 degrees centigrade. Such color changes may have an effect on the results of extractability testing, so the use of accelerated aging should be approached with caution until the effect of heating ink to 100 degrees centigrade has been further investigated."); J. Levinson, Questioned Documents: A Lawyer's Handbook at 118 (Academic Press 2001) (criticizing "claims that particular methods can differentiate between `fresh' writing and `older' writing").
While there do exist cases where a court has overruled objections and allowed expert testimony regarding relative ink age comparison tests, those cases are usually conclusory and typically undertake no in-depth analysis of the testing methodology. This Court's careful analysis of Speckin's testing methodology in this case leaves it convinced that Speckin cannot offer an expert opinion that meets the requirements of Daubert and Fed.R.Evid. 702.
See, for example, Janopoulos v. Harvey L. Walner Assocs., Ltd., 866 F. Supp. 1086, 1095-96 (N.D.Ill. 1994) where the court concluded, with no explanation, that "[r]elative age solvent extraction comparison is also a reputable method of dating ballpoint inks." The Janopoulos court did not mention the phenomenon of "reverse extraction." The undersigned finds somewhat chilling the fact that, in other cases, a civil, or even criminal, verdict may have hinged on the type of expert opinion offered in this case.
V. Conclusion.
Speckin's deposition testimony suggests there may be other methods to determine the age of a document that would be admissible in this case. These methods might include, for example: (1) evidence that the document was written using a certain formulation of ink (possibly including ink tags) that was manufactured only during certain dates; (2) evidence that the paper in question was manufactured only during certain dates; (3) evidence of "press-through" impressions on the document, caused when someone writes on another piece of paper of known date that is on top of the questioned document; and (4) evidence of "write-overs," where someone writes in ink and then someone else writes over it using an ink that is later-manufactured.
But Speckin did not offer an expert opinion in this case in reliance on any of these methods. Rather, Speckin attempted to reach a conclusion regarding the age of the Mora letter using: (1) an unsound scientific technique — using his "accelerated aging method" to determine the relative age of ink, and using questionable comparison-test results; and (2) an unsound mode of data interpretation — imprecise and inaccurate statistical analysis that is insufficiently sensitive to error and to alternative explanations. Accordingly, the EEOC's motion in limine to exclude the testimony of Ethan Allen's designated expert, Erich Speckin, is granted.
IT IS SO ORDERED.