Opinion
CR. No. 10-2734 JCH
02-07-2013
UNITED STATES OF AMERICA, Plaintiff, v. JOHN CHARLES McCLUSKEY, Defendant.
MEMORANDUM OPINION AND ORDER
The subject of this Memorandum Opinion and Order is Defendant John McCluskey's Motion to Exclude Firearm Identification Evidence, and Request for Daubert Hearing [Doc. 418]. Beginning on August 21, 2012, the Court held a three-day evidentiary hearing on the motion, at which Defendant was present. At the hearing, the Court heard testimony from two witnesses, John Murdock and Katharina Babcock, and admitted into evidence approximately 80 exhibits totaling more than 2,000 pages.
LEGAL STANDARD
The Supreme Court has held that trial courts have a "gatekeeping responsibility" to "ensure that any and all scientific testimony or evidence admitted is not only relevant, but reliable." Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579, 589 n. 7 (1993) (citing authorities). In Daubert, the Supreme Court provided a list of specific factors bearing on reliability that trial courts could consider in executing the gatekeeping obligation. These factors—sometimes referred to as the "Daubert factors"—can be summarized as follows: (1) whether a theory or technique has been or can be tested; (2) whether the theory or technique has been subjected to peer review and publication; (3) the technique's known or potential rate of error and the existence and maintenance of standards controlling the technique's operation; and (4) whether a particular technique or theory has gained general acceptance in the relevant scientific community. However, the Daubert Court did not "presume to set out a definitive checklist or test," noting that "[m]any factors" might bear on the Rule 702 inquiry. Daubert, 509 U.S. at 593.
In Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999), the Supreme Court addressed the question of "how Daubert applies to the testimony of engineers and other experts who are not scientists." Id. at 141. The Supreme Court clarified that the "gatekeeping" obligation applied "not only to testimony based on 'scientific' knowledge, but also to testimony based on 'technical' and 'other specialized' knowledge." Id. However, while Kumho Tire held that trial courts considering the reliability of the testimony of non-scientists could consider the Daubert Factors, it reminded trial courts that they were not required to do so because the "list of factors was meant to be helpful, not definitive." Id. at 151. "[T]hehe test of reliability is flexible, and Daubert's list of specific factors neither necessarily nor exclusively applies to all experts or in every case. Rather, the law grants a district court the same broad latitude when it decides how to determine reliability as it enjoys in respect to its ultimate reliability determination." Id. at 141-42, 119 S.Ct. 1167. Many courts have recognized that the list of factors the Supreme Court outlined in Daubert "may not perfectly fit every type of expert testimony, particularly technical testimony based primarily on the training and experience of the expert." United States v. Monteiro, 407 F.Supp.2d 351, 357 (D. Mass. 2006) (compiling cases).
In 2000 and again in 2011, Rule 702 was amended in response to Daubert and "the many cases applying [it], including Kumho Tire Advisory Committee Notes relating to the 2000 Amendments to Rule 702. In its current form, Rule 702 provides:
A witness who is qualified as an expert by knowledge, skill, experience, training, or
education may testify in the form of an opinion or otherwise if:
(a) the expert's scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;
(b) the testimony is based on sufficient facts or data;
(c) the testimony is the product of reliable principles and methods; and
(d) the expert has reliably applied the principles and methods to the facts of the case.
BACKGROUND
The facts, as the Government intends to prove them at trial, are as follows. On August 4, 2010, outside of Santa Rosa, New Mexico, law enforcement personnel discovered a burned out camper trailer containing human remains, which were later identified to be those of Gary and Linda Haas. Later that day, authorities located the Hass' pickup truck in Albuquerque. On August 9, 2010, police in Wyoming arrested co-defendant Tracy Province, who possessed a 9mm handgun identified as belonging to Gary and Linda Haas. On August 19, 2010, Defendant and co-defendant Casslyn Welch were arrested at a campsite in eastern Arizona. Welch was in possession of a .38 caliber handgun, identified as belonging to Gary and Linda Haas. Also found at the campsite were a .45 caliber handgun that belonged to the Haases and a .40 caliber Smith & Wesson handgun. According to the Government, both co-defendants have identified the latter as the murder weapon. The .40 caliber Smith & Wesson was marked as IB22.
All the foregoing firearms were taken into evidence and, along with numerous cartridge casings and possible bullet fragments found within the burned out camper, presented to Katharina Babcock for ballistics analysis. Babcock is a firearms and toolmark examiner for the New Mexico Department of Public Safety ("NMDPS"). She discovered that three of the casings—labeled G7, G11, and G13—had a headstamp reading "BLAZER 40 S & W." These markings indicated that the casings contained ammunition of the same brand and caliber as the handgun marked IB22 that had been found with the Defendant at the time of his arrest. Using the same type of ammunition that was recovered from the burned out camper, Babcock test-fired firearm IB22. Then, using a comparison microscope, she compared the three casings found at the crime scene with the test-fired casings. Babcock identified one of the casings, G13, as having been fired from handgun IB22. One of her colleagues, Alina Sanchez, later reviewed and confirmed this identification. Due to a lack of individual characteristics, Babcock could not conclude whether casings G7 and G11 had been fired by handgun IB22. However, she was able to conclude that none of the other firearms that had been collected as evidence had fired casings G7 or G11, thereby excluding them. Babcock also eliminated the other three firearms collected into evidence as having fired any of casings G7, G11, or G13.
Babcock also compared ten previously fired .40 Smith & Wesson caliber cartridge cases labeled pw1 through pw10 with each other and with the various firearms collected into evidence in this case. She identified cartridge cases pw1, pw2, pw3, pw7, and pw9 as being fired from gun IB72, a .40 caliber Smith & Wesson model SW40VE semi-automatic pistol bearing serial number PDB5557. Babcock further identified cartridge cases pw4, pw5, pw6, pw8, and pw10 as being fired in a firearm labeled Colorado 1, which is a .40 Smith & Wesson caliber, Hi-Point Firearms brand, model 4095 semi-automatic rifle bearing serial number H18087. Babcock's colleague, Alina Sanchez, confirmed these identifications.
It is not clear to the Court from the record presently before it where authorities found evidence items pw1 through pw10, IB72, and Colorado 1.
Babcock concluded that another piece of evidence recovered from the burned camper, item G3, was a heavily charred bullet. Based on its diameter, she concluded that it was a .40 caliber bullet. However, due to the bullet's damaged condition she was unable to determine whether it had been fired by any of the .40 caliber weapons collected into evidence in this case.
DISCUSSION
I. Discovery Issues
McCluskey argues that the expert ballistics testimony should be excluded because the Government failed to adequately and timely summarize the basis and substance of Babcock's opinions, as well as to provide all foundational data. He contends that this alleged failure prevents him from effectively challenging Babcock's opinions and cross examining her.
The record shows that the Government provided McCluskey with Babcock's five-page report in February of 2012. That report summarizes the conclusions that Babcock reached regarding the firearms evidence presented to her for analysis. In addition, in February of 2012 the Government provided McCluskey with Babcock's entire case file, which contains case notes, worksheets, and photographs of the evidence, including side-by-side comparison photographs of the evidence taken with a microscope. The case file describes the work that Babcock did as to each piece of evidence, the analysis that she conducted, and the conclusions she reached. The Court concludes that the Government provided McCluskey with all the information it was required to provide under Rule 16 of the Federal Rules of Criminal Procedure and Rule 702 of the Federal Rules of Evidence.
As for McCluskey's complaint that "there are no points of similarity documented on the photographs and no way to know what marks on the breechface led to the conclusion that G13 and the test fired cartridge were fired from the same weapon," which he repeats for all of Babcock's conclusions, the Court finds it to be without merit. The Government has provided McCluskey with the evidence Babcock examined, the methodology she applied, and her conclusions regarding what characteristics of the evidence led her to find the existence of a match or an exclusion. It appears that McCluskey desires that Babcock identify in writing and photographic documentation every single mark, striation, impression, and grove that led to each of her conclusions. The Court is unaware of any authority requiring that level of detail. In any event, to the extent such a requirement exists, it has been satisfied in that McCluskey has had the opportunity to cross examine Babcock on the witness stand about each of her conclusions in this case.
II. Firearms and Toolmark Identification Generally
Forensic toolmark identification is a discipline that is concerned with the matching of a toolmark to the specific tool that made it. Firearm identification is a specialized area of toolmark identification dealing with firearms, which involve a specific category of tools. Gov't Ex. 5, Richard Grzybowski, et al., Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards, at 3. "Toolmarks are generated when a hard object (tool) comes into contact with a relatively softer object." Gov't Ex. 1 and Deft. Ex. E, National Research Council, Strengthening Forensic Science in the United States: A Path Forward, at 150 (National Academies Press 2009). Toolmarks associated with a firearm may occur in the commission of a crime when "the internal parts of a firearm make contact with the brass and lead that comprise ammunition." Id. "The manufacture and use of firearms produces an extensive set of specialized toolmarks." Id. at 150-51.
In United States v. Otero, 849 F. Supp. 2d 425, 427-28 (D.N.J. 2012), the court provided a helpful and concise description of the "AFTE" (Association of Firearms and Toolmark Examiners) theory of firearm identification. Toolmark identification is based on the theory that tools used in the manufacture of a firearm leave distinct marks on various firearm components, such as the barrel, breech face, or firing pin. The theory further posits that the marks are individualized to a particular firearm through changes the tool undergoes each time it cuts and scrapes metal to create an item in the production of the weapon. Toolmark identification thus rests on the premise that any two manufactured products, even those produced consecutively off the same production line, will bear microscopically different marks. With regard to firearms, these toolmarks are transferred to the surface of a bullet or cartridge casing in the process of firearm discharge. Depending on the tool and the type of impact it makes on the bullet or casing, these surface marks consist of either contour scratch lines, known as striations (or striae), or impressions. For example, rifling (spiraled indentations) inside of a gun barrel will leave raised and depressed striae, known as lands and grooves, on the bullet as it is fired from the weapon, whereas the striking of the firing pin against the base of the cartridge, which initiates discharge of the ammunition, will leave an impression but not striae.
Comparing a test bullet or cartridge fired from a firearm of known origin to another bullet or cartridge of unknown origin, the examiner seeks to determine congruence in the pattern of marks left on the examined specimens. This process is known as "pattern matching." When the marks consist of striations, the identification process can involve a method that observes and counts consecutively matching striae ("CMS"). When one is comparing striated tool marks, CMS is a convenient way to describe for other examiners the extent of agreement the examiner has observed. Gov't Ex. 6, Ronald Nichols, "The Scientific Foundations of Firearms and Tool Mark Identification—A Response to Recent Challenges." An examiner observes three types of characteristics on spent bullets or cartridges: class, subclass and individual. Class characteristics are features common to most if not all bullets and cartridge cases fired from a type of firearm—for example, the caliber and the number of lands and grooves on a bullet. Individual characteristics are microscopic markings produced in the manufacturing process by the random imperfections of tool surfaces (the constantly changing tool as described above) and by use of and/or damage to the gun post-manufacture. According to the theory of toolmark identification espoused by AFTE, individual characteristics "are unique to that tool and distinguish it from all other tools." Theory of Identification as it Relates to Toolmarks, AFTE Journal, Vol. 30, No. 1, Winter 1998, at 87. Subclass characteristics generally fill the gap between the class and individual characteristics categories. They are produced incidental to manufacture but apply only to a subset of the firearms produced, for example, as may occur when a batch of barrels is formed by the same irregular tool.
The firearms examiner uses a comparison microscope to examine the markings on at least two cartridge casings, one of which is known to have been test fired from a particular weapon. If the individual markings on the two casings show sufficient similarity, the examiner can conclude that the cartridges were fired from the same weapon. Sufficient similarity exists when the casings, viewed by a trained and experienced firearms examiner, evince sufficient duplication of markings that they can be considered individual characteristics, and the likelihood that another gun could have made them is so remote that it can be discounted.
III. ADMISSIBILITY OF FIREARM IDENTIFICATION ANALYSIS GENERALLY
Defendant challenges the admissibility of the expert firearms identification opinion insofar as it asserts that a particular discharged bullet or spent shell was fired from a particular gun, to the exclusion of all other guns. This individualized identification, Defendant argues, is based on a theory—that is, that firearms-related toolmarks are unique and reproducible—that has not been proven scientifically, has no quantifiable or definable standards, and is vulnerable to the subjective judgment of the examiner. In support of this argument, Defendant points to discussions in the 2009 National Research Council's report entitled Strengthening Forensic Science in the United States: A Path Forward (the "NRC Forensic Science Report") (Gov't Ex. 1 and Deft. Ex. E) and its 2008 Ballistic Imaging Report (Gov't Ex. 10 and Deft. Ex. F), each of which called into question the validity of the assumptions about toolmarks that underlie firearms identification. The NRC Forensic Science Report focused on the challenges and limitations faced by a number of forensic science disciplines, including autopsies and medical examinations, DNA analysis, controlled substance analysis, toolmark and firearms identification and many others. It identified deficiencies in the forensic sciences and concluded that generally, the forensic identification disciplines, other than nuclear DNA analysis, lack sufficient grounding in scientific research to verify the accuracy and validity of their methodologies. National Research Council, Strengthening Forensic Science in the United States: A Path Forward, at 12-13, 87. The Ballistic Imaging Report expressly focused on assessing the feasibility and accuracy of a national ballistics database. According to the report itself, its scope did not include any commentary on the discipline of firearms identification or opinion "on the admissibility of firearms-related toolmark evidence in legal proceedings." Gov't Ex. 10 at 3. Furthermore, the report states, "we accept a minimal baseline standard regarding ballistic evidence. Although they are subject to numerous sources of variability, firearms-related toolmarks are not completely random and volatile; one can find similar marks on bullets and cartridge cases from the same gun." Id.
Defendant criticizes the theory and methodology of toolmark identification as, essentially, amounting to pseudo-science at best. Kumho Tire, however, instructs that the reliability of expert testimony does not turn on the grounding of the expert's opinion in scientific principles. Rather, the Supreme Court stressed that a district court has "considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable." Kumho Tire, 526 U.S. at 152, 119 S.Ct. 1167. Moreover, Kumho Tire makes clear that expert testimony on matters of a technical nature or related to specialized knowledge, albeit not scientific, can be admissible under Rule 702, so long as the testimony satisfies the Court's test of reliability and the requirement of relevance. Id. at 149, 119 S.Ct. 1167; see also United States v. Mitchell, 365 F.3d 215, 234 (3d Cir. 2004) (holding that "Kumho Tire extended Daubert's 'general principles' to all of 'the expert matters described in Rule 702' " and applying those principles to determine the admissibility of expert testimony on fingerprint identification). This Court expresses no opinion on whether the practice of firearms and toolmark identification constitutes a "scientific" discipline because that is not the question before the Court. Rather, the Court must consider whether the Government's proffered expert opinion is reliable according to the principles of Kumho Tire. Thus, this Court will analyze the reliability of Babcock's proposed opinion based on the applicable Daubert factors.
1. Whether the Theory Can Be or Has Been Tested
Babcock testified that she employs the theory of toolmark identification adopted by AFTE, the leading international organization for firearms and toolmark examiners. At the August 21, 2012 hearing, the Government offered the testimony of John Murdock, a firearm and toolmark examiner employed at the Contra Costa County Sheriff's Office crime laboratory in Martinez, California. Murdock is the former director of that crime laboratory, as well as a former firearms and toolmark examiner for the Bureau of Alcohol, Tobacco, Firearms, and Explosives crime laboratory in Walnut Creek, California. He has testified extensively in state and federal courts on the subject of firearms and toolmarks. The Court recognized him as an expert in firearm and toolmark identification, as well as the literature in the field, without objection from McCluskey.
The AFTE theory of toolmark comparison permits an examiner to conclude that two bullets or two cartridges are of common origin, that is, were fired from the same gun, when the microscopic surface contours of their toolmarks are in "sufficient agreement." The AFTE theory defines the "sufficient agreement" standard as follows:
This sufficient agreement is related to the significant duplication of random toolmarks as evidenced by a pattern or combination of patterns of surface contours. Significance is determined by the comparative examination of two or more sets of surface contour patterns comprised of individual peaks, ridges and furrows. Specifically, the relative height or depth, width, curvature, and spatial relationship of the individual peaks, ridges and furrows within one set of surface contours are defined and compared to the corresponding features in the second set of surface contours. Agreement is significant when it exceeds the best agreement demonstrated between toolmarks known to have been produced by different tools and is consistent with agreement demonstrated by toolmarks known to have been produced by the same tool. The statement that "sufficient agreement" exists between two toolmarks means that the agreement is of a quantity and quality that the likelihood another tool could have made the mark is so remote as to be considered a practical impossibility.Govt. Ex. 5, Richard Grzybowski, et al., Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards, AFTE Journal, Vol. 35, No. 2 (2003) at 4 (emphasis in original). The theory acknowledges that there is a subjective component to the determination of "sufficient agreement," which must necessarily be based on the examiner's training and experience. Pursuant to the AFTE theory of identification, when an examiner concludes that a particular, individual firearm's toolmarks have produced the markings on an examined bullet or shell, he or she "is basing this opinion on the fact that the nature of toolmark agreement, whether impressed or striated, exceeds the best known non-matching agreement that has ever been personally observed, seen in the literature, or discussed with other examiners." Id. at 5. Though the methodology of comparison and the AFTE "sufficient agreement" standard inherently involves the subjectivity of the examiner's judgment as to matching toolmarks, the AFTE theory is testable on the basis of achieving consistent and accurate results. Id. The literature in the field of firearms and toolmark identification documents that the theory has been repeatedly tested. Industry standard, moreover, dictates that one examiner's findings must be reviewed by another examiner to confirm, or possibly disagree, with those findings. Babcock testified that this process, required for accredited forensic science laboratories, is known as "peer review."
As to this factor in the Daubert analysis, the Court holds that the Government has demonstrated that the AFTE theory is testable and has been tested. Studies have been conducted to test the validity of the theory. The literature shows that the many studies demonstrating the uniqueness and reproducibility of firearms toolmarks have been conducted. See, e.g., Gov't Ex. 50, Ronald G. Nichols, Firearms and Toolmark Identification Criteria: A Review of the Literature, Part I, J. Forensic Sci., Vol. 42, No. 3, 1997, at 466; and Ronald G. Nichols, Firearm and Toolmark Identification Criteria: A Review of the Literature, Part II, J. Forensic Sci., Vol. 48, No. 2 (2003) at 1; Gov't Ex. 5, Grzybowski, et al. at 4. Some of these "validation studies" seek to validate the theory that one can individualize tools, even when comparing marks made by tools of the greatest possible similarity, such as those involved in the consecutive manufacture of various firearms of the same make. James E. Hanby, et al., The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries, AFTE J., Vol. 41, No. 2 (2009) (Gov't Ex. 11) ("Hanby study"). The Hanby study was designed to determine if trained firearm and toolmark examiners could identify 15 "unknown" fired bullets to the correct one of ten consecutively rifled gun barrels. Approximately 500 participants from around the world examined a total of 7,605 "unknown" bullets fired from one of the ten consecutively manufactured firearms. Of these, they correctly identified 7,597 to the correct firearm. Of the remaining eight bullets, three were considered too damaged for identification and the remaining five could not be linked to any of the "known" firearms. The examiners in the Hanby study made no false positive identifications. Id. at 107.
Other studies focus on determining whether distinguishing toolmarks made by firearms are reproducible, such that over time the marks continue to be individualized to a particular firearm. For example, in The Scientific Foundations of Firearms and Tool Mark Identification-A Response to Recent Challenges, 13-14, CACNews 2nd Quarter 2006 (Gov't Ex. 6), author Ronald Nichols discusses various studies in which hundreds or even thousands of bullets were fired from the same gun. In each case examiners found that, while some changes were observed over the course of the numerous firings, it was nonetheless possible to identify the first and last bullet (or the first and last cartridge casing) as having been fired from the same gun. Id. Admittedly, these do not appear to have been "blind" studies; rather, the examiners who found that an identification was possible knew the bullets had all been fired from the same gun. Nonetheless, these kinds of results do suggest at least some level of reproducibility.
Furthermore, Murdock testified that industry standards generally require an examiner to document in detail, through note-taking and photographs, the basis for his or her findings. Tr. 8/21/12, Doc. 676, at 121, 137-39. Murdock also testified that industry standards require confirmation by at least one other examiner when the first examiner reaches an identification. Id. at 12, 138, 204-05. See also United States v. Monteiro, 407 F.Supp. 2d 351, 368 (D. Mass. 2006) (noting same); United States v. Diaz, No. CR 05-167, 2007 WL 485967, at *5 (N.D. Cal. Feb. 12, 2007) (noting same). These factors, too, indicate at least some significant level of testability and reproducibility.
The literature, together with Murcock's testimony regarding validity studies and the peer review process of confirming identifications, demonstrates that the AFTE firearms and toolmark identification theory is testable and has been tested. This factor weighs in favor of admission of the evidence.
2. Peer Review and Publication of AFTE Theory
In Daubert, the Supreme Court stated that "publication in a peer reviewed journal [is] a relevant, though not dispositive consideration in assessing the validity of a particular technique or methodology on which an opinion is premised." Daubert, 509 U.S. at 594, 113 S.Ct. 2786. As Murdock explained, the Association of Firearm and Toolmark Examiners ("AFTE"), the principal professional organization for firearms and toolmark examiners, publishes a peer-reviewed journal, the AFTE Journal. Studies testing the AFTE theory are subject to peer review through submission to and publication by the AFTE Journal. The AFTE Journal publishes articles, studies and reports concerning firearm and toolmark evidence. It has a formal process for the submission of articles, including "specific instructions for writing and submitting manuscripts, assignment of manuscripts to other experts within the scientific community for a technical review, returning of manuscripts to authors for clarification or re-write, and a final review by the Editorial Committee." Gov't Ex. 5, Grzybowski, et al., at 12. There is also a formal post-publication peer review process, allowing AFTE members and any other interested individuals to comment on previously published articles. Id. Most of the validation studies discussed in this opinion, which the Government has submitted to demonstrate the testability and viability of AFTE identification theory, were published in the AFTE Journal. In addition, the Journal of Forensic Sciences publishes peer-reviewed articles on toolmarks and firearm identification. Id. The Court thus concludes that the Government has presented sufficient evidence in support of this factor such that it weighs in favor of admissibility.
3. Known or Potential Rate of Error
Daubert directs that, "in the case of a particular scientific technique, the Court ordinarily should consider the known or potential rate of error." 509 U.S. at 594, 113 S.Ct. 2786. McCluskey argues that, "because the process [toolmark identification] is so subjective and qualitative, it 'is not possible to calculate an absolute error rate for routine casework.' " Doc. 418 at 83 (quoting Monteiro, 407 F.Supp. 2d at 367 (quoting Richard Grzybowski et al., Firearm/Toolmark Identification: Passing the Reliability Test Under Federal and State Evidentiary Standards, 35 AFTE J. 209, 213 (2003)).
Commenting on other researchers' analyses of data supplied by the Collaborative Testing Service ("CTS") on international proficiency testing in the firearm and toolmark identification discipline, Grzybowski's 2003 AFTE Journal article (Gov't Ex. 5) summarized error rate calculations derived from the CTS proficiency testing results. He reported that CTS data for the period 1978 to 1997 (firearms) and 1981 to 1997 (toolmarks) demonstrated that false-identification error rates were 0.6% for firearms and 1.5% for toolmarks, and for the period 1978 to 2002, were 1.0% for firearms and 1.3% for toolmarks. Grzybowski at 9. These percentages, he noted, do not include false eliminations or an examiner's determination of "inconclusive." Grzybowski's article also points out that there are other limitations to using the proficiency testing data to calculate an error rate, such as the inherent motivation in such a test for the examiner, whose proficiency is being evaluated, to tend toward conservative results and the lack of peer review or other quality control measure applied to such examinations. Murdock's testimony was in accord. Tr. 8/21/12, Doc. 676, at 186-188.
The error rate reported by the Grzybowski article is, moreover, consistent with Murdock's testimony, that based on his knowledge of validation studies the error rate, defined as the percentage of false positive identifications, hovered around 1 to 2%. Id. at 105-06. However, Murdock acknowledged that these studies were based on tests of firearm examiners asking them to evaluate relatively pristine bullets and casings, and therefore the tests were much easier than real-world forensic ballistic examinations, which often involve bullets and casings that are dirty or damaged. Id. at 100-01. Furthermore, the tests were not blind insofar as the examiners knew they were taking a test rather than performing a "real world" forensic examination. Id. at 61-63, 100. Murdock identified further problems with error rates derived from the results of the proficiency testing program administered by the Forsensic Sciences Foundation in the chapter he co-authored in a treatise on modern forensics. See Gov't Ex. 16, Ch. 35, Firearms and Toolmark Identification, Modern Scientific Evidence, Vol. 4, Ch. 35 (West 2009), at 691-92. The suggestion, then, is that errors by examiners working in the field are probably higher. Murdock testified that a study done in Europe that mimics the imperfect samples often found in the field yielded an error rate of around 5%. Tr. 8/21/12, Doc. 676, at 101, 189-90, 210.
In light of all the foregoing, the Court concludes that insufficient data exists to calculate a definitive error rate. The information that does exist—derived from the proficiency testing— indicates an error rate of 5% or less, which is not excessively high. However, that number is subject to valid criticism. Thus, the evidence presented on error rates leads the Court to conclude that this Daubert factor weighs slightly in favor of admitting the challenged expert testimony.
4. Existence and Maintenance of Standards Controlling the Technique's Operation
As discussed above, the AFTE Theory contains considerable subjectivity in determining the existence of a match. This is perhaps McCluskey's most strident objection to the admission of the firearms identification evidence. As Judge Johnson of this Court recognized:
The AFTE Theory, thus, does not provide any uniform numerical standard examiners can use to determine whether or not there is a match and, indeed, Mr. Nichols indicated in his testimony that most AFT examiners do not use any numerical standard. Instead, the AFTE theory is circular. An examiner may make an identification when there is sufficient agreement, and sufficient agreement is defined as enough agreement for an identification. The conclusion that there is a match between a recovered bullet and a particular gun is, therefore, necessarily a subjective one, held in the mind's eye of the examiner and based largely on training and experience in observing the difference between known matching and known non-matching impression toolmarks.United States v. Taylor, 663 F. Supp. 2d 1170, 1177 (D.N.M. 2009) (internal citations and quotations omitted). And as another federal district court observed, the comparison of test bullets and cartridges to those of unknown origins involves "the exercise of a considerable degree of subjective judgment." United States v. Glynn, 578 F.Supp.2d 567, 573 (S.D.N.Y. 2008). The Glynn court pointed out that because "[t]he bullets and/or shell casings recovered from the crime scene may be damaged, fragmented, crushed, or otherwise distorted in ways that create new markings or distort existing ones," an examiner must rely on his or her experience to distinguish the undistorted toolmarks from other markings. Id. That process is made more complicated by the fact that comparison microscopes "produce flat images despite the fact that a bullet or casing is round, thereby producing distortions not unlike those on a map showing the globe." Id. at 573, n. 11. See also United States v. Monteiro, 407 F.Supp.2d 351, 362-63 (D. Mass. 2006) (quoting Grzybowski et al., supra, at 213).
Despite its inherent subjectivity, the AFTE theory of firearms-related toolmark identification, which has as its primary objective the determination of whether "sufficient agreement" exists between examined bullets or cartridges to enable a toolmark examiner to conclude that there is a "match," has been generally accepted within the field of toolmark examiners. Furthermore, the AFTE training courses and CTS proficiency testing (with all of its limitations) demonstrate the existence of standards governing the methodology of firearms-related toolmark examination to enable a properly trained examiner to provide in-court technical testimony that will be sufficiently reliable and helpful to a lay jury to assist the jurors in determining whether bullets or cartridges have been fired from a particular firearm.
5. General Acceptance of the Theory
Courts have observed that the AFTE theory of firearms and toolmark identification is widely accepted in the forensic community and, specifically, in the community of firearm and toolmark examiners. See United States v. Diaz, No. CR 05-167 WHA, 2007 WL 485967, at *11 (N.D. Cal. Feb. 12, 2007). Even courts which have criticized the bases and standards of toolmark identification have nevertheless concluded that AFTE theory and its identification methodology is widely accepted among examiners as reliable and have held the expert identification evidence to be admissible, albeit with limitations. United States v. Taylor, 663 F. Supp.2d 1170, 1178 (D.N.M. 2009); United States v. Monteiro, 407 F.Supp.2d 351, 372 (D. Mass. 2006); United States v. Green, 405 F.Supp.2d 104, 122-24 (D. Mass. 2005). This Court also concludes that the evidence submitted by the Government demonstrates the general acceptance of the AFTE theory among both professional examiners and federal courts as a reliable method of firearms and toolmark identification.
6. Affidavit of Adina Schwartz
The Court has considered the views offered by Professor Adina Schwartz on the admissibility of firearm identification, and it is not persuaded that they cast doubt on the reliability or admissibility of Babcock's testimony. Dr. Schwartz is an associate professor in the Department of Law, Police Science and Criminal Justice Administration at John Jay College of Criminal Justice and in the Ph.D. Program in Criminal Justice of The Graduate School and University Center, City University of New York (CUNY). To begin with, Professor Schwartz's academic background does not appear to be one that would have prepared her to analyze the reliability of toolmark identification. Her curriculum vitae (Deft. Ex. B) indicates that she received a doctorate in philosophy from Rockefeller University, which is an institution dedicated to biomedical research. See also United States v. Otero, 849 F. Supp. 2d 425, 435-36 (D.N.J. 2012) ("if the issue before the Court were whether or not toolmark identification procedures satisfy scientific standards, her academic background would not appear to give her any particular expertise in whether the discipline adheres to scientific precepts. Moreover, her admitted lack of experience in participating in any firearms identification examination or study concerning firearms-related toolmarks further limits any claim of expertise in the field of firearms and toolmark analysis."). Federal Rule of Evidence 702 does permit one to be qualified as an expert through various means, including study of the literature relating to a field. Based on the record currently before the Court, it appears that Professor Schwartz is well versed in the relevant literature. However, it also appears that she has analyzed the literature not as an objective analyst, but rather as an advocate for the non-admission of ballistics evidence. This bias tends to undermine her credibility. Other courts have found similarly. See United States v. Otero, 849 F. Supp. 2d 425, 435-36 (D.N.J. 2012) (citing cases). Furthermore, the accuracy and honesty of Schwartz's scholarly analysis has been questioned by this Court. In United States v. Taylor, Judge Johnson granted the Government's motion to exclude Professor Schwartz's expert testimony, finding, among other things, that she lacked qualifications to critique the findings of the government's firearms examiner and that her testimony was not reliable under Rule 702. See United States v. Taylor, 704 F.Supp.2d 1192 (D.N.M. 2009). The court pointed out "serious criticisms, not just of Dr. Schwartz's conclusions, but of the integrity of her scholarship," and it concluded that she was not qualified to give an expert opinion regarding the firearms examinations that had been conducted in that case. Id. at 1200. See also Otero, 849 F. Supp. 2d at 436-37. In light of the foregoing, the Court gives little weight to the criticisms of the AFTE method of firearm examination discussed herein.
Defendant offered Dr. Schwartz' opinions through her affidavit and through articles she has written on the subject. She did not testify at the hearing.
7. Other Issues
As Judge Johnson pointed out in Taylor, 663 F. Supp. 2d at 1178-79, there is an issue with firearms examination, not necessarily neatly encapsulated by any one of the Daubert factors. Generally, as was done in this case, the examiner is handed only one or a handful of suspect weapons and the recovered projectiles or cartridges. As one district court has pointed out, this method of testing is, "in effect, an evidentiary 'show-up,' not what scientists would regard as a 'blind test.' " United States v. Green, 405 F. Supp. 2d at 104, 107-08 (D. Mass. 2005). Indeed, McCluskey's unrefuted argument is that "neither the firearm community, specifically, nor the forensic science community, generally, have ever conducted double-blind independent proficiency tests aimed at determining the accuracy (or inaccuracy) of firearm examiners." See Doc. 418 at 83 (citing Alfred Biasotti & John Murdoch, 3 Modern Scientific Evidence 143, 217-18 (David L. Faigman et al. eds. 1997)). This practice, like show-up identifications of suspects, creates a potentially significant "observer effect" whereby the examiner knows that he is testing a suspect weapon and may be predisposed to find a match. See Doc. 418 at 40.
There is also a question as to what degree of certainty Babcock may offer the jury regarding her conclusions. Murdock testified that identifications made using the AFTE theory are made to a "practical certainty," not to an absolute certainty because firearms examiners are unable to compare a bullet or casing to every gun in the world. Tr. 8/21/12, Doc. 676, at 52-53. However, in her report (Deft's Ex. C) Babcock stated her conclusions without any limitations as to certainty. For example, she states that "item G13 fired cartridge case is identified as being fired in the item 1B22 pistol." However, in her testimony at the hearing Babcock acknowledged that under the most recent standards in her profession, she must testify only that her conclusions are made with "practical certainty." Tr. 8/23/2012, Doc. 677, at 40-41.
Increasingly, courts have recognized this issue and have limited the manner in which firearms examiners may testify, requiring them to explicitly inform the jury that they cannot make ballistics findings with absolute certainty. In Glynn, the court ordered that ballistics opinions could be stated in terms of 'more likely than not,' but nothing more. 578 F. Supp. 2d at 574-75. Similarly, in United States v. Diaz, the court allowed the firearms identification expert testimony, but ordered that "[t]he experts may not ... testify to their conclusions 'to the exclusion of all other firearms in the world.' They may only testify that a particular bullet or cartridge case was fired from a firearm to a 'reasonable degree of certainty in the ballistics field.' " 2007 WL 485967, at *14. In Monteiro, the court concluded that "the expert may testify that the cartridge cases were fired from a particular firearm to a reasonable degree of ballistic certainty. However, the expert may not testify that there is a match to an exact statistical certainty." 407 F. Supp. 2d at 375. In Green, the court held that "[W]hile I will allow [the firearms expert] to testify as to his observations, I will not allow him to conclude that the match he found by dint of the specific methodology he used permits 'the exclusion of all other guns' as the source of the shell casings." 405 F. Supp. 2d at 124. Finally, in Taylor the court held that
[B]ecause of the limitations on the reliability of firearms identification evidence discussed above, Mr. Nichols will not be permitted to testify that his methodology allows him to reach this conclusion as a matter of scientific certainty. Mr. Nichols also will not be allowed to testify that he can conclude that there is a match to the exclusion, either practical or absolute, of all other guns. He may only testify that, in his opinion, the bullet came from the suspect rifle to within a reasonable degree of certainty in the firearms examination field.663 F. Supp. 2d at 1180. In this case, Murdock testified that the phrase "reasonable degree of ballistic certainty" is undefined and that even he had no idea what it meant. Tr. 8/22/2012, Doc. 679, at 33. The Court therefore concludes that Babcock may testify that she has reached her conclusions to "a practical certainty," or to a "practical impossibility" of dissimilar origin, and nothing more.
Finally, the Court agrees with McCluskey that Babcock may not testify as to inconclusive results, such as her statements in her report that neither fired casing G7 nor G11 could be identified or eliminated as having been fired in pistol 1B22. To be admissible as relevant evidence under the Federal Rules of Evidence, evidence (expert or otherwise) must make a fact that is of consequence to the determination of the action more or less probable than it would be without the evidence. Fed. R. Evid. R. 401. Because such inconclusive findings do no make a fact more or less probable than it would be without them, they are not relevant may not be admitted into evidence.
These are intended as examples, not as an exhaustive list.
--------
8. Conclusion
Based upon the evidence before the Court, it finds that when a bullet is fired from a gun, the gun will impart to the bullet a set of markings that is, at least to some degree, unique to that gun. The evidence further indicates that using a comparison microscope, a trained and experienced firearms examiner can make observations of those markings, using a method that has been peer-reviewed, that allow him, in some cases, to form an opinion that a particular bullet or casing was or was not fired from a particular gun. The Court therefore concludes that the firearms identification testimony is admissible under Rule 702 and Daubert.
IV. ADMISSIBILITY OF BABCOCK'S TESTIMONY
Having concluded that expert testimony on toolmark identification is admissible under Daubert and Kumho Tire, the Court must next determine whether Babcock's testimony is admissible based on her qualifications and methodology.
First, the Court concludes that Babcock qualifies as an expert under Rule 702 by her training, education, and experience in the field of firearm identification. Babcock began working at NMDPS in 1999, after she graduated from the University of New Mexico with a bachelor's degree in criminology. Initially, she did written correspondence course work in firearms identification. In 2000, she attended the Bureau of Alcohol, Tobacco, and Firearms National Firearms Examiner Academy in Rockville, Maryland for one year. After completing the ATF program, Babcock returned to NMDPS' northern forensics laboratory, where she did casework with a trainer for several months. She then took and passed the AFTE certification test in firearms evidence and examination, making her one of only approximately 90 members who are AFTE certified. Since that time, Babcock has continued to attend classes, workshops, and seminars in firearms identification. She currently is a supervising forensic scientist at NMDPS' forsensics laboratory, where she oversees the day-to-day operations in the firearms unit of the lab. Her primary responsibility is to conduct casework, supervise other forensic scientists in their work, perform technical and administrative reviews of the forensics work of others, and ensure quality assurance in the lab. She has done firearms examination work in 635 cases. See generally Govt. Ex. 20 and Transcript 8/22/12 (Doc. 679) at 153-161. Furthermore, Babcock has testified as an expert on firearm and toolmark identification approximately 20 times. Tr. 8/23/12, Doc. 677, at 7. The Court notes that Defendant did not challenge Babcock's qualifications as an expert in the area of firearms and toolmark identification, either in his written motion or at the hearing. Based on all the foregoing, the Court concludes that Babcock is qualified to offer expert testimony on firearms and toolmark identification in this case.
Next, the question is whether Babcock's conclusions are based on sufficient facts or data. The facts and data that are the basis of her conclusions are set forth in her five-page report, Deft's Ex. C, as well as over 109 pages of Babcock's testing notes, results, worksheets, and photographs, including side-by-side comparison photographs. Govt. Ex. 15, Deft. Ex. 15. These documents describe and depict the evidence that Babcock examined, the comparisons she conducted, and the conclusions she reached. In some cases, there are brief notes that explain Babcock's thought process as she reached certain conclusions. The Government provided these materials to McCluskey in February of 2012. In her testimony on August 22-23, 2012, Babcock described in more detail the physical qualities of the evidence that led her either to make identifications or exclusions, or to opine that other comparisons were inconclusive. Based on all the foregoing, the Court concludes that Babcock's conclusions are based on sufficient facts or data.
Finally, the Court must ask whether Babcock reliably applied the AFTE theory of identification in her firearm identification analysis work in this case. In his written motion, McCluskey did not challenge Babcock's application of the AFTE methodology, only the principles of the methodology itself. This was due at least in part, it seems, to McCluskey's position that at the time he filed his motion the Government had not provided him with adequate materials to understand Babcock's application of the AFTE theory to the evidence in this case. Those issues are discussed in Part I of the Discussion, supra. However, since filing his motion, McCluskey has had the opportunity to hear Babcock describe her work and her conclusions on the witness stand, and also to cross examine her about them. At no time after the conclusion of her testimony on August 23, 2012 has McCluskey elected to supplement his motion with an explanation of his objection to Babcock's methodology. As a result, the Court concludes he has no such objection.
Nevertheless, the Court has reviewed Babcock's case file and her testimony at the hearing. That review leads the Court to conclude that her application of the AFTE methodology to the evidence in this case was reliable. Babcock explains in detail how she received, cleaned, and examined each piece of evidence. Her notes and testimony then explain the comparisons she did under microscope, and the file contains side-by-side photographs of the evidence she compared. Finally, a second firearms examiner, Alina Sanchez, performed a technical peer review of Babcock's work. The Court finds nothing in those materials to indicate that Babcock did not reliably apply the AFTE method of identification to her work in this case. Thus, her testimony will be admitted into evidence at trial, though subject to the limitations described above.
IT IS THEREFORE ORDERED that, subject to the limitations discussed herein, Defendant John McCluskey's Motion to Exclude Firearm Identification Evidence, and Request for Daubert Hearing [Doc. 418] is DENIED.
____________________
UNITED STATES DISTRICT JUDGE