U.S. v. Frabizio

Full title: UNITED STATES OF AMERICA, v. RUDY FRABIZIO, Defendant

Court: United States District Court, D. Massachusetts

Date published: Aug 11, 2006

445 F. Supp. 2d 152 (D. Mass. 2006)

Opinion

Crim. No. 03-10283-NG.

August 11, 2006.

Seth P. Berman United States Attorney's Office, Boston, MA, Dana M. Gershengorn, U.S. Department of Justice, Washington, DC, for Plaintiff.

Miriam Conrad, Federal Defender's Office, Boston, MA, for Defendant.

MEMORANDUM AND ORDER RE: MOTION TO EXCLUDE EXPERT TESTIMONY

GERTNER, District Judge.

I. INTRODUCTION

The government has charged defendant Rudy Frabizio ("Frabizio") with possession of child pornography under the Child Pornography Prevention Act ("CPPA"), 18 U.S.C. § 2252(a)(4)(B). To obtain a conviction under § 2252(a)(4)(B), the Supreme Court has held that the government must prove the defendant possessed images of real children, rather than images of virtual children, engaging in sexually explicit conduct. Ashcroft v. Free Speech Coalition, 535 U.S. 234 (2002); United States v. Hilton, 386 F.3d 13, 18 (1st Cir. 2004). The task is a difficult one, becoming more difficult by the minute, as digital imaging techniques become increasingly sophisticated.¹

1 For example, modern computer technology has enabled graphic artists to manipulate images by air-brushing photographs, cutting and pasting from one image onto another, changing colors, stretching shapes, and even creating images from scratch on the computer.

To meet its burden of proof, the government seeks to introduce the testimony of Thomas Musheno ("Musheno"), a forensic examiner of photographic evidence in the FBI's Forensic Audio, Video, and Image Analysis Unit. Specifically, the government offers testimony that, after visually examining the photographs Frabizio allegedly possessed, and without using any specialized equipment, Musheno concluded that six of the nineteen images definitely depict real children and ten others "appear to be" real children.²

2 Musheno offers testimony that is the polar opposite of the type of expert testimony the government first proposed. The government initially offered Professor Hany Farid, a Dartmouth College professor of computer science and neuroscience. Professor Farid sought to distinguish real and computer-generated images through a computer, rather than using visual inspection. Farid's computer program purported to measure statistical consistencies within photographs and computer-generated images to determine whether or not an image was real.
After one day of a hearing, the government withdrew Dr. Farid as an expert witness. Defense counsel noted that 30 percent of the time, Farid's program classified a photograph [i.e. a real image] as a computer-generated image, and she highlighted these errors. One stood out in particular: an image of a cartoon character, "Zembad," a surrealistic dragon, falsely labeled "real."

Frabizio moves [docket entry # 87] to exclude the proffered testimony under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), on the grounds that Musheno's technique is unreliable because it has not been tested, has not been subjected to independent peer review, has an unknown error rate, has no real standards or controls, has not achieved general acceptance, and satisfies no other set of reasonable reliability criteria. See Daubert, 509 U.S. at 593-94; Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 158 (1999). Accordingly, I held a hearing to determine the admissibility of Musheno's proffered testimony on May 5, 6, and 11, 2005. See Daubert, 509 U.S. 579, 113 S.Ct.2786, 125 L.Ed.2d 469.

After the hearing, however, I noted a far more fundamental issue lurking beneath the questions about Musheno's particular methodology or error rate. However skilled he may be, he bases his testimony on observations of the photographs. The threshold question is whether visual observation is at all appropriate to the task at hand: distinguishing real images from virtual ones.

If it is possible to distinguish the real from the virtual with the naked eye, then the specialized observational skills of aDaubert-qualified photograph expert could help the jurors make their own observations of the evidence. As such, the photograph expert's testimony might be admissible, assuming Daubert's other requirements were satisfied. See United States v. Shay, 57 F.3d 126, 132 (1st Cir. 1995) (noting that Federal Rule of Evidence 702 requires that, to be admissible, expert testimony must "assist the trier of fact").

On the other hand, if visual observation, even by a seasoned observer, cannot distinguish real and computer-generated images in this case, then observation alone fails to address the threshold question and is therefore irrelevant. The testimony of a photograph expert would be inadmissible; an individual with expertise in computer-generated images would be required.

The First Circuit has left this issue open. In the instant case, I am obliged to resolve it. Based on the evidence presented to me about the current state of technology and the specific images involved here, I conclude that neither an expert witness nor a lay jury, using only visual means, can determine whether the images in this case are real or virtual to the level of certainty required in a criminal prosecution. I therefore find that Musheno's testimony is not helpful and must be excluded.³II. THE STATE OF TECHNOLOGY AND VISUAL OBSERVATION OFIMAGES

3 This opinion could be construed to indicate that Musheno's testimony alone would not be sufficient to meet the government's burden at trial to prove the pictures are real. Given the current posture of this case, this Court does not reach the issue of sufficiency of the evidence. Instead, I find that because it may be impossible to evaluate the images in this case based only on visual observation, Musheno's testimony, standing alone, is inadmissible.
To be sure, if the government were to offer a computer expert whose testimony arguably eliminated the possibility that the images in this case were wholly computer-generated, then I would reconsider the admissibility of Musheno's testimony. In that case, Musheno's testimony would be relevant to determining the types of manipulated images involved — for example, whether the images depict real children in pornographic poses, as opposed to real adults air-brushed to look like children or real children manipulated into pornographic poses.
Obviously, even if another expert excluded the possibility that the images are wholly virtual, Musheno would still have to meet the Daubert standard. He cannot. His methodology is not adequate to support the conclusions he so confidently offers; that the images downloaded from the internet are real children or appear to be real children. I would only allow Musheno to testify as an additional expert (supplementing a computer expert), subject to the restrictions described in Section III,infra.
On the current record, however, with Musheno offered as the government's sole expert, his testimony is not adequate to answer the question before the jury: whether images are real or virtual. It is therefore inadmissible in its entirety.

Because the threshold question in determining the admissibility of Musheno's testimony is whether it is possible to create wholly computerized images that are visually indistinguishable from real photographs, my analysis begins here.⁴ This question raises both factual and legal issues. The factual inquiry involves the capabilities of modern computer programs as well as the particular images in the case at bar. The legal question concerns how much uncertainty about the efficacy of visual observation can be tolerated in a criminal case, where the burden of proof weighs heavily on the government.

4 This preliminary issue is key because the Daubert inquiry depends upon the particular task to which Musheno's testimony is directed. As the Supreme Court explained in Kumho Tire, the reasonableness of any given expert's approach depends upon "the particular matter to which the expert testimony [is] directly relevant." Kumho Tire, 526 U.S. at 154. See also, D. Michael Risinger, "Navigating Expert Reliability: Are Criminal Standards of Certainty Being Left on the Dock?" 64 Alb. L.Rev. 99, 99 n. 2 (2000) (observing that Kumho Tire stands, in part, for the proposition that the Daubert inquiry must be made "in regard to the particular `task at hand'"). The task at hand here is excluding, as a threshold matter, the possibility that the images are wholly virtual.

After three days of hearings and multiple briefings on theDaubert issue, I have serious doubts as to whether a person visually studying the images in this case can distinguish real pictures from manipulated or wholly virtual ones with the level of confidence required in a criminal prosecution.

The Supreme Court acknowledged this difficulty two years ago inFree Speech Coalition, even as it held that the government must prove that the images are real:

[T]he Government says that the possibility of producing images by using computer imaging makes it very difficult for it to prosecute those who produce pornography by using real children. Experts, we are told, may have difficulty in saying whether the pictures were made by using real children or by using computer imaging. The necessary solution, the argument runs, is to prohibit both kinds of images. The argument, in essence, is that protected speech may be banned as a means to ban unprotected speech. This analysis turns the First Amendment upside down.

The Government may not suppress lawful speech as the means to suppress unlawful speech. Protected speech does not become unprotected merely because it resembles the latter.

Ashcroft v. Free Speech Coalition, 535 U.S. at 254-55.

Although several circuits have held that expert testimony and extrinsic evidence are not required to meet the government's burden of proof, the First Circuit is not among them. See, e.g., United States v. Irving, 432 F.3d 110, 27-28 (2d Cir. 2006) ("We decline to hold the government is required . . . to present expert testimony proving the children in the unlawful images are in fact real children. . . . [A] reasonable jury could conclude that the images depicted real children solely on the basis of the images themselves," but limiting this holding to cases involving video images); United States v. Kimler, 335 F.3d 1132, 1142 (10th Cir. 2003) cert. denied,Kimler v. United States, 540 U.S. 1083 (2003) ("Juries are still capable of distinguishing between real and virtual images."); United States v. Slanina, 359 F.3d 356, 357 (5th Cir. 2004).

When the First Circuit first visited this issue in United States v. Hilton, the court held that the "government must introduce relevant evidence in addition to the images to prove the children are real." 363 F.3d 58, 64 (1st Cir. 2004), vacated, 2004 U.S. App. LEXIS 19528 (1st Cir. 2004) (emphasis added). Further, the court acknowledged that the "vast technological revolution underway . . . has made undeniable the fact that sexually explicit images portraying children can be produced by artificial means." Id. at 64-65. More recently, the court again suggested that this question has yet to be resolved in this circuit:

In the First Circuit, it remains an open question whether, to prove the reality of an image, the government must present relevant evidence in addition to the image. The decisions of other circuit courts of appeals addressing this question are not binding in this court.

United States v. Rodriguez-Pacheco, CA No. 05-1815, Order entered April 4, 2006 (internal quotation and citations omitted).

The government has previously conceded the difficulty of distinguishing real from virtual or manipulated images. In its brief in the Free Speech Coalition case before the Supreme Court, the government acknowledged Congressional findings on digital imagery:

Congress found that `new photographic and computer [imaging] technologies make it possible to produce * * * visual depictions of what appear to be children engaging in sexually explicit conduct that are virtually indistinguishable to the unsuspecting viewer from unretouched photographic images of actual children engaging in sexually explicit conduct.'

Brief for the Petitioners at 4, Ashcroft v. Free Speech Coalition, 535 U.S. 234 (2002) (No. 00-795), 2001 WL 432538 (citing 18 U.S.C. § 2251 note (Supp. V 1999) (Finding 5)).

Even Musheno, the government's supposed-expert in this case, acknowledged that a trained artist might be able to create realistic-looking images. He testified: "I don't know if I've ever seen an image, a painting that is so photorealistic that it appears to be real, although is it possible to generate that? I guess it's possible."⁵ (Daubert Hr'g Tr. 20, 37-38, May 11, 2005.)

5 Musheno also testified that he cannot determine whether a single image is real because "it is possible to generate imagery or to manipulate imagery that's undetectable." (Daubert Hr'g Tr. 29, May 11, 2005.) However, he believed that if he looked at a series of images of the same person, he could detect manipulation because the difficulty of manipulating images in a way that is not obvious increases when there are more pictures. (Daubert Hr'g Tr. 25, May 6, 2005.) Although he may be able to detect manipulation of an actual photograph by reviewing multiple images, I conclude, as he grudgingly conceded, that he could not detect an image wholly generated on the computer. Moreover, even Musheno's judgment that he can detect manipulation is not meaningfully tested. See Section III, below.

A significant body of literature also indicates that digitally manufactured images may be confused with real photographs. Faculty in the Department of Computer Science at Dartmouth College, for example, have noted "photorealistic images can be created that are nearly impossible to differentiate from photographic images." S. Lyu and H. Farid, "How Realistic is Photorealistic?" 53(2) IEEE Transactions on Signal Processing (2005) (available at http://www.cs.dartmouth.edu/farid/publications).⁶ Other articles suggest that such virtual image creation can be achieved using current technology and that even "experts cannot know whether a digital image is real or virtual." Timothy J. Perla, Attempting to End the Cycle of Virtual Pornography Prohibitions, 83 B.U. L.Rev. 1209, 1216 (2003). See also, A.C. Popescu and H. Farid, "Exposing Digital Forgeries by Detecting Traces of Re-Sampling," 53(2)IEEE Transactions on Signal Processing (2005) (available athttp://www.cs.dartmouth.edu/farid/publications) ("[D]igital images can be easily manipulated and altered. Digital forgeries, often leaving no visual clues of having been tampered with, can be indistinguishable from authentic photographs"); Caught On Camera, NEW SCIENTIST, Sept. 6, 2003 at 5 ("Warnings about the potential for faking digital images are not new. But the proliferation of cheap digital cameras and computers, together with programs for altering photos and editing video footage, is turning that potential into reality. Where once a specialist was needed to alter analogue images, even beginners can now create digital fakes good enough to fool discerning experts."). But see Susan S. Kreston, Defeating the Virtual Defense in Child Pornography Prosecutions, 4 J. High Tech. L. 49, 62 (2004) ("Creating realistic images of people . . . continues to be very difficult, with the difference between a real picture and one created by a computer, even using today's best technology, being discernable to the human eye.").

6 Notably, the co-author of this article, Professor Hany Farid, is the same computer expert whom the government initially offered to prove the photographs were real. See discussion of Farid's testimony in note 1, supra.

In addition, the defendant has submitted examples of wholly-computer-generated images that this Court finds to be nearly indistinguishable from real photographs. See, e.g., http://forums.cgsociety.org/showthread.php?t=141461; (depicting a nude woman in the fetal position);http://forums.cgsociety.org/showthread.php?t=361465 (showing a remarkable likeness of actress Jennifer Garner); see also http://forums.cgsociety.org/showthread.php?t=160012 (revealing a woman's nude back). Although this final image shows greater signs of being computer-generated than the other two, the fact that someone created an image of this quality in less than a week using Photoshop suggests the range of technological possibilities.

In this case, the government has not made the threshold showing that a visual observer can reliably evaluate the relevant pictures for signs of manipulation and computer-generation. Rather, the evidence strongly suggests that it is extremely difficult, if not impossible, for a photographic expert, let alone a lay observer, to determine whether the images involved in the instant case are real images or images created or manipulated through digital technology.

To the extent doubt remains on this threshold issue, I must resolve that doubt against the government. This is a criminal case. Defendant's liberty hangs in the balance, and the government bears the burden of proof.⁷See United States v.Monteiro, 407 F. Supp. 2d 351, 356 (D. Mass. 2006) (noting in the Daubert context that the "burden of proof with respect to reliability remains on the proponent of the evidence"). Furthermore, as the First Circuit recently held, the defendant has no obligation to refute the reality of the images. Rather, it is the government's burden to establish that the pictures are real:

7 Although the rules of evidence apply equally in civil and criminal cases, the Court must be especially vigilant in applying evidentiary rules in the criminal context, given the stakes for the defendant and the fact that, ultimately, the government bears the burden of proof beyond a reasonable doubt. See, e.g., Barefoot v. Estelle, 463 U.S. 880, 916 (1983) (Blackmun, J., dissenting) (suggesting that expert testimony allowed in a civil case should not necessarily be allowed in a capital case because "when a person's life is at stake . . . a requirement of greater reliability should prevail"); Michael H. Gottesman, "Scientific and Technological Evidence: Admissibility of Expert Testimony after Daubert: The `Prestige Factor,'" 43 Emory L.J. 867, 877 (Summer, 1994) (observing that "[t]he standards for admitting expert opinion testimony must be calibrated" to the relevant standard of proof). See also, Bonnie J. Davis, "Admissibility of Expert Testimony After Daubert and Foret: A Wider Gate, A More Vigilant Gatekeeper," 54 La. L.Rev. 1307, 1333-34 ("The prejudicial effect against the defendant of expert testimony presented by the prosecution is very high, and thus this testimony should be very reliable before it could pass the balancing test" for admissibility).

It bears repeating that the government is not released from its burden of proof by a defendant's failure to argue, or by an absence of evidence otherwise suggesting, the artificiality of the children portrayed. That the children in the images are real amounts to an element of the crime which the government must prove, the burden of which should not be displaced to the defendant as an affirmative defense.

United States v. Hilton, 386 F.3d 13, 18 (1st Cir. 2004). Similarly, it is the government's burden to establish that the naked eye could distinguish the real from the virtual. It has not met that burden here.

I therefore conclude that visual observation is not adequate to the task of differentiating the real from the virtual in this case. My conclusion leads to two others: First, jurors cannot decide based on their own visual observations whether the images allegedly possessed by the defendant depict real children, as required by the Supreme Court's holding in Free Speech Coalition. Some expert testimony is required.⁸ Secondly, even a seasoned observer, such as an expert in photography, cannot make this distinction in the case at bar. Given current computer technology and the images involved here, visual observation, by either a jury or an expert, cannot identify which images depict real children and which have been manipulated or generated by computer.⁹ An expert with greater knowledge of computers must speak to that fundamental, threshold issue. III. MUSHENO'S PROFFERED TESTIMONY

8 My conclusions are, of course, limited to the images in this case. There may be cases in which the images involve crude manipulations of a child's image that do not require a computer expert's testimony, but this case is not among them.

9 This case began in 2003, and technology has undoubtedly advanced since then. Even based on evidence from 2003, however, I find that the government has not met its burden to show that it is possible to visually distinguish between real and virtual images.

As discussed above, I find that visual observation alone is inadequate to the task of evaluating the images in this case. If photographic experts as a general matter are inadequate to the task of identifying computer-generated images, then no level of experience in that field will suffice to qualify one as an expert. Indeed, allowing Musheno to testify would be like allowing a dentist to identify the causes of glaucoma.

If the government offered an expert who eliminated the possibility of such imagery in this case, then Musheno's testimony might be admissible.¹⁰ To address that possibility and to underscore the inadequacies of Musheno's technique as applied to the sophisticated task at hand, I will consider the application of Daubert to Musheno's proffered testimony.

10 That is, Musheno's testimony might be relevant to the question of whether the images were real photos that had been manipulated. See note 1, supra.

Even if the government established that the images here were not wholly virtual, Musheno's methodology does not meet theDaubert standard. His technique amounts to little more than "eyeballing" the evidence and recording his conclusions on a checklist. Indeed, the Daubert discussion below underscores the limitations of Musheno's approach, especially with respect to sophisticated, downloaded computer graphics. A. Background

Thomas Musheno conducts forensic examinations of photographic evidence for the FBI's Forensic Audio, Video, and Image Analysis Unit ("FAVIAU"). Despite his experience with photographs,¹¹ Musheno lacks an important piece of evidence that would substantiate his supposed expertise: proficiency testing. In addition, his computer training is decidedly dated. His most recent course on Photoshop, perhaps the dominant image manipulation software, was in 2000. (Daubert Hr'g Tr. 43, 45, May 6, 2005.)

11 Musheno holds a bachelor's and a master's degree in photography. Daubert Hr'g, Ex. 3. He has taken courses in related fields, such as electronic imaging, and has been certified by the FBI as an Examiner of Questioned Photographic Evidence. (Daubert Hr'g Tr. 12, May 5, 2005). Musheno also trains other FBI examiners in image authentication. Id. at 10.

Musheno's examinations can be broken down into four categories, including comparison analysis (for example, matching the image of a bank robber caught on tape and a person in custody); photogrammetry analysis (calculating measurements, such as a robber's height, based on photographs); and information extraction (enhancing an image to reveal additional details). (Daubert Hr'g Tr. 7-8, May 5, 2005.) The fourth category, the only one of Musheno's techniques directly relevant to this case and the area in which Musheno's experience is most limited, is image authentication: Musheno examines an image to determine whether it has been manipulated or digitally created from whole cloth. Id.

Musheno's image authentication analysis has several steps, each of which involves simply looking at the pictures.¹² Upon receiving an allegedly pornographic image to examine, he first attempts to determine if the image depicts a known child victim. He compares the image with photographs from the National Center for Missing and Exploited Children database and images in his unit's Child Exploitation Obscenity Reference File ("CEORF") database.¹³Id. at 19, 20. If the image matches a child in either database, Musheno ends his examination because there is no question that the image depicts a real child.¹⁴Id.

12 Although Musheno looks at the size of the digital file and whether it has been compressed, he does not do any more substantial computer-based analysis.

13 The CEORF database contains pornographic images produced prior to 1986, when — according to Musheno — many northern European publications legally published pictures of real children that would be considered pornographic under the CPPA. (Daubert Hr'g Tr. 20, May 5, 2005.) Because the publications were legal, they are presumed to contain unmanipulated images of real children. Id. at 10.

14 Musheno conducted this analysis in this case and determined that three of the images depicted known child victims. I excluded those images on other grounds. See Order dated May 9, 2005 [doc. #90].

Next, Musheno examines the image for artifacts consistent with manipulation, using what he called the "FAVIAU Single Image Evaluation Form." Id. at 21. However fancy the title, the form is nothing more than a checklist Musheno's unit created to provide a contemporaneous record of the examiner's observations. Daubert Hr'g, Ex. 1. The examiner records, for example, the media type in which the image was submitted; whether the image is in color, black and white, or monotone;¹⁵ and the size of the digital file. Id. at 22-23. He looks for compression artifacts, which are visible signs that a digital image has been condensed into a smaller file.¹⁶Id. at 24.

15 A monotone image appears in white and a single other tone. For instance, rather than appearing in black and white, an image could be in green and white. (Daubert Hr'g Tr. 22-23, May 5, 2005.)

16 In fact, Musheno's testimony in this regard was telling. When asked whether compression was like a zip file, Musheno responded, "Oh, boy, I'm not a computer examiner, but basically what compression does, it will take a file and reduce the actual size of that, and in doing that you throw out detail." (Daubert Hr'g Tr. 25, May 5, 2005.)

The checklist requires the examiner to search for visible inconsistencies in seventeen listed categories, including the image's content, perspective, scale, shadows, quality of light, contrast, textures and patterns, sharpness, focus, and pixel values.¹⁷Id. at 25, 27. An image would have inconsistent content, for example, if it depicted an elephant on a diving board and the diving board was not bent. Id. at 25. The examiner also inspects the image for rough edges that indicate digital cutting and pasting, as well as inconsistent grain structure¹⁸ and resolution.¹⁹Id. at 27-28.

17 Pixel values are the building blocks of digital photographs; they have differing lightness and darkness values, as well as color values. (Daubert Hr'g Tr. 14, May 6, 2005.)

18 Grain structure is the physical structure of the grains or dye clouds in a negative. (Daubert Hr'g Tr. 28, May 5, 2005.)

19 Resolution is the fineness of detail in an image, which would be inconsistent if a digital image were cut and pasted into another digital image with a different resolution. (Daubert Hr'g Tr. 28, May 5, 2005.)

After noting any inconsistencies, the examiner considers fourteen listed human characteristics, asking a broad question — whether the image "look[s] human," looks "cartoonish," or falls somewhere in between. (Daubert Hr'g Tr. 10, May 6, 2005). This could not be a more subjective inquiry; it considers the subject's skin tones and textures, skeletal structure, and flesh and muscle movement to see what "appears to be real." Id. at 10-11.

If the image in question is part of a series of images,²⁰ Musheno conducts additional analysis. He compares the facial and body features shown in the images, as well as the clothing and background detail, to determine if the same person is depicted in each image in the series. Id. at 16, 19, 26.

20 This situation could arise if the files submitted to the examiner contain multiple images apparently of the same person or if existing files in the FAVIAU unit appear similar to the image in question. (Daubert Hr'g Tr. 17, May 6, 2005.)

After completing the examination, Musheno drafts a report summarizing his observations and conclusions. Id. at 15-16, 20. There are four possible conclusions: that the person depicted is real, that the person "appears to be real but no conclusion" can be drawn, that the person is not real, or that no conclusion is possible. Id. at 20-21. An examiner might reach the "appears to be real" conclusion if the image appears to depict a real child but there is insufficient detail to reach a definitive conclusion. Id. No conclusion at all can be drawn, on the other hand, if, for example, multiple compressions have degraded the image. Id. at 21.

Significantly, Musheno conceded that it is possible to alter an image in a manner that is undetectable.²¹ He testified that because of this, he never opines that an image is real on the basis of a single image. Id. at 23. In such cases, he limits his conclusions to "appears to be real" and no conclusion. Id. at 24-25. He assigns the "real" label only if there are "multiple images and they're of great detail and they're complex and there are no artifacts consistent with manipulation or creation."²² Given the sophistication of this field, however, that approach is hardly adequate.

21 At the same time, however, Musheno testified that he has never seen a child pornography image altered in a way that is not detectable. (Daubert Hr'g Tr. 29, May 6, 2005.) Such a conclusion, of course, begs the question whether Musheno has seen an altered pornographic image and simply been unable to tell.

22 Of course, even the presence of inconsistencies alone would not necessarily lead to a conclusion that the image is not real because "some inconsistencies . . . are explainable." (Daubert Hr'g Tr. 22, May 6, 2005.) Musheno said it "depend[s] on the image." Id. at 25.

Musheno completes the authentication analysis by subjecting his conclusions to review, id. at 16, but this review is perfunctory. Rather than reviewing the images independently in a blind test, coming to his own conclusions and then comparing them to Musheno's, the reviewing examiner goes over Musheno's completed checklist, his report, and the images contemporaneously. Id. at 30-31. If the reviewer agrees with Musheno's conclusion, the process ends there. Id. at 31-32. If the reviewer disagrees, the reviewer and Musheno discuss the disagreement, which could lead to the drafting of a revised report that would also be reviewed. Id. at 32-33.

With regard to the images in this case, Musheno testified that he received an initial group of images for review in the summer of 2004. Id. at 34. He conducted the analysis outlined above, except that he did not even look for a series of images because it was not standard procedure at the time. Id. at 34-36. In approximately February 2005, he received additional images for review. Id. at 36. This time, he considered the images, along with images from another case, as a series. Id. at 37-38.

Musheno concluded that six of the images at issue depicted real children and another ten appeared to be real children.²³ Another examiner confirmed Musheno's conclusions using only the review process described above. Id. at 38.

23 Musheno did not examine the remaining three images because the government asserts that they depict identified child victims.

B. Daubert Analysis

As I have explained, I find that the government has not established that visual expertise, such as that of a photograph expert, can distinguish the real from the virtual in the face of modern computer technology, however, even if visual observation were sufficient, Musheno's methodology does not meet the reliability standards that the Supreme Court delineated inDaubert.1. Framework for the Analysis of Expert Testimony

The FBI has only been studying pictures of child pornography for signs of manipulation since the mid-to-late 1990s. (Daubert Hr'g Tr. 15, May 5, 2005.) In addition, the basis for the alleged expertise is constantly in flux because of rapidly changing computer technology. Thus, as with "new" scientific techniques, "the Daubert analysis is critical, and the court should be an especially vigilant gatekeeper."²⁴Hines, 55 F. Supp. 2d 62, 65 (D. Mass. 1999); United States v. Monteiro, 407 F. Supp. 2d 351, 358 (D. Mass. 2006) ("The Court's vigilant exercise of this gatekeeper role is critical because of the latitude given to expert witnesses to express their opinions on matters about which they have no firsthand knowledge, and because an expert's testimony may be given greater weight by the jury due to the expert's background and approach.").

24 The Court recognizes that it is not its role to decide whether it agrees with a particular theory. It is the Court's duty, however, to ensure that the jury hears only qualified expert testimony. See Ruiz-Troche v. Pepsi Cola, 161 F.3d 77, 85 (1st Cir. 1998) ("Daubert neither requires nor empowers trial courts to determine which of several competing scientific theories has the best provenance. It demands only that the proponent of the evidence show that the expert's conclusion has been arrived at in a scientifically sound and methodologically reliable fashion."). Thus, the Court's focus is on the reliability of the expert's approach.

Where, as here, the expert relies on his experience to justify his methodology, the Court must examine the experience underlying his conclusions. As the Advisory Committee noted in amending Federal Rule of Evidence 702, "no one denies that an expert might draw a conclusion from a set of observations based on extensive and specialized experience." Fed.R.Evid. 702 advisory committee's notes to 2000 Amendment (quoting Kumho Tire, 526 U.S. at 156). At the same time:

If the witness is relying solely or primarily on experience, then the witness must explain how that experience leads to the conclusion reached, why that experience is a sufficient basis for the opinion, and how that experience is reliably applied to the facts. The trial court's gatekeeping function requires more than simply `taking the expert's word for it.'

Id. (citing Daubert v. Merrell Dow, 43 F.3d 1311, 1319 (9th Cir. 1995)).

Specifically, Daubert outlined the following, non-exhaustive list of factors to consider in determining admissibility: 1) whether the expert's technique or theory can be or has been tested; 2) whether the expert's technique or theory has been subject to peer review and publication; 3) the known or potential error rate; 4) the existence and maintenance of standards and controls; and 5) whether the technique has been generally accepted in the relevant scientific community. 509 U.S. at 593-94.

Kumho Tire extended Daubert to non-traditional sciences, notably expertise based on observational, clinical, and technical judgments.²⁵ 526 U.S. at 147, 149 (holding that the courts' "basic gatekeeping obligation" applies to "all expert testimony" and concluding that "[t]he trial judge's effort to assure that the specialized testimony is reliable and relevant can help the jury evaluate that foreign experience, whether the testimony reflects scientific, technical, or other specialized knowledge").

25 In Kumho Tire, the Supreme Court approved the district court's exclusion of proffered expert testimony based on observational experience similar to that offered in this case. 526 U.S. at 153-58. Much of the plaintiffs' case rested on the proffered testimony of a supposed "expert in tire failure analysis," whose testimony was based on what the Supreme Court termed "small observational differences." Id. at 142, 157. The district court excluded the testimony because it found that none of the Daubert factors indicated that the testimony was reliable. Id. Moreover, the district court found no countervailing factors in favor of admissibility that could outweigh those identified in Daubert. Id. at 156. Emphasizing that there was no indication in the record that other experts in the industry used the proffered technique or that it had been validated in any articles, the Supreme Court ruled that the exclusion was well within the district court's discretion. Id. at 157-58.

The Kumho Tire Court further instructed that the trial court may consider the Daubert factors in determining the admissibility of such testimony. Id. at 150, 119 S.Ct. 1167. The Court emphasized, however, that theDaubert factors are not a "definitive checklist."Id. (citing Daubert, 509 U.S. at 593), 113 S.Ct. 2786). The gatekeeping inquiry must be "tied to the facts of a particular case," and the law grants the trial judge "broad latitude" to identify the factors relevant to admissibility.²⁶Id. at 153, 150 (quoting Daubert, 509 U.S. at 591 (internal quotation omitted)).

26 The Federal Rules of Evidence have been amended to incorporate the teachings of Daubert and Kumho Tire. Rule 702, in particular, was amended in 2000 and now reads as follows:

If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.

Fed.R.Evid. 702.
This amended version of the Rule specifically

rejects the premise that an expert's testimony should be treated more permissively simply because it is outside the realm of science. An opinion from an expert who is not a scientist should receive the same degree of scrutiny for reliability as an opinion from an expert who purports to be a scientist. . . . The trial judge in all cases of proffered expert testimony must find that it is properly grounded, well-reasoned, and not speculative before it can be admitted.

Fed.R.Evid. 702 advisory committee's notes to 2000 Amendment (citation omitted).

I addressed the application of the Daubert/Kumho Tire framework to a technical field in Hines, where the government sought to introduce expert testimony about handwriting, and I concluded that handwriting analysis suffered a number of flaws.²⁷Hines, 55 F. Supp. 2d 62. It had never been subjected to meaningful validity testing or to financially disinterested peer review. I also took into account the fact that the testimony about similarities and differences in handwriting could be "understood and evaluated by the jury" and that the witness "can be cross examined." Id. at 70. Accordingly, I admitted the proffered expert to testify about his observations.

27 A growing body of literature addresses the application ofDaubert/Kumho Tire to technical fields. See, e.g., Michael J. Saks, Merlin and Solomon: Lessons from the Law's Formative Encounters with Forensic Identification Science, 49 Hastings L.J. 1069 (1998) (questioning the reliability of handwriting, fingerprint, and toolmark analyses, among others); Jessica M. Sombat, Latent Justice: Daubert's Impact on the Evaluation of Fingerprint Identification Testimony, 70 Fordham L.Rev. 2819 (2002).

I reached the opposite conclusion with respect to the expert's conclusions. I noted that there were no studies of the technique's validity, no standards as to how many similarities it took to declare a match. Id. at 69. The error rate was unknown.Id. Unlike his observations, which could easily be understood and evaluated by the jury, the ultimate conclusion would be based only on the expert's experience — a leap that "may not at all be justified by the underlying data." Id. at 70. Accordingly, I excluded testimony about the expert's conclusions.²⁸

28 But see United States v. Mooney, 315 F.3d 54, 63 (1st Cir. 2002) ("[O]nce a trial judge determines the reliability of the proffered expert's methodology and the validity of his reasoning, the expert should be permitted to testify as to the inferences and conclusions he draws from it, and any flaws in his opinion may be exposed through cross-examination or competing expert testimony.").

It is with this framework in mind that I examine Musheno's proffered testimony. 2. Application of the Daubert Factors

I will examine each of the factors delineated in Daubert before considering what other factors, if any, speak to the technique's reliability. a. Testing

The first inquiry under Daubert is whether the technique in question can be or has been tested. See Daubert, 509 U.S. at 593 (deeming testing a "key question"). Although observational fields such as image authentication need not be subjected to the laboratory tests that would be required of a traditional science,see Hines, 55 F. Supp. 2d at 68-69 (considering the lack of testing for proffered handwriting analysis testimony), the lack of testing here is startling.

During the Daubert hearing, I repeatedly asked Musheno whether his technique had ever been tested through, for instance, the use of blind testing, in which he would be asked to identify a variety of samples to determine whether they were real. The examiner, knowing which samples were, in fact, real and which were virtual, could then evaluate Musheno's performance. See (Daubert Hr'g Tr. 64, May 6, 2005). Musheno responded, "I've never been tested."²⁹Id.

29 The failure to conduct blind testing leads to the related problem of examiner bias. See Section III(B)(2)(b), infra.

Absent this type of proficiency testing, neither this Court nor the jury can assess the reliability of Musheno's work. This is particularly true where, as here, the field as a whole has no known error rates, making it impossible to guess how often Musheno is likely to be right or wrong. See Daubert, 43 F.3d at 1319 ("We've been presented with only the experts' qualifications, their conclusions and their assurances of reliability. Under Daubert, that's not enough."). Quite clearly, the first Daubert factor does not indicate that Musheno's technique is reliable.

b. Peer Review

The second Daubert factor considers peer review. The "peer review" process Musheno described leaves much to be desired.See (Daubert Hr'g Tr. 16, 30, May 6, 2005). Rather than conducting an independent examination of the images, Musheno's co-worker analyzed the images contemporaneously with Musheno's checklist and report, fully aware of Musheno's conclusions. Id. at 30. Musheno testified that no reviewer had ever disagreed with his conclusions — a result that could indicate either a flawless record or, equally likely, a review process that functions as a rubber stamp. Id. at 121-22. Indeed, the review process Musheno described runs a substantial risk of "examiner bias," a phenomenon by which an examiner who expects a particular result tends to find it. See William C. Thompson Michele Nethercott,The Challenge of Forensic Science, 28 Champion 50, 50 (Sept.-Oct. 2004) (observing that "[t]he failure of forensic labs to use `blind' procedures for interpreting test results contributes to the production of inaccurate conclusions in lab reports and courtroom testimony"). See also D. Michael Risinger et al., TheDaubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion, 90 Cal. L.Rev. 1 (2002).

Technical fields need not be held to the standard of peer review applicable to traditional sciences, which are often considered in scholarly journals. See Daubert, 509 U.S. at 593-94, 113 S.Ct. 2786. It is nevertheless appropriate to consider whether the fields have been subjected to meaningful peer review in determining its reliability. See Hines, 55 F. Supp. 2d at 68. The "peer review" conducted here is a far cry from the type of independent review that would bolster the technique's reliability.

c. Error Rate

The third Daubert factor concerns the known or potential error rate. This is perhaps the most problematic factor in this case, as the Court has no way of determining whether Musheno's years of experience add up to actual expertise. Although he has looked at many images in his career, no one knows whether his technique produces reliable results. Indeed, even if Musheno were the world's most talented assessor of image manipulation, this would mean little if the error rate in the field were particularly high. This problem is of particular concern where, as here, any expertise in the field of image manipulation would not address the possibility of an image that was not a real photograph manipulated by computer but rather was wholly digital. On the current record, there is no way of knowing whether Musheno's technique can recognize such computer-generated images.³⁰ Lack of a known error rate also prevents the jury from assessing the proper level of deference to accord the expert's conclusions. Because Musheno's technique has never been tested, its error rate is unknown and therefore does not support a finding of reliability. See Hines, 55 F. Supp. 2d at 69.

30 Indeed, I have found that in this case it is not possible to distinguish real photographs from those that are wholly digital based on visual observation. See Section II, supra. This finding emphasizes the potential error rate of Musheno's technique, which assumes that images are either entirely real or real images that have been manipulated and discounts the possibility of wholly virtual images.

d. Standards and Controls

The fourth Daubert factor considers the existence and maintenance of standards and controls. As noted above, Musheno bases his analysis on the FAVIAU checklist. He described this form as an "ever-changing document." See (Daubert Hr'g Tr. 93, May 6, 2005.) Rather than providing true standards for authentication, Musheno acknowledged that the form is an "internal checklist for the FBI" that has never been subjected to outside peer review.³¹Id. at 99. The form simply lists characteristics that examiners thought might be important in detecting virtual images and manipulations. Id. at 92-93. Nonetheless, the existence of the checklist provides a standard methodology and some level of consistency among FBI examiners.

31 Explaining the origin of the form, Musheno stated that the form was initially created for examinations of conventional imagery: "It existed in the unit for a number of years, and then when the digital age hit, it was changed to accommodate some of the digital items and then [h]as since evolved, there's been a number of different evolutions of the form." (Daubert Hr'g Tr. 26, May 5, 2005.)

More disturbingly, no standards govern the conclusions that examiners draw from the completed form. There are no guidelines for the number or type of factors that should be present to conclude that an image is real.³² (Daubert Hr'g Tr. 35-36, May 11, 2005.) Indeed, Musheno testified that he would conclude that an image is not real if it "looks cartoonish." (Daubert Hr'g Tr. 22, May 6, 2005.) He went on to explain that he would reach the same conclusion "if there were something so gross, if the perspectives were so out of whack" and noted that his conclusions are "image specific."³³Id. While the checklist provides a modicum of standardization for Musheno's observations, no standardization at all governs his final determinations. This degree of precision is insufficient to qualify as expertise when a defendant's liberty is at stake.

32 Musheno testified as follows:

Q: Are there a particular number of factors that need to exist for you to be able to conclude that an image is real?

A: Particular number of factors?

Q: For example, on that form that you're looking at?

A: No, No.

Q: And why not?

A: Well, it's image dependent. It has to do with a lot of things . . .

(Daubert Hr'g Tr. 35, May 11, 2005.)

33 The government argues that Musheno and his colleagues are, in fact, governed by standards because "the FBI requires 100% certainty before opining that an image is real." Gov. Mem. at 6 (citing (Daubert Hr'g Tr. 29-30, May 6, 2005)). But Musheno himself acknowledged that one "can't quantify this type of examination," belying the notion that the FBI can enforce a 100% certainty requirement on its examiners. See (Daubert Hr'g Tr. 96, May 6, 2005).

Likewise, Musheno testified that there are no objective criteria for determining when degradation has made it impossible to reliably analyze an image. See (Daubert Hr'g Tr. 40, May 11, 2005). An analysis of the images at issue in this case reveals the significance of this shortcoming. Musheno acknowledged that all of the images were compressed digital images in JPEG format. See (Daubert Hr'g Tr. 102-03, May 6, 2005). He further agreed that compression destroys information from the original image. Id. at 103. If an image is manipulated before being transformed into a JPEG file, some of the artifacts of the manipulation could "possibly" be discarded. Id. Musheno testified that he could not tell how many times (or when) the images in question had been compressed; he could only say for certain that compression artifacts were present. Id. at 106, 114. He also conceded that all six images that he identified as "real" had added text, a form of manipulation that could result in the further loss of information from the original image. Id. at 105-06. As a result, Musheno may draw conclusions from incomplete information without knowing what details were missing or how or by whom the information had been removed.

It appears that the only objective standard to which Musheno adheres in making his final determinations is his practice of not declaring an image real on the basis of a single image.³⁴Id. at 23. There are serious problems with even that criterion. Musheno testified that he will not declare a single image real because he "know[s] it is possible to create imagery or to manipulate imagery that's undetectable. . . ." Id. He is, however, comfortable declaring an image real "if there are multiple images and they're of great detail and they're complex. . . ." Id. at 25. This is not because the analysis itself differs when multiple images are being considered.³⁵Id. Rather, Musheno stated that he is "aware of how incredibly difficult it is to either manipulate or generate one image alone. If you start to have multiples, then that difficulty increases." Id.

34 Musheno's unit also uses four documents that set out standard operating procedures relating to image analysis. (Daubert Hr'g Tr. 33, May 6, 2005.) The relevant form for the analysis Musheno performed here is entitled "Image Content Authentication." See Daubert Hr'g, Ex. 2. It does not, however, provide true standards and controls, as it simply memorializes the internally standardless analysis that Musheno performed in this case.

35 Specifically, when asked whether the analysis differed when considering multiple images, Musheno responded that "there's more information and also . . . if you're looking at multiple images, you are comparing them to see if they are the same individual, which is also added information." (Daubert Hr'g Tr. 25, May 6, 2005.)

But as Frabizio rightly argues, this conclusion is based on the assumption that the computer artist started with a single image, rather than, for instance, alterable images of a single adult in multiple poses or from an entirely virtual image. This single and highly questionable criterion alone surely cannot satisfy theDaubert/Kumho Tire interest in the existence and maintenance of standards and controls. See Hines, 55 F. Supp. 2d at 69 (noting that one cannot "compare the opinion reached by [a handwriting analyst] with a standard protocol subject to validity testing, since there are no recognized standards"). See also Fed.R.Evid. 702 advisory committee's notes to 2000 Amendment ("The more subjective and controversial the expert's inquiry, the more likely the testimony should be excluded as unreliable.").

e. General Acceptance

Finally, there is no evidence that others in the field generally accept this technique. See Kumho Tire, 526 U.S. at 151 (citing general acceptance among its examples of appropriate applications of the Daubert standard to technical, experience-based fields); Hines, 55 F. Supp. 2d at 68-69 (applying the general acceptance factor to handwriting analysis). Musheno testified that his unit at the FBI has been conducting image authentication analysis since the mid-to-late 1990s. (Daubert Hr'g Tr. 15, May 5, 2005.) He later clarified, however, that his unit only began applying the technique in earnest within the last two to three years. (Daubert Hr'g Tr. 86, May 6, 2005.) It is clear, then, that Musheno's technique cannot claim a history of general acceptance.³⁶ Rather, this technique is apparently the product of a group of FBI employees who endorse one another's work.

36 Musheno has testified as an expert on image authentication in child pornography cases only four times, see (Daubert Hr'g Tr. 16, May 5, 2005), and the other members of his unit have testified in child pornography cases a combined total of twelve times (though he did not clarify whether this testimony related to the specific inquiry of image authentication). See id. at 17. Furthermore, Daubert did not suggest that acceptance by a legal, rather than a scientific or technical, community would suffice.

The criticism that the technique has only been considered by other FBI examiners is applicable to a wide range of technical fields. See, e.g., Hines, 55 F. Supp. 2d at 68 (observing that to the extent handwriting analysis had been "generally accepted," it was not by a "financially disinterested independent community, like an academic community") (quoting United States v. Starzecpyzel, 880 F. Supp. 1027, 1038 (S.D.N.Y. 1995). Many of these fields have withstood Daubert challenges. See, e.g., United States v. Crisp, 324 F.3d 261, 270 (4th Cir. 2003) (rejecting Daubert challenges to handwriting and fingerprint evidence). But fields such as handwriting and fingerprint analysis, though certainly not without their own problems, see Hines, 55 F. Supp. 2d at 68, can claim both more analytic rigor than image authentication and certainly a more venerable tradition of acceptance, id. at 69, 67. In Hines, for instance, I noted that expert testimony regarding handwriting analysis has been generally accepted for decades. Id. at 66-67.³⁷ The same cannot be said of Musheno's image authentication technique. f. Helpfulness to the Jury

37 Consideration of a history of acceptance could, of course, be equated with "grandfathering old irrationality." D. Michael Risinger et al., Exorcism of Ignorance as a Proxy for Rational Knowledge: The Lessons of Handwriting "Expertise", 137 U. Pa. L. Rev. 731, 771 n. 182 (1989). Thus, the Court must be wary of placing too much weight on this factor.

The First Circuit has held "`[t]he ultimate purpose of the Daubert inquiry is to determine whether the testimony of the expert would be helpful to the jury in resolving a fact in issue.'" Hochen v. Bobst Group, Inc., 290 F.3d 446, 452 (1st Cir. 2002) (quoting Cipollone v. Yale Indus. Prod., Inc., 202 F.3d 376, 380 (1st Cir. 2000)). In making this determination, the Court must consider, "given the proffered expert's background, whether the scientific, technical, or other specialized knowledge he offers will assist the trier better to understand a fact in issue." Gaydar v. Sociedad Instituto Gineco-Quirurgico y Planificacion Familiar, 345 F.3d 15, 24 (1st Cir. 2003) (internal quotation omitted). Although this inquiry is related to the question of the expert's reliability, the First Circuit has described assistance to the trier of fact as a "distinct" requirement of Federal Rule of Evidence 702. United States v. Shay, 57 F.3d at 132.

Clearly, Musheno's observational skills could be helpful to the jury, if the threshold concern about the possibility of wholly virtual images were addressed. His familiarity with Photoshop and the characteristics of images that have been manipulated would provide tools to assist jurors in their deliberations.³⁸ Essentially, Musheno's experience with image manipulation would guide the jury. Unlike his observations, however, Musheno's conclusions assert a level of certainty unjustified by his methodology and experience. See, e.g., United States v. Green, 405 F. Supp. 2d 104 (D. Mass. 2005). In addition, excluding these conclusions, which are of questionable merit, would reduce the danger that the jury would cede its decision-making entirely to him.³⁹

38 See note 1, discussing possible manipulation techniques.

39 The possibility of the jury ceding to an expert's findings is particularly of concern where, as here, the images are so unpleasant to look at that jurors might be tempted to avoid the task altogether by simply believing the expert without question.

On the current record, however, I cannot allow Musheno to testify even to his observations. As discussed in section II,supra, I find that the current state of technology prevents a lay juror, or any other visual observer, from assessing whether the images in this case depict real children, as opposed to computer-generated or digitally-manipulated images. Therefore, Musheno's observations — without the conclusions, which I must exclude as discussed above — are not useful to the jury.

g. Other Indicia of Reliability

In line with the Supreme Court's instruction that the Daubert factors should be applied flexibly to technical fields, it is appropriate to consider other indicia of reliability. Kumho Tire, 526 U.S. at 149-153, 156 (noting with approval that the district court had considered that there were "no countervailing factors operating in favor of admissibility . . . [that] outweigh[ed] those identified in Daubert"). The parties here have pointed to no additional indicia of reliability, but one additional element of Musheno's technique concerns the Court.

Musheno explained that, when evaluating images, he looks for signs of manipulation — that is, signs that someone had tampered with a picture of a real person. (Daubert Hr'g Tr. 90-91, May 6, 2005.) Again, this assumes that real photographs have been manipulated. As defendant suggested in cross-examination, however, a wholly virtual picture would not show the same signs of manipulation as would a real picture. Id. Thus, the possibility of creating images that are entirely computer-generated and which therefore do not demonstrate the inconsistencies Musheno said he would expect in a real photo that had been digitally doctored, creates substantial doubts about the reliability of Musheno's technique.⁴⁰IV. CONCLUSION

40 Indeed, it is because this Court finds that it is feasible to create such computer-generated images that Musheno's testimony must be excluded in its entirety.

In a world of rapidly changing technology, where the availability and use of Photoshop and other, similar programs is widespread, substantial evidence suggests it may be possible to digitally create or manipulate photographs in a manner the naked eye cannot detect. The government has not shown otherwise. Under these circumstances, it is unreasonable to expect a lay jury to differentiate the real from the computer-generated. The government must therefore present an expert or other extrinsic evidence to prove that the images in question depict real children.

Whether the images in this case are real or virtual cannot be determined based on mere observation, however, even by a photographic expert. More specialized, computer-based knowledge is required to exclude the possibility that the pictures are wholly virtual. Furthermore, even if visual observation were sufficient, I would find that Musheno's qualifications and expertise do not justify the conclusions he proposes to make. I would allow him to testify to his observations, but not to his ultimate conclusions.

Here, however, the government has not established that the pictures at issue in this case are not wholly virtual or that visual observation can reliably detect manipulation, if any. I therefore GRANT defendant's Daubert motion, and pursuant to Federal Rule of Evidence 702, exclude Musheno's testimony in its entirety.

SO ORDERED.