The question at this early phase in the proceedings—the "general causation" phase—is whether a reasonable jury could conclude that glyphosate, a commonly used herbicide, can cause Non-Hodgkin's Lymphoma ("NHL") at exposure levels people realistically may have experienced. If the answer is yes, the case moves to the next phase, which addresses whether each particular plaintiff's NHL was caused by glyphosate. If the answer is no, none of the plaintiffs' cases may proceed. And the answer must be no unless the plaintiffs can present at least one reliable expert opinion in support of their position.

There are two significant problems with the plaintiffs' presentation, which combine to make this a very close question. First, the plaintiffs (along with some of their experts) rely heavily on the decision by the International Agency for Research on Cancer ("IARC") to classify glyphosate as "probably carcinogenic to humans." This classification is not as helpful to the plaintiffs as it might initially seem. To render a verdict for a plaintiff in a civil trial, a jury must conclude, applying the "preponderance of the evidence" standard, that the plaintiff's NHL was more likely than not caused by exposure to glyphosate. And at this general causation phase, the question is whether a reasonable jury could conclude by a preponderance of the evidence that glyphosate can cause NHL at exposure levels people realistically could have experienced. The IARC inquiry is different in kind—it is a public health assessment, not a civil trial. Public health assessments generally involve two steps: (1) an effort to identify hazards ; and (2) an evaluation of the risk that the hazard poses at particular exposure levels. The first step essentially asks whether a substance is cause for concern, while the second step asks how concerned we should be. As IARC takes pains to point out, its decision that a substance is "probably carcinogenic to humans" is a hazard assessment—merely the first step in determining whether the substance currently presents a meaningful risk to human health. IARC leaves the second step—risk assessment—to other public health entities. Moreover, even with its hazard assessment, IARC makes clear that although it uses the word "probably," it does not intend for that word to have any quantitative significance. Therefore, the public health inquiry does not map nicely onto the inquiry required by civil litigation. And the hazard assessment IARC undertakes is too limited and too abstract to fully serve the plaintiffs' purposes here. A substance could be cause for concern, such that it can and should trigger preventive public health measures and further study, even when it is not so clearly dangerous as to allow a verdict in favor of a plaintiff.

The second problem with the plaintiffs' presentation is that the evidence of a causal link between glyphosate exposure and NHL in the human population seems rather weak. Some epidemiological studies suggest that glyphosate exposure is slightly or moderately associated with increased odds of developing NHL. Other studies, including the largest and most recent, suggest there is no link at all. All the studies leave certain questions unanswered, and every study has its flaws. The evidence, viewed in its totality, seems too equivocal to support any firm conclusion that glyphosate causes NHL. This calls into question the credibility of some of the plaintiffs' experts, who have confidently identified a causal link.

However, the question at this phase is not whether the plaintiffs' experts are right. The question is whether they have offered opinions that would be admissible at a jury trial. And the case law—particularly Ninth Circuit case law—emphasizes that a trial judge should not exclude an expert opinion merely because he thinks it's shaky, or because he thinks the jury will have cause to question the expert's credibility. So long as an opinion is premised on reliable scientific principles, it should not be excluded by the trial judge; instead the weaknesses in an unpersuasive expert opinion can be exposed at trial, through cross-examination or testimony by opposing experts.

The three expert opinions most helpful to the plaintiffs at this phase in the proceedings were offered by Dr. Christopher Portier, Dr. Beate Ritz, and Dr. Dennis Weisenburger. A jury may well reject these opinions at trial, finding the opinions too results-driven or concluding that the evidence behind those opinions is too weak. But applying the standard set forth in the case law for admission of expert testimony, the Court cannot go so far as to say these experts have served up the kind of junk science that requires exclusion from trial. And the testimony of these three experts is directly on topic, because they (in contrast to some other experts) went beyond the inquiry conducted by IARC, offering independent and relatively comprehensive opinions that the epidemiological and other evidence demonstrates glyphosate causes NHL in some people who are exposed to it. Accordingly, their opinions are admissible, which means the plaintiffs have presented enough evidence to defeat Monsanto's summary judgment motion. These proceedings thus move on to the next phase, which will involve an attempt by individual plaintiffs to present enough evidence to warrant a jury trial on whether glyphosate caused the NHL they developed. Given how close the question is at the general causation phase, the plaintiffs appear to face a daunting challenge at the next phase. But it is a challenge they are entitled to undertake.

This ruling is organized as follows: Section I provides background information relevant to these lawsuits. Section II describes the legal standard that applies to the admissibility of expert testimony, and explains why the IARC classification is insufficient to get the plaintiffs over the general causation hurdle. Section III provides an overview of the important epidemiological studies, highlighting the strengths and weaknesses of those studies and explaining why Monsanto's criticisms of the studies more helpful to the plaintiffs are not fatal to the plaintiffs' case. Section IV introduces the evidence addressing the carcinogenic effects of glyphosate on rodents. Section V briefly discusses evidence on the effects of glyphosate at the cellular level. Section VI examines each of the plaintiffs' experts' opinions, and analyzes whether those opinions synthesize all this evidence reliably enough to be admissible at trial. Finally, Section VII addresses the plaintiffs' motion to exclude some of Monsanto's experts.

I. BACKGROUND

Glyphosate is the active ingredient in Roundup, an herbicide manufactured by Monsanto. Roundup became commercially available in 1974, and glyphosate-based herbicides are now widely used across the United States and much of the world, on large-scale farms and in backyards. The U.S. Environmental Protection Agency does not currently consider glyphosate likely to cause cancer.

See U.S. Environmental Protection Agency Office of Pesticide Programs, Revised Glyphosate Issue Paper: Evaluation of Carcinogenic Potential 12-13, 143-44 (Dec. 12, 2017) [Daubert Ex. 873].

In 2015, IARC, which is the specialized cancer agency of the World Health Organization, convened a "working group" to assess whether several pesticides, including glyphosate, can cause cancer. Since 1971, IARC has regularly convened working groups to evaluate whether chemicals or other environmental factors are capable of causing cancer in humans. These working groups compile "Monographs" that examine the available scientific evidence and then come to conclusions about the carcinogenic potential of these different agents. The working group examining glyphosate concluded that the pesticide is "probably carcinogenic to humans," a designation whose meaning will be discussed later in this ruling.

IARC, Some Organophosphate Insecticides and Herbicides: Volume 112 , at 398 (2015) [Daubert Ex. 1030] ("Monograph").

IARC's designation addressed cancer in general, but the working group's report paid particular attention to human studies concerning a particular cancer, NHL, in reaching its conclusion. NHL is a cancer that affects lymphocytes, a type of white blood cell that is part of the immune system. Farmers have long had an elevated risk of NHL, even before glyphosate went on the market.

See Kenneth P. Cantor et al., Pesticides and Other Agricultural Risk Factors for Non-Hodgkin's Lymphoma Among Men in Iowa and Minnesota , 52 Cancer Research 2447, 2448 (1992).

After IARC classified glyphosate as a probable carcinogen, a wave of lawsuits followed. These lawsuits, which now number in the hundreds, were dispersed among state and federal courts across the country, but the claims against Monsanto raised similar issues. In particular, a central question in all these cases is whether Monsanto's glyphosate-based herbicides can cause NHL.

The Judicial Panel on Multidistrict Litigation, a panel of judges empowered to coordinate proceedings in federal cases where doing so "will be for the convenience of parties and witnesses and will promote the just and efficient conduct" of the cases, determined that coordination in these cases was warranted. 28 U.S.C. § 1407(a). The Panel therefore created this Multidistrict Litigation to centralize management of all the federal cases, and assigned to this Court all pretrial proceedings in the Multidistrict Litigation. As is common in such proceedings, the Court appointed a group of plaintiffs' counsel to serve as leaders and to represent all the plaintiffs' interests. Dkt. No. 62. Many additional cases have since been transferred to this district as part of the Multidistrict Litigation, and more than 400 cases are now pending.

The Court decided to bifurcate the pretrial proceedings. Dkt. No. 25. The motions at issue here arise during the first phase, which addresses "general causation." As noted, the question at the general causation phase is whether glyphosate is capable of causing NHL at exposure levels humans might have experienced. The second phase will involve, among other things, the issue of "specific causation." The specific causation inquiry focuses on whether individual plaintiffs' exposure to glyphosate-based herbicides caused the NHL they developed.

II. THE DAUBERT STANDARD AND THE GENERAL CAUSATION INQUIRY

To carry their burden during this phase of the litigation, the plaintiffs must put forward admissible evidence supporting their claim that glyphosate is capable of causing NHL at exposure levels humans might have experienced. If the plaintiffs cannot provide admissible evidence supporting this proposition—and enough admissible evidence to allow a reasonable jury to find in favor of the plaintiffs on this question—Monsanto is entitled to summary judgment in all the cases. See Anderson v. Liberty Lobby, Inc. , 477 U.S. 242, 249-50, 106 S.Ct. 2505, 91 L.Ed.2d 202 (1986).

The evidence at issue here is expert witness testimony. The plaintiffs have retained six experts they contend will provide opinions that satisfy the plaintiffs' burden at the general causation phase. These experts are: Dr. Beate Ritz, Dr. Christopher Portier, Dr. Alfred Neugut, Dr. Charles Jameson, Dr. Dennis Weisenburger, and Dr. Chadhi Nabhan. Broadly speaking, each of these experts reviewed the available scientific evidence and concluded that glyphosate is capable of causing NHL in humans. Monsanto has moved to exclude the plaintiffs' experts and has put forward seven retained experts of its own, each of whom provides a contrary view of the science. Before ruling on these motions, the Court held seven days of hearings to assess the testimony of many of these experts. Pursuant to the Cameras in the Courtroom pilot project, these hearings were video recorded. The recordings are publicly available on the U.S. Courts website.

In re Roundup Products Liability Litigation , U.S. Courts, http://www.uscourts.gov/cameras-courts/re-roundup-products-liability-litigation [https://perma.cc/YHJ8-Y7YP].

A. Legal Standard

Experts may not automatically testify before a jury. First, the district court must act as a "gatekeeper" and screen the experts' testimony under the standards set by the Federal Rules of Evidence and the Supreme Court's decision in Daubert v. Merrell Dow Pharmaceuticals, Inc. (Daubert I ), 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993). Federal Rule of Evidence 702, which governs this inquiry, provides that expert opinion testimony is admissible if: (1) the witness is qualified to testify about the topics she intends to address; (2) the expert's specialized knowledge will help the jury "to understand the evidence or to determine a fact in issue"; (3) "the testimony is based on sufficient facts or data"; (4) "the testimony is the product of reliable principles and methods"; and (5) "the expert has reliably applied the principles and methods to the facts of the case." The burden is on the plaintiffs to establish the admissibility of their experts' testimony. See Building Industry Association of Washington v. Washington State Building Code Council , 683 F.3d 1144, 1154 (9th Cir. 2012).

To be qualified, the expert must have sufficient "knowledge, skill, experience, training, or education" to offer the opinion. Fed. R. Evid. 702. So long as the expert's testimony is "within the reasonable confines of his subject area," a lack of particularized expertise generally goes to the weight of the testimony, not its admissibility. D.F. ex rel. Amador v. Sikorsky Aircraft Corp. , No. cv-00331-GPC-KSC, 2017 WL 4922814, at *14 (S.D. Cal. Oct. 30, 2017) (quoting Avila v. Willits Environmental Remediation Trust , 633 F.3d 828, 839 (9th Cir. 2011) and citing United States v. Garcia , 7 F.3d 885, 889-90 (9th Cir. 1993) ); see also Hopkins v. Dow Corning Corp. , 33 F.3d 1116, 1124 (9th Cir. 1994).

Aside from the qualification requirement, there are two questions at the heart of the admissibility determination: whether the testimony is relevant and whether it is reliable. See City of Pomona v. SQM North America Corp. , 750 F.3d 1036, 1043 (9th Cir. 2014). "Expert opinion testimony is relevant if the knowledge underlying it has a valid connection to the pertinent inquiry." Id. at 1044 (citation omitted). In other words, the expert testimony must "fit" the question the jury must answer. Daubert v. Merrell Dow Pharmaceuticals, Inc. (Daubert II ), 43 F.3d 1311, 1321 n.17 (9th Cir. 1995). This bar is cleared where the evidence "logically advances a material aspect of the proposing party's case." Messick v. Novartis Pharmaceuticals Corp. , 747 F.3d 1193, 1196 (9th Cir. 2014) (citation omitted).

Expert evidence "is reliable if the knowledge underlying it has a reliable basis in the knowledge and experience of the relevant discipline." City of Pomona , 750 F.3d at 1044 (citation omitted). In deciding whether to permit an expert to testify, courts face the difficult task of "determin[ing] whether the analysis undergirding the experts' testimony falls within the range of accepted standards governing how scientists conduct their research and reach their conclusions." Daubert II , 43 F.3d at 1317. Among the factors courts consider in making this determination are (1) whether the expert's theory or method is generally accepted in the scientific community; (2) whether the expert's methodology can be or has been tested; (3) the known or potential error rate of the technique; and (4) whether the method has been subjected to peer review and publication. Id. at 1316 (citing Daubert I , 509 U.S. at 593-94, 113 S.Ct. 2786 ). Courts should also consider whether the expert's testimony springs from research independent of the litigation. Id. at 1317. If not, the expert should point to other evidence that the testimony has a reliable basis, like peer-reviewed studies or a reputable source showing that the expert "followed the scientific method, as it is practiced by (at least) a recognized minority of scientists in their field." Id. at 1317-19. These factors are not a mandatory or inflexible checklist, and the Court has broad discretion to determine which factors are most informative in assessing reliability in the context of a given case. See Kumho Tire Co., Ltd. v. Carmichael , 526 U.S. 137, 141-42, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999) ; United States v. Alatorre , 222 F.3d 1098, 1102 (9th Cir. 2000).

The focus of the reliability inquiry is on the principles and methodology an expert uses in forming her opinions rather than the expert's conclusions. But in conducting the reliability analysis, the Court must also consider whether, for a given conclusion, "there is simply too great an analytical gap between the data and the opinion proffered." General Electric Co. v. Joiner , 522 U.S. 136, 146, 118 S.Ct. 512, 139 L.Ed.2d 508 (1997). In short, both unsound methods and unjustified extrapolations from existing data can require the Court to exclude an expert.

The Ninth Circuit has placed great emphasis on Daubert 's admonition that a district court should conduct this analysis "with a ‘liberal thrust’ favoring admission." Messick , 747 F.3d at 1196 (quoting Daubert I , 509 U.S. at 588, 113 S.Ct. 2786 ). Accordingly, the Ninth Circuit has emphasized that the gatekeeping function is meant to "screen the jury from unreliable nonsense opinions, but not to exclude opinions merely because they are impeachable." Alaska Rent-A-Car, Inc. v. Avis Budget Group, Inc. , 738 F.3d 960, 969 (9th Cir. 2013). That is because "[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence." Daubert I , 509 U.S. at 596, 113 S.Ct. 2786 ; see, e.g. , Murray v. Southern Route Maritime SA , 870 F.3d 915, 925 (9th Cir. 2017) ; Wendell v. GlaxoSmithKline LLC , 858 F.3d 1227, 1237 (9th Cir. 2017). This emphasis has resulted in slightly more room for deference to experts in close cases than might be appropriate in some other Circuits. Compare Wendell , 858 F.3d at 1233-38, and City of Pomona , 750 F.3d at 1043-49, with In re Zoloft (Sertraline Hydrochloride) Products Liability Litigation , 858 F.3d 787, 800 (3d Cir. 2017), and McClain v. Metabolife International, Inc. , 401 F.3d 1233, 1244-45 (11th Cir. 2005). This is a difference that could matter in close cases.

B. The Relevance of the IARC Classification

Although much of this ruling concerns itself with the reliability prong of the Daubert analysis, relevance is also important here. It's not sufficient for the plaintiffs to present evidence that glyphosate could cause NHL if humans were exposed to glyphosate at the kinds of massive doses, administered in the kinds of ways, that laboratory animals alone have experienced. A "general causation" phase that focused on this question would be a waste of time—it would be too far afield from the ultimate question whether any of the plaintiffs in these cases got NHL from glyphosate. That is why, to defeat Monsanto's summary judgment motion on the issue of general causation, it is not enough for the plaintiffs merely to present evidence that glyphosate is capable of causing cancer in the abstract.

By the same token, however, the inquiry at the general causation phase is not whether glyphosate gave NHL to any of the particular plaintiffs who brought these lawsuits, and the plaintiffs need not establish any particular level of exposure. It's enough in this litigation, at this stage, for the plaintiffs to show that glyphosate can cause NHL when people are exposed to the highest dose people might plausibly experience. See In re Hanford Nuclear Reservation Litigation , 292 F.3d 1124, 1133 (9th Cir. 2002). Picture, for instance, a professional gardener who has applied Roundup without using protective equipment several times per week, many hours per day, for decades.

The distinction between glyphosate's capacity to cause NHL at any hypothetical dose and its capacity to cause NHL at a human-relevant dose is important here, in light of the plaintiffs' heavy reliance on IARC's classification of glyphosate. Throughout much of this case, the plaintiffs seem to have operated under the assumption that they can clear the general causation hurdle simply by showing that IARC's decision to designate glyphosate a probable human carcinogen is scientifically sound. Accordingly, they have put forward some expert opinions that largely parrot IARC's analysis and conclusions. But whether glyphosate is "probably carcinogenic to humans" as IARC defines that phrase is not what's directly at issue here.

IARC engages in a standardized inquiry that it describes in detail in the Preamble to the Monograph addressing glyphosate. In short, IARC seeks to identify cancer hazards. The organization explains the "important" distinction between hazard identification and risk assessment, stating that "[a] cancer ‘hazard’ is an agent that is capable of causing cancer under some circumstances, while a cancer ‘risk’ is an estimate of the carcinogenic effects expected from exposure to a cancer hazard." Monograph at 10. As a result, the Monograph explains, the IARC classification process is only the "first step in carcinogen risk assessment," because the Monographs "identify cancer hazards even when risks are very low at current exposure levels, because new uses or unforeseen exposures could engender risks that are significantly higher." Id. Putting this definition into practice, Dr. Portier (one of the plaintiffs' experts) wrote a letter urging the EPA to "declare glyphosate a probable human carcinogen and go on to do a risk assessment to determine if human exposure is sufficient to warrant concern." Expert Report of Dr. Portier, App. Doc. 2 at 4, Wagstaff Decl. ISO Pls.' Opp'n to Def.'s Mot. for Summ. J. & Daubert Mot. ("Pls.' Opp'n") Ex. 5 [Dkt. No. 648-5 at 151].

To make its hazard assessment, an IARC working group looks first at studies in humans and then at studies in animals and at other available data, including studies on the mechanisms by which a particular agent affects organisms at the cellular level. The working group determines, "using standard terms," the strength of the evidence for carcinogenicity in both humans and animals. Monograph at 27. Here, IARC concluded that there is "limited" evidence in humans that glyphosate causes cancer, meaning that "[a] positive association has been observed between exposure to the agent and cancer for which a causal interpretation is considered ... to be credible, but chance, bias or confounding could not be ruled out with reasonable confidence." Id. at 27, 398. IARC further concluded there was "sufficient" evidence of carcinogenicity in experimental animals, that is, that "a causal relationship has been established between [glyphosate] and an increased incidence of malignant neoplasms or of an appropriate combination of benign and malignant neoplasms" in animal studies. Id. at 28, 398. The label IARC settled upon for glyphosate, "probably carcinogenic to humans," automatically follows from these evaluations. A substance is deemed a probable carcinogen, also known as a "Group 2A" agent, where IARC concludes the evidence in humans is limited and evidence in animals is sufficient. Id. at 30. A Group 2A classification can also be made when the working group concludes there is "inadequate" evidence—that is, not even limited evidence—that the agent causes cancer in humans but sufficient evidence that it does so in animals, where there is also strong evidence that it causes cancer in animals by a mechanism that operates in humans. Id. For comparison, a "Group 2B" classification of "possibly carcinogenic to humans" usually follows where the working group concludes there is "limited evidence of carcinogenicity in humans" and "less than sufficient evidence of carcinogenicity in experimental animals," or alternatively, where there is "inadequate evidence of carcinogenicity in humans" but "sufficient evidence of carcinogenicity in experimental animals." Id. All this is to say that IARC conducts its inquiry at a higher level of generality than what the Court must do here. Although IARC's assessment is not entirely divorced from real-world exposure levels, IARC sorts agents into different categories based on a fairly rigid formula that seeks to identify whether an agent is capable of causing cancer "under some circumstances." Id. at 10. Here, although there is no need to specify precisely the circumstances under which each plaintiff was exposed to glyphosate, only evidence supporting the conclusion that glyphosate causes NHL in doses within the realistic realm of actual human exposure can get the plaintiffs past summary judgment. It's worth acknowledging that, even at the end of this ruling, precisely what the range of actual human exposure is will remain vague, a product of bifurcated proceedings where the hundreds of individual plaintiffs' experiences remain on the periphery for now. But it's enough at this point to say that IARC's hazard assessment considers the evidence for a different purpose, and without the attention to the effects of current human exposure the Court must pay here. Moreover, it is not enough for the evidence in this case to go merely to the causal relationship between glyphosate and cancer in general; it must go to the relationship between glyphosate and NHL in particular. Perhaps most importantly, the question in a court case at this stage is whether a reasonable jury could conclude by a preponderance of the evidence that glyphosate can cause NHL at human-relevant doses—that is, whether a jury could conclude it is "more likely than not" that glyphosate can cause NHL in the human population. IARC's use of the word "probably" has "no quantitative significance." Id. The inquiry in this case therefore fits neatly into neither the hazard identification nor the risk assessment boxes as IARC defines them.

Neoplasms are tumors. Id. at 10.

IARC also notes that, "[e]xceptionally, an agent may be classified in this category solely on the basis of limited evidence of carcinogenicity in humans," if the agent clearly belongs, based on mechanistic evidence, to a class of agents some of which already have been classified as carcinogenic or probably carcinogenic to humans. Id.

A Group 2B classification can also in some instances follow where there is "inadequate evidence of carcinogenicity in humans and less than sufficient evidence of carcinogenicity in experimental animals together with supporting evidence from mechanistic and other relevant data." Id. "[S]trong evidence from mechanistic and other relevant data" can also support classification in this category. Id.

As a result, expert opinions that simply parrot IARC's analysis and conclusions are somewhat off topic and are unduly limited, rendering them insufficient to satisfy the plaintiffs' burden at the general causation phase. A "hazard assessment," as IARC and other public health bodies define that inquiry, is not what the jury needs to conduct when deciding whether glyphosate actually causes NHL in people at past or current exposure levels. An expert who recites IARC's conclusions and analysis therefore may be offering a sound scientific opinion, but not an opinion that speaks squarely to the issue the jury must decide. And in addition to the fact that such opinions are not enough to get the plaintiffs past the general causation hurdle, there is a significant possibility that, if there ever is a jury trial (that is, if any plaintiff can get past summary judgment on the issue of specific causation), expert opinions that go no further than IARC's analysis will be excluded. An expert opinion of this sort may not "fit" the general causation inquiry closely enough to be helpful to the jury in the way Rule 702 requires; it may serve primarily to confuse the jury, causing the trial to devolve into an abstract discussion about the differences between what public health organizations do and what juries do. In any event, for current purposes, the point is that to the extent the plaintiffs have offered opinions from experts who merely reiterate the IARC analysis, those opinions do not allow the plaintiffs to avoid summary judgment. Beyond that, the relevance and admissibility of these opinions at any eventual trial can be addressed as these cases develop.

III. EPIDEMIOLOGY

Epidemiology is "the field of public health and medicine that studies the incidence, distribution, and etiology of disease in human populations." As the parties acknowledge, epidemiology is central to the general causation inquiry, and where such evidence exists, it must be addressed by the experts. See Norris v. Baxter Healthcare Corp. , 397 F.3d 878, 882 (10th Cir. 2005) ; Def.'s Daubert & Summ. J. Mot. 10 [Dkt. No. 545] ("Def.'s Mot."); Pls.' Opp'n 19-20 [Dkt. No. 647]; cf. Milward v. Acuity Specialty Products Group, Inc. , 639 F.3d 11, 24 (1st Cir. 2011). None of the plaintiffs' experts base their opinions exclusively on the epidemiology research, but all discuss it to varying degrees.

Michael D. Green et al., Reference Guide on Epidemiology , in Reference Manual on Scientific Evidence 551, 551 (3d ed. 2011) ("Reference Manual").

A. The Bradford Hill Criteria

Epidemiology studies examine whether an association exists between an agent like glyphosate and an outcome like NHL. Whether that agent causes the outcome, however, cannot be proven by epidemiological studies alone; an evaluation of causation requires epidemiologists to exercise judgment about the import of those studies and to consider them in context. Once epidemiologists have concluded from the studies that there is an association between an agent and an outcome, they often assess causation through a framework called the "Bradford Hill criteria." These criteria are named for Sir Austin Bradford Hill, who wrote a 1965 article that articulated nine "viewpoints" now generally accepted to be relevant to assessing causation. Broadly, these factors are: (1) the strength of the association; (2) consistency; (3) specificity; (4) temporality; (5) biological gradient or dose response; (6) biological plausibility; (7) coherence with other scientific knowledge; (8) experimental evidence; and (9) analogy. Both parties' experts considered these criteria, which are introduced here to frame the discussion that follows, and they will be explained in more detail in Section VI.

Austin Bradford Hill, The Environment and Disease: Association or Causation? , 58 Proceedings of the Royal Society of Medicine 295 (1965), Wagstaff Decl. ISO Pls.' Opp'n Ex. 47 [Dkt. No. 649-17] ("Bradford Hill").

B. Case-Control Studies and Meta-Analyses

The first step in assessing causation is determining whether an association exists between exposure to glyphosate and NHL. In concluding that studies have shown such an association, the plaintiffs' experts emphasize case-control studies. A case-control study is one of two primary types of observational epidemiological studies. This kind of study starts with a group of people who have the disease of interest (the "cases"), selects a similar population of people without the disease (the "controls"), and then compares the groups on the basis of past exposure to the chemical the investigators are studying. In contrast, a cohort study, the other primary type of observational epidemiological study, selects a study population without the disease of interest, sorts that population into exposed and unexposed groups, and then measures the incidence of disease in the exposed and unexposed groups after observing them for a period of time. Frequently touted benefits of case-control studies are their comparatively low cost and ability to identify associations relevant to rare diseases. See, e.g. , Reference Manual at 556-60; Expert Report of Dr. Mucci 12, Hollingsworth Decl. ISO Def.'s Mot. Ex. 18 [Dkt. No. 546-18] ("Mucci Report"); Expert Report of Dr. Ritz 13, Wagstaff Decl. ISO Pls.' Opp'n Ex. 3 [Dkt. No. 648-3] ("Ritz Report").

Case-control studies report an odds ratio as the measure of association between the variables the investigators are studying. "In a case-control study, the odds ratio is the ratio of the odds that a case (one with the disease) was exposed to the odds that a control (one without the disease) was exposed." Reference Manual at 568. An odds ratio greater than 1.0 indicates an association, as it suggests those with the disease are more likely to have been exposed to the substance of interest.

Odds ratios are typically reported with confidence intervals that seek to capture the likely effects of random error. A 95% confidence interval, the standard interval, is a range that would capture the actual odds ratio 95% of the time if the study were conducted repeatedly. Generally, larger sample sizes produce narrower confidence intervals. When the lower bound of the 95% confidence interval exceeds 1.0, the results of the study are considered to show an association that is "statistically significant" at the .05 level. Id. at 580-81. The purpose of assessing statistical significance is to determine how likely it is that an observed odds ratio is merely due to chance, rather than indicative of a true association. The line delineating what constitutes a statistically significant result is necessarily somewhat arbitrary, and the experts dispute how much weight to give studies reporting odds ratios above 1.0 that are not statistically significant at the .05 level. Although there may be a causal association even in the absence of statistically significant results, statistical significance remains a useful metric for determining whether the results of a given study likely show a real association. In re Zoloft , 858 F.3d at 793.

When assessing whether an epidemiological study can form a reliable basis for an expert's opinion, a court must determine whether the study adequately considered confounding variables and possible sources of bias. In re Abilify (Aripiprazole) Products Liability Litigation , 299 F. Supp. 3d 1291, 1322-23 (N.D. Fla. 2018). Confounding arises where a factor not accounted for by the study wholly or partially explains an apparent association between the agent under study and the outcome. A factor is a confounder where it is independently related to both the exposure and the disease of interest. Failure to control for true confounding variables can skew the results of a study, producing an observed association where none exists or an observed association that is stronger or weaker than the actual association. Reliable epidemiological studies should account for confounders where they are identified, although "failure to control for every conceivable potential confounder does not necessarily render the results of an epidemiological study unreliable." In re Abilify , 299 F. Supp. 3d at 1322. One way to account for confounders is through study design; for instance, matching controls to cases by age would ameliorate concerns about confounding resulting from the age of study participants. Confounders can also be addressed during data analysis, using methods like stratification or multivariable analysis, so long as information about potential confounders was obtained during the study. See Reference Manual at 591-97. One important possible source of confounding in the studies relevant here is exposure to other pesticides.

See Kenneth J. Rothman et al., Modern Epidemiology 129-34 (3d ed. 2008) ("Rothman").

Bias occurs where the results of a study are subject to systematic—in other words, non-random—error. Study design, data collection, and data analysis can all give rise to bias. Id. at 583. Most relevant in this case is the possibility of information bias resulting from inaccurate information about study participants' exposure to glyphosate. One type of information bias, recall bias, occurs where people with a disease (the "cases" in a case-control study) are differently able to recall past exposures than are people who never get sick; generally, the assumption is that the cases will recall greater levels of exposure, as those who become ill are more likely to ruminate about the possible causes of their disease. See id. at 585-86.

Concerns about recall bias and about study accuracy more generally may be heightened where studies rely on proxy respondents. Proxy respondents or surrogates, often spouses or next of kin, are used when the study participants themselves are not available, typically because they have died or are too ill to participate. Proxy respondents are generally considered less reliable than the study participants themselves. Mucci Report 20-21.

With this background in mind, the following is an overview of the some of the most important and frequently discussed case-control studies.

One key publication is a pooled analysis of three separate case-control studies conducted by the National Cancer Institute in the Midwestern United States between 1979 and 1986. In a pooled analysis, the study authors combine the raw, participant-level data from earlier studies and then analyze these data as one combined dataset. See Ritz Report 6; Mucci Report 25. Pooling allows for uniform analysis of the data in the underlying studies and increases the statistical power of the earlier, smaller studies. The experts identify this study as "De Roos (2003)," by the lead author's last name and its year of publication.

A.J. De Roos et al., Integrative Assessment of Multiple Pesticides as Risk Factors for Non-Hodgkin's Lymphoma Among Men , 60 Occupational & Environmental Medicine 1 (2003), Wagstaff Decl. ISO Pls.' Opp'n Ex. 55 [Dkt. No. 652-9].

De Roos (2003) aggregated the data from the three studies and analyzed the effects of 47 different pesticides on the incidence of NHL. Id. at 1. The authors sought to isolate the effect of each pesticide by controlling for the use of all 46 other pesticides, in addition to age and study site, in their models assessing association. Id. at 2. The authors reported results using both a more conventional logistic regression model and a more conservative hierarchical regression model that took into account values estimating the prior distributions of the other pesticides. Id. Using the logistic regression model, the odds ratio for those exposed to glyphosate was 2.1, with a 95% confidence interval of 1.1 to 4.0. Using the hierarchical regression model, the odds ratio was 1.6 (0.9, 2.8), no longer a statistically significant result. Id. at 5. Thirty-six of the cases and 61 of the controls in this analysis were exposed to glyphosate. Id. The study authors considered proxy responses. Id. at 4.

Going forward, the 95% confidence interval will be reported in the following format: odds ratio (lower bound, upper bound).

Another study discussed at length by the experts focused on a population-based case-control study in Canada. They refer to this study as "McDuffie (2001)." NHL diagnoses in this study occurred between 1991 and 1994, and 51 cases and 133 controls were exposed to glyphosate. Id. at 1158. Proxy respondents were not used. Id. at 1156. This study reported an overall odds ratio for glyphosate of 1.2 (0.83, 1.74). This estimate was adjusted for medical variables associated with NHL outcomes (like a positive family history of cancer or past cancer ), age, and province of residence, but not for use of other pesticides. Id. at 1158. The study also sought to capture an estimate of NHL risk that reflected frequency of exposure to glyphosate. It reported that when glyphosate was used between zero and two days per year, the odds ratio was 1.00 (0.63, 1.57). When glyphosate was used more than two days per year, the odds ratio was 2.12 (1.20, 3.73). These estimates likewise appear not to have been adjusted for use of other pesticides. Id. at 1161.

Helen H. McDuffie et al., Non-Hodgkin's Lymphoma and Specific Pesticide Exposures in Men: Cross-Canada Study of Pesticides and Health , 10 Cancer Epidemiology, Biomarkers & Prevention 1155 (2001), Wagstaff Decl. ISO Pls.' Opp'n Ex. 60 [Dkt. No. 652-14].

The North American Pooled Project ("NAPP") aggregated the data from the three case-control studies included in De Roos (2003) and the Canadian data from McDuffie (2001). The results of this pooled analysis have not been published in a peer-reviewed journal, but the parties highlighted results presented in an abstract and two slide decks prepared for conferences. The more recent analysis is presented in a slide deck for an August 2015 presentation, although this slide deck, like the other NAPP materials, does not detail the methods used by the study authors. These slides presented an overall odds ratio for glyphosate use of 1.13 (0.84, 1.51), when adjusted for use of three other pesticides and several other potential confounders. Id. at 10. When proxy respondents were removed from the data, the odds ratio dropped to 0.95 (0.69, 1.32). Id. at 26. The odds ratios reported for subjects who reported using glyphosate for seven lifetime days or fewer were lower than those who reported use for more than seven days, but none of these odds ratios were statistically significant. For subjects who reported using glyphosate for less than or equal to two days per year, without proxy respondents, the odds ratio was 0.66 (0.39, 1.12), and with proxy respondents it was 0.74 (0.46, 1.19). The greater-than-two-days-per-year odds ratio without proxy respondents was 1.77 (0.99, 3.17), and when proxy respondents were included, the result was 1.73 (1.02, 2.94). Id. Monsanto argues that the NAPP study, although still unpublished, should supersede the earlier De Roos (2003) and McDuffie (2001) studies, as it is a more recent and complete analysis.

Manisha Pahwa et al., An Evaluation of Glyphosate Use and the Risk of Non-Hodgkin Lymphoma Major Histological Sub-Types in the North American Pooled Project (Aug. 31, 2015) [Daubert Ex. 1278].

A further publication, "Eriksson (2008)," addresses the results of a Swedish population-based case-control study, with NHL cases collected between 1999 and 2002. There were 29 glyphosate-exposed cases and 18 controls included in this study. Id. at 1659. Proxy respondents were not used. Id. at 1660. The authors analyzed the data using a multivariate model controlling for six other pesticides, age, sex, and year of diagnosis or enrollment, and reported a non-statistically significant odds ratio of 1.51 (0.77, 2.94) for glyphosate. Id. at 1661. This study also reported a more detailed set of numbers unadjusted for use of other pesticides. The overall odds ratio for glyphosate was 2.02 (1.10, 3.71). Breaking this down, the unadjusted results showed statistically significant associations for glyphosate and NHL for those who were exposed to glyphosate for greater than ten days—2.36 (1.04, 5.37), versus 1.69 (0.70, 4.07) for those exposed for less than ten days. Id. at 1659. For those who developed cancer more than ten years after exposure to glyphosate, the odds ratio was 2.26 (1.16, 4.40), compared to 1.11 (0.24, 5.08) for those who developed cancer less than ten years after exposure. Id. at 1658-59. One possible cause for concern in this study is the authors' choice of the control group for the univariate analysis, that is, the analysis not adjusted for use of other pesticides. See Mar. 5, 2018 Tr. [Ritz] 34-35 [Dkt. No. 1172]; Mucci Report 53. The authors used as the control group for this part of the analysis people who were not exposed to any of the pesticides included in the study. Eriksson (2008) at 1658.

Mikael Eriksson et al., Pesticide Exposure as Risk Factor for Non-Hodgkin Lymphoma Including Histopathological Subgroup Analysis , 123 International Journal of Cancer 1657 (2008), Wagstaff Decl. ISO Pls.' Opp'n Ex. 54 [Dkt. No. 652-8].

Dr. Ritz sought to offer an opinion that, had the study authors used a more appropriate comparison group, the results would not have changed materially. See Apr. 4, 2018 Tr. [Ritz] 22-27 [Dkt. No. 1352]; see also Def.'s Apr. 9, 2018 Supp. Br. 5 [Dkt. No. 1354] (objecting to this opinion). In light of her own tentativeness about her adjustment and the absence of any detailed explanation of her method in her reports or her live testimony, Dr. Ritz's opinion regarding how the Eriksson results would change after altering the control group is not admissible.

The plaintiffs also emphasize meta-analyses of the available epidemiological studies. Meta-analysis combines the results of several studies, giving them different weights that take into account, for instance, the size of the study population. Reference Manual at 607. Unlike a pooled analysis, which uses the underlying raw data, meta-analysis uses the reported summary statistics from the earlier studies. See Ritz Report 6; Mucci Report 24. The value of a meta-analysis, like the value of a pooled analysis, depends upon the quality of the underlying studies, and meta-analyses can be uninformative when the studies included in the analysis are very different from one another. Although these meta-analyses take into account one cohort study, which will be discussed shortly, they are introduced here since the bulk of the included studies are case-control studies.

Three meta-analyses of the data on glyphosate and NHL have been discussed during these proceedings. The first was published in 2014 by Schinasi and Leon, but this analysis did not use the odds ratios from some of the underlying studies that were most fully adjusted for confounders. The IARC working group updated Schinasi and Leon's meta-analysis to use the more fully adjusted numbers and reported a meta-risk-ratio of 1.3 (1.03, 1.65). Monograph at 350. A later published meta-analysis, by Chang and Delzell in 2016, likewise took into account the most fully adjusted results from the earlier studies, and reported a meta-risk-ratio of 1.3 (1.0, 1.6). Chang & Delzell (2016) also conducted sensitivity analyses that swapped out the hierarchical regression in De Roos (2003) for the logistic regression and replaced McDuffie (2001) with a 2011 analysis of the Canadian data. See id. at 416. The results for the four models they tested were very similar, all falling between 1.3 (1.0, 1.6) and 1.4 (1.0, 1.8). Id.

Leah Schinasi & Maria E. Leon, Non-Hodgkin Lymphoma and Occupational Exposure to Agricultural Pesticide Chemical Groups and Active Ingredients: A Systematic Review and Meta-Analysis , 11 International Journal of Environmental Research & Public Health 4449 (2014), Wagstaff Decl. ISO Pls.' Opp'n Ex. 67 [Dkt. No. 653-7] ("Schinasi & Leon (2014)").

The risk ratio or relative risk, which is used to assess whether an association exists in cohort studies, is the ratio of the risk of disease among people exposed to those who are unexposed. For relatively rare diseases, the odds ratio approximates the relative risk and, as with an odds ratio, a number above 1.0 indicates an association between the exposure and the disease. Reference Manual at 625, 627.

Ellen T. Chang & Elizabeth Delzell, Systematic Review and Meta-Analysis of Glyphosate Exposure and Risk of Lymphohematopoietic Cancers , 51 Journal of Environmental Science & Health 402 (2016), Wagstaff Decl. ISO Pls.' Opp'n Ex. 68 [Dkt. No. 653-8] ("Chang & Delzell (2016)").

Monsanto and its experts raise concerns about basing a causation assessment on the case-control studies and meta-analyses. For instance, Monsanto's epidemiology experts highlighted concerns about recall bias. Mucci Report 36; Expert Report of Dr. Rider 3, Wagstaff Decl. ISO Pls.' Opp'n Ex. 116 [Dkt. No. 656-11] ("Rider Report"). The plaintiffs' experts acknowledged that recall bias is a potential concern in case-control studies, but disputed that it is a major issue here. Ritz Report 7-8; Revised Expert Report of Dr. Portier 7, 18, Hollingsworth Decl. ISO Def.'s Mot. Ex. 8 [Dkt. No. 546-8] ("Portier Report"). The plaintiffs' experts explained that, at the time the cases were assessed, the participants had no reason to suspect that glyphosate exposure could cause cancer, and therefore they were unlikely to have over-reported their exposure. Apr. 4, 2018 Tr. [Ritz] 51-53. To demonstrate that participants didn't generally over-report glyphosate use when these studies were conducted, they pointed out that epidemiology studies on the whole observed associations only between glyphosate and NHL, and not between glyphosate and the other cancers about which participants were asked. If participants were predisposed to think that glyphosate caused cancer and exhibited recall bias as a result, they explained, one would expect to see associations reported for glyphosate and other cancers. Id. at 50-51; see also Mar. 5, 2018 Tr. [Weisenburger] 192-93 [Dkt. No. 1172]; March 9, 2018 Tr. [Mucci] 945-46 [Dkt. No. 1186]. The plaintiffs' experts also pointed to studies that sought to validate self-reports of pesticide exposure and that found similar recall accuracy between cases and controls. Ritz Report 19; Portier Report 8, 11; Mar. 5, 2018 Tr. [Weisenburger] 182-83. Ultimately, in response to these points, one of Monsanto's epidemiology experts conceded at the Daubert hearing that she was "not quite as worried about recall bias in the context of this body of literature," except in the McDuffie (2001) study. Mar. 9, 2018 Tr. [Mucci] 946. On the whole, concerns about recall bias in these studies do not demand that a reliable expert opinion meaningfully discount the body of case-control studies when assessing causation.

Monsanto's experts also attacked the plaintiffs' experts' reliance on case-control studies they contend reflect inadequate latency periods, that is, periods between exposure and diagnosis. See Mucci Report 7, 36-40, 49, 69; Rider Report 32-33, 35, 38-39, 45-46. Specifically, Monsanto pointed to the case-control studies conducted in Kansas and Iowa/Minnesota, which are included in the pooled analyses reported in De Roos (2003) and the NAPP. The Kansas cases were identified between 1979 and 1981 and the Iowa/Minnesota cases between 1980 and 1983. De Roos (2003) at 1-2. Monsanto and its experts argued that these studies focused on people diagnosed with NHL too soon after glyphosate was put on the market in 1974 to capture cases caused by glyphosate, as cancer typically takes many years to develop. The plaintiffs' experts recognized that inadequate latency periods could be cause for concern, and at least implicitly acknowledge that latency could be an issue with the studies that generated many of the numbers that are most helpful to the plaintiffs. See Ritz Report 17 ("Although a short latency period does not completely exclude the possibility of exposure-disease relationships in cancer, a longer latency period increases confidence in results due to increased biological plausibility[,] i.e.[,] typically we would generally expect a 5-10 year minimum latency between exposure and disease onset for blood system related cancers."); id. at 18-19 (acknowledging that the Iowa/Minnesota study had what "is considered an inadequate latency period"); Mar. 6, 2018 Tr. [Weisenburger] 267-70 [Dkt. No. 1175]; Portier Report 5 ("Because the latency period for cancers can be long (years), evaluation of studies should consider whether the exposure occurred sufficiently long ago to be associated with cancer development."); Apr. 6, 2018 Tr. [Portier] 142, 148-53 [Dkt. No. 1353].

Although the latency concern is legitimate, three of the plaintiffs' experts, Drs. Ritz, Portier, and Weisenburger, explained that this concern was mitigated to a degree by steps taken by some study authors. One reason a study might show an association between glyphosate and NHL shortly after glyphosate was put on the market is confounding; if one of the pesticides that was frequently used before glyphosate came on the market causes NHL, those who later switched to glyphosate might simply be manifesting the NHL triggered by those other pesticides. However, in some of the studies, the authors adjusted for other pesticides. The plaintiffs' experts explained that, although it is always possible that an observed association is the result of confounding for which the authors did not account, the adjustment for many other pesticides used by De Roos (2003), in particular, made it significantly less likely that a pesticide other than glyphosate explained the observed association. If the studies accounted for likely confounders, they explained, there is little reason to discount the studies, notwithstanding the relatively short latency periods they captured. See Mar. 6, 2018 Tr. [Weisenburger] 282-83; Apr. 4, 2018 Tr. [Ritz] 15-18, 30-31; Apr. 6, 2018 Tr. [Portier] 149-53.

The plaintiffs' experts also sought to downplay the latency concern in other ways. Dr. Ritz asserted that, in a case-control study, an association observed after a short latency period might be something of an alarm bell, as those cases might reflect outcomes in people who experienced heavy exposures or who developed particularly aggressive cancers. Apr. 4, 2018 Tr. [Ritz] 9-11. As a result, Dr. Ritz argued, one might expect to see an even stronger association had the studies allowed for longer latency periods. Id. ; see also Apr. 6, 2018 Tr. [Portier] 154. Dr. Ritz also hypothesized that quicker onset of NHL in case-control studies might reflect the older average age of case-control study participants versus participants enrolled in the cohort study discussed below. Overall, the latency concern raised by Monsanto is a legitimate one that makes a causal account of the American case-control studies, in particular, more difficult to swallow. But, at least for the studies that adjust for other pesticide exposures, the relatively short period between glyphosate exposure and cancer development is not a concern so significant as to disqualify an expert who gives significant weight to the case-control studies in rendering a causation opinion.

For the most part, Dr. Ritz introduced her views on latency in her original expert report and discussed them at her deposition. See Wagstaff Decl. ISO Pls.' Opp'n Ex. 58 [Dkt. No. 652-12] (discussing latency in the context of the De Roos (2003) study). To the extent she strayed into new territory regarding latency in the epidemiological studies during the second round of Daubert hearings—as is arguably the case with her opinion that the older average age of participants in case-control studies might have some explanatory power—the Court is persuaded that exclusion is not warranted, as Dr. Ritz's testimony was responsive to the Court's questions, and Monsanto will have an adequate opportunity between now and trial to refine its cross-examination on this point. But see id. at 187-89 (discussing the relationship between age and latency generally). The Court reaches the same conclusion regarding Dr. Portier's illustration of the different latency concerns associated with case-control and cohort studies. See Apr. 6, 2018 Tr. [Portier] 34-40.

Monsanto also argues that reliance on some of the case-control studies is inappropriate because they did not adequately account for the important possible confounder of exposure to other pesticides. Some glyphosate users, like farmers and landscapers, are likely exposed to many pesticides, and these other pesticides may also be associated with elevated incidences of NHL. As discussed, the case-control studies adjusted for possible confounders to different degrees, and when study authors provided both unadjusted and adjusted numbers, the odds ratios adjusted for use of other pesticides were closer to 1.0, and often not statistically significant. See, e.g. , Eriksson (2008) at 1659, 1661. The possibility of confounding arising from exposure to other pesticides is a serious consideration and one that must be accounted for in a reliable expert report assessing the epidemiology evidence.

C. Cohort Study

Instead of the case-control studies, Monsanto's experts focus on the results of the Agricultural Health Study (AHS), a cohort study. Recall that cohort studies, unlike case-control studies, select participants without the disease of interest and follow them for a period of time to see what diseases develop in the exposed and unexposed cohorts. Advantages of such studies include that they can conclusively establish the temporal relationship between exposure to a chemical and a disease, and that they avoid the possibility of recall bias by selecting participants before they develop the disease. See Reference Manual at 557-58; Supplemental Expert Report of Dr. Ritz 3, Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 7 [Dkt. No. 1136-7] ("Ritz Supp. Report"). The AHS is a cohort study of more than 57,000 licensed pesticide applicators from Iowa and North Carolina. The study participants were first surveyed between 1993 and 1997, and were at that time asked about their use of 50 pesticides, including glyphosate. Id. at 2. Participants were asked not only about years of use and days of use per year, but also about other features of their pesticide application that could affect the intensity of their exposure, including use of personal protective equipment and application method. Id. Sixty-three percent of the participants completed a follow-up telephone interview approximately five years later. Id. That survey asked about the participants' pesticide use during the most recent year in which they farmed. Id. Cancer outcomes for members of the cohort were determined through cancer registries. Id. When the initial round of expert reports in this case was prepared, the most recent published study addressing the relationship between glyphosate and NHL as observed in the AHS was a 2005 study, whose lead author was again De Roos. The De Roos (2005) study was published before data from the follow-up surveys were analyzed. It reported no statistically significant association between glyphosate use and NHL, considering 92 total observed cases of NHL—a fully adjusted odds ratio of 1.1 (0.7-1.9) for ever having used glyphosate, with no evidence of higher rates of disease with more days of exposure. Id. at 51-52. The meta-analyses mentioned previously—Schinasi & Leon (2014), Chang & Delzell (2016), and IARC's meta-analysis—incorporated this study.

Gabriella Andreotti et al., Glyphosate Use and Cancer Incidence in the Agricultural Health Study , 110 Journal of the National Cancer Institute 1 (2018), Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 1 at 1-2 [Dkt. No. 1136-1] ("Andreotti (2018)").

See Anneclaire J. De Roos et al., Cancer Incidence Among Glyphosate-Exposed Pesticide Applicators in the Agricultural Health Study , 113 Environmental Health Perspectives 49 (2005), Wagstaff Decl. ISO Pls.' Opp'n Ex. 72 [Dkt. No. 653-12] ("De Roos (2005)").

Several of the experts discussed an unpublished reanalysis of the AHS data during the first round of expert reports, "Alvanja (2013)." Chang and Delzell, authors of the 2016 meta-analysis, prepared an unpublished "technical memorandum" revisiting their meta-analysis, replacing the AHS (2005) data with data from Alvanja (2013) and incorporating the unpublished NAPP data into their sensitivity analyses. Wagstaff Decl. ISO Pls.' Opp'n Ex. 56 [Dkt. No. 652-10]. Data from these unpublished studies were provided by Monsanto. The primary meta-risk ratio reported in this memorandum for ever having used glyphosate was 1.2 (0.91-1.6). Id. at 5.

A few months before the Daubert hearing, an update of the De Roos (2005) study was published in the Journal of the National Cancer Institute. This update, which is known as "Andreotti (2018)," included data gathered using the follow-up telephone interviews and considered 575 individuals who developed NHL. Andreotti (2018) at 5. Like the 2005 study, Andreotti (2018) reported no association between glyphosate use and NHL. Id. at 4-5. The study broke the cohort into quartiles based on how intensively the study participants had used glyphosate, using a formula that included number of days of use, lifetime years of use, use of protective equipment, and other factors to determine the "intensity-weighted lifetime days of use" for each participant. The results ranged from rate ratios of 0.83 (0.59, 1.18) for the lowest quartile, to 0.88 (0.65, 1.19) for the third-highest quartile. Id. The study also reported results that took into account different possible latency periods, and these results likewise showed no statistically significant association. Id. at 6. At the Court's request, the parties submitted supplemental briefs addressing the import of this newly-published study. Dkt. No. 761. All of the plaintiffs' experts submitted supplemental reports addressing the study, and Monsanto's epidemiology experts did the same.

The plaintiffs' experts identified concerns with this study. First among these is the risk of exposure misclassification. Dr. Ritz highlighted potential problems with both the way pesticide exposure was assessed during the initial survey and the way the follow-up survey was conducted. See Ritz Supp. Report 2-7. Inaccurate exposure assessments were likely during the initial survey, she argued, because the initial data were obtained from people applying for pesticide applicator licenses who were asked on the spot to recall their use of many pesticides over the past several decades. They did not have an opportunity to check their records or otherwise verify their answers. Id. at 2-3.

Dr. Ritz also highlighted problems with the questionnaire's inquiry about the use of personal protective equipment, noting that the survey asked only about the use of protective equipment generally, not about the use of such equipment when applying glyphosate. Because the intensity-weighted results for each pesticide relied on the same generic response regarding protective gear, participants were likely classified into incorrect exposure groups. Mar. 5, 2018 Tr. [Ritz] 72-76. For example, if a farmer had in mind the protective gear he used for his most toxic pesticides when he answered the question, even if he used no protective gear when applying glyphosate, he would be placed in a lower exposure group for the intensity-weighted analysis of glyphosate. Another consideration noted by Dr. Ritz is that, because study participants were in the process of applying for their pesticide applicator licenses, they may have felt an incentive to portray themselves in their responses as using protective gear properly even if they did not actually do so. Id. at 74-75.

The plaintiffs' experts also contend there is a particular risk of misclassification where glyphosate is concerned (compared to other pesticides studied in the AHS), because use patterns changed so dramatically in the mid-1990s. Ritz Supp. Report 5. Glyphosate use greatly increased during that period with the introduction of glyphosate-resistant genetically engineered crops. Dr. Ritz elaborated on this concern at some length, arguing that the change in glyphosate use patterns was not adequately captured by the follow-up study for two main reasons. First, the follow-up survey asked only about pesticide use during the last year of farming prior to the interview, rather than asking about all the intervening years. Id. at 6. Second, the study authors imputed the exposures of the approximately thirty-seven percent of participants who did not respond to the follow-up survey using a mathematical model. That imputation, the plaintiffs' experts argued, is also susceptible to error. Id. at 6-7; Supplemental Expert Report of Dr. Portier 2-4, Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 22 [Dkt. No. 1136-22] ("Portier Supp. Report"). They highlighted a published paper evaluating the AHS imputation method that reported the model underestimated glyphosate exposure when tested against a sample of those who had responded to the survey. According to Dr. Portier, use of this imputation method likely resulted in differential exposure misclassification. Apr. 6, 2018 Tr. [Portier] 49-50, 56-57. Moreover, Dr. Portier contended, the differences in the total percentage of people exposed could have masked a misclassification of much larger magnitude, had the imputation model also misclassified some of the exposed people as unexposed. Apr. 6, 2018 Tr. [Portier] 53-56. In addition, the model assumed that non-response to the follow-up survey was random, leaving open the possibility that non-responders were meaningfully different from those who responded to the survey. See Heltshe (2012) at 8.

During his second round of Daubert testimony, Dr. Portier presented a series of hypothetical examples seeking to explain the risk of exposure misclassification associated with the imputation method in the most recent AHS study. Apr. 6, 2018 Tr. [Portier] 51-57. These examples were not included in his supplemental expert report. However, Dr. Portier explained in his supplemental report that the imputation method could have resulted in differential exposure misclassification and pointed to the 2012 study from which he obtained the numbers he used in his Daubert presentation. Portier Supp. Report 3. Under ordinary pretrial circumstances, it would be a closer question whether to exclude these new hypotheticals; there would be an argument that Monsanto lacked sufficient time to prepare to address them before the jury trial. However, in the context of these MDL proceedings, the Court concludes they need not be excluded. Cf. In re Seroquel Products Liability Litigation , No. 6:06-md-1769-Orl-22DAB, 2009 WL 3806435, at *13 (M.D. Fla. June 23, 2009). For one, Monsanto will have adequate time to prepare further cross-examination relevant to these charts between now and the next phase of the proceedings. For another, although the charts themselves reflected additional analysis, that analysis elaborated on Dr. Portier's previously disclosed opinions.

See Apr. 6, 2018 Tr. [Portier] 49-50, 56-57; Sonya L. Heltshe et al., Using Multiple Imputation To Assign Pesticide Use for Non-Responders in the Follow-Up Questionnaire in the Agricultural Health Study , 22 Journal of Exposure Science & Environmental Epidemiology 1, 11, 18 (2012), Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 31 [Dkt. No. 1136-31] ("Heltshe (2012)") (showing an observed prevalence of glyphosate exposure of 52.73%, compared to an imputed prevalence of 45.42% in a holdout dataset used to test the accuracy of the model).

Monsanto's experts mounted a strong defense of this study, pointing out that it considered by far the largest number of NHL cases across a broad range of exposures and for the longest period of time. Supplemental Expert Report of Dr. Mucci 6, Hollingsworth Decl. ISO Def.'s Supp. Br. Ex. 7 [Dkt. No. 1137-8] ("Mucci Supp. Report"); Supplemental Expert Report of Dr. Rider 6, 9, Hollingsworth Decl. ISO Def.'s Supp. Br. Ex. 8 [Dkt. No. 1137-9] ("Rider Supp. Report"). In addition, Monsanto argues, the results are appropriately controlled for confounding by lifestyle factors and other pesticides. Mucci Supp. Report 7. To rebut the critiques of the plaintiffs' experts, Monsanto's experts highlighted the sensitivity analyses used by the study authors, as well as the efforts taken to validate the imputation method used to estimate the missing responses and to demonstrate that selection bias with respect to those who completed the follow-up interview was not a serious concern. Id. at 3-7; Rider Supp. Report 4, 10; Mar. 9, 2018 Tr. [Mucci] 905-09. In short, Monsanto's experts reasonably consider the most recent AHS publication to be the most powerful evidence regarding the relationship between glyphosate and NHL. Because this study shows no association, Monsanto argues, there is no basis for finding a causal relationship.

* * *

The upshot of all this is that the epidemiology evidence is open to different interpretations, and the potential flaws in the data from the case-control studies and meta-analyses are not overwhelmingly greater than the potential flaws in the data from the AHS study. An expert operating "within the range of accepted standards governing how scientists conduct their research and reach their conclusions" could thus place less weight on the AHS study, and could conclude that the analyses of the case-control studies support an association between glyphosate exposure and NHL, even if this is not necessarily the best interpretation of the evidence. Daubert II , 43 F.3d at 1317. As a result, an expert who places more weight on the case-control studies than the AHS study cannot be excluded as categorically unreliable for doing so.

IV. LABORATORY ANIMAL CANCER STUDIES

In addition to the epidemiological evidence, the plaintiffs seek to support their general causation arguments with opinions addressing studies of cancer in rodents.

Monsanto objects to the plaintiffs' experts' reliance on these studies to support their causation opinions, arguing that any opinions based upon these data fail the relevance or "fit" requirement of the Daubert inquiry. In effect, Monsanto argues that for opinions addressing this evidence to be admissible, the plaintiffs must show that it is appropriate to extrapolate directly from increased incidences of particular rodent tumors to an increased incidence of NHL in humans at human-relevant exposure levels. That is not necessary. It's true that, where animal studies provide the best available evidence of causation, the experts seeking to rely upon such evidence must explain why the results in animals are relevant to humans. See Domingo ex rel. Domingo v. T.K. , 289 F.3d 600, 606 (9th Cir. 2002) ; In re Silicone Gel Breast Implants Products Liability Litigation , 318 F. Supp. 2d 879, 891 (C.D. Cal. 2004) ("Animal studies are not generally admissible where contrary epidemiological evidence in humans exists."). But the parties don't face that scenario here.

It is sufficient for purposes of the Rule 702 relevance inquiry that the evidence "logically advance[ ] a material aspect of the proposing party's case." Daubert II , 43 F.3d at 1315. Demonstrating that a chemical is carcinogenic in rodents would logically advance the plaintiffs' argument that glyphosate is capable of causing NHL in humans, because it is pertinent to, at least, the biological plausibility criterion that is part of the Bradford Hill analysis. See , e.g. , Mar. 7, 2018 Tr. [Jameson] 429 ("This is a premise that is generally accepted in the scientific community, that if an agent causes a[ ] cancer in animals, that it's biologically plausible to be a human carcinogen.") [Dkt. No. 1181]. Rodent cancer studies are routinely conducted to learn information that is useful in assessing whether substances cause cancer in humans. So, while the rodent studies would not be sufficient on their own to satisfy the plaintiffs' burden (at least in this case), the rodent studies nevertheless "speak[ ] clearly and directly to an issue in dispute in the case," and they will not mislead the jury when properly contextualized. Daubert II , 43 F.3d at 1321 n.17.

See Bernard D. Goldstein & Mary Sue Henifin, Reference Guide on Toxicology , in Reference Manual on Scientific Evidence 633, 637 (3d ed. 2011) ("[T]he toxic responses in laboratory animals are useful predictors of toxic responses in humans."); see also In re Silicone Gel Breast Implants Products Liability Litigation , 318 F. Supp. 2d at 890.

For these reasons, although IARC's overall conclusion that glyphosate is a "probable human carcinogen" is not squarely relevant to the general causation question in this case, IARC's narrower conclusion about carcinogenicity in lab animals is quite relevant. If there is sufficient evidence that glyphosate causes cancer in animals, as IARC concluded, that would support the plaintiffs' case. And IARC's analysis itself suggests that such a conclusion is within the mainstream of scientific views regarding how to interpret the available animal cancer studies. See Reference Manual at 564 n.46.

As with the epidemiological studies, the parties' experts generally agree about which underlying animal studies are worthy of close consideration. The studies at issue are cancer bioassays that assess the development of tumors (both benign and malignant) in rodent subjects after chronic exposure to different doses of glyphosate over most of their lifetimes. See, e.g. , id. at 640-41, 644-45; Expert Report of Dr. Rosol 3, Wagstaff Decl. ISO Pls.' Opp'n Ex. 97 [Dkt. No. 655-7 at 124] ("Rosol Report"); Expert Report of Dr. Jameson 19, Wagstaff Decl ISO Pls.' Opp'n Ex. 6 [Dkt. No. 648-6] ("Jameson Report"). Included in these studies is a control group subject to the same conditions—regarding food, light exposure, or exercise, for example—as the experimental group in every respect except for exposure to the chemical of interest. See Reference Manual at 640. The rodents in these long-term studies are typically exposed to doses that are significantly higher, relative to body mass, than what humans realistically would experience, as the goal is to maximize the studies' ability to detect the chemical's capacity to cause cancer. Id. at 644-45.

In contrast to the epidemiology studies, much of the data on experimental animals were not presented in studies published in peer reviewed journals. Instead, the data tend to come from studies submitted by manufacturers to regulatory agencies. To the extent the data underlying these studies are public, the data are generally considered by IARC, and they were considered by the experts in this case. See Monograph at 12; Apr. 6, 2018 Tr. [Portier] 186. One source of much of the data for the experts here was a supplement to a review article published in 2015, which included tumor incidence tables from many of the regulatory submissions.

Helmut Greim et al., Evaluation of Carcinogenic Potential of the Herbicide Glyphosate, Drawing on Tumor Incidence Data from Fourteen Chronic/Carcinogenicity Rodent Studies , 45 Critical Reviews in Toxicology 185, 185 (2015) ("Greim (2015)"). Although IARC reviewed Greim (2015), it was unable to evaluate in detail several of the studies considered by the experts here. Mar. 7, 2018 Tr. [Jameson] 455-56; Monograph at 354.

As with the epidemiology, the experts also broadly agreed on the method to be employed in evaluating animal toxicology studies. They conducted literature reviews and assessed study quality, excluding those studies about which inadequate information was available or that had serious methodological problems. Although there is some disagreement at the margins, the experts focused primarily on seven rat studies and five mouse studies. See Jameson Report 28-29; Portier Report 50; Rosol Report 9-19; Expert Report of Dr. Foster 13-25, Wagstaff Decl. ISO Pls.' Opp'n Ex. 37 [Dkt. No. 649-7] ("Foster Report"). Then, broadly speaking—although the details differ—the experts assessed the tumors that arose in the studies for statistical and biological significance. Relevant to the first aspect of this analysis is both whether there was a statistically significant increase in tumor development in a particular dose group, as compared to the control group, and whether the numbers of tumors that developed in the treated groups showed a statistically significant trend as the dosage of glyphosate increased. The experts also agreed that data from concurrent controls—the rodents in the control group of the same study—were most important. But they acknowledged that the rate of tumor incidence in historical control groups—control groups used in previous, similar studies—was relevant as an indicator of how many spontaneous tumors could be expected. The experts disagreed, however, about how and to what extent to consider historical control information. A further important consideration in assessing whether a chemical causes cancer in rodents is whether particular tumor findings were replicated across gender, subtype, species, or study. Monsanto does not dispute the reliability of this method on the whole, instead critiquing specific aspects the plaintiffs' experts' application. These critiques will be discussed in Section VI.

See U.S. Environmental Protection Agency, Guidelines for Carcinogen Risk Assessment 2-19 (2005), https://www.epa.gov/sites/production/files/2013-09/documents/cancer_guidelines_final _3-25-05.pdf [https://perma.cc/G878-YJLC] ("EPA, Guidelines for Carcinogen Risk Assessment"). The EPA Guidelines go on to explain:

Trend tests and pairwise comparison tests are the recommended tests for determining whether chance, rather than a treatment-related effect, is a plausible explanation for an apparent increase in tumor incidence. A trend test such as the Cochran-Armitage test (Snedecor and Cochran, 1967) asks whether the results in all dose groups together increase as dose increases. A pairwise comparison test such as the Fisher exact test (Fisher, 1950) asks whether an incidence in one dose group is increased over that of the control group. By convention, for both tests a statistically significant comparison is one for which p is less than 0.05 that the increased incidence is due to chance. Significance in either kind of test is sufficient to reject the hypothesis that chance accounts for the result.

V. MECHANISTIC DATA

The final category of evidence the plaintiffs seek to put before the jury addresses possible mechanisms at the cellular level by which glyphosate could cause cancer. The plaintiffs identify two possible mechanisms they contend are supported by the scientific literature: genotoxicity and oxidative stress.

Monsanto again disputes the relevance of this body of literature, arguing that the objectives of studies at the cellular level are far afield from the question of general causation. However, for much the same reason that the experts' opinions on the rodent studies are relevant, the plaintiffs' experts' opinions regarding the mechanistic evidence are also relevant: the mechanistic evidence pertains to biological plausibility. Evidence that glyphosate causes damage to the genetic material in cells (genotoxicity ) or an imbalance between the production of reactive oxygen species and antioxidant defenses in a cell (oxidative stress ) supports the plaintiffs' argument that it is biologically plausible that glyphosate acts as a carcinogen. See In re Denture Cream Products Liability Litigation , 795 F. Supp. 2d 1345, 1356 (S.D. Fla. 2011) ("When mechanistic evidence is present it can greatly strengthen a causal inference, but when it is absent it does not necessarily undermine the inference." (citation and alteration omitted) ). This is not a scenario where the plaintiffs are relying on mechanistic studies alone to justify their experts' causal inferences; mechanistic evidence "may supplement the more substantial evidence of general causation in this case." In re Abilify , 299 F. Supp. 3d at 1339.

Monsanto further argues that any opinion that relies upon two human studies—which the parties refer to as "Paz-y-Miño (2007)" and "Bolognesi (2009)"—must be excluded because the methodologies of those studies are so flawed that any opinion based on them is necessarily unreliable. These studies considered possible genotoxic effects of glyphosate in people following aerial spraying in Colombia and Ecuador. They are "in vivo" studies of cells in whole, living organisms, as opposed to "in vitro" studies of cells outside their normal biological contexts.

César Paz-y-Miño et al., Evaluation of DNA Damage in an Ecuadorian Population Exposed to Glyphosate , 30 Genetics & Molecular Biology 456 (2007), Wagstaff Decl. ISO Pls.' Opp'n Ex. 109 [Dkt. No. 656-4] ("Paz-y-Miño (2007)"); C. Bolognesi et al., Biomonitoring of Genotoxic Risk in Agricultural Workers from Five Colombian Regions: Association to Occupational Exposure to Glyphosate , 72 Journal of Toxicology & Environmental Health 986 (2009), Wagstaff Decl. ISO Pls.' Opp'n Ex. 110 [Dkt. No. 656-5] ("Bolognesi (2009)").

Studies are not admissible simply because they are published. See In re Viagra Products Liability Litigation , 658 F. Supp. 2d 936, 945 (D. Minn. 2009). The two human in vivo studies Monsanto targets have flaws, some of which are acknowledged by the study authors themselves. See Bolognesi (2009) at 995 (acknowledging the possibility of misclassification of exposures and "the need to use better procedures to estimate the exposure"). For instance, there was a delay between glyphosate exposure and the genotoxicity assessment in Paz-y-Miño (2007), and some of the study participants showed symptoms suggesting acute illness. See Paz-y-Miño (2007) at 457. But none of these flaws is so glaring that an expert who relies on the studies in assessing all the evidence going to whether glyphosate has a genotoxic effect, as the plaintiffs' experts and IARC did, is necessarily unreliable.

VI. CONCLUSIONS REGARDING THE PLAINTIFFS' EXPERTS

The parties' experts offer contrasting takes on how to assess the evidence discussed in the three preceding sections. It is a given that there will be disagreement among reasonable scientists about which evidence to emphasize in cases where the evidence does not point unequivocally toward a particular conclusion. See, e.g. , Milward , 639 F.3d at 18. The question here is whether the plaintiffs' experts' analysis of these studies "falls within the range of accepted standards governing how scientists conduct their research and reach their conclusions." Daubert II , 43 F.3d at 1317.

Although the plaintiffs' experts specialize in various scientific disciplines, they all engage in some version of a Bradford Hill analysis (perhaps with the exception of Dr. Jameson, as discussed below). Recall that the Bradford Hill approach to assessing whether an association is causal takes into account: strength of association, consistency across studies, specificity of the association, temporality, dose response, biological plausibility, coherence, experimental evidence, and analogous compounds.

As mentioned in Section III, the Bradford Hill criteria are generally associated with epidemiology, and a reliable assessment that an association between glyphosate and NHL exists in the epidemiological literature is a prerequisite to application of the criteria. See Reference Manual at 597. As a practical matter, however, application of these criteria requires an expert to consider more than the epidemiology literature. In particular, by inquiring about biological plausibility and coherence with other knowledge, the Bradford Hill framework asks experts to survey all the available evidence that might support or disprove causation. A broad survey of the available evidence is neither unusual in expert testimony nor necessarily inappropriate. See, e.g. , Milward , 639 F.3d at 19-20 ; In re Neurontin Marketing, Sales Practices & Products Liability Litigation , 612 F. Supp. 2d 116, 158-59 (D. Mass. 2009). However, this feature of the Bradford Hill methodology poses some challenges for a reviewing court, as the sweep of an expert's opinion is likely to be quite broad, the inquiry involves the exercise of subjective judgment, and an expert may opine on matters outside of her core area of expertise.

To the extent the Daubert question is whether consideration of the Bradford Hill factors is a reliable method for determining causation as a general matter, the answer is yes. See, e.g. , Wendell , 858 F.3d at 1235 n.4 ; In re Zoloft , 858 F.3d at 795-97. Although it is not the sort of scientific process that is amenable to objective testing, or that has a known or potential error rate, none of the experts dispute that this method of evaluating scientific evidence is generally accepted in the field of epidemiology. See Daubert II , 43 F.3d at 1316 ; Lust By & Through Lust v. Merrell Dow Pharmaceuticals, Inc. , 89 F.3d 594, 597 (9th Cir. 1996) (noting that "testing and rate of error ... do not apply, however, when the expert has not done original research, but rather has surveyed available literature and drawn conclusions that differ from those presented by the scientists who performed the original work"). What matters more in this case is whether the way the experts assessed each of the Bradford Hill factors is reliable in light of the underlying evidence. The experts must also show that the analytical leaps required to reach their ultimate conclusions regarding glyphosate's ability to cause NHL in humans are supportable, in light of the evidence on which they relied. See Joiner , 522 U.S. at 146, 118 S.Ct. 512.

A. Dr. Portier

Dr. Portier is a biostatistician whose graduate research focused on the design of rodent studies and who, among other things, worked for much of his career at the Center for Disease Control's National Center for Environmental Health and at the National Institutes of Health's National Institute of Environmental Health Sciences. Mar. 7, 2018 Tr. [Portier] 540-41; Portier Report 1-3. Although Dr. Portier's PhD is in biostatistics and his primary focus is on toxicology and mechanistic studies, he has reviewed epidemiology studies throughout his career and has published in the field. Apr. 6, 2018 Tr. [Portier] 13-14, 16-21. Accordingly, although epidemiology is not his core area, he is qualified to examine the epidemiology literature to see whether an association exists and, if so, to engage in a Bradford Hill analysis.

Dr. Portier conducted a literature review of the epidemiological evidence and, as to the epidemiological evidence alone, agreed with IARC's conclusion that the evidence supported a credible causal interpretation but could not definitively rule out chance, bias, or confounding. See Portier Report 6; Mar. 8, 2018 Tr. [Portier] 618-19; Apr. 6, 2018 Tr. [Portier] 78-79. As an initial matter, Monsanto makes a non-frivolous argument that Dr. Portier's description of what the epidemiology evidence shows—a description that several of the plaintiffs' experts shared—entitles it to summary judgment. However, the better conclusion is that the plaintiffs' experts need not derive their causation conclusion exclusively from that body of evidence. If other bodies of knowledge tend to bolster a causal interpretation of studies that could not alone establish causation, an expert need not be excluded based on an opinion that the epidemiology evidence alone is limited in the way Dr. Portier describes.

Although Dr. Portier agreed with IARC's assessment, his expert report did not simply reiterate IARC's conclusions. In analyzing the epidemiology evidence, Dr. Portier emphasized numbers adjusted for use of other pesticides, particularly those from the Chang & Delzell (2016) and IARC meta-analyses and the De Roos (2003) study. He considered the possible roles that chance, confounding, small sample sizes, and recall bias might have played in explaining the observed results. See, e.g. , Portier Report 11. He also explained that he discounted the Andreotti (2018) study in light of possible exposure misclassification arising from the study design, the dramatic increase in glyphosate use over the course of the AHS, and the authors' imputation of exposures for the sizable portion of the cohort that did not respond to the follow-up survey. Portier Supp. Report 3-4.

As noted, reliably identifying an association between NHL and glyphosate is a necessary predicate to reliable application of the Bradford Hill criteria. See Bradford Hill at 295; In re Lipitor (Atorvastatin Calcium) Marketing, Sales Practices & Products Liability Litigation (No. II) , 892 F.3d 624, 640 (4th Cir. 2018). Dr. Portier does not in his report first pause to establish an association. Even though Dr. Portier did not structure his report in this way, however, it is clear that he identified an association between glyphosate and NHL. What primarily persuaded Dr. Portier that an association existed was the consistency of the observed associations across different case-control studies. Portier Report 15. He acknowledged that, using the most highly adjusted numbers, the increases in NHL observed with exposure to glyphosate were "modest"—generally under 2.0—and were not always statistically significant. Id. at 15, 19. But he concluded it was unlikely that so many studies would report results above 1.0, whether statistically significant or not, if there was no true association. Id. at 14-16. This is thus not a scenario where an expert attempted to deploy the Bradford Hill "guidelines to support the existence of causation in the absence of any epidemiologic studies finding an association," given how Dr. Portier interprets the studies. In re Lipitor , 892 F.3d at 640 (citations omitted).

Dr. Portier conducted his Bradford Hill analysis as follows: He concluded that the epidemiology studies addressed exposures occurring prior to disease onset, and therefore that the temporality criterion—the only non-discretionary Bradford Hill factor—was satisfied. Portier Report 75. As to the strength of the observed association, Dr. Portier acknowledged that the observed odds ratios showed a "moderate" association, and that it was therefore "conceivable they are individually due to either chance or bias." Id. at 18. Although the magnitude of the observed association in each individual study was not especially large, another Bradford Hill criterion, consistency, allayed his concerns about chance and bias, leading him ultimately to conclude that the case-control studies "demonstrate a significant strength of association." Id. at 19. His opinion that the consistency criterion provided strong support for causation emphasized the Chang & Delzell (2016) meta-analysis, which showed little heterogeneity between studies and remained stable after sensitivity analyses. Id. at 15-17. He also considered possible sources of bias or confounding that might explain the consistency but noted, among other things, that several of the studies controlled for other pesticides without erasing the observed association. Id. at 17-18. Dr. Portier further concluded, based on two case-control studies and the AHS, that dose response, or biological gradient, was demonstrated to a moderate degree by the epidemiological studies. Id. at 74-75. Dose response—which refers to whether there is an increased risk of contracting a disease associated with higher levels of exposure to the agent—is strong but not necessary evidence of a causal relationship. Reference Manual at 603.

Monsanto argues that the plaintiffs must be able to show a statistically significant odds ratio of greater than 2.0 to survive summary judgment at the general causation stage. Controlling case law does not support that proposition. See In re Hanford Nuclear Reservation Litigation , 292 F.3d at 1137 ; see also In re Bextra & Celebrex Marketing Sales Practices & Product Liability Litigation , 524 F. Supp. 2d 1166, 1172-73 (N.D. Cal. 2007) (explaining that a relative risk of greater than 1.0 is relevant to general causation, while a relative risk of 2.0 can be probative of specific causation).

Dr. Portier further concluded that the biological plausibility criterion "very strong[ly]" supported causation. Portier Report 77. He focused much of his report on this point, analyzing both the rodent carcinogenicity studies and the studies addressing possible cellular mechanisms of action in conjunction with this factor. Id. at 19-74. He again relied on this evidence, along with studies showing absorption and excretion of glyphosate by exposed humans, in support of Bradford Hill's "coherence" criterion, which asks whether a causal interpretation of the association conflicts with other information known about the disease. Id. at 75-76. He concluded this criterion strongly supported a causal assessment. Id. at 77. Because there are causes of NHL aside from glyphosate, Dr. Portier concluded "[t]here is little support for specificity." Id. at 75; see also id. at 77 (stating that specificity is "[n]ot needed"); Apr. 6, 2018 Tr. [Portier] 75 (stating that specificity "doesn't add to the causation argument"). He did not rely on the criteria considering analogous compounds and evidence from human experimental studies in reaching his causation opinion, citing his lack of information about the former and a lack of data altogether as to the latter. Portier Report 76-78.

With respect to Dr. Portier's epidemiology-related conclusions—both his finding of an association between glyphosate exposure and NHL and his application of the Bradford Hill factors that turn on the epidemiology studies—it is not difficult to conclude that much of his analysis is sufficiently reliable to be admissible. For example, as discussed more fully in Section III, it was reasonable for Dr. Portier to rely more heavily on the case-control studies than the AHS. To briefly recap, there is a legitimate concern about exposure misclassification in the AHS. With respect to the case-control studies, Dr. Portier addressed the most significant concern—the possibility that pesticides other than glyphosate caused the observed cases of NHL—by focusing on data adjusted for potential confounding by various other pesticides. See In re Abilify , 299 F. Supp. 3d at 1322-23. Monsanto's other critiques of the case-control studies, like the possible presence of recall bias or the short period between glyphosate exposure and diagnosis in some of the studies, are not significant enough to require an expert categorically to weight them less heavily than the AHS. And having reasonably decided to rely heavily on the case-control studies, Dr. Portier's conclusion that a true association exists between glyphosate and NHL, as well as his conclusion that the Bradford Hill "consistency" criterion was satisfied, was not an unreasonable logical leap.

On the other hand, some of Dr. Portier's epidemiology-related conclusions follow less clearly from the studies—particularly those relating to strength of association and dose response. Regarding the former, it seems like a stretch to conclude, as Dr. Portier seems to have done, that the association between glyphosate use and NHL is strong. See Apr. 6, 2018 Tr. [Portier] 68. Even if one completely discounted the AHS (which Dr. Portier claims not to have done), virtually all the adjusted odds ratios from the case-control studies are below 2.0, and many of them are not statistically significant. As discussed in Section III, data may well be informative even in the absence of statistical significance, but one would expect a more cautious assessment regarding the strength of association in light of these numbers, particularly when one remembers that the case-control studies have vulnerabilities of their own. And when the AHS is given some weight (as Dr. Portier apparently agrees it should), the overall picture from the data becomes fuzzier still.

With respect to dose response, it's true that some of the data from the case-control studies support Dr. Portier's conclusion, but other data do not, as he acknowledged. Eriksson (2008) reported a higher odds ratio—2.36 (1.04, 5.37)—for those who used glyphosate for greater than ten days than for those who used it for ten or fewer days—1.69 (0.70, 4.07). McDuffie (2001) reported odds ratios of 1.0 (0.63, 1.57) for those who used glyphosate between zero and two days per year, and of 2.12 (1.2, 3.73) for those who used it for greater than two days per year. Dr. Portier also concluded that the rodent carcinogenicity studies demonstrated a dose response. Yet neither of the published AHS studies, which used much more detailed exposure metrics, demonstrated a dose response. See Portier Report 74-75; Apr. 6, 2018 Tr. [Portier] 140. Although the better conclusion might be that these data are inconclusive, Dr. Portier's assessment that the biological gradient criterion is moderately supportive of a causal association does not constitute an unsupported scientific leap. See Joiner , 522 U.S. at 146, 118 S.Ct. 512.

More broadly, Dr. Portier's epidemiology-related conclusions, even tempered as they are by the recognition that the epidemiology evidence alone does not show causation, are far from unassailable. There is one large cohort study (the AHS), with results recently published in a well-regarded scientific journal, suggesting no association between glyphosate use and NHL. There is a series of case-control studies arguably suggesting an association, but a fairly weak one. There are limited data indicating that the association strengthens with greater exposure to glyphosate, but also data to the contrary. And there are legitimate concerns about the reliability of the data from all the studies. Under these circumstances, all one might expect an expert to conclude is that glyphosate exposure is cause for concern, but not that glyphosate is likely causing NHL at realistic human exposure levels.

But, as noted at the beginning of this ruling, the Daubert inquiry does not require (or even allow) a district court to exclude an expert's opinion merely because the court is not persuaded that the expert's read of the evidence is the best one. See, e.g. , City of Pomona , 750 F.3d at 1044 ("The district court is not tasked with deciding whether the expert is right or wrong, just whether his testimony has substance such that it would be helpful to a jury." (citation and alteration omitted) ); Quiet Technology DC-8, Inc. v. Hurel-Dubois UK Ltd. , 326 F.3d 1333, 1341 (11th Cir. 2003) ("[I]t is not the role of the district court to make ultimate conclusions as to the persuasiveness of the proffered evidence."); Daubert II , 43 F.3d at 1318 ("[T]he test under Daubert is not the correctness of the expert's conclusions but the soundness of his methodology."); In re TMI Litigation , 193 F.3d 613, 665 (3d Cir. 1999), amended , 199 F.3d 158 (3d Cir. 2000) (explaining that plaintiffs "do not have to demonstrate to the judge by a preponderance of the evidence that the assessments of their experts are correct, they only have to demonstrate by a preponderance of evidence that their opinions are reliable" (citation omitted) ). It bears repeating that applying the Bradford Hill criteria involves a certain amount of subjectivity, and experts often will disagree when doing so. The job of the district court is merely to ensure that the expert's methods are not so far outside the realm of reasonable scientific practice that his testimony would be unhelpful or misleading to a jury. See Messick , 747 F.3d at 1199. The Court must also assure itself that the expert's conclusions are not based upon unreasonable extrapolations from the existing data. See Joiner , 522 U.S. at 146, 118 S.Ct. 512. Monsanto can cross-examine Dr. Portier on the apparent weaknesses in his analysis, and there is little reason to think that a jury will not understand those weaknesses. But the aspects of his opinion based upon the epidemiology evidence have a sufficiently reliable basis in the methods of that discipline for the jury to consider his testimony about that evidence, including his assessments of the strength, consistency, and dose-response Bradford Hill criteria. See Messick , 747 F.3d at 1197.

Dr. Portier's testimony regarding the contested Bradford Hill factors that do not depend primarily upon epidemiology evidence—namely, biological plausibility and coherence—is also admissible, with one exception.

Dr. Portier first supported his biological plausibility conclusion with a determination that sufficient evidence shows that glyphosate causes cancer in two strains of rats and one strain of mice. Portier Report 52. One of Monsanto's major critiques of this portion of his analysis concerns his decision to use a pooling method to analyze together the results of the various rodent carcinogenicity studies. Dr. Portier's expert report combined the results of similar studies, treating the resulting data as "one big bioassay," then analyzed the results using a Cochran-Armitage trend test. Mar. 7, 2018 Tr. [Portier] 579. In response to critiques from one of Monsanto's experts, he then conducted an additional analysis using logistic regression, which he contended provided similar results. Id. at 579-80. He also conducted sensitivity analyses in conjunction with his pooling that sought to isolate the effects of studies that, for instance, had a very high rate of tumor incidence in the control and all dose groups. Id. at 577-84.

Although some version of Dr. Portier's pooled approach may well gain traction as a means of evaluating the results of multiple rodent studies, it fares poorly under the traditional Daubert criteria. His pooling approach is not subject to objective testing, and it appears to have no identifiable error rate. Although neither of these shortcomings itself requires exclusion, Dr. Portier's method also has not gained general acceptance in the scientific community, nor does it appear to have been subjected to peer review and publication. See Estate of Barabin v. AstenJohnson, Inc. , 740 F.3d 457, 463 (9th Cir. 2014).

Dr. Portier seemed to acknowledge that his approach was novel, but argued it was still a reliable way to assess multiple animal studies. Mar. 8, 2018 Tr. [Portier] 638 [Dkt. No. 1183]; Portier Report 21 ("Methods for the combined analysis of multiple animal cancer bioassays are not available in the scientific literature."). Dr. Portier later pointed to two studies by another scientist that he contended used a similar pooling analysis. Mar. 8, 2018 Tr. [Portier] 635. But it appears that the pooling used in these studies combined male and female rodents from the same study, or that the authors displayed results from studies of different lengths in a single figure to model a dose-response curve, rather than combining rodents from multiple separate studies to determine whether given tumor findings were significant in the way that Dr. Portier does. As evidence that Dr. Portier's method has gained acceptance, the plaintiffs pointed to comments by members of the EPA's Science Advisory Panel indicating that pooling the studies here would be appropriate. However, although some members of the panel evidently found Dr. Portier's proposed approach promising, the report of the panel meeting says only that some "[p]anelists recommend that EPA adopt a pooled analysis approach for combining multiple studies." Wagstaff Decl. ISO Pls.' Opp'n Ex. 10 at 59 [Dkt. No. 648-10]. The report continues, "[a]dopting a pooled analysis approach should include the development of full guidelines for how to conduct and evaluate these analyses," suggesting that the details of what might constitute a reliable way to conduct a pooled analysis remained to be determined. Id. ; see also Mar. 8, 2018 Tr. [Portier] 638.

See Michael L. Dourson et al., Update: Mode of Action (MOA) for Liver Tumors Induced by Oral Exposure to 1,4-dioxane , 88 Regulatory Toxicology & Pharmacology 45, 46-50 (2017), Wagstaff Decl. ISO Pls.' Opp'n, Ex. 104 [Dkt Nos. 655-14]; Michael Dourson et al., Mode of Action Analysis for Liver Tumors from Oral 1,4-dioxane Exposures and Evidence-Based Dose Response Assessment , 68 Regulatory Toxicology & Pharmacology 387, 391, 394 (2014), Wagstaff Decl. ISO Pls.' Opp'n Ex. 105 [Dkt. No. 655-15]; Mar. 8, 2018 Tr. [Portier] 635-36.

That Dr. Portier has staked his reputation on his pooling analysis in regulatory submissions in addition to doing so in this litigation suggests that this litigation isn't the only force behind this portion of his analysis. But Dr. Portier's pooling method has evolved as he has received feedback from his peers, and his regulatory submissions reflected a somewhat different analysis than the one he presents here. Mar. 8, 2018 Tr. [Portier] 626-35. Further, during cross-examination, Dr. Portier acknowledged an error in his expert report, in which he neglected to present one of his pooled sensitivity analyses of thyroid C-cell tumors in male rats, making it appear that he had not consistently applied his method to all the relevant studies. Mar. 8, 2018 Tr. [Portier] 665-66. All this suggests that Dr. Portier's pooling is a good faith work in progress, but does not yet constitute "the scientific method, as it is practiced by (at least) a recognized minority of scientists in their field." Daubert II , 43 F.3d at 1319. The proper place to refine his pooling approach is not in front the jury.

The question thus becomes whether Dr. Portier's opinion as to the animal studies "nonetheless rests on good grounds." Karlo v. Pittsburgh Glass Works, LLC , 849 F.3d 61, 83 (3d Cir. 2017). Although it is a somewhat close call, it appears that Dr. Portier's other analyses and conclusions are separable from his pooling. Dr. Portier acknowledged that the pooling was "part of [his] analysis and evaluation," but he sought to make clear that his conclusions were not dependent upon it. Mar. 8, 2018 Tr. [Portier] 640. He explained, "The pooled analysis is just a tool for me to better understand the strength of the evidence across multiple studies. Like a meta-analysis or the pooled analysis in epidemiology. Not having it doesn't change the core meaning of the data. And so my opinion of the animal carcinogenicity data wouldn't change just because I couldn't use the pooled analysis." Apr. 6, 2018 Tr. [Portier] 181. Indeed, a significant portion of his rebuttal report was dedicated to disputing one of Monsanto's expert's interpretations of the individual studies. See Rebuttal Report of Dr. Portier 12-24, Wagstaff Decl. ISO Pls.' Opp'n Ex. 96 [Dkt. No. 655-6].

Without pooling, the remainder of his analysis evinces relatively minor disagreements with the other toxicology experts on how to interpret the studies, and his positions in these debates do not depart from the realm of reasonable science. Monsanto criticized the way Dr. Portier addressed the possibility that random chance could explain the statistically significant tumor findings he identified, given the large number of possible tumor sites analyzed. See Mar. 8, 2018 Tr. [Portier] 682-89. In addition to disputing the method he used to account for the role of chance, Monsanto highlighted that the total number of tumor sites included in this portion of his initial report was higher than the number included in his rebuttal report. At the Daubert hearing, however, Dr. Portier explained the discrepancy, citing his decision to depart from his original reliance on the tumor site counts in a comment provided by another scientist to the EPA. Id. at 683-89. Dr. Portier provided a reasonable explanation, and to the extent Monsanto seeks to argue that this change makes his opinion less credible, it is free to do so. See Primiano v. Cook , 598 F.3d 558, 566 (9th Cir. 2010) ("Where the foundation is sufficient, the litigant is entitled to have the jury decide upon the experts' credibility, rather than the judge." (internal quotation marks, citation, and alteration omitted) ).

Monsanto also accused Dr. Portier of engaging in "p-hacking," manipulation of data to obtain statistically significant results. Monsanto used Dr. Portier's treatment of renal tumors observed in a 1983 mouse bioassay as an example of this alleged methodological flaw. See Def.'s Mot. 25-26. Monsanto cites his prior analyses of these data in regulatory submissions, noting that his approach has evolved over time. Yet, although Monsanto takes issue with his use of a measure that takes into account historical controls, it does not provide any reason why use of his other measures, the Cochran-Armitage trend and Fisher exact tests, is an unreliable way to evaluate these data. See EPA, Guidelines for Carcinogen Risk Assessment at 2-19; see also Expert Report of Dr. Corcoran 8, Wagstaff Decl. ISO Pls.' Opp'n Ex. 102 [Dkt. No. 655-12] ("Corcoran Report"). As to his incorporation of historical control data for tumors he deemed rare, Monsanto has legitimate critiques of the way he calculated his statistic. But it is within the realm of reasonable toxicological practice to consider historical control data in some fashion, and Monsanto has not demonstrated that another approach to historical controls is the only reliable one. Indeed, there are reasons one might expect a reliable expert to use caution when dismissing tumor findings simply because they fall within the range of tumors observed in historical controls. See, e.g. , EPA, Guidelines for Carcinogen Risk Assessment at 2-20 to 2-21. Again, Monsanto may highlight discrepancies between Dr. Portier's past approaches and the analysis he presents in this case, and may emphasize what it perceives to be flaws in Dr. Portier's use of historical controls. But the concerns about the opinion he presents here are not sufficient to render his opinion inadmissible.

See, e.g. , Foster Report 11, 16 n.2 (evaluating tumor findings in comparison to the range of historical controls rather than the mean); Rosol Report 5-6; Charlotte Keenan et al., Best Practices for Use of Historical Control Data of Proliferative Rodent Lesions , 31 Toxicologic Pathology 679, 690 (2009), Wagstaff Decl. ISO Pls.' Opp'n Ex. 108 [Dkt. No. 656-3].

Monsanto points out that not even Drs. Jameson and Portier, the two plaintiffs' experts who focused at length on the animal studies, could agree on how to analyze the studies and suggests this is evidence of unreliability. Def.'s Mot. 22 n.31. But, by that score, Monsanto's experts would also be unreliable, as they reached somewhat different conclusions regarding some of the studies, too. Compare, e.g. , Rosol Report 17 (noting in the Stout and Ruecker study a statistically significant increase by pair-wise comparison for pancreatic islet cell adenomas in low-dose males, but concluding the tumors were not treatment related); id. at 17-18 (reporting no compound related or biologically relevant changes in any treatment group in the Wood 2009 study); with Foster Report 16 (noting non-statistically significant neoplastic changes in pancreatic islet cells in the Stout and Ruecker study); id. at 18 (noting in the Wood 2009 study a statistically significant trend for mammary gland adenocarcinomas, and for adenomas and adenocarcinomas combined for the highest dose group in the same study, but concluding the tumors were not compound-related). The Court may not "t[ake] sides on questions that are currently the focus of extensive scientific research and debate—and on which reasonable scientists can clearly disagree." Milward , 639 F.3d at 22.

In sum, with the exception of his pooled analysis, Dr. Portier's assessment of the animal carcinogenicity data is admissible. Some of the statistical tests he applied to the data within the expert reports submitted in conjunction with this case are essentially unchallenged. Monsanto disputes the way he incorporated data on historical controls into his analysis and how he sought to address concerns that his observed statistically significant results could be due to chance. But seeking to account for these factors comports with good scientific practice, and Monsanto has not shown that Dr. Portier has taken a scientifically unacceptable, as opposed to a debatable, approach.

Dr. Portier's second opinion supporting his conclusion that it is biologically plausible that glyphosate causes cancer in humans concerns the mechanistic evidence. As discussed in Section V, his reliance on the human in vivo studies does not disqualify his expert opinion. Dr. Portier acknowledged that some of the results he considered were not statistically significant. See, e.g. , Portier Report 56. He also considered a later Paz-y-Miño study, published in 2011, that showed no effect, which Monsanto cites in disputing the plaintiffs' reliance on the other two human in vivo studies. Id. at 55-56. Dr. Portier additionally took into account myriad other mechanistic evidence, effectively unchallenged by Monsanto in its motion, including a published meta-analysis of in vivo assays that found a statistically significant positive mean response. Portier Report 68-69. Dr. Portier explained that he weighted these studies heavily, as they demonstrate DNA damage in living organisms with intact DNA repair mechanisms, making them more probative of potential DNA damage in humans than in vitro studies. Id. at 69.

See César Paz-y-Miño et al., Baseline Determination in Social, Health, and Genetic Areas in Communities Affected by Glyphosate Aerial Spraying on the Northeastern Ecuadorian Border , 26 Reviews on Environmental Health 45 (2011), Wagstaff Decl. ISO Pls.' Mot. Ex. 111 [Dkt. No. 656-6]; Def.'s Mot. 33 n.66.

Monsanto also argues that Dr. Portier's chart summarizing the study results is unreliable, contending he inappropriately added up the positive studies. Def.'s Mot. 35. But Dr. Portier expressly cautioned against relying too heavily on the table Monsanto disputes, noting that it was simply a summary tool. Portier Report 65 (explaining that the table "does not address the subtlety needed to interpret any one study," but instead "summarizes these studies in a simple framework that allows all of the experimental data to be seen in one glance"); cf. Expert Report of Dr. Jay Goodman 31, Wagstaff Decl. ISO Pls.' Opp'n Ex. 38 [Dkt. No. 649-8] ("Goodman Report") ("While there were occasional positives, some of which might have occurred by chance, among the very numerous tests for genotoxicity, these are far outweighed by the overwhelmingly negative results.").

In short, Monsanto's attacks on Dr. Portier's analysis of the mechanistic data probe his application of the scientific method, but do not demonstrate that the principles and methodology he applied in analyzing these data were not grounded in science. See Wendell , 858 F.3d at 1232.

Stepping back and applying the Daubert factors not already accounted for to Dr. Portier's Bradford Hill analysis: Dr. Portier has not sought to publish his conclusions regarding glyphosate and NHL in a peer-reviewed journal. However, the studies underlying his opinion were in large part published in peer-reviewed journals. See Daubert II , 43 F.3d at 1318 ; cf. Metabolife International, Inc. v. Wornick , 264 F.3d 832, 845 (9th Cir. 2001) (concluding that experts who "explain[ed] the methodology of risk assessment and how the data found in peer-reviewed articles and adverse incident reports was used" in their declarations "facially complied with Daubert II 's verification requirement for evidence prepared in anticipation of litigation"). In addition, Dr. Portier has become, in the wake of his participation in the IARC Monograph process, something of an advocate for increased regulatory attention to glyphosate, suggesting his position is not one he has taken solely for purposes of this litigation, even if much of his public commentary occurred after he was retained by counsel for the plaintiffs. See, e.g. , Expert Report of Dr. Portier, App. Docs. 1-2, Wagstaff Decl. ISO Pls.' Opp'n Ex. 5; Daubert II , 43 F.3d at 1316-18 ; Mar. 8, 2018 Tr. [Portier] 626-27. Although these factors do not strongly favor admission, neither do they counsel significantly against it.

* * *

On the whole, Dr. Portier has adequately demonstrated that his opinion regarding general causation is sufficiently "within the range of accepted standards governing how scientists conduct their research and reach their conclusions" to proceed to a jury should any of the plaintiffs get past summary judgment at the next phase. Daubert II , 43 F.3d at 1317. He may present his full Bradford Hill analysis, but may not support his biological plausibility conclusion with the application of his pooling method. Turning from methods to conclusions, perhaps Dr. Portier has read too much into the evidence in certain areas—particularly in the important area of epidemiology. This could cause a jury to reject his conclusions, but it does not warrant keeping his opinion from a jury altogether. Thus, although it's a close question, Dr. Portier's opinion does not involve any logical leaps so great and so lacking in support as to render them inadmissible. Joiner , 522 U.S. at 146, 118 S.Ct. 512.

B. Dr. Ritz

Dr. Ritz is an epidemiology professor at the University of California, Los Angeles. She has a PhD in epidemiology, as well as an MD, and her primary research interests include the health effects of environmental and occupational exposures. Ritz Report 1. Monsanto does not dispute that she is qualified to offer an opinion addressing the epidemiology evidence at issue here.

Like Dr. Portier, Dr. Ritz first conducted a literature search to identify the relevant epidemiology evidence, assessed the quality of each pertinent study, and used her judgment to determine how the results of these studies fit together. Id. at 8-9, 14-23. She concluded that "[t]he epidemiologic studies as a whole support an increased risk of NHL with exposure to glyphosate or glyphosate based formulations." Id. at 25.

She also engaged in a Bradford Hill analysis. Dr. Ritz concluded that the strength criterion was "partially met," in light of the results of the meta-analyses that showed a "weak to moderate size association." Id. at 23. She further concluded that the dose-response criterion was met, gesturing toward the same two studies with higher odds ratios for greater exposures as Dr. Portier. Id. In assessing consistency, she noted briefly that positive associations were observed in different populations, places, and time periods. Id. at 24. She briefly concluded that the temporality criterion was met and, like Dr. Portier, acknowledged there was no supportive human experimental evidence. Id. at 24-25.

Dr. Ritz took a somewhat different tack than Dr. Portier with respect to the specificity, biological plausibility, and coherence criteria. Unlike Dr. Portier, who focused on whether NHL was an outcome associated exclusively with glyphosate exposure, Dr. Ritz asked the inverse question, inquiring whether glyphosate exposure resulted in a specific cancer outcome. Approaching the factor this way, she concluded that the criterion was met—increased incidences of NHL were observed, but not of other cancers— although she acknowledged that it was difficult to assess this criterion. Id. at 24; Apr. 4, 2018 Tr. [Ritz] 53-54. She found coherence not to be a relevant factor, as she considered it to overlap with the question whether there was any experimental evidence in humans to consider, and did not address whether any analogous compounds provided information relevant to the causation inquiry here. Ritz Report 25. Finally, to support her conclusion that a causal relationship between glyphosate and NHL is biologically plausible, Dr. Ritz in her report relied on the mechanistic evidence. As to the mechanistic evidence, she provided a cursory summary of studies on human absorption of glyphosate and studies she concludes demonstrate oxidative stress and genotoxicity. Id. at 24-25; Mar. 5, 2018 Tr. [Ritz] 86-88. Ultimately, she concluded "to a reasonable degree of scientific certainty," that glyphosate and glyphosate-based formulations like Roundup cause NHL. Ritz Report 25.

The Bradford Hill article seems to countenance both these experts' interpretations of the criterion, noting that "[o]ne-to-one relationships" between exposures and diseases are rare. Bradford Hill at 297; see also Rothman at 27 ("The criterion of specificity has two variants. One is that a cause leads to a single effect, not multiple effects. The other is that an effect has one cause, not multiple causes.").

Although Drs. Ritz and Portier generally offered similar opinions regarding the epidemiology evidence, one significant difference is Dr. Ritz's greater emphasis on numbers unadjusted for use of other pesticides. Although she acknowledged the importance of considering results that accounted for confounding variables, Dr. Ritz's analysis emphasized some numbers that did not make this adjustment. See, e.g. , id. at 14, 16. Monsanto attacks her opinion on this ground and argues that, once one focuses on the most fully adjusted numbers from the case-control studies, the results of these studies (combined with the AHS cohort study on which Monsanto relies) cannot justify a conclusion that there is a meaningful association between glyphosate and NHL. This critique of Dr. Ritz is a valid one, and exclusive consideration of numbers unadjusted for other pesticides, when adjusted numbers are available, would be disqualifying. Failing to take account of likely confounders by presenting and relying upon only unadjusted (or minimally adjusted) estimates is a serious methodological concern. See Nelson v. Tennessee Gas Pipeline Co. , 243 F.3d 244, 253 (6th Cir. 2001). This is illustrated by the IARC Monograph, which focused on numbers from epidemiological studies that were adjusted for other pesticides, explaining that "there is high potential for confounding by use of multiple pesticides." Monograph at 50; see also id. at 331. Accordingly, the misleading "Forest plot" from Dr. Ritz's report—which highlighted numbers unadjusted for other pesticides and, moreover, reported the number of cases in the individual studies without taking into account how many of these individuals were exposed to glyphosate—may not be presented to a jury. See Ritz Report 14. And frankly, this portion of her presentation calls her objectivity and credibility into question.

However, although Dr. Ritz did not focus heavily on the adjusted numbers in her reports, she did consider them. Two of the meta-analyses of the case-control studies used the fully adjusted estimates, and both regressions performed in De Roos (2003) adjusted for use of many other pesticides. Ritz Report 16, 19. She cited the numbers from the meta-analyses first and foremost in her causation analysis. See id. at 23. Further, during the hearings, Dr. Ritz professed that, even if she were limited to considering only the numbers adjusted for other pesticides, her conclusion would not change. See Apr. 4, 2018 Tr. [Ritz] 37-42, 92. By way of explanation, Dr. Ritz, like Dr. Portier, pointed to the consistency of the observed associations in case-control studies, which were primarily above 1.0 even if some were not statistically significant. See id. at 39-40. As discussed in Section III, Dr. Ritz critiqued the methodology of the AHS study, the most significant study that does not support her conclusion, and those critiques raise valid concerns. Further, although Dr. Ritz's conclusions do not predate this litigation, there is some evidence that her critiques of the AHS do. See Ritz Supp. Report 8. Although it is again a close question, Dr. Ritz's conclusions regarding the epidemiology evidence are admissible. While her analysis is subject to challenge—something Monsanto's cross-examination during the Daubert hearing made plain—her opinion does not rise to the level of an "unreliable nonsense opinion[ ]." City of Pomona , 750 F.3d at 1044 (citation omitted).

Because Dr. Ritz did consider the adjusted numbers, the Court declines Monsanto's invitation to exclude any opinion based on the adjusted numbers. See Def.'s Apr. 9, 2018 Supp. Br. 3-5.
--------

Dr. Ritz's assessment of the Bradford Hill "strength" criterion as "partially met" based on the "weak to moderate size association" reported in the meta-analyses requires less of a logical leap than does Dr. Portier's assessment. Ritz Report 23. Her conclusion regarding dose response is based on the "effect estimates for longer or more extensive use" between 2 and 3—presumably, the greater-than-two-days-per-year odds ratio in the McDuffie (2001) study and the greater-than-ten-days odds ratio in Eriksson (2008). Id. She does not explain how the contrary results of the AHS impacted her dose-response analysis, but, in light of the two published studies suggesting a biological gradient and the negligible weight she gave to the AHS overall in reaching her epidemiology opinions, her conclusion regarding dose response is admissible, even if it is questionable.

Dr. Ritz's opinion regarding biological plausibility is quite brief, and consists in effect of a series of citations to studies on human absorption of glyphosate and possible genotoxic and cytotoxic effects on humans and in rodents. See id. at 24-25. There is little to her analysis of this criterion, and she has not established that she would be qualified to offer an opinion addressing the toxicology evidence in any detail. However, to the extent she simply opines that, as an epidemiologist engaging in a Bradford Hill analysis, a review of the published mechanistic literature suggested it was biologically plausible that glyphosate could cause NHL in humans, that limited conclusion is admissible. Cf. Rothman at 28-29.

With her Bradford Hill analysis cabined in this way, Dr. Ritz's opinion that glyphosate causes NHL, and has caused NHL in those who have used it in the manner studied, is admissible. Mar. 5, 2018 Tr. [Ritz] 96; Ritz Supp. Report 10. Dr. Ritz's opinion, like Dr. Portier's, goes to the ultimate general causation question and therefore is sufficient to support a denial of summary judgment; it does not simply rehash IARC's analysis. Also like Dr. Portier—and perhaps to a greater extent—there is ample room to challenge both her methods and her conclusions. But, as discussed, the purpose of Rule 702 and the Daubert inquiry is not to "exclude opinions merely because they are impeachable." City of Pomona , 750 F.3d at 1044 (citation omitted).

C. Dr. Weisenburger

Dr. Weisenburger, a physician and pathologist who has focused for much of his career on NHL, also engaged in a Bradford Hill analysis in his expert report. He has significant experience in epidemiology and was a co-author of De Roos (2003), one of the key case-control studies for the plaintiffs. See Mar. 5, 2018 Tr. [Weisenburger] 169-70; Expert Report of Dr. Weisenburger 1-2, Wagstaff Decl. ISO Pls.' Opp'n Ex. 8 [Dkt. No. 648-8] ("Weisenburger Report"). Monsanto does not dispute that he is qualified to opine on the epidemiology evidence. See Def.'s Mot. 11 n.16.

In his expert report, Dr. Weisenburger offered an epidemiology opinion that was relatively brief. He considered the same core set of studies, reporting both the adjusted and unadjusted odds ratios from these studies. Although he did not include the hierarchical regression in De Roos (2003) in his summary chart, he otherwise considered the full picture presented by the published epidemiology studies and concluded, first, that an association existed between NHL and glyphosate use. He explained that neither methodological critiques of the case-control studies nor the results of the AHS were sufficient to persuade him that the association he detected in the case-control studies was spurious. Weisenburger Report 6. When confronted with the data from the Andreotti (2018) update to the AHS, he discounted the study on the basis of nondifferential exposure misclassification, making many of the same arguments in support of this point as the experts above, as well as what he characterized as an insufficient follow-up period. Supplemental Report of Dr. Weisenburger 1-2, Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 16 [Dkt. No. 1136-16] ("Weisenburger Supp. Report").

Monsanto contends Dr. Weisenburger's opinion is unreliable because it relied upon the univariate analysis in Eriksson (2008). Although Dr. Weisenburger did not provide a particularly nuanced analysis of that study in his report, he did include the results of the multivariate analysis in his report, and he frankly acknowledged the benefits of adjusting for potential confounders in that study during his testimony. Weisenburger Report 5; Mar. 6, 2018 Tr. [Weisenburger] 234-37. However, he disputed the inclusion of one of the variables in the study authors' model (arsenic), and did not say, as Monsanto implies, that he found the particular multivariate analysis included in Eriksson (2008), which included that disputed adjustment, to be more reliable. See Mar. 6, 2018 Tr. [Weisenburger] 237; Def.'s June 22, 2018 Br. 4. Monsanto also attacks Dr. Weisenburger's failure to mention the NAPP data in his expert report, even though he is an author of that study. Dr. Weisenburger explained that he elected to include published studies in his expert report, a defensible choice. Mar. 6, 2018 Tr. [Weisenburger] 251. And he was willing and able to discuss the NAPP during the Daubert hearings, acknowledging the wisdom of certain adjustments made in the NAPP study and that some of the odds ratios became statistically insignificant after these adjustments. He also emphasized that the odds ratio for higher-intensity exposure remained statistically significant. Id. at 218-21, 253-55, 257-63. Thus, Dr. Weisenburger's treatment of the Eriksson (2008) and NAPP data is not a reason to deem his epidemiology opinion unreliable.

Dr. Weisenburger's handling of latency gives the Court the most pause. In his initial report, he faulted the first AHS study, De Roos (2005), for its inadequate follow-up period. In doing so, he acknowledged neither that the AHS inquired about exposures occurring prior to the start of the study nor that the case-control studies included in De Roos (2003) could be subject to the same criticism. See Weisenburger Report 5. He continued to fault the AHS for inadequate follow-up periods even after publication of the Andreotti (2018) update. Weisenburger Supp. Report 3. And Dr. Weisenburger repeatedly suggested, including in materials prepared outside of this litigation, that glyphosate-induced NHL was likely to have a long average latency period, on the order of 20 or more years. Weisenburger Report 5; Def.'s June 22, 2018 Supp. Br. Ex. 1 [Dkt. No. 1539-1]. Dr. Weisenburger sought to explain why it might be appropriate to discount negative cohort studies on the basis of latency but not positive case-control studies, but his justification was not entirely satisfying. See Mar. 6, 2018 Tr. [Weisenburger] 278-84.

Although Dr. Weisenburger's discussion of this issue during his Daubert testimony did not answer every question it raised, he ultimately persuaded the Court that he could testify reliably about the latency issue. He admitted during the Daubert hearing that the case-control studies could also be critiqued for having a short latency period. Mar. 5, 2018 Tr. [Weisenburger] 190; Mar. 6, 2018 Tr. [Weisenburger] 282-83. And he continued to acknowledge evidence suggesting that it likely takes many years, on average, for NHL to develop as a result of glyphosate exposure. See Mar. 6, 2018 Tr. [Weisenburger] 245-47, 268-69. While acknowledging these concerns, however, Dr. Weisenburger explained that the adjustments for other pesticides made by De Roos (2003) and the NAPP study would ameliorate the latency concern to a degree; as noted, one possible explanation for elevated odds ratios so soon after glyphosate's introduction would have been use of other pesticides, but these adjustments took account of that possible confounder. Id. at 282-83. So, although there is tension between Dr. Weisenburger's view that, on average, it likely takes more than a decade for NHL to develop as a result of glyphosate exposure and the heavy weight he gives the case-control studies that could only account for a few years, he provided a scientifically plausible reason for continuing to credit the studies that adjusted for other pesticides.

Turning to the remainder of Dr. Weisenburger's opinion, the Court likewise finds no basis for excluding it. Dr. Weisenburger provided a brief rundown of the positive tumor findings identified by IARC, Greim (2015), and the EPA and concluded these findings provide sufficient evidence of carcinogenicity in experimental animals. Weisenburger Report 6-8. He also reviewed the mechanistic evidence and found that these studies supported IARC's conclusion that glyphosate and glyphosate-based herbicides are genotoxic. Id. at 8-9. He further opined that certain mechanistic studies indicated that low-dose exposures can have significant biological effects. Id. at 10.

Dr. Weisenburger's Bradford Hill analysis is admissible in light of his interpretation of the epidemiology studies. He concluded the temporality requirement was met. As to the strength of the association, he highlighted the odds ratios above 2.0 observed for certain subsets of the case-control study data, taking into account whether these results were statistically significant. Id. at 10-11. He focused on the same two case-control studies that sought to capture dose response as the experts above, and found elevated odds ratios to be adequately replicated across case-control studies conducted by different researchers in different regions. Regarding biological plausibility, he emphasized the studies demonstrating genotoxic effects and the occurrence of lymphoma in mice in some of the animal experiments. Id. at 11. Like Dr. Ritz, he concluded the specificity criterion supported causation, as the only disease associated with glyphosate exposure was NHL. Unlike the experts previously discussed, he concluded that glyphosate fell within a class of chemicals others of which have been implicated in causing NHL. Id. at 12. In addition, Dr. Weisenburger considered other possible explanations for the observed results and, among other things, concluded that "confounding due to the use of other pesticides does not fully explain the increased risk estimates for glyphosate" in light of the results in some studies that controlled for use of other pesticides. Id. None of these conclusions offends Daubert 's requirements.

Of note, one feature of Dr. Weisenburger's opinion is particularly helpful to the plaintiffs. Unlike Dr. Ritz and Dr. Portier, who elaborated on what the evidence showed as to real-world exposure levels almost as an afterthought, Dr. Weisenburger's opinions were presented in these terms from the beginning. In his original report, he addressed whether glyphosate or glyphosate-based formulations like Roundup cause "NHL in humans exposed to these chemicals in the workplace or environment." Weisenburger Report 2, 12. In addressing this question, he considered the epidemiological studies as well as studies he determined showed biological effects at relatively low doses. See id. at 10. Thus, Dr. Weisenburger's testimony goes directly to the general causation question, and likewise assists the plaintiffs in surviving Monsanto's summary judgment motion.

D. Dr. Neugut

Another of the plaintiffs' experts who focused on epidemiology, although eminently qualified and refreshingly candid, has not provided admissible testimony.

Dr. Neugut, like the experts discussed above, evaluated each of the key epidemiology studies before engaging in a Bradford Hill analysis that took into account all available data across disciplines. See Expert Report of Dr. Neugut 11-17, 20-23, Wagstaff Decl. ISO Pls.' Opp'n Ex. 4 [Dkt. No. 648-4] ("Neugut Report"). His supplemental report offered many of the same critiques of the AHS that Dr. Ritz offered. See Supplemental Report of Dr. Neugut 6-12, Wagstaff Decl. ISO Pls.' Supp. Br. Ex. 15 [Dkt. No. 1136-15]. The reports themselves are of high quality.

However, Dr. Neugut's testimony at the Daubert hearing was of much lower quality. There were several inconsistencies between his deposition testimony and his testimony at the hearing (significant ones—not just the usual molehills of which lawyers often make mountains). He often seemed unfamiliar with key aspects of the material that purportedly formed the basis of his opinion. He sometimes answered questions in a cavalier fashion, apparently without giving much thought to whether he really knew the answer. And he often needed help from the plaintiffs' lawyers in answering questions. Although the written transcript of Dr. Neugut's testimony reflects these problems to some extent, they were far more apparent in the courtroom (and in the video recording of the hearing). To give a few examples:

• Dr. Neugut sought to characterize IARC's assessment of glyphosate as something other than a hazard assessment, even though the Preamble is quite clear about what the Monographs seek to do. Mar. 6, 2018 Tr. [Neugut] 296; Monograph at 10.

• Dr. Neugut opined that an IARC conclusion that an agent is a probable carcinogen means, as a practical

matter, that the group reached this conclusion with 70-90% certainty, although IARC disclaims any numeric probability associated with its classifications. Mar. 6, 2018 Tr. [Neugut] 301, 356-57; Monograph at 30 ("The terms probably carcinogenic and possibly carcinogenic have no quantitative significance and are used simply as descriptors of different levels of evidence of human carcinogenicity, with probably carcinogenic signifying a higher level of evidence than possibly carcinogenic." (emphasis omitted) ).

• In his deposition, Dr. Neugut agreed with Monsanto's counsel that no statistically significant, pesticide-adjusted odds ratio in the published literature supported an association between glyphosate and NHL. Hollingsworth Decl. ISO Def.'s Mot. Ex. 3 158-59 [Dkt. No. 546-3]. That was an erroneous statement about a critical issue in the case. Neugut later sought to correct that deposition testimony to take account of De Roos (2003), which reported a statistically significant association in the logistic regression model adjusted for other pesticides. Hollingsworth Decl. ISO Def.'s Reply Ex. 4 [Dkt. No. 681-5]; De Roos (2003) at 5.

• In the slide presentation during Dr. Neugut's Daubert testimony, ostensibly prepared by Dr. Neugut himself, there was a slide describing McDuffie (2001). Dr. Neugut was not familiar with all the assertions about McDuffie (2001) that were contained in his own slide. See Mar. 6, 2018 Tr. [Neugut] 330.

• He relied on a certain odds ratio from a 2002 Swedish case-control study whose lead author was Lennart Hardell but demonstrated during his testimony that he did not know much about it. Among other things, he did not know whether and to what extent Hardell considered proxy respondents, and required help from the plaintiffs' lawyer to answer this question. See id. at 334-40.

• In response to a question by the Court about the logistic regression and the hierarchical regression in DeRoos (2003), Dr. Neugut first stated that logistic regression was more "legitimate" and that hierarchical regression modeling "is a fancy-schmancy, sophisticated thing you do to look cool." Id. at 341. The Court responded, "so can you now try and explain the difference between the two, to me?" After a period of fumbling in which it became apparent that Dr. Neugut couldn't explain the difference between the two, counsel for the plaintiff stepped in to point Dr. Neugut to the portion of the study that explained it. Dr. Neugut stated that he didn't know what it meant. Id. at 341-42.

• It appeared from his deposition testimony and his testimony at the Daubert hearing that Dr. Neugut reached his opinion that glyphosate causes NHL after reading only the IARC Monograph, and before reviewing the individual studies. Id. at 353-55. He also seemed to suggest that even IARC's finding of limited evidence of carcinogenicity in humans, without any review of the underlying studies, would have sufficed for him to reach the conclusion he reached in his report. Id. at 355-58.

• At his deposition, Dr. Neugut stated that the epidemiology evidence alone was not sufficient to show causation. During the Daubert hearing, Dr.

Neugut stated he was revisiting that conclusion, even though the only evidence that could have justified a change in his analysis was the Andreotti (2018) study, which showed no association between glyphosate and NHL. Id. at 368-69; cf. Domingo ex rel. Domingo , 289 F.3d at 607.

Each problem with Dr. Neugut's testimony is not sufficient, on its own, to justify exclusion. Reliable experts sometimes make mistakes. They sometimes need to refer to the written materials during their testimony, to refresh their recollection about an issue or perhaps to consider a point raised by counsel for the first time on cross-examination. Even a few instances of misstating the details or failing to recall some aspect of a particular study would not be enough to exclude a witness. But in combination, the problems with Dr. Neugut's testimony lead the Court to conclude that his opinion is not sufficiently reliable to be admissible. See Department of Toxic Substances Control v. Technichem, Inc. , No. 12-CV-05845-VC, 2016 WL 1029463, at *1 (N.D. Cal. Mar. 15, 2016) (noting that "[k]ey factual errors" undermine the reliability of an expert's testimony).

E. Dr. Jameson

Dr. Jameson, a chemist and environmental toxicologist who specializes in cancer, engaged in an IARC-style hazard assessment of glyphosate as it relates to NHL, with a focus on the rodent carcinogenicity studies. See Jameson Report 1, 9-11, 19-29. Dr. Jameson has more than forty years of toxicology experience, and has worked for the National Cancer Institute and National Institute of Environmental Health Sciences. Mar. 7, 2018 Tr. [Jameson] 403. He was for many years responsible for the preparation of the Report on Carcinogens, a congressionally mandated public health report listing agents known or reasonably anticipated to cause cancer in humans. Jameson Report 2-3. He has also been a member of several IARC working groups, including the working group that assessed glyphosate as the chair of the experimental animal subgroup. Mar. 7, 2018 Tr. [Jameson] 404.

With respect to his opinion regarding the epidemiological evidence and its bearing on the general causation question, Dr. Jameson is hamstrung by his decision to conduct an IARC-style analysis. Dr. Jameson first summarizes the relevant IARC report at length. Jameson Report 4-8. He then engages in a "hazard based assessment of glyphosate and/or glyphosate-based formulations[ ] that ... is the same as defined and characterized by IARC." Id. at 9. Dr. Jameson concludes that the human evidence is "limited" in the sense IARC used the term; that there is "sufficient" evidence that glyphosate causes certain tumors in experimental animals; and that there is strong evidence that glyphosate is genotoxic and induces oxidative stress, the two possible cancer-causing mechanisms also identified by IARC. Id. at 19, 29, 30-31. Ultimately, he opines "to a reasonable degree of scientific certainty that glyphosate and glyphosate-based formulations are probable human carcinogens," and that "glyphosate and glyphosate-based formulations cause NHL in humans." Id. at 31-32. IARC does not explicitly reach Dr. Jameson's second conclusion but, having characterized his inquiry throughout the report as parallel to IARC's, there is no basis for reading Dr. Jameson's statements regarding glyphosate's ability to cause NHL in humans to mean anything more than that glyphosate is an NHL "hazard" in the sense IARC defines that term. See id. at 9; Mar. 7, 2018 Tr. [Jameson] at 412, 418-19. That conclusion, reached using the methods IARC used, is one that meets Daubert 's reliability requirement, but it does not itself allow the plaintiffs to survive summary judgment and, as discussed in Section II, may not be admissible in this case at all, because it involves too different an inquiry from the one a jury would be required to undertake.

Apparently realizing their mistake before Dr. Jameson's appearance at the Daubert hearing, counsel for the plaintiffs sought to elicit an opinion from Dr. Jameson during the hearing that went beyond the one presented in his report—specifically, an opinion that "exposure to glyphosate not only can cause [NHL], but it is currently doing so, at current exposure levels today." Id. at 405. Dr. Jameson's analysis of the human evidence is not sufficient to support his additional conclusion that glyphosate "is currently" causing NHL "at current exposure levels today." Dr. Jameson's primary focus and meaningful independent analysis concerned the animal toxicology studies, and his analysis was not crafted to support a conclusion that glyphosate is causing NHL in humans at current exposure levels. See id. at 455-57. As a result, "there is simply too great an analytical gap between" his analysis, which effectively duplicates IARC's as to human studies but goes further as to the animal studies, and his new conclusion regarding glyphosate's effects on humans at current exposure levels. Joiner , 522 U.S. at 146, 118 S.Ct. 512 ; see also Domingo ex rel. Domingo , 289 F.3d at 606-07.

Obviously, none of this is the fault of Dr. Jameson—he is a scientist who should not be expected to identify, on his own, the difference between an IARC-style hazard assessment and the evidentiary standard that governs civil lawsuits. But the apparent failure of plaintiffs' counsel to explain this difference to him, and to elicit an opinion from him that goes beyond a hazard assessment, means that his testimony is insufficient to get the plaintiffs over the general causation hurdle.

Although Dr. Jameson's overall hazard-assessment conclusion may end up not being admissible, he will be permitted to offer testimony (if a case makes it to trial) on the narrower topic of the animal cancer studies. As Dr. Jameson stated repeatedly during the Daubert hearing, the purpose of conducting studies in laboratory animals like the ones at issue here is to determine whether a substance causes cancer in animals. See, e.g. , Mar. 7, 2018 Tr. [Jameson] 475. As mentioned in Section IV, whether a substance does so is relevant to the general causation inquiry.

Monsanto attacks Dr. Jameson for not adequately addressing what it deems an absence of replicated tumor findings across different experiments, and for relying too heavily on statistical significance, rather than conducting a fuller assessment of biological significance. See Def.'s Mot. 30-31, 31 n.51; Def.'s Reply 29-30. Dr. Jameson did consider replication, agreeing that repeated findings of the same tumors across sexes, studies, or species would strengthen his conclusion that a particular chemical caused tumor development. Mar. 7, 2018 Tr. [Jameson] 449-52, 495. Further, Dr. Jameson concluded that four of the tumors of interest were repeated across studies. Id. at 449-52. Monsanto disagrees with his interpretation of those studies, and pointed out that his conclusions differ in many cases from those of the study authors. But Monsanto's disagreements with how Dr. Jameson weighted different considerations in arriving at his conclusions are fodder for cross-examination, not grounds for exclusion. See Karlo , 849 F.3d at 83 ("The question of whether a study's results were properly calculated or interpreted ordinarily goes to the weight of the evidence, not to its admissibility." (citation omitted) ). F. Dr. Nabhan

Dr. Nabhan is a hematologist and medical oncologist who specializes "in the diagnosis and management of patients with all types of lymphoma." Expert Report of Dr. Nabhan 1, Wagstaff Decl. ISO Pls.' Opp'n Ex. 7 [Dkt. No. 648-7]. Although he stated that he routinely reviews epidemiology and toxicology studies as part of his clinical practice, he did not dispute that his primary focus is on clinical work. Mar. 9, 2018 Tr. [Nabhan] 805 ("I'm a clinician, I'm not an epidemiologist or a statistician, but we're on the front line with patients."); id. at 818 ("Again, I'm not an epidemiologist...."). He summarized many relevant studies, but offered little in the way of critical analysis of these studies. See id. at 820 ("[F]rom a clinician's view, we don't really sit down and re-analyze and re-perform a peer-review process for every single paper that has been published.... My job as a clinician is not to peer-review the entire literature again."). Instead, he deferred to the opinions of other experts, and to IARC in particular, in arriving at his conclusions. See id. at 820-22, 850; see also id. at 822 ("So as a clinician, I will look [at] these epidemiology studies, then I look at bodies such as the IARC, I look at the history, and it's hard to argue, with all of the data that the IARC looked at and with the history, so I tend to obviously believe the data that came out of IARC."); id. at 837 ("I didn't review this particular evidence, but if the IARC says this particular aspect of the mechanism of action is weak, then it's weak."); id. at 844 (agreeing that he "rel[ied] heavily on IARC for [his] opinion").

"[M]edical doctors do not need to be epidemiologists in order to testify regarding epidemiological studies," so long as the expert is qualified by training or experience to interpret these studies and his opinions would be helpful to the jury. In re Mirena IUD Products Liability Litigation , 169 F. Supp. 3d 396, 426 (S.D.N.Y. 2016) ; see also In re Abilify , 299 F. Supp. 3d at 1349. The primary problem for the plaintiffs, however, is Dr. Nabhan's uncritical reliance on IARC's conclusions. During the Daubert hearing, Dr. Nabhan all but admitted that he reached his conclusion regarding glyphosate upon reading the IARC report, and that contrary new evidence was unlikely to shake his faith in IARC's conclusion. See Mar. 9, 2018 Tr. [Nabhan] 850 ("Q. At this point, nothing would [ ] shake your conviction. A. At this point, the IARC report is very convincing."). The deference to IARC that Dr. Nabhan demonstrated during the Daubert hearing may well be appropriate clinical practice but, under these circumstances, it is not a reliable way to reach a general causation opinion. Dr. Nabhan's report also did not demonstrate that he engaged in his own objective analysis of the epidemiologic literature. Although he summarized the relevant studies, he said little about how or whether they addressed possible bias or confounding, for instance. See Nabhan Report 11-16.

During the Daubert hearing, Dr. Nabhan also suggested that his opinion regarding whether glyphosate was causing NHL at present-day exposure levels was informed by his clinical practice. See Mar. 9, 2018 Tr. [Nabhan] 805-07. He suggested that a subset of NHL patients developed their NHL as a result of glyphosate exposure, and that he recommends curtailing glyphosate use for patients with NHL, treating it as a "modifiable risk factor." Id. at 810-11, 826-27. Dr. Nabhan may well be able to offer an opinion that glyphosate was responsible for causing a particular patient's NHL, based on that patient's clinical presentation and history, during the specific causation phase of this litigation. And it may well be good medical advice to tell a patient to curtail glyphosate exposure. However, because Dr. Nabhan has not provided a reliable basis for concluding that glyphosate can cause NHL as a general matter, Monsanto's motion to exclude his testimony is granted.

VII. THE PLAINTIFFS' DAUBERT MOTION

In addition to defending their own experts, the plaintiffs seek to exclude certain of Monsanto's experts. These challenges are addressed in the sections that follow.

A. Dr. Rosol

The plaintiffs seek to exclude Dr. Rosol, a veterinary pathologist, because he considered certain documents available only in a "glyphosate reading room" in Brussels that has since been shuttered. See Rosol Report 9, 13-18. They do not object to his general methodology or conclusions aside from this critique. See Mar. 8, 2018 Tr. [Rosol] 731 ("We're actually not really even challenging your conclusions or your methodology too much.").

Dr. Rosol elaborated on what the reading room entailed during cross-examination at the Daubert hearing. He testified that the reading room allowed researchers to sign up for up to four half-day sessions during the weeks it was open, and the researchers could use one of approximately ten old, monochrome computers to review the data from the studies. Id. at 732-36. On the one hand, he testified that he took approximately 50 pages of handwritten notes during his time in the reading room, and he references the material he reviewed in the reading room repeatedly in his report. See id. ; Rosol Report 13-18. On the other hand, he testified that the "Reading Room pathology reports," apparently the only material not accessible through the publicly available Greim (2015) study supplements, "did not influence [his] interpretation" and were not necessary to support his conclusions. Id. at 735.

So long as neither the Court nor the plaintiffs' experts have access to the data available only in the reading room, Dr. Rosol will be precluded from referencing this material in rendering his opinion. However, because he testified that his opinion would stand absent that material, and his opinion is otherwise admissible, his opinion will not otherwise be excluded.

B. Dr. Goodman

Dr. Goodman, a toxicologist, seeks to offer an opinion that glyphosate and glyphosate-based formulations "should be regarded as non-genotoxic materials," and that, although they "might be capable of causing oxidative stress under certain experimental conditions, ... it is not appropriate to use this observation to support a contention that these materials are capable of causing cancer." Goodman Report 3-4.

The plaintiffs challenge the admissibility of Dr. Goodman's testimony on two grounds. First, they argue his opinions discounting two human in vivo studies, Bolognesi (2009) and Paz-y-Miño (2007), are inadmissible because his critiques of the studies are too speculative and contain errors. Some of Dr. Goodman's critiques are less than persuasive bases for discounting the studies—for instance, his concern that more than one person might have analyzed the slides in Paz-y-Miño (2007), which might have introduced subjectivity into the data analysis. See id. at 13. Others are very reasonable, like his observations that other factors might have explained the DNA damage in light of the period that elapsed between the aerial glyphosate spraying and the time when the blood samples were taken, and that study participants appeared to exhibit symptoms of acute illness. See Paz-y-Miño (2007) at 457 (noting the physical symptoms reported by participants and that blood samples were gathered between two weeks and two months after the aerial spraying of glyphosate). Regarding the Bolognesi (2009) study, Dr. Goodman emphasized that the indicator of genotoxicity was highest in a region where glyphosate was not aerially sprayed (although where people were still exposed to pesticides, including glyphosate). Goodman Report 15-17; Bolognesi (2009) at 995. Although the plaintiffs ascribe a different meaning to this aspect of the Bolognesi study, Dr. Goodman's observation is neither incorrect nor irrelevant, in light of the study's focus on the effects of aerial spraying and its extremely limited conclusions. See Bolognesi (2009) at 994-95. The plaintiffs' motion to exclude Dr. Goodman's critiques of the human in vivo studies is therefore denied.

The plaintiffs also mount a broader attack on Dr. Goodman's methodology as results-oriented. Dr. Goodman's methodology emphasized studies conducted on mammals or mammalian cells and those that use the four basic tests used by international agencies for registration or approval of chemicals. Goodman Report 10-11. He dismisses several of the studies as unable to rule out cytotoxicity as the cause of the results observed. See, e.g. , id. at 23, 26-27, 29-30. Although he reaches different conclusions about what the weight of the mechanistic evidence shows, his analysis is not so flawed or one-sided that his opinions need be excluded.

C. Dr. Foster

The plaintiffs further seek to exclude the testimony of Dr. Foster, who is also a toxicologist. They contend he is not qualified to offer an opinion on the rodent studies, because his focus is on reproductive toxicology. Notwithstanding Dr. Foster's focus on reproductive toxicology, he is qualified to opine on the rodent carcinogenicity data at issue here. See D.F. ex rel. Amador , 2017 WL 4922814, at *14. He is a trained in toxicology, served as the one-time acting director of an environmental toxicology program at Health Canada, and has published at least a few peer-reviewed articles on cancer in rodents. Wagstaff Decl. ISO Pls.' Opp'n Ex. 122 at 118-23 [Dkt. No. 656-17 at 32-33].

The plaintiffs additionally argue that Dr. Foster's opinion is unreliable. Among the alleged flaws they identify are his comparisons across studies the plaintiffs consider insufficiently similar. For example, they point to his comparison of the results concerning interstitial testicular tumors in the Lankas (1981) study with those in the Atkinson and Suresh studies, noting that, in the latter two studies, not all the low-dose animals were fully examined. Pls.' Opp'n 67-68. The plaintiffs also argue that he inappropriately dismissed certain tumors because no tumor progression was observed or, in the case of the Knezevich and Hogan study, because of alleged weight loss in the high-dose group of mice. Id. at 68-69.

Dr. Foster, like the plaintiffs' experts, conducted a literature review and evaluated the quality of each of the studies. Unlike the plaintiffs' experts, he explained away the statistically significant tumor findings, pointing to a lack of reproducibility between studies, an abnormally low number of tumors in certain control groups, lack of dose response, decreased survival of certain control animals (which would result in older treated animals, and thus likely more spontaneous tumors), and evidence of systemic toxicity in one high-dose group. See Foster Report 27. As discussed above, different interpretations of these studies are not necessarily evidence of unreliability, and Dr. Foster's interpretations of the same core studies evaluated by the plaintiffs are sufficiently grounded in scientific principles to be admissible. The plaintiffs may raise their concerns via cross-examination.

D. Drs. Rider and Mucci

Monsanto proffered two epidemiology experts, Drs. Rider and Mucci. The plaintiffs object to their opinions because they relied heavily and, the plaintiffs argue, uncritically on the various iterations of the AHS study. As discussed above, the AHS study, like the case-control studies, is open to valid critiques. Like the plaintiffs' experts who focused on epidemiology, Drs. Rider and Mucci assessed the strengths and weaknesses of the relevant epidemiology studies, but weighed the studies differently and reached different conclusions. Dr. Rider briefly acknowledged the possibility of exposure misclassification in the AHS study, but concluded that, in light of the observed odds ratios below 1.0, it would not have obscured any positive association. Rider Supp. Report 3-4. She also explained why she was not concerned about the imputation method used by the AHS study authors, citing methodological and sensitivity analyses. Id. at 4-5. Dr. Mucci likewise acknowledged the possibility of nondifferential misclassification of glyphosate exposure and explained how the authors of the Andreotti (2018) study assuaged any concerns she might have about the imputation method. Mucci Report 33, 35; Mucci Supp. Report 2-4, 7; Mar. 9, 2018 Tr. [Mucci] 905 ("[W]e should be, as epidemiologists, concerned with the fact that there is 37 percent missing data. We do want to rule out that there are not biases that are systematic as a result of this missing data."). Both experts offered rebuttals to the plaintiffs' concerns. See Mucci Supp. Report 7-10; Rider Supp. Report 8-11; Mar. 9, 2018 Tr. [Mucci] 863-919. As suggested earlier, the disputes between the experts evaluating these epidemiology studies are reasonable disputes. Dr. Mucci and Dr. Rider used sufficiently reliable methods to reach conclusions about the epidemiology evidence that require no unduly great leap from their analyses. The plaintiffs' Daubert motion to exclude their testimony is therefore denied.

E. Dr. Corcoran

Dr. Corcoran, a biostatistician, critiques Dr. Portier's statistical analysis of the rodent carcinogenicity studies. The plaintiffs argue only that Dr. Corcoran is not qualified to offer an opinion on the data at issue here, because his research has focused on dementia and other aging-related diseases. As proof, they point to an exchange in which he purportedly did not know the difference between primary and secondary tumors, the latter of which the plaintiffs contend should be excluded from tumor counts in animal bioassays. Pls.' Opp'n 56 n.165; Pls.' Reply 12. That Dr. Corcoran's research has not focused on cancer or animal bioassays does not require his exclusion. See Avila , 633 F.3d at 839. As a trained biostatistician, he is qualified to offer a critique of Dr. Portier's statistical analysis, and the plaintiffs are free to dispute his treatment of secondary tumors through Dr. Portier's testimony and during cross-examination. The motion to exclude Dr. Corcoran is denied.

VIII. CONCLUSION

It's a close question whether to admit the expert opinions of Dr. Portier, Dr. Ritz, and Dr. Weisenburger that glyphosate can cause NHL at human-relevant doses. Therefore, it's a close question whether to grant or deny Monsanto's motion for summary judgment. But the Court concludes that the opinions of these experts, while shaky, are admissible. They have surveyed the significant body of epidemiological literature relevant to this question; identified at least a few statistically significant elevated odds ratios from case-control studies and meta-analyses; identified what they deem to be a pattern of odds ratios above 1.0 from the case-control studies, even if not all are statistically significant; emphasized that studies of glyphosate have focused on many different types of cancer but found a link only between glyphosate and NHL; given legitimate reasons to question the results of the primary study on which Monsanto relies; and concluded, in light of all the available evidence, that a causal interpretation is appropriate. Their opinions may be bolstered by Dr. Jameson's narrower opinions regarding glyphosate's ability to cause cancer in animals. Therefore, the plaintiffs have presented evidence from which a reasonable jury could conclude that glyphosate can cause NHL at human-relevant doses. Monsanto's motion for summary judgment is denied.

In re Roundup Prods. Liab. Litig.

Summary

Opinion

I. BACKGROUND

II. THE DAUBERT STANDARD AND THE GENERAL CAUSATION INQUIRY

A. Legal Standard

B. The Relevance of the IARC Classification

III. EPIDEMIOLOGY

A. The Bradford Hill Criteria

B. Case-Control Studies and Meta-Analyses

C. Cohort Study

IV. LABORATORY ANIMAL CANCER STUDIES

V. MECHANISTIC DATA

VI. CONCLUSIONS REGARDING THE PLAINTIFFS' EXPERTS

A. Dr. Portier

B. Dr. Ritz

C. Dr. Weisenburger

D. Dr. Neugut

E. Dr. Jameson

VII. THE PLAINTIFFS' DAUBERT MOTION

A. Dr. Rosol

B. Dr. Goodman

C. Dr. Foster

D. Drs. Rider and Mucci

E. Dr. Corcoran

VIII. CONCLUSION

IT IS SO ORDERED.

Id.

In re Roundup Prods. Liab. Litig.

In re Roundup Prods. Liab. Litig.

Case Details

Citations

Citing Cases

In re Roundup Prods. Liab. Litig.

Summary

Opinion

I. BACKGROUND

II. THE DAUBERT STANDARD AND THE GENERAL CAUSATION INQUIRY

A. Legal Standard

B. The Relevance of the IARC Classification

III. EPIDEMIOLOGY

A. The Bradford Hill Criteria

B. Case-Control Studies and Meta-Analyses

C. Cohort Study

IV. LABORATORY ANIMAL CANCER STUDIES

V. MECHANISTIC DATA

VI. CONCLUSIONS REGARDING THE PLAINTIFFS' EXPERTS

A. Dr. Portier

B. Dr. Ritz

C. Dr. Weisenburger

D. Dr. Neugut

E. Dr. Jameson

VII. THE PLAINTIFFS' DAUBERT MOTION

A. Dr. Rosol

B. Dr. Goodman

C. Dr. Foster

D. Drs. Rider and Mucci

E. Dr. Corcoran

VIII. CONCLUSION

IT IS SO ORDERED.

Id.

In re Roundup Prods. Liab. Litig.

In re Roundup Prods. Liab. Litig.

Case Details

CitationsCopy Citation

Citing Cases

Citations