National Credit Union Administration Board v. Credit Suisse Securities (USA) LLC et al

MEMORANDUM OF LAW in Support re: 76 MOTION in Limine to Admit Expert Statistical Sampling Testimony.. Document

S.D.N.Y.April 4, 2014

IN THE UNITED STATES DISTRICT COURT FOR THE SOUTHERN DISTRICT OF NEW YORK NATIONAL CREDIT UNION ADMINISTRATION BOARD, as Liquidating Agent of Southwest Corporate Federal Credit Union, and Members United Corporate Federal Credit Union, Plaintiff, v. MORGAN STANLEY & CO., INC. and MORGAN STANLEY CAPITAL I INC., Defendants. And other NCUA cases. ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) Case Nos. 13 Civ. 6705 (DLC) 13 Civ. 6719 (DLC) 13 Civ. 6721 (DLC) 13 Civ. 6726 (DLC) 13 Civ. 6727 (DLC) 13 Civ. 6731 (DLC) 13 Civ. 6736 (DLC) MEMORANDUM OF LAW IN SUPPORT OF NCUA’S MOTION IN LIMINE TO ADMIT EXPERT STATISTICAL SAMPLING TESTIMONY April 4, 2014 Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 1 of 20 TABLE OF CONTENTS Page TABLE OF AUTHORITIES ......................................................................................................................... ii STATEMENT ................................................................................................................................................... 2 ARGUMENT..................................................................................................................................................... 3 I. Dr. Cowan’s Sampling Methodology Complies with Rule 702 ..................................................... 4 A. Dr. Cowan Is Qualified as a Statistics Expert .................................................................... 4 B. Statistically Valid Random Sampling Is Well-Accepted and Reliable ............................. 5 C. Dr. Cowan Has Applied His Statistical Sampling Methodology Reliably ...................... 8 D. The Proposed Sampling Methodology Is Relevant and Helpful to the Trier of Fact ......................................................................................................................................... 11 E. The Court Should Rule Now on the Admissibility of the Statistical Sampling Methodology ......................................................................................................................... 12 CONCLUSION ............................................................................................................................................... 14 Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 2 of 20 ii TABLE OF AUTHORITIES CASES Assured Guar. Mun. Corp. v. Flagstar Bank, FSB, 920 F. Supp. 2d 475 (S.D.N.Y. 2013) ......................................................................................................................... 3, 8, 12 Awad v. Merck & Co., 99 F. Supp. 2d 301 (S.D.N.Y. 1999), aff’d sub nom., Washburn v. Merck & Co., 213 F.3d 627 (2d Cir. 2000) ........................................................................................ 4 Campbell ex rel. Campbell v. Metropolitan Prop. & Cas. Ins. Co., 239 F.3d 179 (2d Cir. 2001) ........................................................................................................................................ 3 Cities Serv. Co. v. Derby & Co., 654 F. Supp. 492 (S.D.N.Y.), aff’d, 835 F.2d 1429 (2d Cir. 1987) ..................................................................................................... 6 Countrywide Fin. Corp. Mortg.-Backed Sec. Litig., In re, Nos. 11-ML-2265-MRP MANx et al., -- F. Supp. 2d --, 2013 WL 6231713 (C.D. Cal. Dec. 2, 2013)....................................................... 5 Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311 (9th Cir. 1995), aff’d, Washburn v. Merck & Co., 213 F.3d 627 (2d Cir. 2000) .......................................................... 4 Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) ............................................................................... 3 Feske v. MHC Thousand Trails Ltd. P’ship, No. 11-CV-4124-PSG, 2012 WL 1123587 (N.D. Cal. Apr. 3, 2012) ................................................................................... 13 FHFA v. JPMorgan Chase & Co., Nos. 11 CIV. 6188 DLC et al., 2012 WL 6000885 (S.D.N.Y. Dec. 3, 2012) ........................................................... 4, 5, 7, 9, 12, 13 FHFA v. UBS Ams. Inc., Nos. 11 Civ. 5201 (DLC) et al., 2013 WL 3284118 (S.D.N.Y. June 28), recon. denied, Nos. 11 Civ. 6188(DLC) et al., 2013 WL 5354212 (S.D.N.Y. Sept. 25, 2013) ................................................................................ 12 MBIA Ins. Corp. v. Countrywide Home Loans, Inc., No. 602825/08, 2010 WL 5186702 (N.Y. Sup. Ct. Dec. 22, 2010) .....................................................................8, 12 Massachusetts Mut. Life Ins. Co. v. Residential Funding Co., Civ. A. No. 11-30035-PBS, 2013 WL 6490125 (D. Mass. Dec. 9, 2013)........................................................................... 5, 7, 13 Nielsen Co. (US) v. Truck Ads, LLC, No. 08 C 6446, 2011 WL 3857122 (N.D. Ill. Aug. 29, 2011) .................................................................................................................. 6-7 Rosado v. Wyman, 322 F. Supp. 1173 (E.D.N.Y. 1970), aff’d, 437 F.2d 619 (2d Cir. 1970), aff’d, 402 U.S. 991 (1971) ..................................................................................................................... 7 Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 3 of 20 iii Spears v. First Am. eAppraiseIT, No. C-08-00868 RMW, 2012 WL 1438709 (N.D. Cal. Apr. 25), appeal dismissed, No. 12-80109 (9th Cir. July 19, 2012) ......................................................................................................................... 8 Syncora Guarantee Inc. v. EMC Mortg. Corp., No. 09 C 3106(PAC), 2011 WL 1135007 (S.D.N.Y. Mar. 25, 2011) .................................................................................................................... 8 U.S. Info. Sys., Inc. v. International Bhd. of Elec. Workers Local Union No. 3, AFL-CIO, 313 F. Supp. 2d 213 (S.D.N.Y. 2004) ................................................................................................ 4 United States v. Lahey Clinic Hosp., Inc., 399 F.3d 1 (1st Cir. 2005) ................................................................ 7 United States v. United Shoe Mach. Corp., 110 F. Supp. 295 (D. Mass. 1953), aff’d, 347 U.S. 521 (1954) ..................................................................................................................... 7 United States v. Williams, 506 F.3d 151 (2d Cir. 2007) .................................................................................... 3 Vigilant Ins. v. Sunbeam Corp., 231 F.R.D. 582 (D. Ariz. 2005) ..................................................................... 6 Washington Mut. Mortg. Backed Sec. Litig., In re, No. C09-37 MJP, 2012 WL 2995046 (W.D. Wash. July 23, 2012) ...........................................................................................................8, 12 World Trade Ctr. Disaster Site Litig., In re, 598 F. Supp. 2d 498 (S.D.N.Y. 2009) ...................................... 13 Zurich Am. Ins. Co. v. Ace Am. Reins. Co., No. 05 Civ. 9170 RMB JCF, 2006 WL 3771090 (S.D.N.Y. Dec. 22, 2006) ................................................................................. 13 STATUTE AND RULES 13 U.S.C. § 195 ................................................................................................................................................... 6 Fed. R. Civ. P. 1 ...........................................................................................................................................2, 12 Fed. R. Evid. 702 ..................................................................................................................... 1, 3, 4, 7, 12, 14 OTHER MATERIALS FDIC, Risk Management Manual of Examination Policies (2005) ...................................................................... 6 Fin. Crisis Inquiry Comm’n, Final Report of the National Commission on the Causes of the Financial and Economic Crisis in the United States (2011) ...................................................................... 7 David H. Kaye & David A. Freedman, Reference Manual on Scientific Evidence (Fed. Judicial Ctr. 3d ed. 2011) ........................................................................................................... 7 Manual for Complex Litigation (Fed. Judicial Ctr. 4th ed. 2004) ...................................................... 1, 6, 7 Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 4 of 20 iv OCC, Comptroller’s Handbook, Sampling Methodologies (Aug. 1998) ................................................................. 6 U.S. Census Bureau, Current Population Survey, Sampling (2013) .................................................................... 6 1 Kenneth S. Broun, McCormick on Evid. (7th ed. 2013) ............................................................................... 6 Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 5 of 20 NCUA moves in limine for a ruling in the above-captioned Cases admitting the expert testimony of Dr. Charles D. Cowan regarding statistical sampling under Federal Rule of Evidence 702. As set forth in the expert report of Dr. Cowan, attached as Exhibit A to the Declaration of Wan J. Kim, statistical sampling provides a reliable method for determining whether the loans collateralizing the Certificates at issue were originated in accordance with the applicable underwriting guidelines. Indeed, underwriters like Defendants often performed due diligence on a sample of the loans that they were securitizing into RMBS. Courts also have long accepted that statistical sampling provides a highly reliable method for determining the characteristics of a given population based on a subset of that population. E.g., Manual for Complex Litigation § 11.493 (Fed. Judicial Ctr. 4th ed. 2004) (“Acceptable sampling techniques, in lieu of discovery and presentation of voluminous data from the entire population, can save substantial time and expense, and in some cases provide the only practicable means to collect and present relevant data.”). Based on the massive number of individual loan files at issue in this litigation - nearly 300,000 individual loans collateralize the 82 unique Certificates in the above-captioned Cases - statistical sampling is the only practicable means to evaluate certain representations in the Offering Documents. Dr. Cowan explains that a statistical sample of 7,900 loans (with a reserve sample of an additional 7,900 loans) will be sufficient to determine the percentage of loans that did not comply with the applicable underwriting guidelines to a reasonable degree of certainty. Statistical sampling will reduce significantly the number of loan files needed from Defendants and non-parties across the country, and will markedly cut the costly and time-consuming process of engaging experts to “re- underwrite” all of these loans. Collecting and re-underwriting every loan file (all of which are in the possession of Defendants or third parties) could not be done in any reasonable time frame, and certainly not by January 2016, when expert discovery will close in these cases. Even if the Court allotted several Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 6 of 20 2 more years for discovery, it still would be a prohibitively expensive undertaking for the parties (and the non-parties who possess many loan files).1 Nor could such a massive amount of evidence, even if successfully collected and analyzed by experts, be presented to a jury at trial. For these reasons, courts routinely have approved of the collection and re-underwriting of statistically valid samples of loan files. Indeed, this Court approved of Dr. Cowan’s sampling methodology in the FHFA cases, as did Judge Wu in the related NCUA v. Goldman Sachs case. Consistent with Federal Rule of Civil Procedure 1, a ruling on this issue now, at the onset of discovery, will promote the “just, speedy, and inexpensive determination” of this complex litigation. As this Court recognized in the FHFA litigation, and Judge Wu recognized in NCUA v. Goldman Sachs, an early ruling will create massive efficiencies by preventing unnecessary and wasteful efforts to seek discovery of more loan files than necessary for a scientifically valid statistical sample. STATEMENT The Offering Documents represented that the loans backing the residential mortgage-backed securities (“RMBS”) sold to the (since failed) Credit Unions were originated pursuant to certain underwriting guidelines, and that any exceptions to these guidelines were justified by compensating factors. Those representations were false. Many loans violated the applicable underwriting guidelines without compensating factors that warranted their approval. As part of its proof at trial, NCUA will present expert analysis of the loans collateralizing the Certificates that were sold to the now-defunct Credit Unions. That expert analysis will determine, among other things, whether the loans were originated in accordance with the applicable underwriting guidelines. To do so, NCUA must obtain the loan files and the applicable 1 Another bank-defendant estimates that the average loan file has 300 pages. Hearing Tr., In re FHFA, No. 11-5201 (DLC), ECF No. 30, at 34:12-20 (S.D.N.Y. Dec. 2, 2011) (excerpt attached as Exhibit A). Under that estimate, the 280,000-plus loan files would comprise over 84 million pages. NCUA estimates the cost of engaging an expert to re-underwrite a single loan at approximately $350, so re-underwriting the 280,000-plus loans would cost in excess of $98 million. Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 7 of 20 3 underwriting guidelines. NCUA must then engage experts to “re-underwrite” the loans to determine whether they were originated in accordance with the applicable underwriting guidelines. See generally Assured Guar. Mun. Corp. v. Flagstar Bank, FSB, 920 F. Supp. 2d 475 (S.D.N.Y. 2013) (describing process and concluding, after bench trial, that plaintiff ’s re-underwriting expert correctly identified numerous loans issued in violation of applicable underwriting guidelines). Each Certificate was collateralized by hundreds or thousands of individual loans - collectively, nearly 300,000 loans underlie the 75 RMBS from which the 94 Certificates (including 82 unique CUSIPs) were purchased. Such sampling allows the jury to draw conclusions about the characteristics of the Supporting Loan Group (“SLG”) - that is, all the loans directly collateralizing a particular Certificate - with a high, and known, degree of confidence. As Dr. Cowan explains in his expert report, re-underwriting statistically valid random samples of the loans backing the Certificates provides a reliable way to determine whether other loans in the pools were properly underwritten. Because these samples will be randomly selected, the results of NCUA’s re-underwriting can be extrapolated to the SLGs as a whole with a known degree of precision - here, at a 95% confidence level with a maximum margin of error of ±10%. These statistical sampling and extrapolation principles have long been accepted by the scientific community. Dr. Cowan’s sampling methodology mirrors the methodologies applied in nearly every other similar RMBS case. ARGUMENT Rule 702 assigns the district court “ ‘the task of ensuring that an expert’s testimony both rests on a reliable foundation and is relevant to the task at hand.’ ” United States v. Williams, 506 F.3d 151, 160 (2d Cir. 2007) (quoting Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 597 (1993)). “Pertinent evidence based on scientifically valid principles will satisfy those demands.” Campbell ex rel. Campbell v. Metropolitan Prop. & Cas. Ins. Co., 239 F.3d 179, 185 (2d Cir. 2001) (internal quotations omitted). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 8 of 20 4 “The focus [of the reliability inquiry] must be solely on principles and methodology, not on the conclusions that they generate.” Daubert, 509 U.S. at 595. “[A] significant consideration is whether research was conducted independently or for the sole purpose of litigation.” Awad v. Merck & Co., 99 F. Supp. 2d 301, 304 (S.D.N.Y. 1999) (citing Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1316-17 (9th Cir. 1995) (“Daubert II”)), aff’d sub nom., Washburn v. Merck & Co., 213 F.3d 627 (2d Cir. 2000). The former provides “objective proof that the [methodology] comports with the dictates of good science.” Daubert II, 43 F.3d at 1317. Indeed, because the statistical methods are so widely used, such expert testimony is routinely admitted. See, e.g., U.S. Info. Sys., Inc. v. International Bhd. of Elec. Workers Local Union No. 3, AFL-CIO, 313 F. Supp. 2d 213, 232 (S.D.N.Y. 2004) (“Accordingly, small sample size goes to the weight rather than to the reliability (and admissibility) of a [statistical sampling] study.”); FHFA v. JPMorgan Chase & Co., Nos. 11 CIV. 6188 DLC et al., 2012 WL 6000885, at *6 (S.D.N.Y. Dec. 3, 2012) (“[D]efendants do not challenge Dr. Cowan’s qualifications or the scientific reliability of statistical sampling generally.”). Dr. Cowan’s statistical sampling methodology easily satisfies Federal Rule of Evidence 702 and the Daubert standards. Several courts - including this one - have found his expert testimony admissible under indistinguishable circumstances. See FHFA, 2012 WL 6000885, at *5; Civil Minutes, NCUA v. Goldman, Sachs & Co., No. 11-6521, ECF No. 259 (C.D. Cal. Feb. 10, 2014) (granting NCUA’s motion in limine to admit Dr. Cowan’s statistical sampling expert testimony) (attached as Exhibit B). I. Dr. Cowan’s Sampling Methodology Complies with Rule 702 A. Dr. Cowan Is Qualified as a Statistics Expert Dr. Charles Cowan is eminently qualified by knowledge, skill, experience, training, and education to testify as to his statistical sampling methodology. Cowan Sampling Report ¶¶ 9-26 & Exh. 1. He holds Bachelor of Arts and Masters of Arts degrees in Economics from the University Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 9 of 20 5 of Michigan, and a doctorate in Mathematical Statistics from George Washington University. Id. ¶ 9. Dr. Cowan has served as director of financial research at PriceWaterhouseCoopers, LLP, and as Chief Statistician for the Federal Deposit Insurance Corporation (“FDIC”) and Resolution Trust Corporation (1991-96) and the U.S. Department of Education, National Center for Education Statistics (1986-89). Id. ¶¶ 14-15, 18. He is an Adjunct Full Professor in the Department of Biostatistics in the School of Public Health at the University of Alabama and is the author or co- author of two books and numerous articles on statistical methods and sampling, including several articles relating specifically to the use of statistical sampling or financial analysis by and for lending institutions and loans. Id. ¶¶ 20, 22-23. Dr. Cowan is the Managing Partner of Analytic Focus, LLC, a firm that provides expert witness and litigation consulting services. Id. ¶ 12. He has considerable experience developing and implementing statistical sampling methodologies in RMBS-related cases. Id. & Exh. 2. Every court to consider the issue has deemed Dr. Cowan to be a qualified expert on this subject. See, e.g., FHFA, 2012 WL 6000885, at *6 (approving Cowan sampling methodology); In re Countrywide Fin. Corp. Mortg.-Backed Sec. Litig., Nos. 11-ML-2265-MRP MANx et al., -- F. Supp. 2d --, 2013 WL 6231713, at *3 (C.D. Cal. Dec. 2, 2013) (“Dr. Cowan is therefore qualified to serve as an expert witness in the areas of economics and statistics and is further qualified to design and implement statistical sampling studies.”); Massachusetts Mut. Life Ins. Co. v. Residential Funding Co., Civ. A. No. 11-30035-PBS, 2013 WL 6490125, at *2 (D. Mass. Dec. 9, 2013) (“MassMutual”) (“Defendants do not challenge his qualifications.”). B. Statistically Valid Random Sampling Is Well-Accepted and Reliable Statistical sampling is a scientifically valid, common, and widely accepted technique for making reliable estimates about the characteristics of large populations. Cowan Sampling Report ¶¶ 36-47. Statisticians can estimate the characteristics of a population with a known degree of Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 10 of 20 6 precision based on analysis of a representative sample of that population. As a leading treatise on evidence explains, “[t]he researcher tries to collect information from a manageable portion (a sample) of a larger group (a population) in order to learn something about the population.” 1 Kenneth S. Broun, McCormick on Evid. § 208 (7th ed. 2013); see also Manual for Complex Litigation § 11.493 (“Statistical methods can often estimate, to specified levels of accuracy, the characteristics of a ‘population’ or ‘universe’ of events, transactions, attitudes, or opinions by observing those characteristics in a relatively small segment, or sample, of the population.”). The U.S. government commonly uses statistical sampling to draw conclusions about large populations. For instance, Congress has authorized the U.S. Census Bureau to use sampling for a variety of purposes.2 The FDIC routinely uses statistical samples in regulating financial institutions because it would be impossible to examine all of the loans held by such institutions.3 As the Office of the Comptroller of the Currency (“OCC”) has stated, statistical sampling “is a valid sampling procedure for determining adherence to a requirement, such as a bank’s underwriting standards,” and the OCC has “endorsed” the use of sampling to “evaluate . . . credit portfolios such as . . . residential real estate loans.”4 Private industry likewise uses sampling for a variety of purposes.5 It 2 See 13 U.S.C. § 195 (“Except for the determination of population for purposes of apportionment of Representatives in Congress among the several States, the Secretary shall, if he considers it feasible, authorize the use of the statistical method known as ‘sampling’ in carrying out the provisions of this title.”); U.S. Census Bureau, Current Population Survey, Sampling (2013), available at http://www.census.gov/cps/methodology/sampling.html. 3 See FDIC, Risk Management Manual of Examination Policies §§ 1.1, 3.2 (2005), available at http://www.fdic.gov/regulations/safety/manual/. 4 OCC, Comptroller’s Handbook, Sampling Methodologies at 6 (Aug. 1998), available at http://www2.occ.gov/publications/publications-by-type/comptrollers-handbook/sampmeth.pdf. 5 See, e.g., Vigilant Ins. v. Sunbeam Corp., 231 F.R.D. 582, 587 (D. Ariz. 2005) (insurance industry samples damaged goods to determine total losses in a fire); Cities Serv. Co. v. Derby & Co., 654 F. Supp. 492, 497-98 (S.D.N.Y.) (noting that American Society for Testing and Materials/American Petroleum Institute developed industry standards for sampling petroleum and petroleum products), aff’d, 835 F.2d 1429 (2d Cir. 1987); Nielsen Co. (US) v. Truck Ads, LLC, No. 08 C 6446, 2011 WL Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 11 of 20 7 was a common practice in the RMBS industry, for example, to select a sample of loans for “due diligence” review.6 As with private industry and other branches of government, federal courts employ statistical sampling to manage the scope of discovery. “Such mathematical and statistical methods are well recognized by the courts as reliable and acceptable in determining adjudicative facts.” Rosado v. Wyman, 322 F. Supp. 1173, 1180 (E.D.N.Y. 1970), aff’d, 437 F.2d 619 (2d Cir. 1970), aff’d, 402 U.S. 991 (1971).7 The Federal Judicial Center expressly endorses statistical sampling. See Manual for Complex Litigation § 11.422 (“When limits are placed on discovery of voluminous transactions or other events, consider using statistical sampling techniques to measure whether the results of the discovery fairly represent what unrestricted discovery would have been expected to produce.”); id. § 11.493 (“Acceptable sampling techniques, in lieu of discovery and presentation of voluminous data from the entire population, can save substantial time and expense, and in some cases provide the only practicable means to collect and present relevant data.”); see generally David H. Kaye & David A. Freedman, Reference Manual on Scientific Evidence, Reference Guide on Statistics (Fed. Judicial Ctr. 3d ed. 2011), available at http://www.fjc.gov/public/pdf.nsf/lookup/sciman02.pdf/$file/sciman02.pdf. For these reasons, courts have consistently ruled that statistical sampling methodologies in RMBS litigation satisfy Rule 702. E.g., FHFA, 2012 WL 6000885, at *11 (denying defendants’ Daubert motion to exclude Dr. Cowan’s testimony); MassMutual, 2013 WL 6490125, at *2, *9 (same); 3857122, at *2 (N.D. Ill. Aug. 29, 2011) (describing Nielsen’s statistical sampling process to measure the audience for television programs). 6 See Fin. Crisis Inquiry Comm’n, Final Report of the National Commission on the Causes of the Financial and Economic Crisis in the United States at 165-68 (2011), available at http://www.gpo.gov/fdsys/pkg/GPO-FCIC/content-detail.html. 7 See also United States v. Lahey Clinic Hosp., Inc., 399 F.3d 1, 18 n.19 (1st Cir. 2005) (“Further, sampling of similar claims and extrapolation from the sample is a recognized method of proof.”); United States v. United Shoe Mach. Corp., 110 F. Supp. 295, 305-06 (D. Mass. 1953) (“If anti-trust trials are to be kept manageable, samples must be used, and a sample which is in general reasonable should not be rejected in the absence of the offer of a better sample.”), aff’d, 347 U.S. 521 (1954). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 12 of 20 8 Assured Guar., 920 F. Supp. 2d at 501 (“[T]he sample size suggested by Dr. Lipshutz provided an adequate basis for assessing whether the Trusts as a whole complied with or breached Flagstar’s representations and warranties.”); In re Washington Mut. Mortg. Backed Sec. Litig., No. C09-37 MJP, 2012 WL 2995046, at *6 (W.D. Wash. July 23, 2012) (“Defendants have not shown . . . that the statistical method employed by Dr. Cowan is unacceptable.”); see also MBIA Ins. Corp. v. Countrywide Home Loans, Inc., No. 602825/08, 2010 WL 5186702, at *4-*5 (N.Y. Sup. Ct. Dec. 22, 2010) (concluding that plaintiff ’s expert sampling testimony was reliable and admissible under New York’s “general acceptance” test).8 Several of these cases involved sampling methodologies designed by Dr. Cowan. C. Dr. Cowan Has Applied His Statistical Sampling Methodology Reliably The process of sampling and re-underwriting loan files involves four general steps: (1) design a methodology to obtain a statistically valid random sample of representative loans from each SLG; (2) apply that methodology to each SLG at issue to select the loans composing the random samples; (3) obtain and re-underwrite the loan files for the loans selected for the sampling; and (4) extrapolate the re-underwriting results to the SLGs. Dr. Cowan’s Sampling Report resolves the first two steps. As set forth below, Dr. Cowan selected random samples of 100 loans per SLG, with stratification based on Fair Isaac Corporation credit scores (“FICO scores”), which will be representative at a 95% confidence level with a maximum ±10% margin of error. Notably, this Court considered and approved nearly identical statistical sampling methodology in the FHFA litigation. There, as here, Dr. Cowan proposed to randomly select 100 loans from each SLG, 8 See also Syncora Guarantee Inc. v. EMC Mortg. Corp., No. 09 C 3106(PAC), 2011 WL 1135007, at *4, *6-*7 (S.D.N.Y. Mar. 25, 2011) (granting summary judgment to the plaintiff, in breach of contract case brought by insurer of RMBS offering, based on random sample of 400 of the 9,871 loans); Spears v. First Am. eAppraiseIT, No. C-08-00868 RMW, 2012 WL 1438709, at *6 (N.D. Cal. Apr. 25) (on class certification in case involving alleged conspiracy to inflate home appraisals, holding statistical sampling could be used to prove aggregate characteristics of the 300,000 appraisals), appeal dismissed, No. 12-80109 (9th Cir. July 19, 2012). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 13 of 20 9 involving a 95% confidence level and with a maximum ±10% margin of error. FHFA, 2012 WL 6000885, at *5. 1. Random Sample of 100 Loans : As Dr. Cowan details in his Sampling Report, it is statistically appropriate to obtain 100 loans randomly selected from SLGs collateralizing each of the 82 unique Certificates.9 The relevant characteristics of these random samples will match the characteristics of the SLGs with a known degree of accuracy - specifically, to a confidence level of 95% with a maximum margin of error of ±10%. Because it may not be possible to obtain loan files for the 100 randomly selected loans (some loan files may be lost, misplaced, or otherwise unavailable), Dr. Cowan also randomly selected 100 supplemental (or “backup”) loans for each SLG.10 In the event that any loan files in the 100-loan sample are unavailable, they will be replaced with loans from the backup sample according to a preselected, random order that precludes any sort of cherry-picking. 2. Confidence Level and Margin of Error : When a sample is randomly selected, it provides an unbiased description of the population from which the sample was drawn. Cowan Sampling Report ¶ 36. The descriptive precision of a random sample is measured using the statistical concepts of “confidence level” and “margin of error.” Id. ¶¶ 37-38. Dr. Cowan explains that a sample size of 100 loans per SLG will generate results at a 95% confidence level with a maximum 9 The “SLG” refers to the particular group of loans directly collateralizing a Certificate. In some RMBS, a single SLG collateralizes all the Certificates. In other RMBS, Certificates may be collateralized by different SLGs. 10 This specification of backup samples is an addition to Dr. Cowan’s proposed methodology from the FHFA case. Although this Court ruled that the lack of a “substitution procedure for loans included in the sample whose files are missing or unobtainable” was not “a valid basis for exclusion of the sample” in FHFA, 2012 WL 6000885, at *11, Dr. Cowan has addressed that issue explicitly in this expert report. Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 14 of 20 10 margin of error of ±10%. Id. ¶ 57.11 The confidence level is a measure of how frequently the sample reflects the actual value for the population. Id. ¶ 38. Thus, at a 95% confidence level, there is a 95% likelihood that the actual value for the population will be within a specified range of the sample value. Id. The margin of error defines that range. Id. For example, suppose that a re- underwriting expert determines that 50% of loans in a sample (50/100) were not originated in compliance with the applicable underwriting guidelines. A 95% confidence level and 10% margin of error means there is a 95% probability that 40% to 60% (that is, 50% plus or minus the 10% margin of error) of the loans in the SLG were not originated in compliance with the underwriting guidelines. 3. Stratification by FICO: Dr. Cowan also used “stratification” by FICO score. Stratification is a tool used by statisticians that may increase (but cannot decrease) precision. Id. ¶ 41. To accomplish the FICO stratification, the SLGs were divided into equal size subgroups by FICO score. Equal size samples were selected from each of these subgroups (25 per subgroup, with 25 backup loans per subgroup). Id. ¶¶ 65-66. This stratification helps to ensure that the FICO scores of the loans in the sample corresponded to the FICO scores of the loans in the SLG. 4. Implementation: Implementation of Dr. Cowan’s sampling methodology was a mechanical process. See id. ¶ 68. Each SLG was divided into four equal subgroups by FICO score. Id. ¶ 66. Next, Dr. Cowan used standard statistical software to assign a random number to each loan. Id. Finally, he reordered each subgroup by the random number. Id. The sample consists of 11 As Dr. Cowan explains, the maximum margin of error is 10%. The actual margin of error may be less. See Cowan Sampling Report ¶¶ 58-66 & Chart 2 (explaining that the percentage of loans which did not comply with underwriting guidelines will affect the margin of error); id. ¶ 57 (explaining that FICO stratification may decrease the margin of error). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 15 of 20 11 the first 25 loans in each of the four subgroups, and the backup sample consists of the next 25 loans in each subgroup. Id.12 After the random samples were selected, Dr. Cowan confirmed that they were representative of the SLGs. He compared each random and backup sample with the corresponding SLG on up to 11 key variables: FICO score, debt-to-income ratio (DTI), LTV ratio, CLTV ratio, note rate, current loan amount, original term, documentation type, occupancy type, property type, and loan purpose. Id. ¶ 69. For continuous variables (those for which the values are numeric and increasing or decreasing in value, such as FICO score or LTV ratio), he compared the mean of the samples to the mean of the SLG distribution using t- and z-tests, both common statistical methods. For categorical variables (those for which the values are categories, such as documentation type), he compared the distribution of the categories in the samples to the distribution of the categories in the SLG using a Goodness of Fit Chi-square test, another common statistical method. Id. Because of the 95% confidence interval, one would expect that 5% of these tests would fail by pure chance. In fact, fewer than 5% of the tests failed, indicating the samples were representative of the SLGs as a whole. Id.13 D. The Proposed Sampling Methodology Is Relevant and Helpful to the Trier of Fact The sheer quantity of loans collateralizing the 82 unique Certificates - approximately 280,000 - precludes re-underwriting every loan file, and therefore collecting loan files beyond those 12 If the loan file for a loan in a random sample is unavailable in discovery, that loan will be replaced by the first numbered loan in the relevant sample supplement. Id. ¶ 32. For example, if one loan file is missing in each of the lowest and highest FICO score subgroups, the first available loan in those respective subgroups will be used instead. 13 As of the filing date of this motion, Defendants had failed to produce loan tapes for the GMAC 2006-HE5 Certificate in Case No. 13-6726. NCUA has subpoenaed third parties to obtain the associated loan tapes. Once NCUA has obtained these loan tapes, Dr. Cowan will draw samples for that Certificate pursuant to the same sampling methodology explicated in detail in his expert Sampling Report and used to draw the lists of samples appended to Dr. Cowan’s report for every other Certificate for which Defendants provided loan tapes to NCUA. Id. ¶ 56. Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 16 of 20 12 required for sampling is needless and wasteful.14 Because some (but not all) loan files will be re- underwritten, two options are available: statistically valid random sampling or some other method of re-underwriting an unspecified number of loans. Compared to the alternative, NCUA’s statistically valid sampling methodology will help the trier of fact “to understand the evidence” and “to determine a fact in issue,” Fed. R. Evid. 702, because sampling allows the trier of fact to estimate the characteristics of the entire loan pool with a known degree of accuracy (i.e., 95% confidence level and ±10% margin of error). Cowan Sampling Report ¶ 5. As this Court ruled, “the use of [Dr. Cowan’s] sampling will assist the fact finder in determining liability not only for any one securitization with a ‘known level of accuracy,’ but also in analyzing liability for a combined set of securitizations in any one case with ‘a much higher level of confidence’ when the estimated total number of breaches across all securitizations in one case are counted.” FHFA, 2012 WL 6000885, at *6. Similarly, every other court to have ruled on the issue in similar RMBS cases has found that Dr. Cowan’s and similar sampling methodologies assist the trier of fact. See MBIA, 2010 WL 5186702, at *4-*5; Washington Mut., 2012 WL 2995046, at *6; Assured Guar., 920 F. Supp. 2d at 502. E. The Court Should Rule Now on the Admissibility of the Statistical Sampling Methodology A ruling on the admissibility of Dr. Cowan’s statistical sampling methodology at the onset of discovery will facilitate “the just, speedy, and inexpensive determination” of this case. Fed. R. Civ. P. 1. This Court recognized that efficient management of RMBS litigation requires prompt 14 In the FHFA litigation, this Court noted that although defendants refused to limit discovery to FHFA’s statistical samples of the relevant loan pools, “[t]o date, however, it appears that none of the defendants will be relying on a sample different from that identified by FHFA for the purpose of performing the task of re-underwriting.” FHFA v. UBS Ams. Inc., Nos. 11 Civ. 5201 (DLC) et al., 2013 WL 3284118, at *14 n.13 (S.D.N.Y. June 28), recon. denied, Nos. 11 Civ. 6188(DLC) et al., 2013 WL 5354212 (S.D.N.Y. Sept. 25, 2013). During a hearing on the NCUA cases, Defendants confirmed “that the litigation of FHFA is progressing based upon samples selected by FHFA.” Hearing Tr., NCUA v. Morgan Stanley & Co., Nos. 13 CV 6705 DLC et al., ECF No. 47, at 18:13-14 (S.D.N.Y. Mar. 13, 2014) (excerpt attached as Exhibit C). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 17 of 20 13 resolution of the statistical sampling issue. See FHFA, 2012 WL 6000885, at *3 (“Early vetting of the parties’ sampling protocols is particularly important in this case, as the plaintiff and defendants should not be required to begin the costly and time-consuming process of reunderwriting without some assurance that the samples will be deemed admissible.”). Judge Wu reached the same conclusion in NCUA’s action against defendant Goldman in the Central District of California. Hearing Tr., NCUA v. Goldman, Sachs & Co., No. 2:11-cv-06521, at 30:16-31:4, 34:3-17 (C.D. Cal. Sept. 5, 2013) (noting that the sampling issue “has to be done earlier [rather] than later,” because “if you’re not going to be allowed to do something of that sort, you pretty much better know early on”) (excerpt attached as Exhibit D); see also MassMutual, 2013 WL 6490125, at *5 (“Early resolution of the viability of the sampling methodology makes sense as a case management matter.”); Order Regarding Discovery Schedule, In re Countrywide Fin. Corp. Mortg.-Backed Sec. MDL, No. 2:11-ml- 02265-MRP, ECF No. 125 (C.D. Cal. May 20, 2013) (attached as Exhibit E) NCUA already has issued scores of subpoenas to third parties to collect the loan files for re- underwriting. A ruling that Dr. Cowan’s testimony is admissible will allow NCUA to limit the production requests to the sampled loans only, thereby greatly reducing the burden on third parties. Numerous courts have recognized that statistical sampling can serve this purpose.15 Critically, Defendants will not be prejudiced by an early ruling on this issue. Rather, an early ruling will benefit 15 See, e.g., Feske v. MHC Thousand Trails Ltd. P’ship, No. 11-CV-4124-PSG, 2012 WL 1123587, at *2 (N.D. Cal. Apr. 3, 2012) (“Between the two extremes of burdening all 100,000-plus members, and none of them, the balance is appropriately struck by allowing Plaintiffs discovery of a statistical[ly] significant sample of members.”); Zurich Am. Ins. Co. v. Ace Am. Reins. Co., No. 05 Civ. 9170 RMB JCF, 2006 WL 3771090, at *2 (S.D.N.Y. Dec. 22, 2006) (“The parties shall therefore propose a protocol for sampling R & Q’s claim files . . . . To the extent that R & Q objects to any sampling proposal advanced by Zurich, it shall support its objections with specific evidence of the cost and burden involved.”); In re World Trade Ctr. Disaster Site Litig., 598 F. Supp. 2d 498, 503, 505 (S.D.N.Y. 2009) (sua sponte requiring sampling for both liability and damages in coordinated proceeding). Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 18 of 20 14 Defendants by allowing them to decide whether to rely on NCUA’s samples, create their own separate samples, or adopt some other course. In sum, uncertainty on this issue benefits no one and will result in a more protracted, burdensome, and expensive discovery process. As this Court has observed: Obviously this early Daubert motion practice has been necessitated by the enormity of the task that confronts all of us. . . . [A]n enormous amount of work by the plaintiff and defendants is going to be driven by the plaintiff ’s protocol for choosing this sample and by the identification of the individual loan files. All of you need to know now whether this sampling protocol is so fatally flawed that it should be stricken on Daubert grounds. Hearing Tr., FHFA v. JPMorgan Chase & Co., Nos. 11-5201 (DLC) et al., ECF No. 223, at 87:25- 88:14 (S.D.N.Y. Oct. 15, 2012) (excerpt attached as Exhibit F). CONCLUSION For the foregoing reasons, NCUA respectfully requests a ruling in limine admitting Dr. Cowan’s expert testimony regarding statistical sampling pursuant to Federal Rule of Evidence 702. Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 19 of 20 15 Dated: April 4, 2014 Erik Haas Peter W. Tomlinson Philip R. Forlenza Henry J. Ricardo PATTERSON BELKNAP WEBB & TYLER LLP 1133 Avenue of the Americas New York, NY 10036 Tel: (212) 336-2000 Fax: (212) 336-2222 ehaas@pbwt.com pwtomlinson@pbwt.com pforlenza@pbwt.com hjricardo@pbwt.com David H. Wollmuth Frederick R. Kessler Steven S. Fitzgerald Ryan A. Kane WOLLMUTH MAHER & DEUTSCH LLP 500 Fifth Avenue, 12th Floor New York, NY 10110 Tel: (212) 382-3300 Fax: (212) 382-0050 dwollmuth@wmd-law.com fkessler@wmd-law.com sfitzgerald@wmd-law.com rkane@wmd-law.com /s/ David C. Frederick David C. Frederick Wan J. Kim Gregory G. Rapawy Andrew C. Shen KELLOGG, HUBER, HANSEN, TODD, EVANS & FIGEL, P.L.L.C. Sumner Square 1615 M Street, N.W., Suite 400 Washington, D.C. 20036 Tel: (202) 326-7900 Fax: (202) 326-7999 dfrederick@khhte.com wkim@khhte.com grapawy@khhte.com ashen@khhte.com George A. Zelcs KOREIN TILLERY LLC 205 North Michigan Avenue, Suite 1950 Chicago, IL 60601 Tel: (312) 641-9760 Fax: (312) 641-9751 gzelcs@koreintillery.com Stephen M. Tillery Greg G. Gutzler Peter H. Rachman Robert L. King KOREIN TILLERY LLC 505 North Seventh Street, Suite 3600 St. Louis, MO 63101 Tel: (314) 241-4844 Fax: (314) 241-3525 stillery@koreintillery.com ggutzler@koreintillery.com prachman@koreintillery.com rking@koreintillery.com Attorneys for Plaintiff National Credit Union Administration Boar Case 1:13-cv-06736-DLC Document 77 Filed 04/04/14 Page 20 of 20