7,813,596 B2 et al.Download PDFPatent Trials and Appeals BoardJun 1, 20212018007745 (P.T.A.B. Jun. 1, 2021) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 95/000,684 08/17/2012 7,813,596 B2 7571 23363 7590 06/01/2021 Lewis Roca Rothgerber Christie LLP PO BOX 29001 Glendale, CA 91209-9001 EXAMINER FOSTER, ROLAND G ART UNIT PAPER NUMBER 3992 MAIL DATE DELIVERY MODE 06/01/2021 PAPER Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ GOOGLE INC. Requester, v. Patent of VEDERI, LLC. Patent Owner ____________ Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 Technology Center 3900 ____________ Before DENISE M. POTHIER, ERIC B. CHEN, and IRVIN E. BRANCH, Administrative Patent Judges. POTHIER, Administrative Patent Judge. DECISION ON APPEAL I. STATEMENT OF CASE This proceeding returns to us on remand from the Federal Circuit, vacating our previous decisions for this proceeding mailed August 15, 2016, September 28, 2018, and February 1, 2019. See Vederi, LLC v. Google LLC, 813 F. App’x 499, 501, 505 (Fed. Cir. 2020). As background, Requester requested an inter partes reexamination (“the Request”) of U.S. Patent No. 7,813,596 (“the ’596 patent”). The ’596 patent Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 2 claims priority to U.S. Applications: (1) 11/761,361 (now U.S. Patent No. 7,577,316), filed June 11, 2007, (2) 11/130,004 (now U.S. Patent No. 7,239,760), filed May 16, 2005, and (2) 09/758,717 (now U.S. Patent No. 6,895,126), filed on January 11, 2001. The ’596 patent, code (60). Pursuant to 35 U.S.C. § 154(a)(2), the term of the ’596 patent ended twenty (20) years from the filing date (i.e., January 11, 2001) of the earliest application for which a benefit is claimed under 35 U.S.C. §§ 120 and 121. See 35 U.S.C. § 154(a)(2) (2013); see also the Manual of Patent Examining Procedure (MPEP) § 2701(I). Thus, the ’596 patent expired on January 11, 2021.1 “No amendment may be proposed for entry in an expired patent.” 37 C.F.R. § 1.530(j); see also 37 C.F.R. § 1.121(j) (referring to § 1.530). That is, “[a]lthough the Office actions will treat proposed amendments [during a reexamination proceeding] as though they have been entered, the proposed amendments will not be effective until the reexamination certificate is issued and published.” 37 C.F.R. § 1.530(k). Notably, “no amendment, other than the cancellation of claims, will be incorporated into the patent by a certificate issued after the expiration of the patent.” 37 C.F.R. § 1.530(j). Accordingly, the reexamination proceeding will now be based on the original patent claims of the ’596 patent. Patent Owner’s proposed amendments (see, e.g., the January 3, 2013 Amendment (“Jan. 2013 Amendment”)) to the claims, including new claims 63–75 (Jan. 2013 Amendment 15–17; PO Reopen Request 19), are thus improper at this time. See MPEP § 2666.01. On the other 1 The MPEP states the Office should “refuse to express to any person any opinion as to . . . the expiration date of any patent, except to the extent necessary to carry out: . . . (C) a . . . reexamination proceeding to reexamine the patent.” MPEP § 1701 (9th ed. rev. 10.2019 June 2020) (emphases added). Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 3 hand and even though the ’596 patent has expired, Patent Owner’s proposed claim amendments to cancel claims 1, 2, 10, 15, 16, 20, 23, 35, 40, 45, and 50 (see Jan. 2013 Amendment 4–7, 10–12) are permitted. See MPEP § 2666.01. Additionally, claims 3, 5–9, 11–14, 17–19, 22, 24–34, 36–39, 41–44, 46–49, and 51–62 are not subject to reexamination. See RAN 1 (box 1b).2 Based on the foregoing, the reexamination proceeding will be based on original patent claims 4 and 21. Upon review, we REVERSE the rejections adopted by the Examiner but present new grounds of rejection for patent claims 4 and 21 pursuant to 37 C.F.R. § 41.77(b). Related Matters The parties indicate that the ’596 patent was the subject of the following litigation: Vederi, LLC v. Google Inc., Case No. 2:10-CV-07747 (C.D. Cal.), Vederi, LLC v. Google Inc., Case Nos. 13-1057, and Vederi, LLC v. Google Inc., Case No. 13-1296.3 PO Appeal Br. 2; 3PR Appeal Br. 1, 20–21, Related 2 Throughout this Opinion, we refer to: (1) the Action Closing Prosecution (ACP) mailed September 24, 2013, (2) the Right of Appeal Notice (RAN) mailed June 4, 2014, (3) Patent Owner’s Appeal Brief (PO Appeal Br.) filed September 3, 2014, (4) Requester’s Respondent Brief (3PR Resp. Br.) filed October 2, 2014, (5) Patent Owner’s Rebuttal Brief (PO Reb. Br.) filed May 22, 2015, (6) the Requester’s Appeal Brief (3PR Appeal Br.) filed September 8, 2014, (7) the Examiner’s Answer (Ans.) mailed April 21, 2015, (8) Patent Owner’s Request to Reopen Prosecution (PO Reopen Request) filed September 16, 2016, (9) the Examiner’s Determination Under 37 C.F.R. 41.77(d) (Ex. Deter.) mailed May 17, 2018, and (10) the Request for Inter Partes Reexamination (“Request”) filed August 17, 2012. 3 Cases Nos. 13-1057 and 13-1296 were decided on March 14, 2014 and concerned U.S. Patent Nos. 7,577,316 B2, 7,805,025 B2, and 7,239,760 and the ’596 patent. Vederi, LLC v. Google, Inc., 744 F.3d 1376 (Fed. Cir. 2014). The Federal Circuit reversed the claim construction of the district court, vacated the judgement, and Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 4 Proceedings App. Additionally, the parties indicate that this appeal may be related to: (1) U.S. Patent No. 7,805,025 B2, which is the subject of inter partes reexamination having been assigned Control No. 95/000,681, (2) U.S. Patent No. 7,239,760 B2, which is the subject of inter partes reexamination having been assigned Control No. 95/000,682, and (3) U.S. Patent No. 7,577,316 B2, which is the subject of inter partes reexamination having been assigned Control No. 95/000,683. PO Appeal Br. 2; 3PR Appeal Br. 1. The opinions in these proceedings were similarly vacated. Vederi, 813 F. App’x 501. Parties Appeals Patent Owner appealed the decision in the RAN rejecting claims 4 and 21. See PO Appeal Br. 3, 6. Requester responded, and Patent Owner rebutted. Requester cross-appealed the decision in the RAN determining now- improper claims of the ’596 patent are patentable. 3PR Appeal Br. 2. Patent Owner responded, and Requester rebutted. The Examiner’s Answer incorporated the RAN (Ans. 1), which rejected claims 4 and 21 on various grounds. RAN 1, 6–19. An oral hearing was conducted on April 27, 2016. The transcript of the hearing has been made of record. After the August 2016 Decision, Patent Owner requested reopening prosecution. See PO Reopen Request 1. The remand was granted in part for the Examiner’s consideration of now-improper claims. May 23, 2017 Order 3–5. The Examiner determined that the rejections of the now-improper claims were not remanded for further proceedings. See id. at 1384; see also PO Appeal Br. 3. The disputed claim language addressed by the Federal Circuit in this case differs from the disputed language in the instant appeal. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 5 overcome. Ex. Deter. 2. We, subsequently, rendered a second opinion on September 28, 2018, and a decision on rehearing on February 1, 2019. The Federal Circuit vacated our decisions. Vederi, 813 F. App’x at 501. In its opinion, the court construed three phrases found in the claims of U.S. Patent No. 7,805,025 (“the ’025 patent”). Vederi, 813 F. App’x at 501–505. These phrases are: (1) “composite image,” (2) “moving” within the phrase “image frames acquired by an image recording device moving along a trajectory,” and (3) “web page for the retail establishment” within the phrase “accessing a web page for the retail establishment; and invoking by the computer system a display of the web page on the display screen.” The court specifically agreed with how the panel construed the phrase “composite image” (id. at 503), did not fully adopt how the panel construed the phrase “moving” (id. at 503–504), and disagreed with how the panel construed the phrase “web page for the retail establishment” (id. at 504– 505). Given the claim construction addressed in Vederi, we reevaluate the rejections of claims 4 and 21. In reaching our decision, we consider the record as a whole. Claimed Subject Matter Canceled claim 1 and dependent claim 4 are reproduced below: 1. (Canceled) In a system including an image source and a user terminal having a screen and an input device, a method for enabling visual navigation of a geographic area from the user terminal, the method comprising: receiving a first user input specifying a first location in the geographic area; retrieving from the image source a first image associated with the first location, the image source providing a plurality of images depicting views of objects in the geographic area, the views being Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 6 substantially elevations of the objects in the geographic area, wherein the images are associated with image frames acquired by an image recording device moving along a trajectory; retrieving a map of at least a portion of the geographic area; displaying the retrieved first image on a first display area of the screen and the retrieved map on a second display area of the screen; receiving a user selection of a position on the displayed map; determining a second location based on the user selected position; and retrieving from the image source a second image associated with the second location. 4. The method of claim 1, wherein the first and second images are each a composite image, wherein each composite image is created by processing pixel data of a plurality of the image frames. The ’596 patent 15:42–64 (emphasis added), 16:6–9 (emphasis added). Prior Art Relied Upon The record relies on the following as evidence of unpatentability: Name Reference Date Lachinski4 US 5,633,946 May 27, 1997 Murphy US 6,282,362 B1 Aug, 28, 2001 4 Requester indicates that Lachinski was cited in its Comments to rebut Patent Owner’s response and to explain how Yee’s four-view images are created. 3PR Resp. Br. 2, 18–19 (citing page 27 of the “May Supplemental Third-Party Comments”); see also RAN 19–20 (discussing Lachinski is properly cited under 37 C.F.R. §1.948(a)(2)). Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 7 Michael J. Shiffer, Augmenting Geographic Information with Collaborative Multimedia Technologies, 11 Proc. Auto Carto. 367–376 (1993) (“Shiffer”). Frank Yee, GPS & Video Data Collection In Los Angeles County: A Status Report, Proc. IEEE Position Location and Navigation 388–393 (1994) (“Yee”). Toru Ishida et al., Digital City Kyoto: Towards A Social Information Infrastructure, 1652 Lecture Notes in Artificial Intelligence from Int’l Workshop on Cooperative Info. Agents 23–355 (1999) (“Ishida”). J. Dykes, An Approach To Virtual Environments For Visualization Using Linked Geo-referenced Panoramic Imagery, 24 Computers, Env’t & Urb. Sys. 127–152 (2000) (“Dykes”). Kheir Al-Kodmany, Using Web-Based Technologies and Geographic Information Systems in Community Planning, 7 J. Urb. Tech. 1–31 (2000) (“Al-Kodmany”). Nada Bates-Brkljac & John Counsell, Issues in Participative Use of an Historic City Millennial Web Site, IEEE Proc. Int’l Conf. Info. Visualization 119–125 (July 2000) (“Bates”). Current Rejections The Examiner rejects the claims as follows: Reference(s) Basis6 Claims RAN Dykes § 102(a) 4 6–8 Yee § 102(b) 4 8–10 Al-Kodmany § 102(a) 4 10–12 5 Ishida is not numbered but we refer to the pages sequentially starting with page 23 like the Request. See, e.g., Request 16, 212–215. 6 The Leahy-Smith America Invents Act, Pub. L. No. 112-29, 125 Stat. 284 (2011) (“AIA”), amended 35 U.S.C. §§ 112, 102, 103, and 305. Changes to §§ 102 and 103 apply to applications filed on or after March 16, 2013. Because this application has an effective filing date before March 16, 2013, we refer to the pre- AIA versions of §§ 102 and 103. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 8 Bates § 102(a) 4 12–13 Murphy and Yee § 103(a) 4 14–15 Shiffer and Yee § 103(a) 4 16–17 Ishida and Dykes § 103(a) 4, 21 17–19 II. MAIN ISSUE ON APPEAL We review the appealed rejections for error based upon the issues identified by Patent Owner, and in light of the arguments and evidence produced thereon. Cf. Ex parte Frye, 94 USPQ2d 1072, 1075 (BPAI 2010) (precedential) (citing In re Oetiker, 977 F.2d 1443, 1445 (Fed. Cir. 1992)). “Any arguments or authorities not included in the brief permitted under this section or [37 C.F.R.] §§ 41.68 and 41.71 will be refused consideration by the Board, unless good cause is shown.” 37 C.F.R. § 41.67(c)(1)(vii). Based on the record, the major issue on appeal is whether, as presented, the Examiner erred in rejecting patent claims 4 and 21? III. ANALYSIS A. Claim Construction As previously noted, the ’596 patent has expired. Because the ’596 patent has expired, we give its claims’ recitations “their ordinary and customary meaning” as would have been understood by “a person of ordinary skill in the art in question at the time of the invention.” Phillips v. AWH Corp., 415 F.3d 1303, 1312–13 (Fed. Cir. 2005); see also MPEP § 2258(I)(G) (citing Phillips, 415 F.3d at 1316; Ex parte Papst-Motoren, 1 USPQ2d 1655 (BPAI Dec. 23, 1986)). Additionally, “[c]laims ‘must be read in view of the specification, of which they are a part’” (Phillips, 415 F.3d at 1315 (quoting Markman v. Westview Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 9 Instruments, Inc., 52 F.3d 967, 979 (Fed. Cir. 1995) (en banc))), and “the specification ‘is always highly relevant to the claim construction analysis’” (id. (quoting Vitronics Corp. v. Conceptronic, Inc., 90 F.3d 1576, 1582 (Fed. Cir. 1996))). The parties discuss limitations of canceled claims 1 and 15, claim 4, and claim 21 of the ’596 patent in their briefings. PO Appeal Br. 6–23; see also 3PR Appeal Br. 16–18; 3PR Resp. Br. 2–9. Although claims 1 and 15 are canceled (see the Jan. 2013 Amendment 4, 6), claim 4 of the ’596 patent depends from canceled claim 1 and claim 21 ultimately depends from claim 15. Thus, each appealed claim includes either canceled claim 1’s or claim 15’s recitations. 1. The Image Frames Limitation of Canceled Claims 1 and 15 Canceled claims 1 and 15 recite “wherein the images are associated with image frames acquired by an image recording device moving along a trajectory” (the Image Frames Limitation7). The ’596 patent 15:52–54, 17:18–20. Patent Owner’s arguments focus on the word “moving” in the Image Frames Limitation. PO Appeal Br. 8–19. Specifically, Patent Owner contends that it is unreasonable to construe this claim limitation broad enough to encompass situations where all the images are captured while the image recording device is stationary. Id. at 9. Patent Owner asserts that the claim term “moving” in this limitation should be given “meaning, [such that] image frames must be acquired while the recording device is in motion” (id.) and “along the trajectory at the time of the acquisition” (id. at 10). Patent Owner further argues that this understanding is the only construction consistent with the ’596’s patent disclosure. Id. at 10–17 (citing the 7 Requester refers to the quoted limitation as “the ‘Image Frames Limitation.’” 3PR Appeal Br. 16; 3PR Resp. Br. 2. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 10 ’5968 patent, code (57), 2:27–31, 2:45–48, 3:54–60, 4:50–62, 5:18–22, 5:52–54, 6:56–62, 7:58–64, and U.S. Provisional App. No. 60/238,490 (“the ’490 provisional application”), pp. 8–9). The Examiner and Requester disagree. RAN 21–26 (citing 3PR November 25, 2013 Comments (“3PR Nov. 2013 Comments”) 7, 9); 3PR Resp. Br. 2–7. The Examiner explains that canceled claims 1 and 15 do not recite that “[the] image frames [are] acquired by the device while moving along a trajectory (i.e., recording and moving at the same time).” RAN 21 (underlining omitted); see id. at 25–26 (referring to 3PR Nov. 2013 Comments 14–15). The Examiner asserts that the claims and Patent Owner’s admission “embrace alternate embodiments [as] disclosed in the [’596] patent,” including those of both recording while moving and stopping along a trajectory. Id. at 21; see id. at 21–24 (citing 3PR Nov. 2013 Comments 8–11; the ’596 patent 3:58–60, 5:52–6:23, Fig. 9; the ’490 provisional application, pp. 8–9); 3PR Resp. Br. 3–4, 7 (citing the ’596 patent 3:51–53, 4:56– 58, 4:61–5:8, 7:55–64, 10:15–19, Fig. 9; the ’490 provisional application 8–9; ACP 20). The court in Vederi construed the term “moving” within the phrase “image frames acquired by an image recording device moving along a trajectory” found in claim 21 of the ’025 patent. Vederi, 813 F. App’x at 501, 503–504. The court found “the claims to cover (1) image recording devices that acquire images while moving; (2) image recording devices that acquire images both while moving and while stationary,” but not “(3) image recording devices that acquire images only while stationary (although the image recording device moves along a trajectory at 8 Patent Owner refers to “the ’316 patent.” PO Appeal Br. 10–11. We presume that Patent Owner intended to refer to the ’596 patent. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 11 other times).” Id. at 504. Each of canceled claims 1 and 15 of the ’596 patent includes the same recitation as claim 21 of the ’025 patent addressed by the court. Compare the ’025 patent 17:51–53, with the ’596 patent 15:52–54, 17:18–20. Accordingly and as explained below, we apply a similar claim construction for the Image Frames Limitation in canceled claims 1 and 15 of the ’596 patent. The Vederi court applied the “broadest reasonable interpretation” to the claims at issue, not the ordinary and customary meaning as understood by an ordinarily skilled artisan as set forth in Phillips. Vederi, 813 F. App’x at 504 (stating “[t]he broadest reasonable interpretation requires that the claim construction be reasonable in light of the specification”). Even so, the court considered the disclosure of the ’025 patent in arriving at its construction. Id. (citing the ’025 patent 2:27–29, 3:47–49, 3:54–57, 4:50–53, 4:55–58, 5:18–19, 5:52–54, 6:58–61, Fig. 9). Similar passages to those cited by the court in the ’025 patent are found in the Specification of the ’596 patent at issue in this appeal. In particular, the Specification states “an image recording device moves along a path recording images of objects along the path” (the ’596 patent 2:27–29), “[m]ovement to the camera 10 is provided by a base, platform, or motor vehicle moving at an average speed of preferably about 20 miles/hour” (id. at 4:55–57), and “the camera 10 moves along the path” (id. at 5:18). See also id. at 4:51–53 (discussing a camera moving along a path); 5:53–54 (same), 6:59–61 (same). The Vederi court also states “the [S]pecification contemplates that some photos may be taken while the vehicle is stopped, for example, at an intersection.” Vederi, 813 F. App’x at 504 (citing the ’025 patent, Fig. 9); see also the ’596 patent, Fig. 9. When read in view of the Specification of the ’596 patent, we determine that the ordinary and customary meaning of “moving” within the Image Frames Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 12 Limitation in canceled claims 1 and 15 as understood by an ordinarily skilled artisan at the time of the invention includes an image recording device that acquires images associated with image frames (1) while moving and (2) both while moving and while stationary as long as some images are associated with image frames acquired while the image recording device is moving. Also, “Patent Owner agrees that the ‘trajectory’ [recited in the Image Frames Limitation of canceled claims 1 and 15] need not be preplanned.” PO Appeal Br. 19; see id. at 18–19. The Examiner and Requester similarly agree. RAN 25; see also 3PR Resp. Br. 3 n.1 (noting that Patent Owner concedes the “trajectory” need not “be preplanned”9); 3PR Appeal Br. 17. Patent Owner also contends that “when the images are being retrieved in response to a user input, the trajectories of the image acquisition device are known.” PO Appeal Br. 19. Requester contends the Image Frames Limitation does not require a known or specific trajectory. 3PR Appeal Br. 17–18; see also 3PR Resp. Br. 3 n.1. The ’596 patent describes a trajectory as synonymous with a path. The ’596 patent 3:56 (describing cameras “moving along a trajectory/path.”). The ’596 patent further provides an “illustration of a trajectory” in Figure 9 where a camera is moved along a path (e.g., 110 including streets or blocks) making turns at intersections and circling around streets. Id. at 3:14–15; see also id. at 7:58–64, Fig. 9. Even presuming, without agreeing, that the path of streets in this example (e.g., 110 in Figure 9) may have been known prior to recording, we note that there exists a level of randomness when driving down a street, such as lane shifting, 9 Requester indicates that the Examiner took a contrary position in the proceedings for Control Nos. 95/000,681 and 65/000,682 when construing the term “trajectory” for the claims in the ’025 patent and U.S. Patent No. 7,239,760. 3PR Resp. Br. 3 n.1; see also 3PR Appeal Br. 17–18. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 13 which deviates from any purported, known or specific path. Furthermore, the Figure 9 example in the Specification is described as “an illustration” of a trajectory, whereas the claim’s scope is not limited to this illustration. Compare the ’596 patent 15:54, with id. at 3:14–15 (stating “FIG. 9 is an illustration of a trajectory”), 7:58–64 (describing Figure 9). Thus, under its ordinary meaning and consistent with the Specification of the ’596 patent, the recited “trajectory” in canceled claims 1 and 15 need not be a preplanned, specific, or known trajectory. In sum, we find that the Image Frames Limitation (“wherein the images are associated with image frames acquired by an image recording device moving along a trajectory”) in canceled claims 1 and 15 requires the image recording device moves along a path or route, which is not necessarily preplanned, specific, or known, and that the image recording device acquires images that “are associated with image frames acquired by an image recording device” (1) while moving or (2) both while moving and while stationary as long as some image frames are acquired while the image recording device is moving. 2. Composite Image of Claim 4 Claim 4 depends from claim 1 and adds “the first and second images are each a composite image, wherein each composite image is created by processing pixel data of a plurality of the image frames.” The ’596 patent 16:6–9. The Examiner finds the phrase, “composite image,” includes synthesizing multiple images and combining four images into a single image. See RAN 28 (citing Lachinski 5:25–31). For support, the Examiner also refers to the Requester’s Comments filed November 25, 2013. Id. (citing 3PR Nov. 2013 “Comments” 5–6, 17–19). In these comments, Requester argues Patent Owner improperly imported a particular embodiment found in the ’596 patent’s Specification into the claims. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 14 3PR Nov. 2013 Comments 18; see also 3PR Resp. Br. 8 (stating that Patent Owner’s construction “improperly imports limitations from Figure 2”). Requester also states a plain meaning of “composite image” does not require a single, new view and can include a four-view image. 3PR Nov. 2013 Comments 18–19; see also 3PR Resp. Br. 8. Patent Owner argues the Examiner’s interpretation is unreasonable. PO Appeal Br. 19–2310; see also PO Reb. Br. 7–10. Specifically, Patent Owner argues that: The composite image depicts a single new view of the objects in the geographical area. The single new view is different from any of the views depicted in any one of the image frames from which the composite image is created, e.g., it can be a wider view. Moreover, the new view is from a single location as if the viewer [was] at that location. PO Appeal Br. 19; see also id. at 20–21 (reproducing the ’596 patent, Figs. 2, 16 (in part); U.S. Provisional Application No. 60/238,490, Fig. 11) (citing U.S. Provisional Application No. 60/238,490, p. 8), 23; PO Reb. Br. 7 (citing the ’596 patent 1:50–54, 1:58–60). Patent Owner further contends that “[n]othing in the ’596 patent suggests that two or more separate and independent images depicting separate and distinct views of different objects is a ‘composite image’ as used in the ’596 patent simply because they are displayed simultaneously on a screen.” PO Appeal Br. 23. When considering the disclosure, the ’596 patent discusses creating “composite images” by synthesizing images, image data, or image frames but does 10 Patent Owner refers to claims 13, 18, 23, and 36 when discussing the “composite images” limitation. PO Appeal Br. 19. Notably, claims 13, 18, and 36 are not the subject of this reexamination, and claim 23 has been canceled. RAN 1. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 15 not disclose how the images are synthesized or combined to form the composite image. The ’596 patent, code (57), 2:22–24, 2:34–36, 3:46–49, 5:45–47. The Specification also states image data from each selected image frame 42 in Figure 2 is extracted and combined to form the composite image. Id. at 5:66–6:1, Fig. 2. The plain and ordinary meaning of “composite” includes “something that is made up of different parts.”11 A single image consisting of data from four reduced image frames is something made from different parts (e.g., a composite). An ordinary meaning of (1) “synthesize”12 includes “to make (something) by combining different things” or “to combine (things) in order to make something new,” and (2) “combine,”13 includes “to unite into a single number or expression.” Thus, the phrase “composite image” consistent with the Specification and its ordinary meaning should be construed to mean a single image created by combining different image data or by uniting image data. The Federal Circuit agreed with this claim construction in Vederi, determining the term “composite image,” such as that found in claim 4 of the ’596 patent, can reasonably be construed to include “a single image created by combining different image data or by uniting image data.” Vederi, 813 F. App’x at 503 (citing the ’025 patent, 5:66–6:1). The court also stated “[w]e are not persuaded by Vederi’s argument” that limits the claimed “‘composite image’ to ‘a 11 Composite (noun), Merriam-Webster’s Online Dictionary, available at http://www.merriam-webster.com/dictionary/composite. 12 Synthesize, Merriam-Webster’s Online Dictionary, available at http://www.merriam-webster.com/dictionary/synthesize. 13 Combine, Merriam-Webster’s Online Dictionary, available at http://www.merriam-webster.com/dictionary/combine (def. 1c). Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 16 new image . . . that depicts a single new view (from a single location) of the objects in the geographical area that is different from any of the views depicted in any one of the image frames from which the composite image is created.’” Vederi, 813 F. App’x at 503 (quoting the ’025 patent, 5:66–6:1; Personalized Media Commc’ns, LLC v. Apple Inc., 952 F.3d 1336, 1343 (Fed. Cir. 2020)). Accordingly, “a composite image” in claim 4 does not require the composite image to be an image having a single view from one location, a new view, a different view, or a wider field of view than any acquired image frame as argued by Patent Owner. Stated differently, “although the specification often describes very specific embodiments of the invention, [the Federal Circuit] ha[s] repeatedly warned against confining the claims to those embodiments.” Phillips, 415 F.3d at 1323 (citing Nazomi Commc’ns, Inc. v. ARM Holdings, PLC, 403 F.3d 1364, 1369 (Fed. Cir. 2005); Liebel-Flarsheim Co. v. Medrad, Inc., 358 F.3d 898, 906–08 (Fed. Cir. 2004)). We thus do not confine our understanding of the phrase, “composite image” in claim 4, to the exact representations in the Specification. Patent Owner also discusses “the patent infringement suit” between Patent Owner and Requester. PO Appeal Br. 19 n.1; see also id. at 20 (stating “a composite image 40 . . . has pixel values that are computed from pixel values of each of the image frames from which the composite image is created”). According to Patent Owner, the parties in that proceeding “agreed that ‘a composite image created by processing pixel data of a plurality of the image frames’ meant ‘an image formed by combining two or more image frames at the pixel level.’” Id. at 19 n.1 (citing “Joint Construction of Agreed Terms, Joint Exhibit C to Plaintiff Vederi, LLC's Opening Claim Construction Brief, Exhibit D hereto”). Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 17 The court in Vederi found the phrase “by processing pixel data of a plurality of the image frames,” similar to that found in claim 4 of the ’596 patent (the ’596 patent 16:8–9), specifies “the image may be achieved by combining or uniting image data, meaning at the level of pixel data.” Vederi, 813 F. App’x at 503 (citing the ’025 patent, 19:6–14). But, this recitation does not recite how the pixel data of the images frames are processed, such that pixel values of the composite image are computed from pixel values of two or more image frames. We stress that the ’596 patent states a preference for the composite image to be created by extracting image data from each image frame on a column-by-column basis. See the ’596 patent, code (57), 6:1–15, Fig. 2. But, applying the ordinary meaning of “processing,” claim 4 requires no more than “combining or uniting image data, meaning at the level of pixel data.” Vederi, 813 F. App’x at 503. Accordingly, the limitation of “a composite image” and “each composite image is created by processing pixel data of a plurality of the image frames” in claim 4 collectively means a single image created by combining different image data or uniting image data of image frames at the level of pixel data. 3. Web page for the retail establishment in claim 21 Claim 21 ultimately depends from canceled claim 15 and adds “wherein the particular one of the objects is a retail establishment, the method further comprising: accessing a web page for the retail establishment; and invoking by the computer system a display of the web page on the display screen.” The ’596 patent 18:7–12. Neither party disputes this claim or how this claim should be construed in this proceeding. However, we note the court in Vederi stated “[t]he Board limited a ‘web page for the retail establishment’ to web pages belonging to, owned by, or operated Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 18 by the retail establishment.” Vederi, 813 F. App’x at 504 (citing Google Inc. v. Vederi, LLC, No. 95/000,681, 2016 WL 792285, at *2–3 (PTAB Feb. 26, 2016)). The court found this characterized interpretation as “unduly narrow.” Id. The court indicated “an online Yellow Pages directory may be a web page for a retail establishment in that it shows particular information about the retail establishment for the convenience of a consumer” (id. at 505) and further states “a web page, such as an online Yellow Pages directory, may be associated with a particular retail establishment, but not owned or controlled by that establishment” (id. (citing the ’025 patent 12:53–56)). The Specification of the ’596 patent does not describe an online Yellow Pages directory but does discuss “business establishments” (the ’596 patent 12:48), stating that “the establishment” can be “associated with a particular Web page” (id. at 12:53–54). Consistent with the Specification, an ordinary meaning of the recited “web page for the retail establishment” in claim 21 as understood by an ordinarily skilled artisan at the time invention would have included a web page in which a business or retail establishment is associated. See id. This web page thus may include more than those belonging to, owned by, or operated by the retail establishment. See id.; see also Vederi, 813 F. App’x at 504. Although the court did not provide us an explicit claim construction for the phrase “web page for the retail establishment” found in claim 21, we understand the ordinary meaning of this phrase to include a web page that (1) shows particular information about the retail establishment or (2) is associated with a particular retail establishment. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 19 B. Pending Rejections Claim 4 is rejected under: (1) 35 U.S.C. § 102(b) based on Yee and (2) 35 U.S.C. § 102(a) based on (a) Dykes, (b) Al-Kodmany, and (c) Bates. RAN 6–13. Claim 4 is also rejected under 35 U.S.C. § 103(a) based on (3) Murphy and Yee, (4) Shiffer and Yee, and (5) Ishida and Dykes. Id. at 14–19. Claim 21 is rejected under 35 U.S.C. § 103(a) based on Ishida and Dykes. Id. at 17–19. These rejections were presented on the claims as amended and prior to the ’596 patent’s expiry. We reverse the rejections given the particular circumstances of this proceeding, which include that the dependencies of the claims have changed since the ’596 patent’s expiry, the Federal Circuit provided intervening claim construction for claim terms in the ’025 patent, and the claims are now construed under Phillips as opposed to the broadest reasonable construction. Compare Phillips, 415 F.3d at 1312–13, with Personalized Media Commc’ns, 952 F.3d at 1340. C. New Grounds of Rejection Pursuant to 37 C.F.R. § 41.77(b), we present a new ground of rejection for: (1) claim 4 under 35 U.S.C. § 103(a) based on Yee, Dykes, and Lachinski and (2) claims 4 and 21 under 35 U.S.C. § 103(a) based on Ishida, Yee, and Dykes. Of note, Patent Owner argues that the citation to Lachinski by Requester is improper and should be excluded from consideration because Lachinski was “introduced . . . for what Lachinski discloses in itself; it is not explaining another reference.” PO Appeal Br. 24; see also id. at 23–24; PO Reb. Br. 10–11. Requester contends that its reliance on and discussion of Lachinski is proper under 37 C.F.R. § 1.948(a)(2). See 3PR Resp. Br. 18–19 (contending Lachinski was Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 20 cited to explain Yee’s teachings, including its mobile mapping system) (citing May 22, 2013 3PR Comments 27); see also RAN 20. But, the propriety of whether a reference was properly submitted under § 1.948 is a petitionable matter. Because this issue is not appealable, the Board lacks jurisdiction to decide this issue. See MPEP §§ 1002 and 1201; see also In re Hengehold, 440 F.2d 1395, 1403 (CCPA 1971) (stating that there are many kinds of decisions made by examiners, “which have not been and are not now appealable to the board or to this court when they are not directly connected with the merits of issues involving rejections of claims, but traditionally have been settled by petition to the Commissioner”). Nonetheless, we underscore that “should the Board have knowledge of any grounds not raised in the appeal for rejecting any pending claim, it may include in its opinion a statement to that effect with its reasons for so holding, which statement shall constitute a new ground of rejection of the claim.” 37 C.F.R. § 41.77(b). 1. Claim 4 Because claim 4 depends from canceled claim 1, we adopt the findings and conclusions related to claims 1 and 4 by the Requester and the Examiner when addressing Yee. See Request 79–88 (citing Yee 389–92; Ex. CC-B; Ex. OTH-B 104:16–20, 121:1–3; Ex. OTH-D 17:7–9); see also Ex. CC-B 1–8 (citing Yee 389– 92); RAN 8–9 (citing Yee 389, 391–92, Fig. 1) (incorporating Request 79–97, Ex. CC-B); 3PR Resp. Br. 11–12 (citing Yee 388–89; Lachinski 5:25–40; ACP 22); 3PR Appeal Br. 5–12 (citing Yee 389–92, Abstract; ACP 7, 22; RAN 8–9, 27–28; Lachinski 5:25–31). We highlight that Yee discusses acquiring images as the vehicle, which houses its cameras, moves down the road or the recited “the images are associated with image frames acquired by an image recording device moving Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 21 along a trajectory” (the Image Frames Limitation) as canceled claim 1 recites. Yee 388–390 (describing the van is driving on the streets and recording images in all four directions as the van travels); see also Request 84 (citing Yee 389–90). We also highlight Yee’s “rolling video” or “single-frame images” (Yee 392) depict objects’ elevations and its views in a geographic area, including curbside views, street views, real estate views, houses, street lights, guard rails, and highway striping (id. at 389). See also RAN 8–9 (noting the same). The Request also turns to Yee’s discussion of various views, including “composite of them,” to teach the recited “composite image” and how the image is created. Request 87–88 (quoting Yee 389) (citing Ex. CC-B); see also Ex. CC-B 8 (quoting Yee 389). Patent Owner disputes that Yee teaches the recited first or second “composite image” of claim 4 when properly construed in light of the ’596 patent. PO Appeal Br. 27–30; see also PO Reb. Br. 8–10, 12. Patent Owner refers to “Section VII.A.3” of its brief, contending that Yee fails to disclose “a composite image.” PO Appeal Br. 27. In Section VII.A.3, Patent Owner argues the Examiner relies upon a changed meaning for composites in determining that Yee anticipates claim 4. See id. at 21–22. Patent Owner asserts that the discussion in Yee of composites includes “side-by-side views” or “multiple views, such as a 4- view display.” Id. at 22. Patent Owner contends that each view is depicted in a separate image and cannot be a composite as recited, which requires a single view of objects. Id. at 22–23. As discussed above in Section (III)(A)(2), we determine the phrase “composite image” includes a single image created by combining different image data or by uniting image data but does not have to be “a single new image with a new view” as urged by Patent Owner. Id. at 28. Moreover, as for the remaining Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 22 phrase that “each composite is created by processing pixel data of a plurality of the image frames,” we found that the recitation requires only combining different image data or uniting image data of image frames at the level of pixel data (see Section (III)(A)(2)) but does not recite how the pixel data of the images frames are processed. We determine Yee teaches or suggests the recited “composite image” recitation. Although identity of terminology is not required, Yee actually discloses “composites.” That is, Yee states “[s]ome of the specific data to be collected and made available . . . include: curbside view, front and back; street view, front and back; real estate view left and right; real estate and addresss [sic] zoom, 4-view; and composites of them.” Yee 389 (emphasis added). This portion of Yee explicitly discloses a composite image (i.e., “composite of them”). Id. Notably, the language “composites of them” in Yee is separate from the other described views, including the 4-view, and “them” refers back to the other discussed views, including a curbside view, a street view, and a real estate view. To illustrate, the Examiner identifies, and we agree “a composite display of (for example) a curbside view with a front and back view is a new view manipulated at the pixel level and synthesized from multiple images.” RAN 28. As another example, a composite in Yee may combine or unite image data from (1) the curbside view and the street view or (2) two different street views to produce the disclosed “composite[] of them.” See Yee 389. Additionally, an ordinarily skilled artisan would have recognized Yee’s disclosed “composites” (see id.) would have involved combining or uniting the views at the level of pixel data in some manner so as to form the disclosed “composites” available to the user in Yee. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 23 We thus disagree with Patent Owner that the Examiner changes the meaning of “composite” in determining Yee teaches the recited “composite image” in claim 4. Patent Owner argues that Yee’s composites “teaches away from creating composite images with its process.” PO Appeal Br. 28. This argument contrasts directly with Yee’s explicit disclosure of a process that creates “composites” from collected data. Yee 389. As for Patent Owner’s assertion that Yee’s discussion of composites includes “side-by-side views,” “multiple views,” or “a 4-view display” (PO Appeal Br. 22–23; see also id. at 28 (addressing Lachinski)), this argument is unavailing. As explained above, Yee discuses “composites of them” separate from its “4- view.” Yee 389. This teaching at least suggests to an ordinarily skilled artisan that each of Yee’s “composites of them” is a single image that may be created by combining or uniting image data from a plurality of image frames (e.g., street and curbside views) at the level of pixel data as we construed the phrase “composite image” in Section (II)(A)(2). To the extent that Yee’s “composites of them” are not considered to teach or suggest “each composite image is created by processing pixel data of a plurality of the image frames” as claim 4 recites (for which we do not agree), the rejection further relies on Dykes. Dykes teaches a known technique for creating “composites” by combining and uniting image data (e.g., stitching) to produce a panoramic image. Dykes 132–36, Fig. 2. When substituting Dykes’s known technique of forming a composite (e.g., a panoramic image) for Yee’s composite image forming technique, “the combination must do more than yield a predictable result.” KSR, 550 U.S. at 416. Additionally, Dykes teaches panoramic imagery (1) can assist “with educational aims,” including making more sense of maps when Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 24 looking at panoramic landscape views, or with the urban planning ( Dykes 134 (quoting Shiffer 365)), (2) can evoke a visual experience in an engaging virtual environment (id. at 136), (3) provide an ability to navigate across the virtual space and between recognized features (id. at 139), and (4) permit panning around the landscape touring across the virtual environment (id. at 140). Thus, combining Dykes’s teaching related to creating “a composite image” with Yee would have improved on Yee’s system by providing educational information, evoking a visual and engaging experience, and provide the ability to navigate across a virtual space. See KSR, 550 U.S. at 417. Also, Yee discloses a mobile mapping system for recording and storing images of a geographic area in a visual interface system (VIS) for an end user to locate and retrieve the collected video images. See Yee 391–92. Dykes discloses its process of collecting composite image data is minimal and cheap. Dykes 136, 140, 148. Thus, combining Dykes with Yee to arrive at claim 4’s “composite image” would have improved on Yee’s system by creating an environment that is inexpensive and easy to set up due to minimizing the collected data but still provides a spatial interface for a user to retrieve composite images (see id. at 134– 136). See KSR, 550 U.S. at 417. Patent Owner further reproduces a 4-view example in its appeal brief, asserting this is not a single view and thus not “a composite image” of claim 4. Appeal Br. 22. In concluding that this type of view is not “[a] composite image . . . created by processing pixel data of a plurality of the image frames” as claim 4 recites, Patent Owner presumes the example from “the GeoSpan Brochure”14 (PO 14 We are unable to locate this exhibit in the briefing. Patent Owner refers to “Exhibit E hereto.” Appeal Br. 22 n.2. But, Exhibit E is an Opinion from the Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 25 Appeal Br. 22 n.2) is the only “4-view” that Yee envisions and argues “each view is depicted in a separate image” (id. at 22). See id. at 22–23, 27–28 (discussing Lachinski when arguing that Yee’s “4-view” is not a composite image). Yet, the views described and shown in the GeoSpan Brochure and in Lachinksi are just examples of the data acquired and retrieved by the GeoSpan Corporation discussed in Yee. See, e.g., Yee 389 (describing that data collected). In any event, Yee discusses “images can be displayed as rolling video of four views in a frame” (Yee 392), which suggests a single frame with four views or a single image created by combining different image data of image frames (e.g., from the four views) at the level of pixel data. To the extent that Yee’s “4-view” (Yee 389) is not considered “a composite image” (for which we do not agree), the rejection further relies on Lachinski. Yee discusses a “4-view” example but does not provide details concerning how the view is formed. See id. at 389. Lachinski, which is a patent assigned to GeoSpan Corporation (Lachinski, code (73)), explains: The four-view generator 62 has four inputs 82, allowing signals from four of the video cameras 50 to be input simultaneously. The generator 62 reduces the image represented by each signal to one- fourth of its original size and then combines the reduced images to form a single video image by placing each of the reduced images into one of the four corners of an output image. United States District Court for the Central District of California. We did locate a submission on January 7, 2013 titled “Drive around town on your PC with GEOVISTA” (GEOVISTA) with page 2 appearing to have a similar figure to that reproduced on page 22 of Patent Owner’s Appeal Brief. In any event, the Geospan brochure is not part of Patent Owner’s briefing. Patent Owner also refers to its “January 2, 2013 Reply as Exhibit C” (id.), which is also not part of its briefing before us. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 26 Id. at 5:25–31 (emphasis added), Fig. 3. Thus, the 4-view discussed in Yee (Yee 389) can be produced as a single image that combines four reduced images (e.g., different image data at the pixel data level), one in each of four corners that is reduced in size as taught by Lachinski’s known technique. Yee also teaches that data from the four images, which includes its pixel data, are used to create the reduced-sized images into a single image. That is, each of the “four views in a frame” discussed in Yee (Yee 392) or the “single video image” with four reduced views (e.g., one-fourth of its original size that form “reduced images” as explained in Lachinski (Lachinski 5:25–31)) is a single image that is made up of different parts or image frames (e.g., image data from multiple views) and combines pixel image data from each of the different view image frames collectively to create the single 4-view image. Yee, as evidenced by Lachinski, therefore teaches and suggests another example of “a composite image” as claim 4 recites. Moreover, combining Lachinski’s known 4-view generator technique with Yee would have predictably yielded a “4-view” image (Yee 389) as a single image that combines image data from a plurality of image frames at the level of pixel data (e.g., four reduced images) or the recited “each composite image is created by processing pixel data of a plurality of the image frames” as claim 4 recites. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 416 (2007). Patent Owner next contends that Yee, including its GeoSpan system, would involve “manually review[ing] the raw image frames,” “select[ing] the most appropriate image” (PO Appeal Br. 28) and “add[ing] the step of creating composite images” (id. at 28–29), which would amount to “multiply[ing] the manual labor costs, greatly slow[ing] the process, and add[ing] another level of Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 27 complexity” (id. at 29). See also id. at 28–29.15 The record fails to support Patent Owner’s contentions, which essentially rely on arguments of counsel. See In re Geisler, 116 F.3d 1465, 1470 (Fed. Cir. 1997); see also In re Pearson, 494 F.2d 1399, 1405 (CCPA 1974) (attorney argument is not evidence). Moreover, the limitation “each composite is created by processing pixel data of a plurality of the image frames” in claim 4 does not exclude inputting some data manually, and many of the disputed features (e.g., slow, cost, precision, complexity) (PO Appeal Br. 28–29) are not commensurate in scope with claim 4. In any event, Lachinski, which addresses a GeoSpan system having similarities to Yee, discusses that a manual process is not used to generate composites. Lachinski 5:25–40 (discussing using generator 62 to form a single video image); see also 3PR Resp. Br. 12 (citing Lachinski 5:25–40; Yee 391). Accordingly, claim 4 is newly rejected under 35 U.S.C. § 103(a) based on Yee, Dykes, and Lachinski pursuant to 37 C.F.R. § 41.77(b). 2. Claims 4 and 21—Ishida, Yee, and Dykes Because claim 4 depends from canceled claim 1, we adopt the findings and conclusions related to claims 1 and 4 by Requester and the Examiner when addressing Ishida and Dykes. See Request 210–222 (citing Ishida 26–28, 34, Fig. 1; Dykes 134–36, 139–41, 144, 146, Figs. 2, 4; Ex. CC-G; Ex. OTH-B 58:18–19, 104:16–20, 106; Ex. OTH-D 17:7–9); see also RAN 17–19 (discussing and citing Ishida 26, Fig. 3; Dykes 142, 146–47; Request 211–12) (incorporating the Request for claims 4 and 21; Claim Chart CC-G); 3PR Resp. Br. 17–18 (citing Request 210–12; Ishida 27, Abstract; Dykes 142, 146; ACP 15–16). Because claim 21 15 Patent Owner footnotes a reference entitled “GEN-2 City Tour BBC & CNBC 1995, January 1, 2004” and states the reference was submitted January 7, 2013. PO Appeal Br. 28 n.3. This evidence is not in Patent Owner’s briefing. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 28 ultimately depends from canceled claim 15, we adopt the findings and conclusions related to claims 15, 20, and 21 by Requester and Examiner when addressing Ishida and Dykes. See Request 222–236 (citing Ishida 25–27, 30–31, 34, Figs. 1, 3; Dykes 136, 139–41, 144–46, Figs. 4, 6; Ex. CC-G; Ex. OTH-B 59:18–19, 61:23–25, 104:16–20, 106, 121:1–3; Ex. OTH-D 17:7–9); see also RAN 17–19 (discussing and citing Ishida 26, Fig. 3; Dykes 142, 146–47; Request 211–12) (incorporating the Request for claims 4 and 21; Claim Chart CC-G); 3PR Resp. Br. 17–18 (citing Request 210–12; Ishida 27, Abstract; Dykes 142, 146; ACP 15–16). Concerning claim 21, we repeat that the phrase “web page for the retail establishment” includes a web page (1) that shows particular information about the retail establishment or (2) associated with a particular retail establishment as discussed in Section (III)(A)(3). See Vederi, 813 F. App’x at 505. Ishida teaches a social information infrastructure for a city (e.g., Kyoto) that includes shopping, business, transportation, education, and other information. See Ishida 23–24, Abstract. This infrastructure integrates both World Wide Web archives and real- time information related to the city into WEB and ftp interface (e.g., the interface or second layer) on the Internet. See id. at 23–25, 28. Specifically, Ishida’s Section 4 indicates the digital city integrates WEB and sensory data on a map, which involves registering WEB pages with the digital city, determining the XY coordinate of each WEB page, and retrieving WEB pages. Ishida 28–30. As an example, sensors in Kyoto gather traffic data from buses that send location and route data to the live digital city, and WEB pages for bus stops are retrieved and displayed so that real-time bus data is displayed on the map of Kyoto. See Ishida 29–30, Fig. 5(b). As such, each of these web pages in Ishida (e.g., WEB and ftp Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 29 interface) shows particular information (e.g., bus data) about or associated with a retail establishment (e.g., a transportation company having a bus stop). Ishida therefore teaches or suggests “the particular one of the objects is a retail establishment” (e.g., a bus stop in geographic area,16 like Kyoto), “accessing a web page for the retail establishment” (e.g., Kyoto’s or the digital city’s WEB/ftp interface that contains web page information for the transportation company’s bus stop) and “invoking by the computer system a display of the web page on the display screen” (e.g., displaying the Kyoto bus stop’s real-time information using the WEB/ftp interface) as claim 21 recites. To the extent that a transportation company’s bus stop is not viewed as “a retail establishment,” Ishida also teaches its interface retrieves data related to parking, shopping, and sightseeing (id. at 24), which include information concerning parking lots (e.g., the nearest parking lot), restaurant tables (e.g., whether one can reserve a table at a restaurant), and shopping (e.g., what is on sale at a department store). See id. at 24, 30. Although Ishida further notes that information related to parking lots and restaurants are expected in the future (id. at 30), these teachings in Ishida at a minimum suggest “a web page for the retail establishment” recitations in claim 28 as construed in Section III(A)(3). Turning to the arguments, Patent Owner contends that neither Ishida nor Dykes acquires image frames by “an image recording device moving along a trajectory.” PO Appeal Br. 40–41, 43; see PO Reb. Br. 14. Patent Owner asserts that Dykes’s discussion of students taking images as they travel from one location 16 Claim 21, from which claim 28 depends, recites “a plurality of images depicting views of objects in the geographic area.” The ’025 patent, 17:48–49. Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 30 to another, but that the camera is placed on a tripod and kept stationary. PO Appeal Br. 25–26 (citing Dykes 134, 141, Fig. 4); PO Reb. Br. 11. In Section (III)(A)(1), we determined the Image Frames Limitation requires the image recording device moves along a path, course or route and that the image recording device acquires images (1) while moving or (2) both while moving and while stationary. Ishida teaches its digital city (e.g., Kyoto) interface can be built from 3D Web technology that integrates photos mapped onto 3D blocks and 2D planes to create a realistic 3DML (three-dimensional modeling language) space. Ishida 26–27, cited in Request 214. However, Ishida is silent regarding how its photos or image frames are obtained. Dykes provides a little more detail about how its images are obtained. Dykes 146, cited in Request 216–17. Dykes teaches students record images using a digital camera and obtain images at select locations along a footpath’s slope. Id.; see id. at 127 (discussing data collection devices), 134 (discussing digital cameras capturing images). Although Patent Owner’s assertion that Dykes “appears” to use a tripod and its camera is “stationary” (see PO Appeal Br. 25 (citing Dykes 134)) is not found in Dykes, Dykes is silent regarding whether the camera moves while acquiring images. See id. Thus, to the extent the Request relies on Dykes to teach the Image Frames Limitation (see Request 216–17), we determine the record does not demonstrate whether Dykes teaches these images can be obtained by an image recording device (e.g., the digital cameras) (1) while moving or (2) both while moving and while stationary. But, as explained above in Section (III)(C)(1), Yee teaches a known technique for obtaining image frames using cameras located on a van that travels along roads (Yee 388–390 (describing the van is driving on the streets and recording images in all four directions as the van travels)) in order to “see Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 31 everything” and to “ensure[] no object is lost behind an obstruction” (see id. at 390). Given Ishida’s desire both to obtain images of city realistically and to diminish modeling problems by using photos (see Ishida 26–27 (discussing “us[ing] photos mapped onto 3D blocks and 2D planes” that “significantly diminishes some of [the noted problems]”)), Yee provides a solution that collects photos (e.g., images) completely and accurately for Ishida’s digital city interface. Yee further discusses its system has accurate GPS three-dimensional positioning to ensure accuracy and completeness of the data collected and where the user is located. Yee 390. This teaching in Yee further assists Ishida’s process of determining coordinates for images associated with WEB pages that are part of Ishida’s digital city interface. See Ishida 29. As such, Yee, when combined with Ishida, teaches or suggests the recited “an image recording device moving along a trajectory” in canceled claims 1 and 15 so as to obtain a complete and accurate digital interface for captured environment. Patent Owner also argues that neither Ishida nor Dykes discloses “determining a second location based on the user selected position,” as recited in canceled claim 1 (and similarly recited in canceled claim 15) and refers to “Section VII.C.1.b).” PO Appeal Br. 41. In Section VII.C.1.b) (id. at 26–27), Patent Owner specifically argues that Dykes fails to teach another recitation of “receiving a user selection of a position on the displayed map” in canceled claim 1 because the selection is based on a hot-linked symbol within the panorama and not “on the displayed map,” as recited. Id. at 26. Patent Owner further asserts that the selecting of icons displayed on an overhead image discussed in Dykes is a two-step process that fails to teach the three-step process in claim 1 that includes the above disputed “determining” step. Id. at 26–27 (citing Dykes 142). Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 32 We are not persuaded. The rejection proposes that Dykes, in combination with Ishida, teaches the “receiving a user selection of a position on the displayed map” in canceled claim 1. For example, Figure 4 of Dykes shows a map of Haytor Down on the left (e.g., VFC panoraMap) and linked panoramas on the right (VCF panoraMap:htd-018 and VCF panoraMap:mark58). Dykes 141. Dykes teaches a user clicks on multiple symbols on this map to specify multiple locations (e.g., first and second locations) and obtain multiple views associated with locations. Id. at 140–141, Fig. 4, cited in Request 218–219. These symbols on the panoraMap identify different locations of the panoramas. Id. at 140–141. Each selection on the map by a user thus constitutes “receiving a user selection of a position on the displayed map” as canceled claims 1 and 15 recite. When the symbols on a map are clicked in Dykes, a viewer is revealed (e.g., VCF panoraMap:htd-018 and VCF panoraMap:mark58 on the middle-right and lower-right respectively) and more symbols within the viewer (e.g., down arrows) provide links to other panoramic images. See id., Fig. 4; see also 3PR Resp. Br. 11 (discussing “user clicking on a symbol on the map”). The viewer (right in Figure 4) is separate and distinct from the panoraMap in Dykes (left in Figure 4) (Dykes 141, Fig. 4), which as explained above, permits multiple selections within the displayed panorama. Dykes further discusses the “[s]ymbols on the map identify the locations of panoramas, and reveal the view and angle of view when clicked.” Dykes 139–40. Dykes discusses providing dynamic links between the map and images and that “the direction of imagery is known instantly.” Dykes 140. Based on these teachings, Dykes suggests to an ordinarily skilled artisan that its process includes determining the location or position on the panoraMap associated with the selected symbol in order to reveal the proper panorama. See id. at 139–41. Thus, Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 33 when the user selects a position on panoraMap, Dykes at least suggests a location is determined based on the position in order to retrieve the panorama (e.g., “determining a second location based on the user selected position” in canceled claim 1 and similarly recited in claim 15). Yee additionally teaches a visual interface feature that permits a user to point at specific locations within a city’s map and provides images of the selected location in an efficient manner. Yee 388, 391–92. As such, Yee, when combined with Ishida and Dykes, further teaches or suggests the concept of “receiving a user selection of a position on the displayed map,” “determining a second location based on the user selected position,” and “retrieving from the image source a second image associated with the second location” as canceled claim 1 recites. As for the contention related to a two-step process in Dykes versus the three- step process of the claims, we are not persuaded. We are not sure what three steps Patent Owner refers to, as steps (1) and (3) of the three step process are described the same. PO Appeal Br. 27 (describing both steps (1) and (3) as “retrieving from the image source a second image associated with the second location.”) Also, as explained above, we further disagree that Dykes does not teach “determining a second location based on the user selected position” step in canceled claim 1 and similarly recited in canceled claim 15. Patent Owner further contends that the Examiner has not presented a reason to combine Ishida and Dykes and that Dykes teaches away from a combination with Ishida. PO Appeal Br. 42–43. Patent Owner argues Ishida concerns a two- dimensional (2D) map with a three-dimensional (3D) model of city that a user can navigate. Id. at 42 (citing Ishida 24). In Patent Owner’s view, Dykes concerns a method of storing and displaying multimedia data in association with specific Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 34 locations on map, which is distinct and incompatible with Ishida. Id. (citing Dykes 131; Ishida 27). Patent Owner further contends “that the systems disclosed in Ishida and Dykes used distinct and incompatible methods of visualizing an area” (id.) and “it is unclear how a person of ordinary skill in the art at the time [of] the invention was made would have combined the 3D virtual environment of Ishida with the student media database at specific locations (including panoramas) of Dykes” (id. at 42–43). We are not persuaded. As noted in In re Keller, 642 F.2d 413 (CCPA 1981): The test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. Keller, 642 F.2d at 425. Thus, there is no requirement in an obviousness rejection to show how to incorporate Dykes’s image database within Ishida’s 3D environment. See 3PR Resp. Br. 18 (citing In re Mouttet, 686 F.3d 1322, 1332 (Fed. Cir. 2012)). Additionally, Ishida teaches using 2D photographs to add texture and detail to buildings in its 3D model. Ishida 27, cited in PO Appeal Br. 42. For example, Ishida discusses a 3D interface that uses photos mapped onto 3D blocks and 2D planes to build the Shijo Shopping Street 3DML (3D Modeling Language) implementation. Ishida 27–28, Fig. 3. Based on the foregoing, Ishida obtains photographs of a region that are used to create its 3D interface. See id. Dykes teaches one such known technique for acquiring images of a region along a trajectory that can be integrated with other data. Dykes 146. Similarly, as explained above, Yee teaches yet another known image acquisition technique Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 35 involving an image recording device that moves along a path or trajectory. Yee 388–90. As such, one skilled in the art would have recognized integrating Dykes’s and Yee’s images with Ishida’s interface—not necessarily using Dykes’s database or how Dykes visualizes an area as argued (see PO Appeal Br. 42–43)—is an effective method for building Ishida’s 3D blocks and that using these techniques with Ishida’s 3D interface would have yielded the predictable result of integrating captured images with Ishida’s system in an efficient manner. See RAN 18; see also 3PR Resp. Br. 18. Furthermore, as explained above, Dykes is relied upon to teach and suggest techniques for retrieving user inputs that specify a first and second location, determining the location based on the selection, and retrieving the images from an image source associated with the locations as canceled claims 1 and 15 recite. See also Request 218–220. Thus, an ordinarily skilled artisan would have recognized several additional reasons to combine Dykes’s teachings with Ishida, including to “ensur[e] that [the system’s] data are stored spatially with a minimum of effort” (Dykes 142; see also 3PR Resp. Br. 18 (citing ACP 16 (further citing Dykes 142))) and to provide: (1) “[v]isualization [that] is a particularly suitable technique to help. . . synthesise information and understand the spatial character of collected data” (Dykes 140), (2) links between the map and images that create (a) “a powerful technique and means that the direction of imagery is known” and (b) “realistic pictures [that] can be synthesised with” the map (id. at 140), and (3) “an additional reality check” for the images (id. at 143). Dykes further discusses the images are “geo-referenced” (id. at 139), “geo-referenc[ing] collected data files” (id. at 140), and “stor[ing] geo-referenced panoramas” (id. at 144). See also RAN 18 (citing Dykes 142) (discussing “geo-referencing system of Dykes . . . help[s] Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 36 build the 3D models in Ishida.”). These teachings in Dykes, along with Yee’s discussed above, further assist Ishida’s process of determining coordinates for images associated with WEB pages that are part of Ishida’s digital city interface. See Ishida 29. The Request provides yet another reason to combine the teachings of Dykes with Ishida in order to “provide[s] a system for creating a navigable digital city . . . by providing navigation images.” Request 212; see also 3PR Resp. Br. 18 (stating “a POSITA would be motivated to use the efficient photograph collection and geo- referencing system of Dykes to help build the 3D models in Ishida.”). We find each of the above-provided reasons has a rational underpinning to support an obviousness rejection. See KSR, 550 U.S. at 418. We, thus, disagree that Ishida and Dykes are incompatible. Patent Owner also fails to demonstrate sufficiently that one skilled in the art would not have recognized how to use the panoramic images taught by Dykes within Ishida. See PO Appeal Br. 43. Such panoramic images would further augment Ishida’s 3D interface by providing a greater angle of view of a block, such as the Shijo Shopping Street, to the user. See Dykes 127, 132, 134, 139–41; see Ishida 27–28. Accordingly, the record provides numerous reasons to combine Dykes and Yee with Ishida. Lastly, Patent Owner contends that “Ishida appears to discourage the use of images” because problems exists when downloading GIF (Graphics Interchange Format) or JPEG (Joint Photographic Experts Group) compressed photos. PO Appeal Br. 43 (citing Ishida 27). However, this purported problem is described as existing with “any site on the WEB using many graphics.” Ishida 27. Also, neither Ishida nor Dykes requires compressed GIF or JPEG formats. Yet, even Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 37 presuming without agreeing Patent Owner is correct that problems will exist, this problem may render the combined system somewhat inferior if GIF or JPEG images are used but does not teach away from the Ishida/Dykes combination. See In re Gurley, 27 F.3d 551, 553 (Fed. Cir. 2008) (indicating a known product “does not become patentable simply because it has been described as somewhat inferior to some other product for the same use.”). For the above reasons, we newly reject claims 4 and 21 based on Ishida, Yee, and Dykes. D. Requester’s Cross Appeal Requester appeals the Examiner’s decision not to adopt a proposed rejection of now improper claims (i.e., claims 72 and 73) based on Yee. 3PR Appeal Br. 2, 12–16. Because the claims appealed are improper at present, Requester’s cross- appeal has been rendered moot. IV. CONCLUSIONS Concerning the claims rejected by the Examiner, we determine: Claims Rejected 35 U.S.C. § Reference(s)/ Basis Affirmed Reversed New Ground 4 102(a) Dykes 4 4 102(b) Yee 4 4 102(a) Al-Kodmany 4 4 102(a) Bates 4 4 103(a) Murphy, Yee 4 4 103(a) Shiffer, Yee 4 4, 21 103(a) Ishida, Dykes 4, 21 4 103(a) Yee, Dykes, Lachinksi 4 Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 38 Claims Rejected 35 U.S.C. § Reference(s)/ Basis Affirmed Reversed New Ground 4, 21 103(a) Ishida, Yee, Dykes 4, 21 Overall Outcome 4, 21 4, 21 V. TIME PERIOD FOR RESPONSE This decision contains a new ground of rejection pursuant to 37 C.F.R. § 41.77(b). Section 41.77(b) provides “a new ground of rejection pursuant to this paragraph shall not be considered final for judicial review.” Section 41.77(b) also provides that Patent Owner, within one month from the date of the decision, must exercise one of the following two options with respect to the new grounds of rejection to avoid termination of the appeal proceeding as to the rejected claims: (1) Reopen prosecution. The owner may file a response requesting reopening of prosecution before the examiner. Such a response must be either an amendment of the claims so rejected or new evidence relating to the claims so rejected, or both. (2) Request rehearing. The owner may request that the proceeding be reheard under § 41.79 by the Board upon the same record. The request for rehearing must address any new ground of rejection and state with particularity the points believed to have been misapprehended or overlooked in entering the new ground of rejection and also state all other grounds upon which rehearing is sought. In accordance with 37 C.F.R. § 41.79(a)(1), the “[p]arties to the appeal may file a request for rehearing of the decision within one month of the date of: . . . Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 39 [t]he original decision of the Board under § 41.77(a).” A request for rehearing must be in compliance with 37 C.F.R. § 41.79(b). Comments in opposition to the request and additional requests for rehearing must be in accordance with 37 C.F.R. § 41.79(c)-(d), respectively. Under 37 C.F.R. § 41.79(e), the times for requesting rehearing under paragraph (a) of this section, for requesting further rehearing under paragraph (c) of this section, and for submitting comments under paragraph (b) of this section may not be extended. An appeal to the United States Court of Appeals for the Federal Circuit under 35 U.S.C. §§ 141-144 and 315 and 37 C.F.R. § 1.983 for an inter partes reexamination proceeding “commenced” on or after November 2, 2002 may not be taken “until all parties’ rights to request rehearing have been exhausted, at which time the decision of the Board is final and appealable by any party to the appeal to the Board.” 37 C.F.R. § 41.81. See also MPEP § 2682. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv). Requests for extensions of time in this proceeding are governed by 37 C.F.R. §§ 1.956 and 41.79(e). In the event neither party files a request for rehearing within the time provided in 37 C.F.R. § 41.79, and this decision becomes final and appealable under 37 C.F.R. § 41.81, a party seeking judicial review must timely serve notice on the Director of the United States Patent and Trademark Office. See 37 C.F.R. §§ 90.1 and 1.983. REVERSED 37 C.F.R. § 41.77 Appeal 2018-007745 Reexamination Control 95/000,684 Patent 7,813,596 B2 40 FOR PATENT OWNER: LEWIS ROCA ROTHGERBER CHRISTIE LLP PO BOX 29001 GLENDALE, CA 91209-9001 FOR THIRD-PARTY REQUESTER: O’MELVENY & MYERS LLP IP&T CALENDAR DEPARTMENT LA-1005D 400 SOUTH HOPE STREET LOS ANGELES, CA 90071-2899 Copy with citationCopy as parenthetical citation