Ex Parte Mihelich et alDownload PDFPatent Trial and Appeal BoardAug 28, 201814588515 (P.T.A.B. Aug. 28, 2018) Copy Citation UNITED STA TES p A TENT AND TRADEMARK OFFICE APPLICATION NO. FILING DATE FIRST NAMED INVENTOR 14/588,515 01/02/2015 Patrick Mihelich 129433 7590 08/30/2018 Google Technology Holdings LLC c/o Davidson Sheehan LLP 6836 Austin Center Blvd. Suite 320 Austin, TX 78731 UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www .uspto.gov ATTORNEY DOCKET NO. CONFIRMATION NO. Gl4001 8264 EXAMINER LIU, GORDON G ART UNIT PAPER NUMBER 2612 NOTIFICATION DATE DELIVERY MODE 08/30/2018 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): docketing@ds-patent.com Google@ds-patent.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte PA TRICK MIHELICH and JOHNNY LEE Appeal2017-010038 Application 14/588,515 Technology Center 2600 Before CAROLYN D. THOMAS, BARBARA A. BENOIT, and JOHN F. HORVATH, Administrative Patent Judges. HORVATH, Administrative Patent Judge. DECISION ON APPEAL STATEMENT OF THE CASE Appellants appeal and seek review, under 35 U.S.C. § 134(a), of the Examiner's rejection of claims 1, 5, 7, and 9-30. We have jurisdiction under 35 U.S.C. § 6(b ). We AFFIRM. Appeal2017-010038 Application 14/588,515 SUMMARY OF THE INVENTION The invention is directed to using three-dimensional mapping to facilitate location-based augmented reality overlays in electronic devices. Spec. ,r 2. Claim 1, reproduced below, is illustrative of the claimed subject matter: 1. A method comprising: determining a position and orientation of a first electronic device relative to a second electronic device based on a three- dimensional mapping of an environment of the first and second electronic devices; concurrently displaying a first augmented reality overlay at the first electronic device and displaying a second augmented reality overlay at the second electronic device, the first augmented reality overlay displaying a first perspective of a virtual object in the environment based on a position and orientation of the first electronic device, the virtual object not representative of an actual object in the environment, and the second augmented reality overlay displaying a second perspective of the virtual object based on a position and orientation of the second electronic device; receiving, at the first electronic device, user input representing a manipulation of the virtual object, the manipulation including a movement of the virtual object relative to the environment from the first perspective; and modifying a depiction of the virtual object in the second augmented reality overlay at the second electronic device so as to reflect the movement of the virtual object relative to the environment from the second perspective in response to the manipulation of the virtual object at the first electronic device. 2 Appeal2017-010038 Application 14/588,515 Bozarth et al. Blanchflower et al. Lee Abe et al. REFERENCES US 9,170,318 Bl US 2013/0307874 Al US 2014/0240469 Al US 2015/0356788 Al REJECTIONS Oct. 27, 2015 Nov. 21, 2013 Aug. 28, 2014 Dec. 10, 2015 Claims 1, 5, 7, 11-17, 19, 20, 22-27, 29, and 30 stand rejected under 35 U.S.C. § I03(a) as unpatentable over Bozarth, Blanchflower, and Abe. Final Act. 5. Claim 9, 10, 18, 21, and 28 stand rejected under 35 U.S.C. § I03(a) as unpatentable over Bozarth, Blanchflower, Abe, and Lee. Final Act. 31. ISSUES AND ANALYSIS We have reviewed the Examiner's rejection in light of Appellants' arguments on appeal that the Examiner has erred. We note Appellants' arguments are directed solely to the rejection of claim 1, and do not separately address the rejections of independent claims 11, 19, and 25, or of any dependent claim. See App. Br. 4--12. Therefore, we consider only the Examiner's rejection of claim 1, and treat the rejections of claims 5, 7, and 9-30 as standing or falling together with the rejection of claim 1. See 37 C.F.R. § 4I.37(c)(iv). We disagree with Appellants' contentions that the Examiner erred in rejecting claim 1, and adopt as our own the findings and reasons set forth by the Examiner in the Examiner's Final Action and Answer regarding the rejection of claim 1. We highlight the following for emphasis. 3 Appeal2017-010038 Application 14/588,515 Claims 1, 5, 7, and 9-30 Appellants do not argue the proposed combination of Bozarth, Blanchflower, and Abe fails to teach or suggest any limitation recited in claim 1. Rather, Appellants argue the combination is disallowed because it "would render each of Bozarth and Blanchflower unsatisfactory for their respective intended purposes," and would "fundamentally change the respective operations of Bozarth and Blanchflower." App. Br. 10. Specifically, Appellants argue the combination would render Bozarth inoperable because Bozarth requires a first device to detect a real image projected by a second device, and replacing the real image with Abe's virtual image would prohibit both image projection and image detection. Id. at 9. Similarly, Appellants argue the combination would render Blanchflower inoperable because Blanchflower requires first and second devices to detect a real object in order to trigger the display of a shared virtual reality, and replacing the real object with Abe's virtual object would prohibit Blanchflower' s real object detection. Id. at 9-10. "If references taken in combination would produce a 'seemingly inoperative device,' ... such references teach away from the combination and thus cannot serve as predicates for a prima facie case of obviousness." McGinley v. Franklin Sports, Inc., 262 F.3d 1339, 1354 (Fed. Cir. 2001) (citation omitted); see also In re ICON Health & Fitness, Inc., 496 F.3d 1374, 1382 (Fed. Cir. 2007) ("a reference teaches away from a combination when using it in that combination would produce an inoperative result," but the obviousness analysis must account for "modifications that one skilled in the art would make to a device borrowed from the prior art"). 4 Appeal2017-010038 Application 14/588,515 We disagree with Appellants' arguments, which do not address the combination of references proposed by the Examiner. We explain the Examiner's combination below, and why Appellants arguments are inapposite in view of the combination. Bozarth teaches a first device ( e.g., device 202) that detects an image (e.g., barcode 208) projected by a second device (e.g., device 204) in a real world environment in order to determine the relative positioning and orientation of the first and second devices. See Bozarth, 3:28-35, 6:6-20, Figs. 2, 7(a), 7(b ). The Examiner relies on this disclosure to teach the first limitation of claim 1, namely, "determining a position and orientation of a first electronic device relative to a second electronic device based on a three- dimensional mapping of an environment of the first and second electronic devices." See Final Action 5---6. We agree with this finding, which Appellants do not contest. See App. Br. 4--10. Significantly, the Examiner does not rely on Bozarth to teach the second limitation of claim 1, namely, concurrently displaying augmented reality overlays on the first and second devices including first and second perspectives of a virtual object based on the positions and orientations of the first and second devices. In fact, the Examiner acknowledges that Bozarth "fails to explicitly disclose ... concurrently displaying a first augmented reality overlay at the first electronic device and displaying a second augmented reality at the second electronic device." Final Act. 6. Rather, the Examiner relies on Blanchflower to teach this limitation. Id. at 7. Blanchflower discloses first and second devices ( e.g., devices 360/370) that detect real world objects (e.g., objects 350/355 in bounding area 345) in order to trigger the concurrent display of a virtual reality 5 Appeal2017-010038 Application 14/588,515 overlay ( e.g., augmented reality overlay 410 in bounding area 415) on the first and second devices based on the perspective of each device. See Blanchflower ,r,r 28, 31, Figs. 3C, 4. The Examiner relies on these disclosures to teach the second limitation of claim 1. See Final Act. 7. We agree with this finding by the Examiner, which Appellants do not contest. See App Br. 4--10. The Examiner relies on Abe to teach the third and fourth limitations of claim 1, namely, receiving user input from the first device representing movement of the virtual object relative to the environment from the first perspective, and modifying the depiction of the virtual object in the virtual reality overlay of the second device to reflect this movement from the second perspective of the second device. See Final Act 8-9. In particular, the Examiner finds Abe teaches the virtual object can be an avatar, the user of the first device can move the avatar in the virtual reality overlay displayed on the first device, and this movement will be depicted in the virtual reality overlay displayed on the second device. Id. ( citing Abe ,r,r 157, 160, 170, 198, Figs. 1, 2, 7, 8, 13, and 20). We agree with these findings of the Examiner, which Appellants do not contest. See App. Br. 4--10. To summarize the Examiner's findings, with which we agree, Bozarth teaches determining the position and orientation of a first device relative to a second device based on the real world environment of the first and second devices (i.e., based on the first device detecting a bar code projected onto the environment by the second device). See Bozarth, 3:28-35, 6:6-20, Figs. 1 ( c ), 2. Blanchflower teaches concurrently displaying virtual reality overlays ( e.g., augmented reality overlay 410 in bounding area 415) on first and second devices based on the relative positions and orientations of the 6 Appeal2017-010038 Application 14/588,515 first and second devices upon detecting a real world object in the environment ( e.g., objects 350/355). See Blanchflower ,r,r 28, 31, Figs. 3( c ), 4. Blanchflower's real world objects 350/355 can be, e.g., the bar code projected by Bozarth's second device on a wall. Abe teaches displaying a virtual reality space on first and second devices based on a real world space, where the virtual reality space contains a virtual object (e.g., an avatar) that can be moved by a user of the first device. See Abe ,r,r 157, 170, 198, Figs. 1, 2, 7, 20. Abe's virtual reality space can be, e.g., Blanchflower's augmented reality overlay 410 displayed on the first and second devices based upon the position and orientation of those devices. See Blanchflower ,r 31. Thus, movement of the avatar in the virtual reality overlay displayed on the first device will modify the virtual reality overlay displayed on the second device to reflect the avatar's movement from the perspective of the second device. Our understanding of the Examiner's findings and proposed combination of the teachings of Bozarth, Blanchflower, and Abe is confirmed by the Examiner's Answer, where the Examiner explains: Bozarth, modified by Blanchflower, teaches that two or more users may share identified augmented reality information through identifiers that are real objects, such as bar code image projection captured by the second user. Abe teaches that two or more users may share augmented reality ... and [when] one user moves one virtual object (avatar), the other user may see the change from a second perspective. Answer 4. The Examiner further explains that-like Bozarth and Blanchflower-Abe uses a real world marker MKl to determine where to place virtual object DMl to overlay real world marker MKl. Id ( citing Abe 7 Appeal2017-010038 Application 14/588,515 ,r 344, Fig. 54). Therefore, the Examiner explains, substituting Abe's real world maker for Bozarth's bar code marker would be a "[s]imple substitution of one known element for another to achieve a predictable result." Id. Thus, contrary to Appellants' inoperability argument, the Examiner is not proposing replacing the Bozarth/Blanchflower real image/object with Abe's virtual image. Instead, the Examiner is proposing replacing one real world marker (e.g, Abe's MKl) for another real world marker (e.g., Bozarth's bar code). Appellants fail to demonstrate that such a substitution would be discouraged by one of ordinary skill in the art and therefore teach away. See In re Kahn, 441 F.3d 977, 990 (Fed. Cir. 2006). Instead, Appellants' respond to the explanation provided in the Examiner's Answer by arguing that a combination of Bozarth, Blanchflower, and Abe that substitutes Abe's real world marker for Bozarth' s barcode would fail to teach the third and fourth limitations of claim 1, namely, receiving user input from the first device representing movement of the virtual object relative to the environment from the perspective of the first device, and modifying the depiction of the virtual object in the virtual reality overlay of the second device to reflect this movement from the perspective of the second device. Reply Br. 3. We are not persuaded by this argument as well, which again, does not address the Examiner's proposed combination. The Examiner is not suggesting that Abe's real world maker is the virtual object that is moved and subsequently displayed in the virtual reality overlays of the first and second devices. For that limitation, the Examiner relies on Abe's disclosure of a moveable avatar (see, e.g., Abe Figs. 2, 20), which can be displayed within virtual reality overlays on first and second devices based on the 8 Appeal2017-010038 Application 14/588,515 perspectives of those devices. See Final Act. 7-9; Ans. 4. The suggestion in the Examiner's Answer is that the real world marker that is used to both orient the first and second devices (as taught by Bozarth) and trigger the concurrent display of virtual reality overlays on the first and second devices (as taught by Blanchflower) can be Abe's real world marker MKl instead of Bozarth's bar code 208 or Blanchflower's trigger objects 350/355. Id. For the reasons discussed above, we see no error in the Examiner's findings, which Appellants' do not contest other than to argue that the combination proposed by the Examiner ( 1) renders Bozarth and Blanchflower inoperable, and (2) fails to disclose the third and fourth limitation of claim 1 when Abe's real world marker MKl is substituted for Bozarth's projected bar code 208. See App. Br. 4--10; Reply 3. However, as explained above, Appellants arguments are not persuasive of Examiner error because they do not address the Examiner's proposed combination. Accordingly, we sustain the Examiner's rejection of claim 1. Moreover, because Appellants do not separately argue for the patentability of claims 5, 7, and 9-30, we sustain the Examiner's rejections of these claims for the same reasons. DECISION The rejection of claims 1, 5, 7, 11-17, 19, 20, 22-27, 29, and 30 under 35 U.S.C. § 103(a) as unpatentable over Bozarth, Blanchflower, and Abe is sustained. The rejection of claims 9, 10, 18, 21, and 28 under 35 U.S.C. § 103(a) as unpatentable over Bozarth, Blanchflower, Abe, and Lee is sustained. 9 Appeal2017-010038 Application 14/588,515 No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv). AFFIRMED 10 Copy with citationCopy as parenthetical citation