Nelson George. Publicover et al.Download PDFPatent Trials and Appeals BoardSep 3, 201914937782 - (D) (P.T.A.B. Sep. 3, 2019) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 14/937,782 11/10/2015 Nelson George Publicover 1500-G14023T-USC2 1033 125070 7590 09/03/2019 Google LLC c/o Davidson Sheehan LLP 6836 Austin Center Blvd. Suite 320 Austin, TX 78731 EXAMINER HE, WEIMING ART UNIT PAPER NUMBER 2612 NOTIFICATION DATE DELIVERY MODE 09/03/2019 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): Google@ds-patent.com docketing@ds-patent.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ Ex parte NELSON GEORGE PUBLICOVER, LEWIS JAMES MARGGRAFF, ELIOT FRANCES DRAKE, and SPENCER JAMES CONNAUGHTON ___________ Appeal 2018-0054201 Application 14/937,782 Technology Center 2600 _________ ________ Before JAMES B. ARPIN, HUNG H. BUI, and PHILLIP A. BENNETT, Administrative Patent Judges. ARPIN, Administrative Patent Judge. DECISION ON APPEAL Appellants2 appeal under 35 U.S.C. § 134(a) the Examiner’s final rejection of claims 1, 3–6, 8–11, 13, 14, 17–20, 23–25, and 27–36, all of the pending claims. Final Act. 2. Claims 2, 7, 12, 15, 16, 21, 22, and 26 are canceled. Id. We have jurisdiction under 35 U.S.C. § 6(b). We affirm. 1 In this Decision, we refer to Appellants’ Appeal Brief (“App. Br.,” filed December 11, 2017) and Reply Brief (“Reply Br.,” filed April 26, 2018); the Final Office Action (“Final Act.,” mailed July 12, 2017); the Examiner’s Answer (“Ans.,” mailed February 26, 2018); and the originally-filed Specification (“Spec.,” filed November 10, 2015). Rather than repeat the Examiner’s findings and determinations and Appellants’ contentions in their entirety, we refer to these documents. 2 Appellants assert Google LLC is the real party-in-interest. App. Br. 1. Appeal 2018-005420 Application 14/937,782 2 STATEMENT OF THE CASE Appellants’ recited methods and systems relate[s] generally to systems and methods for Discerning the Intent of a User (DIU) and subsequently controlling and interacting with computing and other devices primarily using one’s eyes with optional ancillary input support. The system utilizes techniques within the fields of Human-Machine Interfaces (HMIs), wearable computing, human physiology, image processing, and deep learning. DIU can be implemented within unobtrusive eye-tracking headwear and/or remote eye- tracking hardware that can optionally be associated with head- mounted displays (HMD), remote displays, and/or other wearable sensors or actuators. The system may provide a user with an easy-to-use, intuitive, and flexible input mechanism to control and interact with local or remote computing devices. Spec. ¶ 3. As noted above, claims 1, 3–6, 8–11, 13, 14, 17–20, 23–25, and 27–36 are pending. Claims 1, 20, 25, and 32 are independent. App. Br. 18, 21–25 (Claims App’x). Claims 3–6, 8–11, 13, 14, 17–19, 35, and 36 depend directly or indirectly from claim 1; claims 23 and 24 depend directly from claim 20; claims 27–31 depend directly or indirectly from claim 25; and claims 33 and 34 depend directly from claim 32. Id. at 18–26. Claim 1, reproduced below with disputed limitations emphasized, is representative. 1. A method for providing a graphical user interface to determine intent of a user, the method comprising: identifying, with a detector, when at least one eye of a user is directed at a starting location; identifying, with the detector, a first saccade of the at least one eye from the starting location towards a first object at a first location on a display; Appeal 2018-005420 Application 14/937,782 3 confirming, with the detector, that the first saccade of the at least one eye is completed to within a predetermined distance from the first location of the first object; in response to the first saccade being completed to within the predetermined distance from the first location, removing the first object from the display, presenting an icon corresponding to the first object that overlaps a portion of the display previously occupied by the first object to reduce a number of pixels changed on the display due to removal of the first object and presentation of the icon corresponding to the first object, and presenting a plurality of second objects on the display at respective second locations of a corresponding plurality of second locations, the plurality of second locations different than the first location of the first object; monitoring, with the detector, the at least one eye to identify whether the at least one eye performs a second saccade from the first location towards a selected second location of the plurality of second locations; confirming, with the detector, that the second saccade of the at least one eye is completed to within a predetermined distance from the selected second location of the plurality of second locations to thereby identify a selected second object; and in response to the selected second object being identified, without waiting for perception of the second object by the user, initiating performance of an action related to one or more of the group consisting of starting location, an object at the starting location, the first object, and the selected second object. Id. at 18 (emphases added). REFERENCES AND REJECTIONS The Examiner relies upon the following references: Name3 Number Publ’d Filed Ambrus US 2015/0212576 A1 Jul. 30, 2015 Jan. 28, 2014 3 All reference citations are to the first named inventor only. Appeal 2018-005420 Application 14/937,782 4 Name Number Publ’d Filed Venable US 2014/0380230 A1 Dec. 25, 2014 Jun. 25, 2013 Leinonen US 2014/0362346 A1 Dec. 11, 2014 Mar. 9, 2012 Henderek US 2014/0247210 A1 Sep. 4, 2014 Mar. 3, 2014 Poulos US 2014/0237366 A1 Aug. 21, 2014 Feb. 19, 2013 Aiden US 2014/0139652 A1 May 22, 2014 Nov. 21, 2012 Bouvin US 2009/0125849 A1 May 14, 2009 Aug. 27, 2008 The Examiner rejects the pending claims on the following grounds. (1) Claims 1, 3–6, 8, 13, 14, 19, 20, 23, 24, and 35 stand rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable and Poulos. Final Act. 5–13. (2) Claims 25, 27, 28, and 30–34 stand rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable, Poulos, and Bouvin. Id. at 13–18. (3) Claim 9 stands rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable, Poulos, and Leinonen. Id. at 18. (4) Claims 10 and 11 stand rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable, Poulos, and Aiden. Id. at 19. (5) Claims 17, 18, and 36 stand rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable, Poulos, and Henderek. Id. at 19–21. (6) Claim 29 stands rejected under 35 U.S.C. § 103(a) as rendered obvious over the combined teachings of Venable, Poulos, Bouvin, and Aiden. Id. at 21. Appeal 2018-005420 Application 14/937,782 5 We review the appealed rejections for error based upon the issues identified by Appellants, and in light of the arguments and evidence produced thereon. Ex parte Frye, 94 USPQ2d 1072, 1075 (BPAI 2010) (precedential). Arguments not made are waived.4 See 37 C.F.R. § 41.37(c)(1)(iv). Unless otherwise indicated, we adopt the Examiner’s findings in the Final Action and the Answer as our own and add any additional findings of fact appearing below for emphasis. We address the rejections below. ANALYSIS A claim is unpatentable under 35 U.S.C. § 103(a) if the differences between the claimed subject matter and the prior art are such that the subject matter, as a whole, would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 406 (2007). 1. Independent Claim 1 The Examiner determines that claim 1 is rendered obvious over the combined teachings of Venable and Poulos. Final Act. 2–8; see Ans. 2–7. Appellants contend that (a) Venable does not teach or suggest the step of, “without waiting for perception of the second object by the user, initiating performance of an 4 Appellants do not argue independent claim 20, 25, or 32 separately from independent claim 1, nor do they argue the dependent claims separately from their independent base claims. App. Br. 16; Reply Br. 8. Thus, the pending claims stand or fall with the rejection to claim 1. Appeal 2018-005420 Application 14/937,782 6 action,” as recited in claim 1 (App. Br. 18 (Claims App’x) (emphasis added)); (b) Venable in combination with Poulos does not teach or suggest the step of “removing the first object from the display, presenting an icon corresponding to the first object that overlaps a portion of the display previously occupied by the first object to reduce a number of pixels changed on the display due to removal of the first object and presentation of the icon corresponding to the first object,” as recited in claim 1 (id. (emphasis added)); and (c) the Examiner fails to show that a person of ordinary skill in the art would have had reason to combine the teachings of Venable with those of Poulos to achieve the methods recited in claim 1. App. Br. 11–16; see Reply Br. 2–8. For the reasons given below, we are persuaded that the Examiner has shown that the combined teachings of Venable and Poulos render claim 1 obvious. a. Venable teaches Initiating “Without Waiting for Perception” As noted above, claim 1 recites “in response to the selected second object being identified, without waiting for perception of the second object by the user, initiating performance of an action.” App. Br. 18 (Claims App’x) (emphasis added). The Specification explains: Many commercial eye-tracking systems require a user to focus on a virtual object (i.e., dwell) for some period of time or to blink as a part of a selection process to indicate intent. Dwell requires perception (requiring about 0.25 seconds[, i.e., about 250 milliseconds (ms)]) and focal maintenance. Often, the indication that a dwell has taken place involves magnifying the region being viewed. This induces further mental image processing during perception by the visual cortex. Blinks take even longer periods Appeal 2018-005420 Application 14/937,782 7 of time, requiring a complex series of muscle contractions. The minimum time for a blink is about 0.3 to 0.4 seconds. Spec. ¶ 228 (emphasis added). Further, the Specification explains: In some situations within some embodiments, it may be desirable to leave an image at the location of a target for a period of time. This time can be such that: 1) there is an awareness of the target, but it is not fully perceived (i.e.[,] generally less than approximately 200 milliseconds[, i.e., 0.2 seconds]) including “flash-like” removal after just a few milliseconds to tens of milliseconds, 2) there is a perception of a target image, but it does not persist or “hang around” any longer than needed for perception, 3) there is clear perception of the target until the eye begins to move away from the location, or 4) the target persists until an action or some other operation is performed. Id. ¶ 288 (emphasis added). Venable teaches: In some embodiments, the predetermined motion condition may include short gaze dwells on the one or more interactive targets. Therefore, the specific sequence of motion interactions may include one or more gaze dwells between gaze movements. Such dwells may be imperceptible to the user (i.e., the user does not feel that he is fixating on a particular object for any unnatural duration), but may be long enough to indicate that the user has sufficiently interacted with the interactive target. As one non-limiting example, the dwells may be between 100-150 ms in duration. However, any suitable dwell length may be used for the predetermined motion condition. Venable ¶ 31 (emphases added). The Examiner determines that Venable teaches (1) the gaze dwells may be “imperceptible” to the user and (2) dwell times of 100 ms to 150 ms, which range is considerably less than the 200 ms Appeal 2018-005420 Application 14/937,782 8 for perception disclosed in Appellants’ Specification. Ans. 3–4; see Final Act. 7. Thus, the Examiner finds that Venable teaches this limitation. Appellants respond that, although the portion of the Specification cited by the Examiner provides an estimate of dwell time required for user perception of objects (e.g., eye focus at a particular object for approximately greater than 0.25 seconds[, i.e., 250 ms]), it should not be assumed that the provided 0.25 seconds represents an exact boundary and that all dwells for periods of time less than 0.25 seconds are not perceived. . . . That is, objects subject to dwell times in the range of 0.20 – 0.25 seconds[, i.e., 200 ms–250 ms,] are still perceivable by the eyes. Reply Br. 4–5. This response, however, fails to address the shorter gaze dwell times, i.e., in the 100 ms to 150 ms range, taught by Venable. These are clearly less than the minimum dwell time, i.e., “generally less than approximately 200 milliseconds,” described in the Specification. Spec. ¶ 288. Consequently, we are persuaded that the Examiner has shown sufficiently that Venable teaches this limitation. b. Venable and Poulos Teach “Presenting an Icon . . . To Reduce a Number of Pixels Changed” Claim 1 also recites “removing the first object from the display, presenting an icon corresponding to the first object that overlaps a portion of the display previously occupied by the first object to reduce a number of pixels changed on the display due to removal of the first object.” App. Br. 18 (Claims App’x). The Specification’s Figures 15B and 15C are reproduced below. Appeal 2018-005420 Application 14/937,782 9 Figures 15B and 15C depict examples of eye-signal, multi-level menu selection during the viewing of a map. Spec. ¶ 66; see id. ¶¶ 423–425. Referring to Figures 15A–C, the Specification explains: Elements within submenus can, in turn, be selected. Repeating this process allows tool selection to be nested as deeply as desired by application developers. The selection of a specific menu item causes unused selections to disappear and the interactable representing the selected function to be replaced by a “back" interactable 1538. . . . When designing layout with a three by four gaze- selectable region, at all times a “back” interactable 1538 can be displayed on the top row 1539a of the central region of the screen (at the column position where a selection was previously made) . . . . Appeal 2018-005420 Application 14/937,782 10 Id. ¶¶ 424, 425 (emphasis added); see App. Br. 2, 6–7 (citing Figs. 15A–15C and the supporting disclosure as written description for the recited limitation). The Specification further explains: Along similar lines, by carefully choosing the location of objects, temporal differences during transitions between objects and/or the distinction between objects and a display background can be exploited to either enhance or suppress attention by the device user. If a newly introduced object is substantially different in size, shape, and/or central location compared with an object being replaced, then both the disappearance of the previous object and the introduction of the new object serve to attract attention. On the other hand, if the newly presented object is positioned such that it is superimposed on the previous object (i.e., minimizing the number of pixels that transition) the potential for distraction can be minimized. Spec. ¶ 487 (emphasis added). Thus, the Specification teaches that overlaying a new icon on previous icon’s location or an object on another object’s location “reduce[s] a number of pixels changed.” From the claim language, we understand that overlaying an icon on an objects location would achieve the same reduction.5 5 Although the Examiner asserts that the cited limitation “reducing the number of pixel changes” is NOT supported by the original filed specification. There is no any written description in the specification to disclose how to reduce the number of pixel changes, or how to identify the reduced number of pixel changes. This limitation is merely appellant’s imagination from the limitation “due to removal of the first object and presentation of the icon corresponding to the first object”. However, this supposition may not be true as appellant’s guess. Ans. 4–5. Nevertheless, the Examiner does not reject the claims as lacking adequate written description, and the adequacy of the written description is not before us in this appeal. Appeal 2018-005420 Application 14/937,782 11 The Examiner determines that Venable teaches selecting an item from a user interface menu. Final Act. 8 (citing Venable ¶ 37). The Examiner further determines Poulos teaches [a] user gazing at first object 410 to select the first object in [Poulos ¶ 30]; “representations of commands associated with the selected object may be overlaid on one or more regions in the augmented reality environment” in [id. ¶ 38]; “Available actions for a selected object also may be displayed on selected objects directly, and may use standardized iconography” in [id. ¶ 39]. Final Act. 8. Poulos further explains: FIG. 4 further shows, at 408, user 106 after ceasing gazing at first object 410 so that first object 410 is no longer a selected object. After user 106 deselects first object 410, first object 410 may remain in its second state, or may change to a different state, as user 106 focuses on and selects a second object 414, as indicated via gaze lines 430. After user 106 selects second object 414, then representations of groups of commands associated with the second object are displayed to user 106. These representations may be displayed adjacent to second object 414 as shown at 434, overlaid on other objects in the environment, e.g., on a wall adjacent to second object 414 as shown at 436, or displayed in any other suitable manner. Poulos ¶ 43 (emphases added). Thus, Poulos teaches that menu options, e.g., icons, may be overlaid on an object and may remain there after the object is deselected. Appellants contend that, because (1) Poulos describes overlaying commands on a “selected object” and because (2) Venable teaches moving over a sequential order of targets without making individual selections, Poulos and Venable cannot be combined without rendering Venable unsuitable for its intended purpose. App. Br. 14–15. According to Appellants, “[i]f Venable is modified by adding Poulos’[s] overlaying of available actions, the resulting combination would be the user gazing at a Appeal 2018-005420 Application 14/937,782 12 sequence of targets to show intent to select an object and displaying available actions for the selected object afterwards.” Id. at 15 (emphasis added). The Examiner responds that Venable teaches selection of user interface elements via eye tracking and launches a program represented by the selected element, for example, selecting an item from a user interface menu. Ans. 7 (citing Venable ¶ 37). The Examiner further determines that a person of ordinary skill would have understood that menu options, e.g., icons, could be overlaid on an object before it is selected, i.e., a “selectable object.” See Final Act. 4 (citing Ambrus ¶ 85 (“In step 646, a menu of options associated with the virtual object is displayed in proximity to the virtual object in response to detecting the selection of the virtual object.” (emphasis added)), Claim 17 (“the displaying one or more options includes displaying a radial menu including the one or more options overlaying the selectable object” (emphasis added)); see also Ambrus ¶ 64 (describing a “selectable object”), Fig. 5 (steps 502 and 504 describing “selectable objects”).6 Thus, the Examiner concludes, and we agree, that “Poulos teaches the presentations of commands associated with the selected object can be located [at] any regions on the display, even directly overlay[ing] on the selected object” (Final Act. 8), but a person of ordinary skill in the art would have understood that the combined teachings of Venable and Poulos teach overlaying icons on selectable objects. See id.; Ans. 6–7. Consequently, we are persuaded that the combination of Venable and Poulos 6 “[T]he absence of specific findings on the level of skill in the art does not give rise to reversible error ‘where the prior art itself reflects an appropriate level and a need for testimony is not shown.’” Okajima v. Bourdeau, 261 F.3d 1350, 1355 (Fed. Cir. 2001) (quoting Litton Indus. Prods., Inc. v. Solid State Sys. Corp., 755 F.2d 158, 163 (Fed. Cir. 1985)). Appeal 2018-005420 Application 14/937,782 13 teaches or suggests this limitation and that the proposed combination of Poulos and Venable would not render Venable unsuitable for its intended purpose. See App. Br. 15. c. Reason to Combine Venable and Poulos The Examiner reasons that: It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the invention of Venable with the teaching of Poulos so as to display a menu item for further sub-menu selection in response one click of object in the control interface. Final Act. 8. Although Venable does not teach explicitly the presentation of the menu overlaying on the selected first object, as discussed above, the Examiner determines that, in the same filed of endeavor, namely object selection by eye tracking, Poulos teaches displaying an icon on a selected or previously selected object. Ans. 7 (citing, e.g., Poulos ¶ 39). In view of the discussion above, a person of ordinary skill in the art would have understood this teaching to extend to “selectable” objects. Final Act. 4. Based on these teachings, the Examiner concludes that: The motivation of combining Venable with the teaching of Poulos may be an obvious to try – choosing from a finite number of predictable solutions, or a known work in one field of endeavor may prompt variations of it for use in the same field based on design incentive or other market forces if the variations would have been predictable to one of ordinary skill in the art. Id. (emphasis added); see KSR, 550 U.S. at 421 (“When there is a design need or market pressure to solve a problem and there are a finite number of Appeal 2018-005420 Application 14/937,782 14 identified, predictable solutions, a person of ordinary skill has good reason to pursue the known options within his or her technical grasp.”). Appellants contend that the Examiner’s reasoning is insufficient because it lacks a specific underlying rationale and because [w]ithout articulating how and why the Venable and Poulos references could be combined in a manner that does not render the cited references unsuitable for their intended purposes, the Examiner appears to be impermissibly using the claims of the present application as a blueprint and extracting discrete teachings from the references to map to the claim features without considering the context of the references’ teachings. Reply Br. 2–3. For the reasons given below, we disagree. Referring to Figures 4A–4C and 5A–5C, Venable teaches following multiple sequential targets and following an interactive target to multiple locations, via “sequential eye gestures,” respectively. See Venable ¶¶ 28, 29, 35. Regarding Figures 1A–1C, Venable further teaches, “the sequential eye gesture is performed between the user interface element and a single depicted interactive target.” Id. ¶ 28; see id., Abstract (“Embodiments are disclosed herein that relate to selecting user interface elements via a periodically updated position signal.”). As Venable explains: The term “sequential eye gesture” as used herein signifies that the location of gaze moves from a starting location to an ending location on the user interface in one or more sequential segments of movement. Such gestures may allow natural eye movements, such as saccades, to be used to perform the gesture. Id. ¶ 26 (emphasis added). Considering Venable’s teachings as a whole, we find Venable broadly teaches selecting user interface elements via position signals derived from sequential eye gestures that may include saccades. Appellants contend, “Poulos is relied upon for teaching overlaying commands over a selected object.” App. Br. 13–14 (citing Poulos ¶¶ 37–39) Appeal 2018-005420 Application 14/937,782 15 (emphasis added). In particular, Appellants assert, “Poulos only speaks to how to display commands after objects have been selected (emphasis added).” Id. at 14. As noted above, this is not accurate. Poulos also teaches the continued display of menus associated with previously selected objects. Poulos ¶ 43. These teachings extend logically to menus, e.g., icons, displayed with selectable objects. Final Act. 4 (quoting Ambrus ¶ 85); but see also Ambrus, Claim 17 (describing “displaying a radial menu including the one or more options overlaying the selectable object” (emphasis added)). For the reasons discussed above, we are persuaded that the Examiner has articulated sufficiently how and why Poulos’s menu display commands, understood in light of Ambrus’s teachings, would have been combined with Venable’s related teachings regarding sequential targets to arrive at the recited methods and systems. KSR, 550 U.S. at 418 (quoting In re Kahn, 441 F.3d 977, 988 (Fed. Cir. 2006)) (“[T]he analysis [determining a reason to combine] need not seek out precise teachings directed to the specific subject matter of the challenged claim, for a court can take account of the inferences and creative steps that a person of ordinary skill in the art would employ.”). Accordingly, on this record, we sustain the Examiner’s obviousness rejection of independent claim 1. 2. Independent Claims 20, 25, and 32 and the Dependent Claims Appellants do not argue independent claims 20, 25, and 32 or the pending dependent claims the dependent claims separately from claim 1 (App. Br. 16), and we sustain the rejections of those claims for substantially the same reasons given for independent claim 1. Final Act. 8–21; see supra, note 4. Appeal 2018-005420 Application 14/937,782 16 CONCLUSIONS (1) The Examiner did not err in rejecting: (a) claims 1, 3–6, 8, 13, 14, 19, 20, 23, 24, and 35 as rendered obvious over the combined teachings of Venable and Poulos; (b) claims 25, 27, 28, and 30–34 as rendered obvious over the combined teachings of Venable, Poulos, and Bouvin; (c) claim 9 as rendered obvious over the combined teachings of Venable, Poulos, and Leinonen; (d) claims 10 and 11 as rendered obvious over the combined teachings of Venable, Poulos, and Aiden; (e) claims 17, 18, and 36 as rendered obvious over the combined teachings of Venable, Poulos, and Henderek; and (f) claim 29 as rendered obvious over the combined teachings of Venable, Poulos, Bouvin, and Aiden. (2) Thus, claims 1, 3–6, 8–11, 13, 14, 17–20, 23–25, and 27–36 are not patentable. DECISION We affirm the Examiner’s rejections of claims 1, 3–6, 8–11, 13, 14, 17–20, 23–25, and 27–36. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv). AFFIRMED Copy with citationCopy as parenthetical citation