Palo Alto Research Center Incorporated

Patent Trials and Appeals Board

Mar 17, 2022

2021000919 (P.T.A.B. Mar. 17, 2022)

UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 16/237,285 12/31/2018 Hsuan-Ting Wu PARC-20180472US01 2022 35699 7590 03/17/2022 PARC c/o Yao Legal Services, Inc. 2800 Fifth Street Suite 110 DAVIS, CA 95618-7759 EXAMINER WANG, JIN CHENG ART UNIT PAPER NUMBER 2613 NOTIFICATION DATE DELIVERY MODE 03/17/2022 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): incoming@yaolegal.com jenny@yaolegal.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE ____________ BEFORE THE PATENT TRIAL AND APPEAL BOARD ____________ Ex parte HSUAN-TING WU and ROBERT R. PRICE ___________ Appeal 2021-000919 Application 16/237,285 Technology Center 2600 ____________ Before CARL W. WHITEHEAD JR., ADAM J. PYONIN and PHILLIP A. BENNETT, Administrative Patent Judges. WHITEHEAD JR., Administrative Patent Judge. DECISION ON APPEAL1 Appellant2 is appealing the final rejection of claims 1-20 under 35 U.S.C. § 134(a). See Appeal Brief 1. Claims 1 and 11 are independent. We have jurisdiction under 35 U.S.C. § 6(b). We AFFIRM IN PART. 1 Rather than reiterate Appellant’s arguments and the Examiner’s determinations, we refer to the Appeal Brief (filed June 24, 2020), the Reply Brief (filed November 18, 2020), the Final Action (mailed January 24, 2020) and the Answer (mailed September 18, 2020), for the respective details. 2 Appellant refers to “applicant” as defined in 37 C.F.R. 1.42(a). (“The word ‘applicant’ when used in this title refers to the inventor or all of the joint inventors, or to the person applying for a patent as provided in §§ 1.43, 1.45, or 1.46.”). Appellant identifies Palo Alto Research Center Incorporated as the real party in interest. Appeal Brief 1. Appeal 2021-000919 Application 16/237,285 2 Introduction According to Appellant, “This disclosure is generally related to the field of computer vision systems. More specifically, this disclosure is related to a system and method for facilitating alignment- and orientation-based task assistance to a user of augmented reality (AR) technology.” Specification ¶ 2. Representative Claim3 (disputed limitations emphasized) 1. A method for facilitating dynamic assistance to a user in an augmented reality (AR) environment of an AR device, comprising: detecting, by the AR device, a first element of an object using an object detector, wherein the object is associated with a task and the first element is associated with a step of the task; determining an orientation and an alignment of the first element in physical world of the user; determining an overlay for the first element, wherein the overlay distinctly highlights one or more regions of the first element, and wherein the overlay indicates how the first element fits in the object based on the one or more regions; and highlighting the one or more regions in the AR environment by placing the overlay on the one or more regions of the first element at the determined orientation in the AR environment. 3 Appellant does not argue independent claims 1 or 11 individually. See Appeal Brief 10 (“The rejection of independent claims 1 and 11 is improper because there is at least one fundamental distinction between the cited reference, Peacock, and the claimed features.”). Accordingly, we select independent claim 1 as representative. See 37 C.F.R. § 41.37(c)(1)(iv) (“When multiple claims subject to the same ground of rejection are argued as a group or subgroup by appellant, the Board may select a single claim from the group or subgroup and may decide the appeal as to the ground of rejection with respect to the group or subgroup on the basis of the selected claim alone.”). Appeal 2021-000919 Application 16/237,285 3 Reference (Reference citation is to the first named inventor only.) Name Reference Date Peacock US 10,366,521 B1 July 30, 2019 Rejections on Appeal Claims 8 and 18 stand rejected under 35 U.S.C. § 112(b) or 35 U.S.C. § 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Final Action 8. Claims 1-20 stand rejected under 35 U.S.C. § 102(a)(2) as being anticipated by Peacock. Final Action 9-30. ANALYSIS 35 U.S.C. § 112 Rejection The Examiner determines: The claim 8 recites “in response to the AR device determining that the step is complete . . . wherein the third element is associated with a subsequent step of the step” at lines 4-6 of the claim 8. The recitation of “the step” ambiguously refers to the recitation of “the step” at line 4 of the claim 8 or “step” within the recitation of “a subsequent step” at line 6 of the claim 8. Clarification is required. The claim 18 is subject to the same rationale of rejection as the claim 8. Final Action 8. Appellant argues: [C]laims 8 and 18 clearly describes two steps: (i) the step, and (ii) a subsequent step of the step. Appellant respectfully points out that, contrary to Examiner's allegation, the mere recitation of the phrase “step” does not render the claims indefinite because the claims Appeal 2021-000919 Application 16/237,285 4 clearly distinguish the two distinct steps with the corresponding qualifiers. Appeal Brief 9. Appellant’s argument is persuasive of Examiner error. The recited step and subsequent step are distinguishable as two distinct steps and therefore not indefinite. See MPEP § 2173.05(e)(“[T]he failure to provide explicit antecedent basis for terms does not always render a claim indefinite. If the scope of a claim would be reasonably ascertainable by those skilled in the art, then the claim is not indefinite.”). 35 U.S.C. § 102 Rejection Appellant contends: Peacock, either expressly or inherently, does not suggest “determining an overlay for the first element, wherein the overlay distinctly highlights one or more regions of the first element, and wherein the overlay indicates how the first element fits in the object based on the one or more regions” and “highlighting the one or more regions in the AR environment by placing the overlay on the one or more regions of the first element at the determined orientation in the AR environment.” Appeal Brief 10. Appellant argues that the “Examiner has not addressed a fundamental difference between the Peacock system and the system of the instant application” whereas “the Peacock system provides textual instructions and identifies an element relevant to the instruction by placing an indicator around the element” conversely “the instant application teaches a highlight directly placed on relevant regions of an element in an AR environment and how the element may fit in an object.” Appeal Brief 11. The Examiner finds: Appeal 2021-000919 Application 16/237,285 5 Peacock depicts in FIG. 1C or FIG. 1D or FIG. 5B that the element 113-1 and 112 is integrated into one single CAD element. Peacock teaches at FIG. 1C or FIG. 1D or FIG. 5B highlighting the end region of an integral element comprising the parts 112 + 113-1 by placing the visual indicator overlay on the one or more end regions of the integral element 112+ 113-1 at the determined orientation in the AR environment. Final Action 13. Appellant points to Peacock, column 2, lines 15-17 and column 3, lines 60-66 and states “which are cited by the Examiner in the Office Action.” Appeal Brief 11. Appellant does not provide a specific cite to the Office Action as to where, specifically, the Examiner relied upon the noted Peacock sections. Appellant contends that: Peacock explicitly suggests that the Peacock system presents an instruction and places an indicator 163 around a part relevant to the instruction. Appellant respectfully points out that the Peacock system does not place distinctive highlights on the parts. Instead, the Peacock system merely instructs a user using textual instructions and indicates which part that textual instructions correspond to (see Peacock, FIGs. 1C and 1D). Appeal Brief 11. Peacock teaches, “The presented information may be in the form of textual and/or graphical information that aids a worker in performing one or more tasks, such as object assembly, at the workstation 102.” Peacock, column 4, lines 35-38 (emphasis added); see Figure 1A. Accordingly, Appellant’s argument is unpersuasive of Examiner error because Peacock is not restricted to only using textual instructions as Appellant contends; rather, Peacock provides the option to use textual, graphical information and/or both. Appellant argues that contrary to Peacock: [T]he instant application explicitly teaches an overlay that (i) distinctly highlights one or more regions of the first element, (ii) Appeal 2021-000919 Application 16/237,285 6 indicates how the first element fits in the object based on the one or more regions, and (iii) is placed on the one or more regions of the first element for highlighting the one or more regions in the AR environment. Appeal Brief 11. Peacock Figure 1B is reproduced below: In the assembly task illustrated in FIG. 1C it is determined and indicated that no tools are needed to complete the assembly task. Peacock, column 5, lines 7-9. Appellant surmises that, “Here, the system of the instant application facilitates an overlay that can directly be placed on regions of an element in such a way that the highlight can indicate how the element may fit into an object without the need for a textual instruction.” Appeal Brief 11. Appellant’s arguments are unpersuasive of Examiner error because Peacock teaches, “Likewise, an indicator 163 may be presented around the position of the parts, Appeal 2021-000919 Application 16/237,285 7 such as the motor cable 112 and the motor arm 113-1 relevant to the presented instructions to provide the user with an augmented reality environment that illustrates to the user the parts and assembly instructions to be performed.” Peacock, column 3, lines 62-67 (emphasis added). Peacock further teaches: [T]he next task is for the user to connect the motor cable to the coupler-1 of the body. Through the use of the augmented realty presentation, the wearable computing device indicates the specific end of the motor cable 112 by a visual indicator 163 and also visually indicates the coupler-1 162-2 so that the worker is provided clear guidance on the assembly task to be performed. Peacock, column 5, lines 9-16 (emphasis added). Appellant states that, “The distinctive overlay of the instant application is determined prior to the placement operation (underlining removed)” where [h]ere, the system of the instant application determines the overlay prior to the placement operation. Hence, even prior to the placement operation, the overlay is determined in such a way that the overlay can (i) distinctly highlight one or more regions of an element, and (ii) indicate how the element fits in an object based on the one or more regions (e.g., based on the shape of the region and using distinct colors, as shown in FIGs. 3B and 3C of the instant application). Appeal Brief 13. In contrast, Appellant argues that: On the other hand, indicator 163 of Peacock is a mere circular highlight without any distinctive marking specific to a region of parts 112 or 113 of Peacock. Therefore, assuming arguendo that indicator 163 supposedly can identify a region of part 112 or 113 of Peacock, indicator 163 cannot distinctly indicate a region of part 112 or 113 of Peacock without being placed around the corresponding region of part 112 or 113. In other words, indicator 163, prior to its placement around a part of Peacock, cannot distinctly highlight one or more regions of an element and indicate how the element fits in an object (e.g., object 110 of Peacock). Appeal Brief 14. Appeal 2021-000919 Application 16/237,285 8 Appellant’s arguments are not persuasive of Examiner error because Appellant’s assessment of Peacock is not accurate. As discussed above, Peacock teaches highlighting specific ends of the motor cable 112 by using a visual indicator 163 and visually indicates the coupler-1 162-2 to guide a worker on the assembly task that is required to be performed. See Peacock, column 5, lines 9- 16; see also Peacock Figure 1C. Appellant contends that a claim element is absent in Peacock because “the Peacock system does not use an object detector to detect an element of an object, such as arm 113 of the WIP [work in progress] object, on a workstation of Peacock.” Appeal Brief 18 (citing Peacock column 2, line 61-column 3, line 1; column 10, lines 4-10). Appellant argument is not persuasive of Examiner error because Peacock teaches, “As is shown in FIG. lA, the sensor 153 evaluates a portion of the workstation 102 to detect objects located at the workstation.” Peacock, column 2, lines 40-42; see Final Action 10. Appellant contends that Peacock does not teach “determining an orientation and an alignment of the first element” as recited in claim 1. Appeal Brief 19. Appellant argues that Peacock cannot use identifier 140 to determine alignment. Appeal Brief 20. Appellant further argues, “Here, the Peacock system uses identifier 140 placed on the parts of a WIP object of Peacock, such as a motor arm, to determine the orientation of the part.” Appeal Brief 20. Appellant’s argument is not persuasive of Examiner error because as discussed above, Peacock teaches highlighting specific ends of the motor cable 112 by using a visual indicator 163 and visually indicates the coupler-1 162-2 to guide a worker on the assembly task that is required to be performed. See Peacock, column 5, lines 9-16 (emphasis added); see also Peacock Figure 1C. Appeal 2021-000919 Application 16/237,285 9 Further, “Appellant respectfully points out that, since an expanded view does not involve determining an overlay, Appellant respectfully submits that Examiner interpretation of expanded view 165 as “dashed circular visual indicator overlay 165” is contradictory to the disclosure of Peacock. Appeal Brief 22. The Examiner finds: Peacock teaches at FIG. 1D and FIG. 5B determining a lighted visual indicator overlay 163 overlaying the specific end of the motor cable 112 of the integral element 112 + 113-1 or the dashed circular visual indicator overlay 165 with the marker 514-2 as an overlay for the integral element 112+ 113-1. Final Action 13-14. Note, Appellant does not explain how the Examiner’s interpretation of Peacock’s element 165 affects the cogency of the Examiner’s anticipation rejection. Further, as reproduced above, the Examiner does not rely upon Peacock’s element 165 alone, the Examiner relies upon Peacock’s element 163 in the alternative. Appellant argues that “without the presence of the corresponding textual instruction, it is unclear how indicator 163 can distinctly highlight one or more regions of arm 113 of Peacock and indicate how arm 113 fits in object 110 based on the one or more regions highlighted by indicator 163.” Appeal Brief 22. Appellant surmises that “without instruction 160-2 and textual identifier 162-2, which are not part of an overlay, indicator 163 may not clearly indicate that a region falling within the circumference of indicator 163 should be coupled to coupler 1 of Peacock prior to the placement of indicator 163.” Appeal Brief 22; see Peacock Figure 1C. Appellant concludes that “Examiner’s allegation is not supported by Peacock, and indicator 163 does not distinctly highlight one or more end regions of arm 113 can fit into coupler 1 prior to the placement of indicator 163.” Appeal Brief 23. Appeal 2021-000919 Application 16/237,285 10 Peacock teaches, “providing augmented reality information to workers to assist the workers in assembly of objects, such as aerial vehicles.” Peacock, Abstract. As discussed above, Peacock teaches, “The presented information may be in the form of textual and/or graphical information that aids a worker in performing one or more tasks, such as object assembly, at the workstation 102.” Peacock, column 4, lines 35-38 (emphasis added); see Figures 1A-1D. Accordingly, there is no foundation for Appellant’s assertion that Peacock requires a textual identifier: as discussed above, Peacock teaches highlighting specific ends of the motor cable 112 by using a visual indicator 163 and visually indicates the coupler-1 162-2 to guide a worker on the assembly task that is required to be performed. See Peacock, column 5, lines 9-16 (emphasis added); see also Peacock, Figure 1C. Appellant contends that the Examiner’s allegations are contrary to Peacock because: [A]s discussed above, item 165 of FIG. 1D of Peacock is an expanded view (see id.). Since zooming-in on a part of Peacock does not involve highlighting a region of that part using an overlay, Appellant respectfully submits that Examiner interpretation of applying expanded view 165 as highlighting is contradictory to the disclosure of Peacock.” Appeal Brief 23-24. As discussed above, the Examiner does not rely upon Peacock’s element 165 alone, the Examiner recites Peacock’s element 163 alternately. See Final Action 13-14. Accordingly, Appellant’s argument is not persuasive of Examiner error. We sustain the Examiner’s anticipation rejection of independent claims 1 and 11. Appeal 2021-000919 Application 16/237,285 11 Claims 2 and 12 Appellant argues: [the] Examiner has already alleged that motor arm 113 of Peacock can supposedly be interpreted as the first element of the instant application (see page 9 of the Office Action). Examiner has also interpreted indicator 163 of Peacock as the overlay of the instant application (see page 11 of the Office Action). By alleging that indicator 163 is a hologram of motor arm 113, Appellant respectfully points out that Examiner has contradicted both prior interpretations. Appeal Brief 29; see Final Action 17-19, 29-30. Appellant further argues, “The ordinary and customary meaning of a hologram of an element is well-known and well-established, and indicates a holographic projection of a replica of the element. However, no such hologram is disclosed in Peacock.” Appeal Brief 29. Here, Appellant’s arguments are persuasive of Examiner error. The Examiner relies upon Peacock’s element 163 to teach projecting a hologram, however Peacock is silent in regard to element 163 being a hologram. See Final Action 17 (citing Peacock Figures 1B-1D, column 3, lines 40-55). As articulated by the Federal Circuit, anticipation under 35 U.S.C. § 102 requires that “‘each and every element as set forth in the claim is found, either expressly or inherently described, in a single prior art reference.”’ In re Robertson, 169 F.3d 743, 745 (Fed. Cir. 1999). The Examiner proffers no explanation as to why Peacock’s element 163 is a hologram. Accordingly, we reverse the Examiner’s anticipation rejection of dependent claims 2 and 12 commensurate in scope. Claims 3 and 13 Appellant contends: [T]hat no part of a WIP object of Peacock that appears in FIGs. lA- 1D and 5A-5B is shown to have distinct markers on different Appeal 2021-000919 Application 16/237,285 12 regions. For example, cable 112 runs through motor arm 113-1 in FIG. lC of Peacock to generate an embedded part. Each end of cable 112 represent a portion (or region) of the embedded part. However, indicator 163 appears only on one end of the embedded part. Hence, each region of the embedded part is not highlighted with a distinct mark. Appeal Brief 30. The Specification discloses, “In this example, system 110 can determine the orientation of element 320 and overlay distinct marks on regions of element 320 that assists user 300 to attach element 320 to element 330 without ambiguity.” Here, Appellant’s argument is not persuasive of Examiner error because Peacock teaches distinct marks on different regions of the WIP such as elements 162-2 and 163 shown in Figure 1C. See Final Action 20. Appellant’s Specification does not provide a description of the claimed distinct marks that distinguishes over or from the distinct marks taught by Peacock. We sustain the Examiner’s anticipation rejection of dependent claims 3 and 13 argued together. Claims 4 and 14 Appellant argues, “that FIG. 1D of Peacock merely shows fasteners (e.g., screws) (see col. 5, lines 29-48). The Peacock system does not determine the presence of the fasteners in the field of view through a wearable computing device 150.” Appeal Brief 31. Here, Appellant argument is not persuasive of Examiner error because Peacock teaches, “the presented information may also indicate the tools needed, as illustrated by the indicator 164-9, task time information for the task, as indicated by indicator 164-8 and/or include other confirmation or motivational information, such as the ‘Confirmed’ indicator 164-7.” Peacock, column 5, lines 43-48, Figure 1D; see Final Action 21. Accordingly, we sustain the Examiner’s anticipation rejection of dependent claims 4 and 14 argued together. Appeal 2021-000919 Application 16/237,285 13 Claims 5 and 15 Appellant contends, “In rejecting claims 5 and 15, on pages 21-22 of the Office Action, Examiner alleged that Peacock supposedly teaches ‘projecting a hologram depicting the first element attached to the one or more elements.’” Appeal Brief 32. Appellant’s argument is persuasive of Examiner error because the Examiner does not indicate where Peacock teaches the employment of a hologram as required in dependent claims 5 and 15. See Final Action 21-22. Claims 6 and 16 Appellant argues that “Peacock does not disclose a symmetry breaking feature. A symmetry-breaking feature or anything equivalent does not appear in the disclosure of Peacock.” Appeal Brief 34-35. The Specification discloses in paragraph 68, “A symmetry-breaking feature indicates how the faces of the element are distinct (e.g., are asymmetric).” Here, Appellant’s argument is persuasive of Examiner error because the rejection merely states, “enhancing a resolution of an image of the first element in the AR environment, wherein the enhancing include[s] increasing, within the AR environment, prominence of a symmetry-breaking feature of the first element,” however the Examiner does not indicate specifically within Peacock where this is taught. Final Action 23. The Peacock citations relied upon by the Examiner fail to teach the disputed limitation. See Final Action 23. We reverse the Examiner’s anticipation rejection of dependent claims 6 and 14 commensurate in scope. Claims 7 and 17 Appellant argues that, “Peacock does not disclose ‘placing the same mark on the first and second regions.’” Appeal Brief 36. However, the Examiner finds, “the same mark to the first and second regions (Peacock teaches at FIGS. 1B-1D and column 3, lines 33-67 and column 4, lines 1-32 that the identifiers are detected.” Final Action 24. Here, Appellant’s argument is Appeal 2021-000919 Application 16/237,285 14 persuasive of Examiner error because Peacock does not teach placing the same mark on the multiple regions as require by dependent claims 7 and 17. Claims 8 and 18 Appellant argues that “Peacock at most suggests determining the completion of the assembly operation.” Appeal Brief 37. Appellant contends: However, determining the completion of the assembly operation does not include determining whether the Peacock system can determine the completion of a task in a set of tasks. Since determining the completion of the assembly operation of Peacock does not involve a “determining whether” step, Appellant respectfully submits that a determination of all the assembly tasks for the WIP object being completed does not indicate the “determining whether ...” operation of the instant application. Appeal Brief 37. Appellant’s argument is not persuasive of Examiner error because Peacock teaches, “When the wearable computer device has detected that the instructed task has been completed, for example, in response to interaction from the worker and/or image processing of the WIP object, the next assembly task of the assembly may be determined and presented to the worker.” Peacock, column 4, lines 54-58, see Final Action 25. Peacock further teaches, “Additional processing may be performed to determine that to correct the error, the motor arm needs to be rotated a quarter turn clockwise. As such, the error and corrective action may be presented on the display of the glasses to the worker, as illustrated by the visual indicator 504B.” Peacock, column 12, lines 2-7. Accordingly, Peacock teaches determining if the AR device is capable of determining the competition of the step as required by claims 8 and 18, argued together. We sustain the Examiner’s anticipation rejection of dependent claims 8 and 18. See Final Action 25. Appeal 2021-000919 Application 16/237,285 15 Claims 9 and 19 Appellant argues that, “a physical object in the field of view through a wearable computing device 150 of Peacock does not generate a 3D model of that object.” Appeal Brief 38. Note, dependent claim 9 recites “obtaining a three- dimensional model.” Appellant concludes that, “Since a physical object in Peacock is not a 3D model, Appellant respectfully submits that Peacock does not disclose ‘obtaining a three-dimensional model of the first element.’” Appeal Brief 38. Here, Appellant’s argument is persuasive of Examiner error because Peacock does not teach obtaining a three-dimensional model as recited by dependent claims 9 and 19. See Final Action 26-27. Claims 10 and 20 Appellant argues that “the assembly operation of Peacock does not involve a task model that defines a respective task associated with the assembly operation. Since a task model is not present in Peacock, Appellant respectfully submits that Peacock does not disclose ‘determining the step of the task from a task model.’” Appeal Brief 39. Appellant’s Specification discloses in paragraph thirty-nine that, “To complete the task, the system may load a task model that indicates the steps and sequences of the task. The task model can include a set of steps and a set of operations.” Appellant’s argument is not persuasive of Examiner error because Peacock teaches, “the worker is being presented AR data that includes assembly instructions for assembly of an aerial vehicle (object), such as an unmanned aerial vehicle.” Peacock, column 3, lines 35-38; see Final Action 28. Accordingly, we sustain the Examiner’s anticipation rejection of dependent claims 10 and 20. Appeal 2021-000919 Application 16/237,285 16 CONCLUSION Claims Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 8, 18 112(b) Indefiniteness 8, 18 1-20 102(a)(2) Peacock 1, 3, 4, 8, 10, 11, 13, 14, 18, 20 2, 5-7, 9, 12, 15-17, 19 Overall Outcome 1, 3, 4, 8, 10, 11, 13, 14, 18, 20 2, 5-7, 9, 12, 15-17, 19 No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(1). See 37 C.F.R. § 1.136(a)(1)(v). AFFIRMED-IN-PART