Google LLCDownload PDFPatent Trials and Appeals BoardMay 17, 20212020001365 (P.T.A.B. May. 17, 2021) Copy Citation UNITED STATES PATENT AND TRADEMARK OFFICE UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 16/009,696 06/15/2018 Andrew Ian Russell 0059-610001 2748 79318 7590 05/17/2021 BRAKE HUGHES BELLERMANN LLP P.O. Box 1077 Middletown, MD 21769 EXAMINER LIU, ZHENGXI ART UNIT PAPER NUMBER 2611 NOTIFICATION DATE DELIVERY MODE 05/17/2021 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): docketing@brakehughes.com uspto@brakehughes.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE _______________ BEFORE THE PATENT TRIAL AND APPEAL BOARD _______________ Ex parte ANDREW IAN RUSSELL _______________ Appeal 2020-001365 Application 16/009,696 Technology Center 2600 _______________ Before ERIC S. FRAHM, JOHN A. EVANS, and BETH Z. SHAW, Administrative Patent Judges. FRAHM, Administrative Patent Judge. DECISION ON APPEAL Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s final rejection of claims 1–21. We have jurisdiction under 35 U.S.C. § 6(b). We affirm in part. DISCLOSED AND CLAIMED INVENTION Appellant’s disclosed invention “relates, generally, to shading images in a three-dimensional system” (Spec. ¶ 1), such as communication systems used for video conferencing, “to fulfill the wish for efficient and natural 1 We use the word “Appellant” to refer to “applicant” as defined in 37 C.F.R. § 1.42. “The word ‘applicant’ when used in this title refers to the inventor or all of the joint inventors, or to the person applying for a patent as provided in §§ 1.43, 1.45, or 1.46.” 37 C.F.R. § 1.42(a) (2017). According to Appellant, Google LLC is the real party in interest (Appeal Br. 1). Appeal 2020-001365 Application 16/009,696 2 long-distance communication” including “a natural person-to-person interaction between two or more people” that is lifelike (Spec. ¶ 2). Appellant achieves these purposes by using depth data (common in three- dimensional image processing) to differentiate and shade different image content and/or portions of objects in an image scene with different shading processes (see Spec. ¶¶ 4, 5; Fig. 5 (first portion 500 shaded differently than second portion 504); claims 1 and 10). Appellant claims a method (claim 1) and system (claim 19) for making a three-dimensional presentation by using depth data to identify first and second image content and to apply different shading thereto (see claims 1, 10, 19). Independent claims 1 and 10 are illustrative of the invention and are reproduced below. 1. A method comprising: receiving three-dimensional (3D) information generated by a first 3D system, the 3D information including images of a scene and depth data about the scene, the images generated by cameras capturing respective views of the scene; identifying, using the depth data, first image content in the images associated with a depth value that satisfies a criterion; generating modified 3D information by applying first shading regarding the identified first image content; and making, using the modified 3D information, a 3D presentation that includes at least a portion of the respective views of the scene. Appeal Br. 7, Claims Appendix (emphasis added). 10. The method of claim 1, wherein the scene contains an object in the images and a first portion of the object has a greater depth value in the depth data than a second portion of the object, and wherein generating the modified 3D information further comprises applying second shading regarding a portion of the images where second image content corresponding to the second portion is located. Appeal 2020-001365 Application 16/009,696 3 Appeal Br. 8, Claims Appendix (emphases added). EXAMINER’S REJECTIONS (1) The Examiner rejected claims 1–5, 14, 18, and 19 under 35 U.S.C. § 103 as being unpatentable over Nguyen et al. (US 2012/0051631 A1; published March 1, 2012) (hereinafter, “Nguyen”) and Dore et al. (US 2017/0347055 A1; published Nov. 30, 2017) (hereinafter, “Dore”). Final Act. 6–13. (2) The Examiner rejected claims 6–10, 13, 20, and 21 under 35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, and Chuang et al. (US 2018/0012407 A1; published Jan. 11, 2018) (hereinafter, “Chuang”). Final Act. 13–17. (3) The Examiner rejected claims 11 and 12 under 35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, Chuang, and Kubota et al. (US 2011/0292018 A1; published Dec. 1, 2011) (hereinafter, “Kubota”). Final Act. 17–18. (4) The Examiner rejected claim 15 under 35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, and Seto (JP 401188176 A; published July 27, 1989). Final Act. 19. (5) The Examiner rejected claim 16 under 35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, and Altberg et al. (US 2008/0263458 A1; published Oct. 23, 2008) (hereinafter, “Altberg”). Final Act. 19–20. (6) The Examiner rejected claim 17 under 35 U.S.C. § 103 as being unpatentable over Nguyen, Dore, Altber, and Lux et al. (US 2016/0140689 A1; published May 19, 2016) (hereinafter, “Lux”). Final Act. 20–21. Appeal 2020-001365 Application 16/009,696 4 ISSUES Because Appellant does not present any separate arguments as to the obviousness rejections of (i) claims 1–5, 14, 18, 19, and 21 over the combination of Nguyen and Dore; (ii) claims 6–9 and 21 over the combination taken with Chuang; and (iii) claims 15–17 over the combination taken with various other tertiary prior art references, Appellant has not shown that the Examiner erred in rejecting claims 1–9, 14–19, and 21, and we sustain the obviousness rejections of claims 1–9, 14–19, and 21 pro forma. Ex parte Frye, 94 USPQ2d 1072, 1076 (BPAI 2010) (precedential) (“[T]he Board will generally not reach the merits of any issues not contested by an appellant.”); 37 C.F.R. § 41.37(c)(1)(iv) (2017) (Each ground of rejection must be treated under a separate heading.). Other than including this prior art rejection of claims 1–9, 14–19, and 21 (see supra Rejections 1, 2, and 4–6) in our conclusion, we will not discuss these rejections further. Based on Appellant’s arguments in the Appeal Brief (Appeal Br. 2–6) and the Reply Brief (Reply Br. 1–5), the following three principal issues are presented on appeal: (1) Did the Examiner err in rejecting claims 10–12 under 35 U.S.C. § 103 as being unpatentable over the combination of Nguyen, Dore, and Chuang, because Chuang, and thus the combination of Nguyen, Dore, and Chuang, fails to teach or suggest a scene containing an object having first and second portions, where the different portions are shaded differently, as recited in claim 10? (2) Has Appellant shown the Examiner erred in rejecting claim 13 under 35 U.S.C. § 103 as being unpatentable over the combination of Appeal 2020-001365 Application 16/009,696 5 Nguyen, Dore, and Chuang, because Nguyen, and thus the combination of Nguyen, Dore, and Chuang, fails to teach or suggest identifying a hole, as recited in claim 13? (3) Has Appellant shown the Examiner erred in rejecting claim 20 under 35 U.S.C. § 103 as being unpatentable over the combination of Nguyen, Dore, and Chuang, because Chuang, and thus the combination of Nguyen, Dore, and Chuang, fails to teach or suggest determining a surface normal and applying shading, as recited in claim 20? ANALYSIS We have reviewed Appellant’s arguments in the Appeal Brief (Appeal Br. 2–6) and the Reply Brief (Reply Br. 1–5), the Examiner’s rejections (Final Act. 6–21), and the Examiner’s response to Appellant’s arguments (Ans. 4–11). With regard to claim 20, we agree with and adopt as our own the Examiner’s findings of facts and conclusions as set forth in the Final Rejection (Final Act. 1, 16) and Answer (Ans. 11). Appellant’s arguments with respect to claim 20 (see Appeal Br. 5–6; Reply Br. 4–5) regarding Chuang’s failure to teach or suggest applying shading of second content based on the determined surface normal are not persuasive. However, Appellant’s arguments have persuaded us of error in the Examiner’s rejection of claims 10 and 13 under 35 U.S.C. § 103 based on Chuang’s failure to teach or suggest applying different shading to first and second portions of an object in a scene as recited in claim 10 (see Appeal Br. 3–4; Reply Br. 1–3), and Nguyen’s failure to teach or suggesting identifying a hole as recited in claim 13 (see Appeal Br. 4–5; Reply Br. 3–4). Appeal 2020-001365 Application 16/009,696 6 Issue (1): Claim 10 Claim 10 stands rejected over Nguyen, Dore, and Chuang, and is reproduced below: 10. The method of claim 1, wherein the scene contains an object in the images and a first portion of the object has a greater depth value in the depth data than a second portion of the object, and wherein generating the modified 3D information further comprises applying second shading regarding a portion of the images where second image content corresponding to the second portion is located. Appeal Br. 8, Claims Appendix (emphases added). In rejecting claim 10, the Examiner relies on Figures 25 and 26 of Nguyen as teaching the limitation “wherein the scene contains an object in the images and a first portion of the object has a greater depth value in the depth data than a second portion of the object,” set forth in claim 10 (see Final Act. 6, 7, 15; Ans. 5–7). Nguyen’s Figure 25 shows a scene, containing an object in the foreground (the man with the glasses and stripe shirt on), and a background containing several objects. Nguyen’s Figure 26 shows the object in the foreground, and also shows the background being shaded out in its entirety. Nguyen’s Figures 25 and 26 thus show shading different objects differently, and shading all the objects in the background differently than the object in the foreground. However, Nguyen’s Figures 25 and 26 do not show a scene containing an object, where the object has different portions of that object shaded differently as required by Appellant’s claims 1 and 10, and as shown in Appellant’s Figure 5 showing an object (person 500) in the foreground having a first portion (rest of torso portion Appeal 2020-001365 Application 16/009,696 7 500) and a second portion 504 shaded differently (see Spec. ¶¶ 63–66) (describing Figure 5). The Examiner maps the limitations of claims 1 and 10 as follows: • “first image content” is mapped to the background content of a scene • “first shading” is mapped to the image processing that involves removing the background objects and coloring the background in gray or a lighter color (see Nguyen, fig. 26) • “a first portion of the object” (Claim 10) is mapped to the background portion of a scene object • “second image content” (Claim 10) is mapped to the foreground content of the scene • “a second portion of the object” (Claim 10) is mapped to the foreground portion of the scene object • “second shading” (Claim 10) is mapped to Chuang's shading method. Ans. 6. Thus, the Examiner maps Nguyen’s (i) background content shown in Figure 25 to the “first image content” of claim 1 and the “first portion of the object” of claim 10; and (ii) foreground content (the man with the glasses and stripe shirt on) shown in Figure 25 to the “second image content” and “second portion of the object” of claim 10. This mapping is unreasonable, because Nguyen’s Figure 25 shows a scene or image containing multiple objects, and shades all objects in the background in one manner, and the foreground (which contains only one object) in another. Thus, Nguyen’s Figure 25 fails to show two different portions of a single object in a scene that are being shaded differently. Appellant argues, and we agree, that “the Examiner’s reasoning regarding Nguyen fails to address whether a person of ordinary skill in the art would consider the ceiling, walls, a table, a standing person, and a seated person, on the one hand, and the person in a striped shirt, on the other hand, Appeal 2020-001365 Application 16/009,696 8 to be portions of the same ‘object’ as required by claim 10” (Reply Br. 3). Appellant contends, and we agree, that Nguyen “does not describe these separate features as all being part of some scene object” (Reply Br. 3). Further, Appellant contends, and we agree, that “the Examiner presents no argument, and makes no evidence of record, that a skilled artisan would have considered these disparate aspects of Nguyen’s FIG. 25 to be portions of the same object” (Reply Br. 3). Appellant’s arguments have persuaded us of error in the Examiner’s rejection of claim 10 over the combination of Nguyen, Dore, and Chuang, and the claims which depend thereupon (i.e., claims 11 and 12 rejected over the same combination of Nguyen, Dore, and Chuang, further in view of Kubota). Accordingly, we do not sustain the Examiner’s rejections of claim 10, or dependent claims 11 and 12, which rely on the same combination. At best, the Examiner’s proposed combination is speculative and not based on record evidence. We will not resort to speculation or assumptions to cure the deficiencies in the Examiner’s fact finding. See In re Warner, 379 F.2d 1011, 1017 (CCPA 1967); Ex parte Braeken, 54 USPQ2d 1110, 1112 (BPAI 1999) (unpublished) (“The review authorized by 35 U.S.C. [§] 134 is not a process whereby the examiner . . . invite[s] the [B]oard to examine the application and resolve patentability in the first instance.”). Appellant’s arguments as to claim 10 are persuasive. Based on the record before us, we do not sustain the Examiner’s obviousness rejection of clam 10. Issue (2): Claim 13 Claim 13 stands rejected over Nguyen, Dore, and Chuang, and is Appeal 2020-001365 Application 16/009,696 9 reproduced below: 13. The method of claim 1, further comprising identifying a hole in at least one of the images, wherein generating the modified 3D information comprises applying second shading regarding the hole. Appeal Br. 8, Claims Appendix (emphasis added). In rejecting claim 13, the Examiner relies on paragraph 70 of Nguyen as teaching the limitations set forth in claim 13. Nguyen’s paragraph 70 describes dilating boundaries of regions of foreground pixels and eroding holes in foreground regions (see Nguyen ¶ 70). The Examiner reasons that pixels within identified foreground pixel areas that are not identified as foreground pixels are considered hole pixels (see Ans. 9–10). Appellant argues that Nguyen fails to teach or suggest identifying a hole (see Appeal Br. 4–5; Reply Br. 3–4). We agree. Even if we agreed with the Examiner that Nguyen teaches a hole in an image, we find no support in the record that Nguyen teaches or suggests identifying a hole in an image, and then applying second shading different than the first shading applied to the first image content, as set forth in claim 1, from which claim 13 depends. Based on the record before us, Appellant’s arguments as to claim 13 are persuasive. Accordingly, we do not sustain the Examiner’s obviousness rejection of clam 13. Issue (3): Claim 20 Claim 20 stands rejected over Nguyen, Dore, and Chuang, and is reproduced below: 20. The system of claim 19, wherein the scene contains an object in the images, and wherein generating the modified 3D Appeal 2020-001365 Application 16/009,696 10 information further comprises determining a surface normal for second image content of the object, and applying second shading regarding the second image content based on the determined surface normal. Appeal Br. 9, Claims Appendix (emphasis added). In rejecting claim 20, the Examiner relies on paragraph 143 of Chuang as teaching applying second shading based on a determined surface normal (see Final Act. 13, 16; Ans. 11). The Examiner reasons that “Chuang’s shading method is ‘based on the determined surface normal’ because paragraph 143 of Chuang recites ‘Contributions from non-occluded images can be weighted by respective dot products of camera orientations and surface normal.’” (Ans. 11). This finding is reasonable and supported by paragraph 143 of Chuang, because applying shading for a surface normal first requires determining the surface normal. To the extent Appellant argues that Chuang’s weighting of dot products of contributions from non-occluded images is not the same as applying shading to image content (see Appeal Br. 6), this argument is conclusory and not persuasive, because Appellant has presented no evidence to support such a proposition. It is well settled that mere lawyer’s arguments and conclusory statements, which are unsupported by factual evidence, are entitled to little probative value. In re Geisler, 116 F.3d 1465, 1470 (Fed. Cir. 1997); In re De Blauwe, 736 F.2d 699, 705 (Fed. Cir. 1984); In re Schulze, 346 F.2d 600, 602 (CCPA 1965) (It is well established that the arguments of counsel cannot take the place of evidence in the record); In re Pearson, 494 F.2d 1399, 1405 (CCPA 1974) (Attorney argument is not evidence); see also In re Rijckaert, 9 F.3d 1531, 1532 (Fed. Cir. 1993) Appeal 2020-001365 Application 16/009,696 11 (When an examiner establishes a prima facie case of obviousness, the burden shifts to the applicant to show non-obviousness). In this light, Appellant’s arguments with respect to claim 20 (see Appeal Br. 5–6; Reply Br. 4–5) regarding Chuang’s failure to teach or suggest applying shading of second content based on the determined surface normal are not persuasive. Accordingly, we sustain the Examiner’s rejection of claim 20. CONCLUSIONS (1) Appellant has not presented any arguments as to claims 1–9, 14– 19, and 21, therefore Appellant has not shown the Examiner erred in rejecting claims 1–9, 14–19, and 21 over the combination of Nguyen and Dore, or the combination in view of various tertiary prior art references. Accordingly, we summarily sustain the Examiner’s obviousness rejections of claims 1–9, 14–19, and 21. (2) Appellant has shown the obviousness rejections of claim 10, as well as claims 11 and 12 depending therefrom and claim 13 separately argued, over the combination of Nguyen, Dore, and Chuang to be in error. Accordingly, we do not sustain the Examiner’s obviousness rejections of claims 10–13. (3) Appellant has not shown the obviousness rejection of claim 20 over the combination of Nguyen, Dore, and Chuang to be in error. Accordingly, we sustain the Examiner’s obviousness rejection of claim 20. In view of the foregoing, we (i) affirm the Examiner’s obviousness rejections of claims 1–9 and 14–21 over the combination of Nguyen and Appeal 2020-001365 Application 16/009,696 12 Dore; and (ii) reverse the Examiner’s obviousness rejection of claims 10–13 over Nguyen, Dore, and Chuang, under 35 U.S.C. § 103. In summary: Claims Rejected 35 U.S.C. § Reference(s)/Basis Affirmed Reversed 1–5, 14, 18, 19 103 Nguyen, Dore 1–5, 14, 18, 19 6–10, 13, 20, 21 103 Nguyen, Dore, Chuang 6–9, 20, 21 10, 13 11, 12 103 Nguyen, Dore, Chuang, Kubota 11, 12 15 103 Nguyen, Dore, Seto 15 16 103 Nguyen, Dore, Altberg 16 17 103 Nguyen, Dore, Altberg, Lux 17 Overall Outcome 1–9, 14– 21 10–13 No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv). AFFIRMED IN PART Copy with citationCopy as parenthetical citation