Ex Parte Lucas-Woodley et alDownload PDFPatent Trial and Appeal BoardMay 21, 201813900987 (P.T.A.B. May. 21, 2018) Copy Citation UNITED STA TES p A TENT AND TRADEMARK OFFICE APPLICATION NO. FILING DATE 13/900,987 05/23/2013 530 7590 05/23/2018 LERNER, DAVID, LITTENBERG, KRUMHOLZ & MENTLIK 600 SOUTH A VENUE WEST WESTFIELD, NJ 07090 FIRST NAMED INVENTOR Thomas Edward Lucas-Woodley UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O. Box 1450 Alexandria, Virginia 22313-1450 www .uspto.gov ATTORNEY DOCKET NO. CONFIRMATION NO. DYOUNG 3.0F-104 6996 EXAMINER LIU, GORDON G ART UNIT PAPER NUMBER 2612 NOTIFICATION DATE DELIVERY MODE 05/23/2018 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address( es): eOfficeAction@ldlkm.com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte THOMAS EDWARD LUCAS-WOODLEY, NICHOLAS ANDREW LORD, WILLIAM OLIVER SYKES, and ADRIEN BAIN Appeal2017-009389 Application 13/900,987 Technology Center 2600 Before BRUCE R. WINSOR, JOSEPH P. LENTIVECH, and KARA L. SZPONDOWSKI, Administrative Patent Judges. LENTIVECH, Administrative Patent Judge. DECISION ON APPEAL Pursuant to 35 U.S.C. § 134(a), Appellants 1 appeal from the Examiner's decision to reject claims 1-20, the only claims pending in the application on appeal. We have jurisdiction over the pending claims under 35 U.S.C. § 6(b). We affirm-in-part. 1 According to Appellants, the real party in interest is the Applicant, Sony Computer Entertainment Europe Limited. App. Br. 2. Appeal2017-009389 Application 13/900,987 STATEMENT OF THE CASE Appellants 'Invention Appellants' invention generally relates to "generating an internally consistent model of the state of a book captured in a video image." Spec., Abstract. Claims 1 and 7, which are illustrative of the claimed invention, read as follows: 1. A method of generating an internally consistent model of a state of a book captured in a video image, the method comprising the steps of: obtaining a plurality of pieces of evidence relating to a current state of a corresponding plurality of different aspects of the book in the video image, the obtained pieces of evidence including an identity of one or more fiduciary markers and an angular position 8 1 of a turning leaf of the book; associating a score with each piece of evidence based on one or more estimates for the piece of evidence; generating an initial model of the current state of the book, wherein the current state of the book is constrained by physical properties of the book and at least the highest scoring piece of evidence from among the obtained plurality of pieces of evidence; and sequentially constraining the model in response to one or more successive pieces of the obtained evidence whose scores meet a respective predetermined first threshold value. 7. A method of generating an augmented reality image, comprising the steps of: capturing an video image comprising an image of a book; generating an internally consistent model of a current state of the book captured in the video image, wherein the current state of the book is constrained by physical properties of the book and at least a highest scoring piece of evidence from among a set of obtained plurality of pieces of evidence, the obtained plurality 2 Appeal2017-009389 Application 13/900,987 pieces of evidence including an identity of one or more fiduciary markers and an angular position 8 of a turning leaf of the book, a score for each piece of evidence representing a confidence value for that piece of evidence based on one or more estimates for the piece of evidence; rendering a computer graphic book responsive to the generated model of the current state of the book; and augmenting the captured video image with the rendered computer graphic book. References The Examiner relies on the following prior art in rejecting the claims: Kashimoto Sakurai et al. Campbel et al. US 2012/0223968 Al US 2012/0306917 Al EP 2 426 641 Al Rejections Sept. 6, 2012 Dec. 6, 2012 July 3, 2012 Claims 1---6, 8, 16, and 20 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over the combination of Kashimoto and Campbel. Final Act. 4--14. Claims 7, 9--15, and 17-19 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over the combination of Kashimoto, Campbel, and Sakurai. Final Act. 14--26. ANALYSIS Claim 1 Appellants contend the combination of Kashimoto and Campbel does not teach or suggest "sequentially constraining the model in response to one 3 Appeal2017-009389 Application 13/900,987 or more successive pieces of the obtained evidence whose scores meet a respective predetermined first threshold value," as recited in claim 1. App. Br. 13-15. Campbel generally relates to augmenting a captured video image of a book with virtual graphic elements. Campbel i-f 20. Campbel describes a process for applying graphical augmentations during the turning of a page of the book (e.g., a pop-up picture folding up or down). See, generally, Campbel i-fi-1 44--77. Campbel teaches "[g]iven the position of the axis of rotation of the turning leaf of the book, and in addition the size of that leaf and the fact that it is substantially rigid, it then becomes possible for the system to hypothesise positions for the leaf as it is turned over in the book." Campbel i-f 55. The Examiner finds Campbel teaches determining a score for each hypothetical position and determining that the hypothetical position with the highest score is representative of the current position of the leaf as the leaf is being turned. Final Act. 6 (citing Campbel, Fig. 5; i-fi-174--75). The Examiner further finds Campbel teaches "[ t ]his process is repeated, for example at the video frame rate of the output display (typically 50 or 60 Hertz), although other frequencies may be considered, for example to reduce computational overhead." Id. The Examiner finds Campbel teaches or suggests the disputed limitation because "the claim does not specify that the constraining is applied to one frame, thus, it is fine that the model is sequentially applied to different frames." Ans. 7. Appellants argue Campbel teaches repeating the process at a particular frame rate, thereby performing a complete page turning estimation process for successive image frames. App. Br. 15 (citing Declaration of 4 Appeal2017-009389 Application 13/900,987 Diannid Campbell under 37 CPR§ 1.132, filed Mar. 9, 2017). Appellants argue performing a complete page turning estimation process for successive image frames, as taught by Campbel, "merely obtains a series of separate hypotheses" and "[t]here is no model that is sequentially constrained as set forth in claim 1." App. Br. 13. We agree. The Specification describes: subsequent evidence needs to be consistent with the [initial constraints] ... [T]he AR application runs through the available evidence in a predetermined order of priority to select evidence that can be classified as 'verified'. Verified evidence is that which has a confidence value (score) that exceeds a respective predetermined threshold for that evidence type ... The verified evidence is compared against the constraints upon the book imposed by the base pose ... In this way, additional information about the state of the book may be used, in conjunction with the physical constraints of the book and the constraints imposed by preceding evidence, to further constrain the possible state of the book. Spec. 19; see also Spec. 25 ("the model of the book is sequentially constrained first by the highest scoring evidence and then by trusted evidence (verified evidence), such that the subsequent evidence is only used to further constrain an existing model of the book when it is consistent with the current constraints of the model of the book, thereby generating an overall internally consistent model of the book based upon strong evidence from the video image."). Considering the broadest, reasonable interpretation in light of the Specification, we construe "sequentially constraining the model" as "further constraining the initial model." 5 Appeal2017-009389 Application 13/900,987 Campbel describes a process for applying graphical augmentations to a leaf of a book as the leaf is being turned. Campbel i-f 45. Campbel teaches hypothesizing a plurality of possible positions for the free edges of the leaf as it is being turned based on an estimated placement of the spine of the book and the known size of the leaf. Campbel i-f 56. Campbel teaches "one hypothesis is given for each degree of rotation of the leaf within a given angular range, thereby generating a plurality of candidate hypotheses for the position of the rigid leaf and its edges." Campbel i-f 57. A captured video image of the book is imaged processed to generate an edge-image in which only the edges of the leaf are identified. Campbel i-f 62. The edges of the leaf identified in the edge-image are compared to the hypothesized plurality of possible positions for the free edges of the leaf to determine the position of the rigid leaf being turned in the book. Campbel i-fi-166-74. Campbel teaches that this process can be repeated at a particular frame rate to track the position of the leaf as the leaf is being turned. Campbel i-fi-175-78. As such, we agree with Appellants (App. Br. 13) that Campbel teaches repeating the process to generate a new model for successive frames of the video image and does not teach or suggest sequentially constraining an initial model in response to one or more successive pieces of the obtained evidence, as required by claim 1. We do not reach Appellants' further allegations of error because we find the issue discussed above to be dispositive of the rejection of claim 1 and of claims 2-6, 8-16, and 18-20, argued together with claim 1. Accordingly, we do not sustain the Examiner's rejection of claims 1- 6, 8-16, and 18-20. 6 Appeal2017-009389 Application 13/900,987 Claim 7 Appellants contend the combination of Kashimoto, Campbel, and Sakurai does not teach or suggest generating an internally consistent model of a current state of the book captured in the video image, wherein the current state of the book is constrained by physical properties of the book and at least a highest scoring piece of evidence from among a set of obtained plurality of pieces of evidence, the obtained plurality pieces of evidence including an identity of one or more fiduciary markers and an angular position 8 of a turning leaf of the book, a score for each piece of evidence representing a confidence value for that piece of evidence based on one or more estimates for the piece of evidence; as recited in claim 7. App. Br. 15-18; Reply Br. 4--6. In particular, Appellants argue the cited references do not teach or suggest that the current state of the book is constrained by at least a highest scoring piece of evidence, as required by claim 7. Id. We are not persuaded. Contrary to Appellants' arguments, claim 7 does not require "scores for pieces of evidence of different respective types (including fiduciary markers and an angular position of a turning leaf of a book)." App. Br. 17. Claim 7 recites "wherein the current state of the book is constrained by ... at least a highest scoring piece of evidence from among a set of obtained plurality of pieces of evidence." Claim 7 further requires that "the obtained plurality pieces of evidence," rather than the set of the obtained plurality of pieces of evidence, includes "an identity of one or more fiduciary markers and an angular position 8 of a turning leaf of the book." The Examiner finds Kashimoto teaches obtaining a plurality of pieces of evidence including an identity of one or more fiduciary markers and the three curves calculated by the curve calculation unit. Final Act. 14--15. 7 Appeal2017-009389 Application 13/900,987 Thus, the three curves constitute a set of evidence of the obtained plurality of pieces of evidence. Kashimoto teaches that "[ o ]nee the three curves have been calculated, the curve calculation unit 3 8 establishes one of the curves as most closely approximating the inter-marker distance computed from the internal marker information." Kashimoto i-f 93. Kashimoto, therefore, teaches, or at least suggests, determining some metric indicating how closely each curve approximates the inter-marker distance and selecting the curve that most closely approximates the inter-marker distance (e.g., the curve with the highest metric). As such, Kashimoto teaches or suggests that the current state of the book is constrained by at least a highest scoring piece of evidence, as required by claim 7. Accordingly, we are not persuaded the Examiner erred in rejecting claim 7 or claim 17, which depends from claim 7 and is not separately argued with particularity. See App. Br. 18. DECISION We reverse the Examiner's rejection of claims 1-6, 8-16, and 18-20. We affirm the Examiner's rejection of claims 7 and 1 7. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a)(l )(iv). AFFIRMED-IN-PART 8 Copy with citationCopy as parenthetical citation