Ex Parte Ernst et alDownload PDFPatent Trial and Appeal BoardFeb 27, 201813216017 (P.T.A.B. Feb. 27, 2018) Copy Citation United States Patent and Trademark Office UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office Address: COMMISSIONER FOR PATENTS P.O.Box 1450 Alexandria, Virginia 22313-1450 www.uspto.gov APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO. 13/216,017 08/23/2011 Steve Ernst 3450.103.01US 3346 61529 7590 Neugeboren O'Dowd PC 1227 Spruce Street SUITE 200 BOULDER, CO 80302 EXAMINER GRANT, MICHAEL CHRISTOPHER ART UNIT PAPER NUMBER 3715 NOTIFICATION DATE DELIVERY MODE 03/01/2018 ELECTRONIC Please find below and/or attached an Office communication concerning this application or proceeding. The time period for reply, if any, is set in the attached communication. Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the following e-mail address(es): craig @ neugeborenlaw .com sean @ neugeborenlaw. com rene @ neugeborenlaw .com PTOL-90A (Rev. 04/07) UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Ex parte STEVE ERNST, CHARLES J. SMITH, GREGORY KLINKEL and ROBERT BURGIN Appeal 2016-008451 Application 13/216,017 Technology Center 3700 Before STEVEN D.A. McCARTHY, MICHAEL W. KIM and BRETT C. MARTIN, Administrative Patent Judges. McCARTHY, Administrative Patent Judge. DECISION ON APPEAL 1 STATEMENT OF THE CASE 2 The Appellants1 appeal under 35 U.S.C. § 134(a) from the Examiner’s 3 decision finally rejecting claims 11-16, 18-21 and 27-41. (See “Appeal 4 Brief under 37 C.F.R. 41.37,” dated May 23, 2016 (“Appeal Brief’ or “App. 5 Br.”), at 4; Final Office Action, mailed October 23, 2015 (“Final Act.”), at 6 2). We have jurisdiction under 35 U.S.C. § 6(b). 7 We AFFIRM. 1 The Appellants identify the real party in interest as Knowledge Factor, Inc. (See App. Br. 4). 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 ISSUES On page 4 of their Appeal Brief, the Appellants say that they have appealed only claims 11, 27 and 37. Nevertheless, the Appellants argue, on page 20 of the Appeal Brief, that claims 12-16, 18-21, 28-36 and 38—41 “are allowable by virtue of their dependence on allowable claims.” Based on this argument, we reach claims 12-16, 18-21, 28-36 and 38—41 in this appeal. Because we conclude that claims 11, 27 and 37 are unpatentable for the reasons articulated by the Examiner, we will affirm the rejections of claims 12-16, 18-21, 28-36 and 38^11, as well. Only those arguments actually made by the Appellants have been considered. Arguments that the Appellants could have made, but chose not to make, have not been considered and are deemed to be waived. See 37 C.F.R. § 41.37(c)(l)(iv); In re Jung, 637 F.3d 1356, 1365 (Fed. Cir. 2011). Four issues are dispositive of this appeal: First, do Etesse (US 2004/0030781 Al, publ. Feb. 12, 2004), Bruno ’920 (US 2006/0029920 Al, publ. Feb. 9, 2006), Antoniak (US 5,456,607, issued Oct. 10, 1995) and Kerfoot (US 2010/0035225 Al, publ. Feb. 11, 2010), in combination, teach or suggest all limitations of appealed independent claim 11, so as to provide a sufficient factual underpinning for rejection of the claim under pre-AIA 35 U.S.C. § 103(a)? (See App. Br. 17 & 18; “Reply Brief to Examiner’s Answer under 37 CFR 41.41,” dated Sept. 7, 2016 (“Reply Br.”), at 16). Second, did the Examiner articulate a proper reason, with some rational underpinning, why one of ordinary skill in the art would have modified the teachings of Etesse, Bruno ’920 and Kerfoot in the fashion claimed in claim 27? {See App. Br. 18 & 19; Reply Br. 16 & 17). 2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Appeal 2016-008451 Application 13/216,017 Third, do Etesse, Bruno ’920, Kerfoot and Altenhofen (US 2003/0152905 Al, publ. Aug. 14, 2003), in combination, teach or suggest all limitations of appealed independent claim 37, so as to provide a sufficient factual underpinning for rejection of the claim under pre-AIA 35 U.S.C. § 103(a)? {See App. Br. 19 & 20; Reply Br. 17 & 18). Fourth, are one or more of independent claims 11, 27 and 37 directed to an abstract idea and, if so, do the claims directed to an abstract idea also include an “inventive concept,” so as to be patent eligible under 35 U.S.C. §101? {See App. Br. 7-17; Reply Br. 3-15). THE CLAIMED SUBJECT MATTER The appealed claims are directed to microprocessor- and network- based testing and learning systems. (Spec., para. 2). The Appellants’ Specification criticizes traditional multiple-choice testing for encouraging students to guess. Under this situation, a successful guess would mask the true extent or the state of knowledge of the learner, as to whether he or she is informed (i.e., confident with a correct response), misinformed (i.e., confident in the response, which response, however, is not correct) or lacked information (i.e., the learner explicitly states that he or she does not know the correct answer, and is not allowed to respond in that fashion). (Spec., para. 3). The Appellants seek to remedy this problem by providing multiple-choice tests having “two-dimensional answers,” that is, answers capable of measuring both the correctness of the test-takers’ responses and the test-takers’ confidence in those responses. {See Spec., para. 10). The Specification defines an “ampObject,” called earlier in the Specification a “learning object,” as a combination of “an individual 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Appeal 2016-008451 Application 13/216,017 question/answer presented to a learner or other user of the assessment and learning system (including introductory material), . . . learning information that is displayed to the learner (explanations and Additional Learning), and [metadata] available to the author and analyst.” (Spec., para. 64). The Specification teaches that: shadow questions may be utilized that are associated with the same competency (learning outcome; learning objective). In one embodiment, the author associates relevant learning objects into a shadow question grouping. If a learner receives a correct score for one question that is part of a shadow question group, then any learning object in that shadow question is deemed as having been answered correctly. The system will pull randomly (without replacement) from all the learning objects in a shadow group as directed by one or more of the algorithms described herein. (Spec., para. 113). For reasons discussed earlier, we address only independent claims 11, 27 and 37. The remaining claims on appeal stand or fall with these three claims. Claim 11 recites: 11. A service-oriented computer structure comprising a multi-tiered services structure adapted to perform a method of knowledge assessment, the method comprising: creating, through an interface to a content management server, a knowledge assessment application; providing the knowledge assessment application to a learner through a learning server; enabling the learner to access the knowledge assessment through a registration and data analytics server; displaying to the learner at a display device a plurality of multiple-choice questions and two-dimensional answers stored at the content management server; transmitting via a communication network to the display device the plurality of multiple-choice questions and two- 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 Appeal 2016-008451 Application 13/216,017 dimensional answers, wherein the answers include a plurality of full-confidence answers consisting of single-choice answers, a plurality of partial-confidence answers consisting of one of more sets of multiple single-choice answers, and an unsure answer; administering an assessment comprising presenting to the learner via the display device the plurality of multiple-choice questions and the two-dimensional answers, the multiple-choice questions grouped into shadow groups, and receiving via the display device the learner’s selected answers to the multiple- choice questions by which the learner indicates both their substantive answer and the level of confidence category of their answer by dragging the substantive answer and the level of confidence across the display to an appropriate area; and scoring the assessment by assigning a knowledge state designation to a shadow group that two or more answered multiple-choice questions are grouped into, and wherein no additional multiple-choice questions in the shadow group are presented to the learner when the knowledge state is a proficient knowledge state or a mastery knowledge state, determining when a learner requires more learning material about a particular topic by comparing a learner’s selected answers to the two-dimensional answers stored at the content management server; providing to the learner, from the content management server, and in response to the determining, one or more learning objects by assembling textual content and one or more of: digital images, videos, and links to internet websites, wherein the one or more learning objects are provided to the learner through the knowledge assessment application on the display device in real time as soon as a determination is made that the learner requires more learning material. 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 GROUNDS OF REJECTION The Examiner rejects claims 11-16, 18-21 and 27—41 under 35 U.S.C. § 101 as being directed to ineligible subject matter (see Final Act. 2). In addition, the Examiner rejects under pre-AIA 35 U.S.C. § 103(a): claims 27-31 and 36 as being unpatentable over Etesse, Bruno ’920 and Kerfoot (see Final Act. 11); claim 32 as being unpatentable over Etesse, Bruno ’920, Kerfoot and Bruno article (Bruno, Admissible Probability Measures in Instructional Management, 14 J. Computer-Based Instruction 23 (Ass’n for the Dev. of Computer-Based Instructional Sys., Winter 1987)) (see Final Act. 11); claims 34 and 35 as being unpatentable over Etesse, Bruno ’920, Kerfoot and Bruno ’592 (US 2003/0190592 Al, publ. Oct. 9, 2003) (see Final Act. 11); claims 37^10 as being unpatentable over Etesse, Bruno ’920, Kerfoot and Altenhofen (see Final Act. 12); claim 41 as being unpatentable over Etesse, Bruno ’920, Kerfoot, Altenhofen and Bruno article (see Final Act. 12); claims 11, 13,21 and 33 as being unpatentable over Etesse, Bruno ’920, Antoniak and Kerfoot (see Final Act. 3 & 11); claim 12 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot, and Bruno article (see Final Act. 8); claims 14-16 and 18 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot and Altenhofen (see Final Act. 9); and claims 19 and 20 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot and Bruno ’592 (see Final Act. 9). 6 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Appeal 2016-008451 Application 13/216,017 FINDINGS OF FACT The record supports the following findings of fact (“FF”) by a preponderance of the evidence. Etesse 1. Etesse describes a system for “transmit[ting] course files including course lectures, textbooks, literature, and other course materials, receiving] student questions and input, and conducting] participatory class discussions using an electronic network such as . . . the Internet.” (Etesse, para. 94). The system is multi-tiered, including a user interface tier 1002, a platform tier 1002 and a data tier 1005. {See Etesse, para. 95 & Fig. IB). Figure 1A of Etesse depicts student workstations 56, 58, 60 interacting with instructor workstations 52, 54 and a system server 100 through the Internet 62. The system includes one or more servers in direct or indirect communication with the student workstations. The user interface tier 1002 includes components permitting users at individual workstations to access, interact with, and retrieve information from, the system via browsers 120. {See Etesse, paras. 38, 96, 127 & 149). 2. Instructor control panels 1602, displayed on the instructor workstations 52, 54, enable instructors to access the system to manage and develop courses. {See Etesse, paras. 132, 169 & 170). In particular, an instructor may access assessment tools 1612 through an instructor control panel 1602. {See Etesse, paras. 170 & 195). The assessment tools enable an instructor to create knowledge assessments such as test, quizzes and surveys. {See Etesse, paras. 139 & 195). 7 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Appeal 2016-008451 Application 13/216,017 3. In particular, the assessment tools enable instructors to create knowledge assessments with multiple choice or matching questions. (See Etesse, paras. 139 & 195). The assessment tools also permit instructors to set parameters for showing the students whether each question in an assessment was answered correctly or incorrectly; and to show the student the correct answer, along with “feedback” entered by the instructor, for each question. (See Etesse, para. 196). Other parameters that instructors might set by means of the assessment tools include parameters permitting students to repeat the knowledge assessment; timing the assessment; or requiring entry of a password to access the assessment. (See id.) 4. Etesse teaches maintaining pools of questions, that is: predefined groups of questions and answer sets that are logically linked, usually by subject matter, so that an instructor may draw from a pool to obtain existing questions and answers sets from other courses, instructors, semesters, etc. and not have to “recreate the wheel” every time they generate or modify a test. (Etesse, para. 197). 5. Students may access the tests or quizzes created by the assessments tools at the student workstations 56, 58, 60 through an assignments web page. (See Etesse, para. 151). The system described by Etesse provides automatic, real-time grading (and, presumably, feedback) through an automatic grading functionality. (See Etesse, paras. 151 & 195). Bruno ’920 6. Bruno ’920 criticizes traditional multiple-choice testing for failing to accurately assess a student’s actual knowledge, by “encourag[ing] individuals to become skilled at eliminating possible wrong answers and making best-guess determinations at correct answers.” (Bruno ’920, para. 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Appeal 2016-008451 Application 13/216,017 6). Bruno ’920 addresses this problem by providing multiple-choice tests having “two-dimensional answers,” that is, answers capable of measuring both the correctness of the test-takers’ responses and the test-takers’ confidence in those responses. (See Bruno ’920, para. 17). 7. In particular, Bruno ’920 teaches “converting a standard multiple choice test comprising three answer (“A”, “B”, and “C”) multiple choice questions into questions answerable by seven options, that cover three states of mind: confidence, doubt, and ignorance.” (Bruno ’920, para. 37). For example, a student responding to a two-dimensional, multiple choice question may choose one of three substantive answers with full confidence; choose any two of the three substantive answers with partial confidence; or answer “I Don’t Know.” (See Bruno ’920, paras. 17 & 39; see also id., Fig. 1). When the test is graded, correct answers selected with full confidence receive full credit; correct answers selected with partial confidence receive partial credit; “I Don’t Know” responses receive no credit; and incorrect answers selected with partial or full confidence are penalized. (Bruno ’920, para. 39). The penalty for incorrect answers selected with partial of full confidence discourages guessing. (Bruno ’920, para. 44). 8. Bruno ’920 teaches that a student may be surprised when informed of incorrect responses; and that “[sjurprise creates a teachable moment, where the mind is more receptive to feedback and new information.” (Bruno ’920, para. 58). In addition, Bruno ’920 teaches that “it is important to provide specific learning materials, immediately, when the learner is ready for them.” (Bruno ’920, para. 60). Such learning materials 9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Appeal 2016-008451 Application 13/216,017 may include explanations for the correct answers, as well as links to Internet websites. (See Bruno ’920, para. 61 & Fig. 4). 9. Bruno ’920 teaches allowing students to retake an assessment as part of the learning process. (See Bruno ’920, para. 63). Furthermore: Questions are developed in a database in which there is a certain set of questions to cover a subject area. To provide true knowledge acquisition and testing of the material, a certain number of questions are presented each time rather than the full bank of questions. This allows the individuals to develop and improve with their understanding of the material over time. (Bruno ’920, para. 64). 10. Bruno ’920 suggests that the same ideas regarding confidence- based testing may be applied to surveys as well as knowledge assessments. (See Bruno ’920, para. 74). Antoniak 11. Antoniak describes a knowledge testing game in which a question is displayed on a computer screen 61, and the player seeks to place each of multiple answers in an order responsive to the displayed question. For each such question, the game displays multiple blocks, each block corresponding to an answer. The player uses a computer pointer selecting input device, such as a “mouse” 62, to move each block to a position on the computer screen corresponding to its position in the correct order. (See Antoniak, col. 4,1. 47 - col. 5,1. 5). Kerfoot 12. Kerfoot describes testing methods for use in “Spaced Education.” (See generally Kerfoot, paras. 5 & 6). For present purposes, 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 “Spaced Education” contemplates electronically delivering test questions or educational materials, related to a particular concept or set of concepts, at spaced intervals. Each time the student is tested, the student’s answers are recorded, and a new level of difficulty, test delivery interval and content area are calculated. The next test delivered to the student reflects the level of difficulty, delivery interval and content area derived from the student’s answers to the previous test. (See Kerfoot, para. 9). 13. Kerfoot teaches that the responses to each test are graded automatically by a central computer server. After the test is completed, the student may be provided with explanations of the correct answers to the test questions. “[FJurther supplementary educational materials, hyperlinks to other educational materials, and additional feedback regarding the learner’s performance on the spaced education program may also be sent to the learner along with the explanations.” (Kerfoot, para. 47). 14. In addition, Kerfoot’s system calculates a content-area proficiency factor based on the student’s responses to the test questions. If the student’s content-area proficiency factor indicates mastery of the concept or set of concepts assessed in a test, the system may deliver questions related to a different content area in the next test. (See Kerfoot, para. 51). Altenhofen 15. Altenhofen teaches organizing course materials for an electronically delivered course in terms of hierarchically-arranged structural items including a course 110, sub-courses 120, learning units 130 and knowledge items 140. (See Altenhofen, para. 25). “Structural items also may be tagged with metadata that is used to support adaptive delivery, 11 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 reusability, and search/retrieval of content associated with the structural element.” (Altenhofen, para. 35). ANALYSIS First Issue Claim 11 recites a “service-oriented computer structure comprising a multi-tiered services structure adapted to perform a method of knowledge assessment.” The method of knowledge assessment the computer structure is adapted to perform includes the step of “creating, through an interface to a content management server, a knowledge assessment application.” In addition, claim 11 recites that “one or more learning objects are provided to the learner through the knowledge assessment application on the display device in real-time as soon as a determination is made that the learner requires more learning material.” The prior art cited by the Examiner teaches both the step of “creating, through an interface to a content management server, a knowledge assessment application;” and wherein “one or more learning objects are provided to the learner through the knowledge assessment application on the display device in real time as soon as a determination is made that the learner requires more learning material.” As to the first of these limitations, Etesse teaches providing an interface in the form of an instructor control panel, permitting an instructor to access assessment tools. The assessment tools permit the instructor to create a knowledge assessment such as a test, quiz or survey. (See FF 2). The Appellants argue that Etesse fails to describe the creation of a “knowledge assessment application.” (See App. Br. 17 & 18; Reply Br. 16). 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 The term “application” is sufficiently broad to encompass any computer program that performs a particular task. (See American Heritage Dictionary of the English Language (Houghton Mifflin Harcourt Publ’g Co., 5th ed. 2016), reproduced at https://www.theffeedictionary .com/Computer+application (last visited Feb. 20, 2018)). Typically, application programs are distinguished from control software, such as operating systems. (See Encyclopedia, https://www.pcmag.com /encyclopedia/term/37892/application (last visited Feb. 20, 2018)). The Appellants have not pointed to any formal definition or clear disclaimer in the Specification that would narrow the interpretation of the term. The Examiner correctly concludes that claim 11 does not limit the format of the “knowledge assessment application.” (See Examiner’s Answer, mailed July 7, 2016 (“Ans.”), at 9). The assessment tools described by Etesse permit an instructor to identify the questions to be included in a quiz or test. In addition, the assessment tools permit the instructor to set parameters instructing the system to perform particular tasks, such as showing the students whether each question in an assessment was answered correctly or incorrectly; showing the student the correct answer, along with “feedback” entered by the instructor, for each question; permitting students to repeat the knowledge assessment; timing the assessment; and requiring entry of a password to access the assessment. (See FF 3). The Examiner correctly finds that Etesse teaches “creating ... a knowledge assessment application” as recited in claim 11. The Appellants also argue that Etesse fails to describe creating the knowledge assessment application “through an interface to a content management server.” (See App. Br. 17 & 18). The Examiner correctly 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 points out that the system described by Etesse includes one or more servers that communicate, directly or indirectly, with the instructor workstations. (See FF 1; Ans. 9). Furthermore, the instructors communicate with the servers in the system, once again either directly or indirectly, through the instructor control panels. (See FF 2; Ans. 9 & 10). Although Etesse does not identify a precise server as a content management server, the instructor control panels would serve as interfaces to the system, in general; and, in particular, to the server within the system that corresponds to the content management server. Turning to the second limitation of claim 11 argued by the Appellants, Etesse, Bruno ’920 and Kerfoot together teach wherein “one or more learning objects are provided to the learner through the knowledge assessment application on the display device in real time as soon as a determination is made that the learner requires more learning material.” Etesse taught providing automatic, real-time grading of tests and quizzes through an automatic grading functionality (see FF 5); as well as providing feedback after tests or quizzes are completed (see FF 3). Bruno ’920 taught that the moment when a student was informed of mistakes in his or her test answers could be a “teachable moment;” and that it was important to provide the student with learning materials, immediately, when the student was ready for the materials. (See FF 8). Both Bruno ’920 and Kerfoot taught providing additional materials, including links to Internet websites, when reporting test or quiz grades. (See FF 8 & 13). These teachings, taken together, would have provided one familiar with the teachings of Etesse, Bruno ’920 and Kerfoot reason to provide one or more learning objects to the student through the knowledge assessment application on the student 14 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 workstation, in real time, as soon as a determination was made that the student required more learning material. (SeeAns. 11). We sustain the rejection of claims 11,13 and 21 under § 103(a) as being unpatentable over Etesse, Bruno ’920, Antoniak and Kerfoot. In addition, we sustain the rejection of claim 12 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot and Bruno article; as well as the rejection of claims 19 and 20 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot and Bruno ’592. Second Issue With respect to independent claim 27, the Appellants argue that “Etesse itself does not disclose real-time adaptive testing and provision of learning materials as a function on its servers.” (See App. Br. 19). In addition, the Appellants argue that one of ordinary skill in the art would not have had reason to modify the system described by Etesse in view of the teachings of Bruno ’920 and Kerfoot to perform these functions. (See App. Br. 19). Etesse taught a system having assessment tools for creating knowledge assessment applications including multiple choice or matching questions. (See FF 3). Bruno ’920 criticized knowledge assessments with one-dimensional, multiple choice questions for encouraging guessing; and suggested addressing the problem through the use of two-dimensional, multiple choice questions, instead. (See FF 6). This would have provided one of ordinary skill in the art reason to modify the assessment tools described by Etesse to permit instructors to create knowledge assessment applications, including two-dimensional, multiple choice questions, and to additionally modify the system described by Etesse to provide automatic 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Appeal 2016-008451 Application 13/216,017 grading of two-dimensional, multiple choice questions according to the teachings of Bruno’920. (SeeAns. 11). In addition, one familiar with the teachings of Etesse, Bruno ’920 and Kerfoot would have had reason to provide one or more learning objects, including textual feedback from the instructor and links to Internet websites, in response to a determination that the student required more learning material about a particular topic. The same findings and reasoning implying that a computer system adapted to provide “one or more learning objects . . . to the learner through the knowledge assessment application on the display device in real time as soon as a determination is made that the learner requires more learning material,” as recited in claim 11, imply the obviousness of a system satisfying this limitation, as well. (See also Ans. 11 & 12). We sustain the rejection of claims 27-31 and 36 under § 103(a) as being unpatentable over Etesse, Bruno ’920 and Kerfoot. In addition, we sustain the rejection of claim 32 under § 103(a) as being unpatentable over Etesse, Bruno ’920, Kerfoot and Bruno article; as well as the rejection of claims 34 and 35 as being unpatentable over Etesse, Bruno ’920, Kerfoot and Bruno ’592. Third Issue Claim 37 recites a computer database system structure including a database of learning materials: comprising a module library and a learning object library, the learning object library comprising a plurality of learning objects each grouped into shadow groups, each of the plurality of learning objects comprising, 16 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 metadata corresponding to the learning object, assessment data corresponding to the learning object, and learning data corresponding to the learning object. The Appellants argue that Altenhofen fails to describe “assessment data corresponding to the learning object.” (See App. Br. 20; Reply Br. 17 & 18). The Appellants correctly interpret the term “assessment data corresponding to the learning object” as “information about test questions themselves, such as ‘an introduction, the questions, a correct answer, and wrong answers.’” (Reply Br. 17 & 18, citing Spec., para. 139; see also Fig. 18). That correct interpretation, however, does not further aid the Appellants with respect to the cited references. Both Etesse and Kerfoot teach maintaining pools or databases of questions and multiple choice answers linked by subject matter. (See FF 4 & 9). Altenhofen would have suggested tagging the questions and multiple choice answers with metadata “so as to provide an organizational data structure which allows for identification of the properties associated with each learning object” (Final Act. 10), thereby facilitating search or re-use of the questions (see FF 15). Likewise, it would have been obvious to associate the learning materials provided to the student in response to a determination, based on test results, that the student required more learning material, in order to facilitate automatic, real-time delivery of the learning materials once the test is graded. The Appellants’ argument is not persuasive. We sustain the rejection of claims 37^10 under § 103(a) as being unpatentable over Etesse, Bruno ’920, Kerfoot and Altenhofen. In addition, we sustain the rejection of claim 41 under § 103(a) as being unpatentable over Etesse, Bruno ’920, Kerfoot, 17 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 Altenhofen and Bruno article; as well as the rejection of claims 14-16 and 18 as being unpatentable over Etesse, Bruno ’920, Antoniak, Kerfoot and Altenhofen. Fourth Issue The Supreme Court has established a two-step analysis for determining whether the subject matter of a claim is eligible for patent protection. First, one must determine whether the claim is “directed to one of [the] patent-ineligible concepts,” such as an abstract idea. Alice Corp. v. CLS Bank Int’l, 134 S.Ct. 2347, 2355 (2014). Second, if so, one must determine if the remainder of the claim recites an “inventive concept,” such that the claim as a whole recites a specific application of the patent- ineligible concept. Id. at 2357 & 2358. Claim 11 Independent claim 11 is properly analyzed as a method claim. The Appellants point out, on page 16 of the Appeal Brief, that the preamble of independent claim 11 recites a “service-oriented computer structure comprising a multi-tiered services structure.” In addition, the Appellants point out that the body of claim 11 recites hardware components. (See App. Br. 16). Nevertheless, in assessing a rejection for ineligible subject matter under § 101, we look not to the name or intended use assigned to the claimed subject matter in the preamble, but to the nature of the claimed subject matter as a whole, to determine whether the claim falls within the “abstract idea” exception. See CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1374 (Fed. Cir. 2011) (“Regardless of what statutory 18 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Appeal 2016-008451 Application 13/216,017 category (‘process, machine, manufacture, or composition of matter,’ 35 U.S.C. § 101) a claim’s language is crafted to literally invoke, we look to the underlying invention for patent-eligibility purposes”). Because the body of claim 11 recites process steps that the structure recited in the preamble is adapted to perform, the claim is properly addressed as a method claim for purposes of determining patent eligibility. Turning to the first step of the analysis, neither the Supreme Court, nor our reviewing court, has defined the term “abstract.” See, e.g., Alice at 2357; Research Corp. Techs., Inc. v. Microsoft Corp., 627 F.3d 859, 868 (Fed. Cir. 2010). Instead, the contours of what constitutes an “abstract idea” have developed on a case-by-case basis. The Appellants argue that the Examiner has not provided case law support for the conclusion that claim 11 is directed to an abstract idea. (See generally App. Br. 7-11). The Examiner correctly characterizes claim 11 as directed to “comparing new information (i.e. the learner’s test results) to stored information (i.e. the test answers) and then using rules to identify options for the learner (i.e. provide additional learning materials).” (Ans. 4). The Examiner also correctly analogizes the subject matter of appealed claim 11 to that at issue in SmartGene, Inc. v. Advanced Bio. Labs., SA, 555 F. App’x 950 (Fed. Cir. 2014). In SmartGene, a patent holder sued to enforce a claim to a system for computerized meal planning: 1. A method for guiding the selection of a therapeutic treatment regimen for a patient with a known disease or medical condition, said method comprising: (a) providing patient information to a computing device comprising: 19 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Appeal 2016-008451 Application 13/216,017 a first knowledge base comprising a plurality of different therapeutic treatment regimens for said disease or medical condition; a second knowledge base comprising a plurality of expert rules for evaluating and selecting a therapeutic treatment regimen for said disease or medical condition; a third knowledge base comprising advisory information useful for the treatment of a patient with different constituents of said different therapeutic treatment regimens; and (b) generating in said computing device a ranked listing of available therapeutic treatment regimens for said patient; and (c) generating in said computing device advisory information for one or more therapeutic treatment regimens in said ranked listing based on said patient information and said expert rules. Id. at 951-52. Our reviewing court held that the claim was directed to an abstract idea, because the claim did not recite an improvement to computer technology; and because it did not “purport to identify any steps beyond those which doctors routinely and consciously perform” when prescribing a treatment regimen. Id. at 955. The method at issue in SmartGene included three recited steps, each such step identified by a letter. Step (a), “providing patient information to a computing device,” is analogous to the step of “receiving via the display device the learner’s selected answers to the multiple-choice questions,” as recited in appealed claim 11, in that both are data-gathering steps. Step (b), “generating in said computing device a ranked listing of available therapeutic treatment regimens for said patient,” is analogous to the steps of “scoring the assessment” and “determining when a learner requires more learning material about a particular topic,” as recited in appealed claim 11, 20 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 in that both steps process the data previously gathered. Step (c), “generating in said computing device advisory information for one or more therapeutic treatment regimens in said ranked listing based on said patient information and said expert rules,” is analogous to the step of “providing to the learner, from the content management server, and in response to the determining, one or more learning objects by assembling textual content and one or more of: digital images, videos, and links to internet websites,” as recited in appealed claim 11, in that both steps select information stored in a memory (that is, in a knowledge base, as in the claim at issue in SmartGene, or a content management server, in appealed claim 11). The analogy indicates that the steps of “receiving via the display device the learner’s selected answers to the multiple-choice questions,” “scoring the assessment,” “determining when a learner requires more learning material about a particular topic” and “providing to the learner, from the content management server, and in response to the determining, one or more learning objects by assembling textual content and one or more of: digital images, videos, and links to internet websites,” in combination, are directed to an abstract idea. The Appellants argue that the claim at issue in SmartGene cannot be analogized to appealed claim 11. According to the Appellants, this is because appealed claim 11 recites steps carried out using a content management server; a learning server; a registration and data analytics server; a display device; and a communication network, while the claim at issue in SmartGene recites a method carried out using a computing device. (See Reply Br. 9). This argument is belied by the Appellants’ Specification. We review claim 11 as a computer-implemented method. “In addressing the first step of the section 101 inquiry, as applied to a computer- 21 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Appeal 2016-008451 Application 13/216,017 implemented invention, it is often helpful to ask whether the claims are directed to ‘an improvement in the functioning of a computer,’ or merely ‘adding conventional computer components to well-known business practices.’” Affinity Labs of Tex., LLC v. Amazon.com Inc., 838 F.3d 1266, 1270 (Fed. Cir. 2016) (quoting Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1338 (Fed. Cir. 2016)). Here, the objects of the claimed subject matter, as set forth in paragraphs 6-9 of the Specification, relate to improving the accuracy with which the knowledge of a student is assessed; the reusability of learning objects; and an integrated reporting capability. Paragraph 51 sums up the disclosure of the first fifty paragraphs: “the system substantially facilitates the construction of non-one-dimensional queries or the conversion of traditional one-dimensional queries into multi dimensional queries,” but does not necessarily improve the efficiency or performance of the system as a computer system. Although Figures 2 and 3, at least at first glance, appear to depict computer structure in schematic form, the Specification fails to describe how the method steps recited in claim 11 might improve the functioning of that network. (See, e.g., Spec., paras. 10, 26-28, 33, 34, 39 and 41). Likewise, paragraph 109 of the Specification says, in general terms, that “the system described herein may be implemented in a variety of stand-alone or networked architectures, including the use of various database and user interface structures,” but does not describe how the recited method steps might improve the performance of the system. Finally, paragraphs 159-164 describe various components that a computer system might possess, but does not describe how these components might be adapted to perform the recited method steps. 22 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Appeal 2016-008451 Application 13/216,017 Considering the disclosure as a whole, the Specification does not describe how the recited method steps constitute an improvement in the functioning of a computer, rather than computerized implementation of the abstract idea of “comparing new information (i.e. the learner’s test results) to stored information (i.e. the test answers) and then using rules to identify options for the learner (i.e. provide additional learning materials).” (Ans. 4). The method steps leading up to, and including, the steps of “administering an assessment comprising presenting to the learner via the display device the plurality of multiple-choice questions and the two- dimensional answers,. . . [and] scoring the assessment by assigning a knowledge state designation to a shadow group that two or more answered multiple-choice questions are grouped into,” may be viewed either as incidental data gathering steps or, as characterized by the Examiner in the Final Office Action, as themselves being directed to the abstract idea of a method of knowledge assessment. (Final Act. 2). In this regard, OIP Technologies, Inc. v. Amazon.com, Inc., 788 F.3d 1359 (Fed. Cir. 2015), is instructive. In OIP Techs., a patent holder sued to enforce a claim to a method for offer-based price optimization: 1. A method of pricing a product for sale, the method comprising: testing each price of a plurality of prices by sending a first set of electronic messages over a network to devices; wherein said electronic messages include offers of said product; wherein said offers are to be presented to potential customers of said product to allow said potential 23 Appeal 2016-008451 Application 13/216,017 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 customers to purchase said product for the prices included in said offers; wherein the devices are programmed to communicate offer terms, including the prices contained in the messages received by the devices; wherein the devices are programmed to receive offers for the product based on the offer terms; wherein the devices are not configured to fulfill orders by providing the product; wherein each price of said plurality of prices is used in the offer associated with at least one electronic message in said first set of electronic messages; gathering, within a machine-readable medium, statistics generated during said testing about how the potential customers responded to the offers, wherein the statistics include number of sales of the product made at each of the plurality of prices; using a computerized system to read said statistics from said machine-readable medium and to automatically determine, based on said statistics, an estimated outcome of using each of the plurality of prices for the product; selecting a price at which to sell said product based on the estimated outcome determined by said computerized system; and sending a second set of electronic messages over the network, wherein the second set of electronic messages include offers, to be presented to potential customers, of said product at said selected price. OIP Techs, at 1361. The court held that the claim was directed to an abstract idea due to the similarity of the recited offer-based price optimization method to the subject matter of other claims held previously to be directed to abstract ideas. See OIP Techs, at 1362-63. 24 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 OIP Technologies indicates that administering a test consisting of one or more questions, and quantifying the user’s responses, is an abstract idea. The offer-based price optimization method recited in the claim at issue in OIP Technologies included the step of “testing each price of a plurality of prices by sending a first set of electronic messages over a network to devices.” The “electronic messages include[d] offers of [a] product.” In this context, each such offer confronted potential customers with a single dimensional, binary-choice question, namely, whether to accept, or not accept, the offer at a given price. The questions implicit in the offers were presented to the potential customers via devices, in the sense that “the devices [were] programmed to receive offers for the product based on the offer terms” via the electronic messages; and “the devices [were] programmed to communicate offer terms, including the prices contained in the messages received by the devices.” Responses to the offers were “scored” in the sense that statistics related to the number of offers accepted by the recipients were processed “to automatically determine, based on said statistics, an estimated outcome of using each of the plurality of prices for the product.” Admittedly, the offers presented to potential customers in the electronic messages recited in the claim at issue in OIP Technologies assessed potential customers’ demand for a product, rather than students’ knowledge; were not multiple-choice questions; did not solicit two- dimensional answers; and were not grouped into shadow groupings. Nevertheless, one of ordinary skill in the art would have understood that the same idea underlies the conduct of both knowledge assessments and surveys. (Cf. FF 10 (Bruno ’920 teaches techniques useful for either a knowledge 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Appeal 2016-008451 Application 13/216,017 assessment or a survey)). The holding of OIP Technologies implies that the idea is abstract.2 It remains to address the second step of the analysis. The Appellants’ arguments regarding the recitation of hardware components in the body of claim 11 were addressed earlier. The Appellants’ argument on page 15 of the Reply Brief, purporting to demonstrate that claim 11 is directed to a technological process, is unpersuasive. The purported reduction in the need for permanent storage in memory on the display device, and the inherent saving of network bandwidth, do not persuade us that the claimed subject matter as a whole is a technological improvement sufficient to impart patent eligibility. The purported reduction in the need for permanent storage in memory on the display device, for example, is a foreseeable efficiency resulting from implementation on a network rather than as a stand-alone application. The purported inherent saving of network bandwidth is merely a by-product of implementing the two-dimensional, multiple choice questions in shadow groups to facilitate adaptive repetition (see Spec., para. 74), rather than a significant improvement in network performance. Therefore, we sustain the rejection of claims 11-16 and 18-21 under § 101 as being directed to ineligible subject matter. 2 We note that it is not necessary for us to find that the method steps recited in the body of claim 11 could be performed by hand in order to conclude, pursuant to the holding of OIP Technologies, that the claim is directed to an abstract idea. 26 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Appeal 2016-008451 Application 13/216,017 Claims 27 and 37 Independent claims 27 and 37 may be disposed of quickly. The only separate discussion of the rejection of claims 27 and 37 is on page 16 of the Appeal Brief. The Appellants point out that the preamble of claims 27 recites a “services-oriented system for knowledge assessment and learning;” and that of claim 37 recites a “computer database system structure configured to deliver to a learner at a client terminal a plurality of multiple- choice questions and two-dimensional answers, and a plurality of learning objects.” Nevertheless, both claims are properly characterized as being directed to the abstract idea of “comparing new information (i.e. the learner’s test results) to stored information (i.e. the test answers) and then using rules to identify options for the learner (i.e. provide additional learning materials)” (Ans. 4); as well as the abstract idea of a method of knowledge assessment” (Final Act. 2). The recitation of various servers in the body of claim 27, or the recitation of servers and a database of learning materials in the body of claim 37, does not constitute a sufficient “something more” such that either claim, as a whole, is patent eligible. Neither the claims themselves, nor the Specification, nor the Appellants’ argument on page 16 of the Appeal Brief, sufficiently detail how the combination of hardware recited in claim 27 or claim 37 constitutes an improvement in the functioning of a computer rather than merely the implementation of knowledge assessment, as well as the processing of information according to rules to identify options for the learner. Therefore, we sustain the rejection of claims 27—41 under § 101 as being directed to ineligible subject matter. 27 1 2 3 4 5 6 7 8 9 10 11 Appeal 2016-008451 Application 13/216,017 DECISION We sustain all grounds of rejection entered by the Examiner. We AFFIRM the Examiner’s decision rejecting claims 11-16, 18-21 and 27—41. No time period for taking any subsequent action in connection with this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R. § 1.136(a)(l)(iv). AFFIRMED 28 Copy with citationCopy as parenthetical citation