Byron, Donna K. et al.

Patent Trials and Appeals Board

Dec 6, 2019

14139456 - (D) (P.T.A.B. Dec. 6, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/139,456 12/23/2013 Donna K. Byron AUS920130284US1 4129
50170 7590 12/06/2019
IBM CORP. (WIP)
c/o WALDER INTELLECTUAL PROPERTY LAW, P.C.
1701 N. COLLINS BLVD.
SUITE 2100
RICHARDSON, TX 75080
EXAMINER
SAINT-VIL, EDDY
ART UNIT PAPER NUMBER
3715
MAIL DATE DELIVERY MODE
12/06/2019 PAPER
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte DONNA K. BYRON, SUZANNE L. ESTRADA,
ALEXANDER PIKOVSKY, and TIMOTHY P. WINKLER
____________

Appeal 2018-000964
Application 14/139,456
Technology Center 3700
____________

Before JEAN R. HOMERE, CAROLYN D. THOMAS, and
IRVIN E. BRANCH, Administrative Patent Judges.

THOMAS, Administrative Patent Judge.

DECISION ON APPEAL

Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the
Examiner’s decision to reject claims 1–20. We have jurisdiction over the
appeal under 35 U.S.C. § 6(b).
We REVERSE.

1 We use the word “Appellant” to refer to “applicant” as defined in
37 C.F.R. § 1.42. Appellant identifies the real party in interest as
International Business Machines Corporation. Appeal Br. 2.
Appeal 2018-000964
Application 14/139,456

2
The present invention relates generally to automatically generating a
set of test questions. See Abstr.
Independent claim 1, reproduced below, is representative of the
appealed claims:
1. A method, in a data processing system comprising
a processor and a memory, for automatically generating a set of
test questions for use in generating a test to be administered to
human test takers, the method comprising:
ingesting, by test question answer extraction
engine logic executing on one or more processors in the data
processing system, a document from a corpus of documents to
select a potential test question answer from the document;
generating, by test question generation engine
logic executing on the one or more processors in the data
processing system, a candidate test question based on the
potential test question answer;
evaluating, by a Question and Answer (QA)
system, the candidate test question to generate a result
indicative of whether the candidate test question should be
maintained for test generation;
storing, by the data processing system, the
candidate test question in the set of test questions in response to
a result indicating that the candidate test question should be
maintained for test generation; and
transmitting, by test generation engine logic
executing on the one or more processors in the data processing
system, to at least one other computing device, the set of test
questions to administer the test to at least one human test taker
via the at least one other computing device.

Appellant appeals the following rejection:
Claims 1–20 are rejected under 35 U.S.C. § 101 because the claimed
invention is directed to patent-ineligible subject matter. Final Act. 2–15
We review the appealed rejections for error based upon the issues
identified by Appellant, and in light of the arguments and evidence produced
Appeal 2018-000964
Application 14/139,456

3
thereon. Ex parte Frye, 94 USPQ2d 1072, 1075 (BPAI 2010) (precedential).
ANALYSIS
Rejection under § 101
An invention is patent-eligible if it claims a “new and useful process,
machine, manufacture, or composition of matter.” 35 U.S.C. § 101.
However, the Supreme Court has long interpreted 35 U.S.C. § 101 to include
implicit exceptions: “[l]aws of nature, natural phenomena, and abstract
ideas” are not patentable. E.g., Alice Corp. v. CLS Bank Int’l, 573 U.S. 208,
216 (2014).
In determining whether a claim falls within an excluded category, we
are guided by the Supreme Court’s two-step framework, described in Mayo
and Alice. Id. at 217–18 (citing Mayo Collaborative Servs. v. Prometheus
Labs., Inc., 566 U.S. 66, 75–77 (2012)). In accordance with the framework,
we first determine what concept the claim is “directed to.” See Alice, 573
U.S. at 219 (“On their face, the claims before us are drawn to the concept of
intermediated settlement, i.e., the use of a third party to mitigate settlement
risk.”); see also Bilski v. Kappos, 561 U.S. 593, 611 (2010) (“Claims 1 and 4
in petitioners’ application explain the basic concept of hedging, or protecting
against risk.”). For example, concepts determined to be abstract ideas, and
thus patent ineligible, include certain methods of organizing human activity,
such as fundamental economic practices (Alice, 573 U.S. at 219–20; Bilski,
561 U.S. at 611); mathematical formulas (Parker v. Flook, 437 U.S. 584,
594–95 (1978)); and mental processes (Gottschalk v. Benson, 409 U.S. 63,
69 (1972)).
Recently, the USPTO published revised guidance on the application
of 35 U.S.C. § 101. USPTO’s 2019 Revised Patent Subject Matter
Appeal 2018-000964
Application 14/139,456

4
Eligibility Guidance, 84 Fed. Reg. 50 (Jan. 7, 2019) (“Revised Guidance”).
Under the Revised Guidance “Step 2A,” the office first looks to whether the
claim recites:
(1) any judicial exceptions, including certain groupings of abstract
ideas (i.e., mathematical concepts, certain methods of organizing human
activity such as a fundamental economic practice, or mental processes); and
(2) additional elements that integrate the judicial exception into a
practical application (see MPEP § 2106.05(a)-(c), (e)-(h)). 84 Fed. Reg. at
51–52, 55.
Only if a claim (1) recites a judicial exception and (2) does not
integrate that exception into a practical application, does the Office then
(pursuant to the Revised Guidance “Step 2B”) look to whether the claim:
(3) adds a specific limitation beyond the judicial exception that is not
“well-understood, routine, conventional” in the field (see MPEP
§ 2106.05(d)); or
(4) simply appends well-understood, routine, conventional activities
previously known to the industry, specified at a high level of generality, to
the judicial exception. 84 Fed. Reg. at 56.
Step 2A, Prong 1 (Does the Claim Recite a Judicial Exception?)
With respect to independent method claim 1, and similarly, computer
readable storage medium claim 11 and apparatus claim 20, the Examiner
determines that the claims are directed to “‘An Idea Of Itself,’ . . . such as
collecting and comparing known information and/or obtaining and
comparing intangible data, and/or comparing new and stored information
and using rules to identify options” (Final Act. 3–4), which we conclude are
mental processes, which is a type of abstract idea.
Appeal 2018-000964
Application 14/139,456

5
Although the previously recognized category of “an idea of itself” is
not one of the currently recognized categories, it is sufficient for the
purposes of the present appeal that the claimed concepts reasonably can be
characterized as falling within the still-recognized category of mental
processes. See, e.g., MPEP § 2106.04(a)(2)(III):
The courts have used the phrase ‘an idea ‘of itself’” to describe
an idea standing alone such as an instantiated concept, plan or
scheme, as well as a mental process (thinking) that “can be
performed in the human mind, or by a human using a pen and
paper.” CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d
1366, 1372 (Fed. Cir. 2011).
This is true even if the claim recites that a generic computer
component performs the acts. See, e.g., Versata Dev. Grp., Inc. v. SAP Am.,
Inc., 793 F.3d 1306, 1335 (Fed. Cir. 2015) (“Courts have examined claims
that required the use of a computer and still found that the underlying,
patent-ineligible invention could be performed via pen and paper or in a
person’s mind.”); see also 2019 Eligibility Guidance 84 Fed. Reg. at 52 n.14
(“If a claim, under its broadest reasonable interpretation, covers performance
in the mind but for the recitation of generic computer components, then it is
still in the mental processes category unless the claim cannot practically be
performed in the mind.”).
Specifically, Claim 1 recites at least the following limitations:
(1) “ingesting . . . a document . . . to select a potential test question answer,”
(2) “generating . . . a candidate test question,” (3) “evaluating . . . the
candidate test question to generate a result,” (4) “storing . . . the candidate
test question,” and (5) “transmitting . . . the set of test questions.” Claim 1.
We agree with the Examiner (see Final Act. 4) that these limitations,
under their broadest reasonable interpretation, encompass acts people can
Appeal 2018-000964
Application 14/139,456

6
perform using their minds or pen and paper because people can perform the
“ingesting . . . a document” and “generating . . . a candidate test question”
steps by simply looking at a document and observing answers from the
document and then use pen and paper to write down candidate test questions.
People can also perform the “evaluating” step by looking at the candidate
test questions and making a judgment as to whether the question should be
maintained. Finally, people can “store” the test question by merely
remembering the questions and/or writing them down, and “transmit” the
test questions by merely passing a piece of paper to one another with the
questions written thereon. As noted by the Examiner, it is common for
teachers/educators to perform such acts. See id.
Appellant’s Specification discloses:
With the increased usage of computing networks, such as
the Internet, humans are currently inundated and overwhelmed
with the amount of information available to them from various
structured and unstructured sources. However, information
gaps abound as users try to piece together what they can find
that they believe to be relevant during searches for information
on various subjects. To assist with such searches, recent
research has been directed to generating Question and Answer
(QA) systems which may take an input question, analyze it, and
return results indicative of the most probable answer to the
input question. QA systems provide automated mechanisms for
searching through large sets of sources of content, e.g.,
electronic documents, and analyze them with regard to an input
question to determine an answer to the question and a
confidence measure as to how accurate an answer is for
answering the input question.
Spec. ¶ 2. In other words, Appellant’s invention is concerned with
automating searching through large sets of content for potential test question
Appeal 2018-000964
Application 14/139,456

7
answers, analyzing such answers, and providing candidate test questions
based thereon.
We agree with the Examiner that the rejected claims “involve data
collection, data comparison[,] and using rules to identify options.” Final
Act. 4. However, information collection and analysis, including when
limited to particular content, is within the realm of abstract ideas. See, e.g.,
Elec. Power Grp. LLC v. Alstom S.A., 830 F.3d 1350, 1353 (Fed. Cir. 2016)
(holding that “collecting information, analyzing it, and displaying certain
results of the collection and analysis” are “a familiar class of claims
‘directed to’ a patent-ineligible concept.”). See also, e.g., Content
Extraction & Transmission LLC v. Wells Fargo Bank, Nat’l Ass’n, 776 F.3d
1343, 1347 (Fed. Cir. 2014) (finding claims directed to collecting data,
recognizing certain data within the collected data set, and storing the
recognized data drawn to an abstract idea and noting that “humans have
always performed these functions”). Thus, Appellant’s claims are
comparable to other claims that recite abstract ideas.
Appellant challenges the Examiner’s determinations on the grounds
that “[t]he FOA does not identify what this alleged abstract idea is” (Appeal
Br. 5), “[t]he FOA merely presents an unsubstantiated conclusion” (id.), and
“[t]he FOA fails to even map any of the features of the claims to specific
elements of an alleged (and unidentified) abstract idea.” Id. Appellant also
contends that they “are not claiming some unspecified abstract idea that can
be performed by a human being using pen and paper as alleged by the FOA”
(id. at 6), and “the FOA is not examining that actual claimed invention, but
rather an oversimplification and generalization.” Id. at 7.
Appeal 2018-000964
Application 14/139,456

8
Contrary to Appellant’s contentions, the Examiner clearly noted the
claims as being similar to concepts such as “collecting and comparing
known information” and that “[i]t is common practice, using pen and paper, .
. . [to] generate test question(s).” Final Act. 3–4. As demonstrated supra,
such a characterization falls squarely under our “mental processes” category
of abstract concepts.
Furthermore, it is plainly evident that the Examiner considered all the
limitations in the claim and the Examiner’s analysis is indeed tethered to
actual claim language (see Final Act. 3–4). We determine the Examiner
properly and reasonably found that claim 1 recites steps that can be
performed by “using pen and paper” (id. at 4) and that that is an abstract
idea, i.e., mental processes. The Examiner simply condensed the abstract
concept. Apple, Inc. v. Ameranth, Inc., 842 F.3d 1229, 1240–41 (Fed. Cir.
2016) (“An abstract idea can generally be described at different levels of
abstraction. As the Board has done, the claimed abstract idea could be
described as generating menus on a computer, or generating a second menu
from a first menu and sending the second menu to another location. It could
be described in other ways, including, as indicated in the specification,
taking orders from restaurant customers on a computer.”)
Whichever way the concept is described, we do not see that Appellant
has adequately shown that claim 1 does not recite an abstract idea. Just
because the claim includes more words than the phrase the Examiner uses as
an articulation of the abstract idea is an insufficient reason to persuasively
argue that claim 1 does not recite an abstract idea. Further, the identified
abstract idea need not encompass every claim limitation. Otherwise, there
would be no need for step two of the Alice inquiry. Alice, 573 U.S. at 218.
Appeal 2018-000964
Application 14/139,456

9
Therefore, for at least the aforementioned reasons, we agree with the
Examiner that claim 1 recites an abstract idea, which we conclude is a
“mental processes.”
Step 2A—Prong 2 (integration into Practical Application) 2
Under the Revised Guidance, we now must determine if additional
elements in the claims integrate the judicial exception into a practical
application (see MPEP § 2106.05(a)–(c), (e)–(h)).
Here, we have discern additional combination of elements recited in
Appellant’s representative claim 1 that integrates the judicial exception into
a practical application. See Revised Guidance, 84 Fed. Reg. at 54–55
(“Prong 2”). For example, Appellant’s claimed additional elements (e.g.,
“data processing system,” “processor,” “memory,” “test question answer
extraction logic,” “test question generation engine logic,” and the “Question
and Answer (QA) system”) do (1) improve the functioning of a computer or
other technology and (2) are applied with a particular machine (beyond a
generic computer). See MPEP § 2106.05(a)–(c), (e)–(h).
Specifically, Appellant contends, and we agree, that “the claimed
invention not only provides a computer system based mechanism for
generating test questions . . . but then also provides a computer system based
mechanism for evaluating the generated test questions” (Appeal Br. 10),
“which includes a Question and Answer (QA) system, for evaluating those
generated test questions.” Id.

2 We acknowledge that some of the considerations at Step 2A, Prong 2,
properly may be evaluated under Step 2 of Alice (Step 2B of the Office
revised guidance). For purposes of maintaining consistent treatment within
the Office, we evaluate them under Step 1 of Alice (Step 2A of the Office
revised guidance). See Revised Guidance, 84 Fed. Reg. at 55 n.25, 27–32.

Appeal 2018-000964
Application 14/139,456

10
Appellant’s Specification discloses:
The illustrative embodiments provide mechanisms for
dynamically generating and validating test questions from a
corpus using a natural language processing system, such as a
Question and Answer (QA) system. The illustrative
embodiments not only generate the test questions that are
candidates to test the knowledge and/or abilities of students
with regard to the subject matter covered by a corpus of
documents, but also provides mechanisms for evaluating these
generated questions to thereby eliminate badly formed
questions that should not be presented to the student or are
unlikely to result in a student providing a correct answer to the
posed test question. A QA system may be augmented to be an
evaluator for synthetically generated questions generated using
the mechanisms of the illustrative embodiments.
Spec. ¶ 15. In other words, Appellant’s Specification describes specific
logic executed by the QA system to evaluate whether candidate test
questions are suitable for test takers and should be maintained, i.e. eliminate
badly formed questions.
Appellant’s Specification also describes problems arising in the
context of static automated mechanisms for generating tests, whereby such
systems are limited to existing sets of questions. Spec. ¶ 17. The
Specification further discusses the need to provide dynamic identification
and validation of test questions from a corpus. Spec. ¶ 1.
Specifically, Appellant’s Specification states:
Current automated mechanisms for generating
tests/quizzes are limited to selecting questions from an already
existing set of test questions that were generated manually and
placed in a database. That is, a set of questions may be
provided along with the instructional material and the
automated aspect of test/quiz generation is limited to simply
selecting a subset of questions from the already provided set of
questions. This requires much effort and expense on the part of
Appeal 2018-000964
Application 14/139,456

11
the provider of the instructional material to generate the set of
questions prior to distributing the instructional material and
question set. Furthermore, such automated mechanisms are
static and limited to the original set of questions provided.
There is no ability to dynamically generate new questions to be
added to the set of questions, or dynamically generate a
completely new set of questions, based on the instructional
material or changes to the instructional material.
Spec. ¶ 17.
The QA system 440 operates in the manner previous
described above but with regard to the candidate test questions
and variants generated by the test question generation engine
430. In so doing, some automatically generated candidate test
questions/variants may be poorly worded or malformed by the
automated mechanisms. As a result, the QA system 440 may
not be able to satisfactorily parse and analyze the candidate test
question/variant. For example, if the question is
ungrammatical, the parsing stage of the QA system 440 will
fail. However, if the question is grammatical, but the question
is phrased in such a way that the QA system is unable to clearly
identify a focus of the question, a lexical answer type of the
question, or any other important characteristics of the question
needed to generate candidate answers, then the question may
fail during the parsing stage of the QA system 440. For
example, the question “How is Darwin’s study about
evolution?” is a grammatical question with poor phrasing that
does not indicate the question’s focus. Such poorly worded or
malformed questions may be immediately discarded from
further consideration by the QA system 440 and will result in
the QA system 440 not performing its further operations for
generating candidate answers for the input question.
Id. ¶ 74.
We determine that the computer-related limitations in claim 1 capture
the improvement discussed in the Specification, and are thereby sufficient to
integrate the judicial exception into a practical application. In particular, the
Appeal 2018-000964
Application 14/139,456

12
Question and Answer (QA) system evaluating the candidate test question, to
generate a result indicative of whether the candidate test question should be
maintained for test generation, integrates the judicial exception into a
practical application. As such, we agree with Appellant that “the present
claims recite an improvement to computer-related technology by improving
the functionality of computer-related technology with regard to the
automatic generation of a set of test questions for use in generating a test to
be administered to human test takers.” Appeal Br. 15, citing Enfish, LLC v.
Microsoft Corp., 822 F.3d 1327, 1335–36 (Fed. Cir. 2016) and McRO, Inc.
v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1313–14 (Fed. Cir. 2016),
see also MPEP § 2106.05(a); 84 Fed. Reg. 55.
For at least the reason noted supra, we determine that although claim
1 recites a judicial exception, the additional combination of elements
integrate that exception into a practical application. Thus, representative
claim 1 is not directed to an abstract idea. Therefore, the Examiner’s
rejection of claims 1–20 under 35 U.S.C. § 101 is reversed.

CONCLUSION
In summary:
Claims
Rejected
35 U.S.C. § Reference(s)/Basis Affirmed Reversed
1–20 101 Eligibility 1–20

REVERSED