Marcel Van Os et al.

Patent Trials and Appeals Board

Oct 2, 2019

14213241 - (D) (P.T.A.B. Oct. 2, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/213,241 03/14/2014 Marcel VAN OS P7392USX1/77870000087401 7195
150004 7590 10/02/2019
DENTONS US LLP - Apple
4655 Executive Dr
Suite 700
San Diego, CA 92121
EXAMINER
GELIN, JEAN ALLAND
ART UNIT PAPER NUMBER
2643
NOTIFICATION DATE DELIVERY MODE
10/02/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
dentons_PAIR@firsttofile.com
patents.us@dentons.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte MARCEL VAN OS, GREGORY B. NOVICK,
and SCOTT M. HERZ
____________

Appeal 2018-002085
Application 14/213,241
Technology Center 2600
____________

Before MAHSHID D. SAADAT, JOHN P. PINKERTON,
and NORMAN H. BEAMER, Administrative Patent Judges.

BEAMER, Administrative Patent Judge.

DECISION ON APPEAL

Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
Final Rejection of claims 1–34. We have jurisdiction over the pending
rejected claims under 35 U.S.C. § 6(b).
We affirm in part.

1 We use the word “Appellant” to refer to “applicant” as defined in
37 C.F.R. § 1.42. Appellant identifies the real party in interest as Apple Inc.
(Appeal Br. 1.)
Appeal 2018-002085
Application 14/213,241

2

CLAIMED SUBJECT MATTER
Appellant’s disclosed and claimed invention is directed to extracting a
proposed action from recorded voicemail, determining associated action
parameters, and presenting a prompt to facilitate undertaking the action
using the parameters. (Abstract.) Claim 1, reproduced below, is illustrative
of the claimed subject matter:
1. A method of operating a digital assistant, comprising:
at a device having one or more processors and
memory:
providing a recorded voice message from a caller
to a user;
automatically, without user input:
extracting from the voice message a
proposed action to be performed by the user;
determining at least one action parameter
for undertaking the action; and
presenting the user with a prompt to
facilitate undertaking the action using the at least
one parameter.

REJECTIONS
The Examiner rejected claims 1, 7, 8, 17, 23, 28, and 32–34 under
35 U.S.C. § 102 as anticipated by Ramamoorthy et al. (US 2007/0286399
A1, pub. Dec. 13, 2007). (Final Act. 2–4.)
The Examiner rejected claims 2, 3, 24, 25, 29, and 30 under 35 U.S.C.
§ 103(a) as being unpatentable over Ramamoorthy and Miller et al. (US
7,251,313 B1, issued July 31, 2007). (Final Act. 5–6.)
Appeal 2018-002085
Application 14/213,241

3

The Examiner rejected claims 4, 6, 16, 26, and 31 under 35 U.S.C.
§ 103(a) as being unpatentable over Ramamoorthy and Cochinwala et al.
(US 2011/0228913 A1, published Sept. 22, 2011). (Final Act. 6–8.)
The Examiner rejected claims 5 and 27 under 35 U.S.C. § 103(a) as
being unpatentable over Ramamoorthy, Cochinwala, and Macwan (US
2009/0092239 A1, published Apr. 9, 2009). (Final Act. 8–9.)
The Examiner rejected claims 9 and 19 under 35 U.S.C. § 103(a) as
being unpatentable over Ramamoorthy and Brown et al. (US 2003/0026392
A1, published Feb. 6, 2003). (Final Act. 9–11.)
The Examiner rejected claim 10 under 35 U.S.C. § 103(a) as being
unpatentable over Ramamoorthy and Trajkovic et al. (US 2003/0048881 A1,
published Mar. 13, 2003). (Final Act. 11–12.)
The Examiner rejected claim 11 under 35 U.S.C. § 103(a) as being
unpatentable over Ramamoorthy and Gamon et al. (US 2013/0007648 A1,
published Jan. 3, 2013). (Ans. 7.)
The Examiner rejected claims 12, 14, 15, and 18 under 35 U.S.C.
§ 103(a) as being unpatentable over Ramamoorthy and Macwan. (Final Act.
12–15.)
The Examiner rejected claim 13 under 35 U.S.C. § 103(a) as being
unpatentable over Ramamoorthy, Macwan, and Upadhyay et al.
(US 8,406,745 B1, issued Mar. 26, 2013). (Final Act. 15–16.)
The Examiner rejected claim 20 under 35 U.S.C. § 103(a) as being
unpatentable over Ramamoorthy, Brown, and Zhang (CN 202035047 U,
published Nov. 9, 2011). (Final Act. 16–17.)
Appeal 2018-002085
Application 14/213,241

4

The Examiner rejected claims 21 and 22 under 35 U.S.C. § 103(a) as
being unpatentable over Ramamoorthy, Brown, and Gamon. (Final Act. 17–
18.)
ISSUES ON APPEAL
Appellant’s arguments present the following issues2:
First Issue: Whether the Examiner erred in finding Ramamoorthy
discloses the independent claim 1 limitation, “extracting from the voice
message a proposed action to be performed by the user,” and the
commensurate limitations of independent claims 19, 23, and 28. (Appeal Br.
14–23.)
Second Issue: Whether the Examiner erred in finding the combination
of Ramamoorthy and Brown taught or suggested the independent claim 19
limitation, “wherein the action is to meet at a specified geographic location,
and the at least one action parameter comprises a name or an address of the
geographic location.” (Appeal Br. 23–25.)
Third Issue: Whether the Examiner erred in finding the combination
of Ramamoorthy and Miller taught or suggested the claim 2 limitation,
“converting the recorded voice message from speech to text; and extracting
the action from the text,” and the commensurate limitations of claims 24 and
29. (Appeal Br. 25–31.)

2 Rather than reiterate the arguments of Appellant and the findings of the
Examiner, we refer to the Appeal Brief (filed June 30, 2017) (“Appeal Br.”);
the Reply Brief (filed Dec. 19, 2017) (“Reply Br.”); the Final Office Action
(mailed June 9, 2016) (“Final Act.”); and the Examiner’s Answer (mailed
Oct. 19, 2017) (“Ans.”) for the respective details.
Appeal 2018-002085
Application 14/213,241

5

Fourth Issue: Whether the Examiner erred in finding the combination
of Ramamoorthy and Brown taught or suggested the claim 9 limitation,
“wherein the at least one parameter includes a specific time period, and the
presenting of the prompt occurs at or before the specific time period,” and in
finding Ramamoorthy, Brown, and Gamon taught or suggested the similar
limitation of claim 22. (Appeal Br. 31–36.)
Fifth Issue: Whether the Examiner erred in finding the combination of
Ramamoorthy and Gamon taught or suggested the claim 11 limitation,
“wherein the action is to check on the user’s availability, and the at least one
parameter is retrieved from a calendar of the user.” (Appeal Br. 36–37;
Reply Br. 19–22.)
Sixth Issue: Whether the Examiner erred in finding the combination of
Ramamoorthy and Macwan taught or suggested the claim 14 limitation,
“wherein the prompt offers to the user an option to store the email address
extracted from the recorded voice message in association with the identified
existing contact in the contact list,” and the commensurate limitation of
claim 18. (Appeal Br. 37–39.)
Seventh Issue: Whether the Examiner erred in finding that
Ramamoorthy disclosed the claim 32 limitation, “wherein extracting the
proposed action comprises determining the proposed action from a plurality
of proposed actions,” and the commensurate limitation of claim 33. (Appeal
Br. 39–41.)
ANALYSIS
First Issue
Each of the independent claims (and by dependency the remaining
claims) require “extracting from the voice message a proposed action to
Appeal 2018-002085
Application 14/213,241

6

be performed by the user.” (Appeal Br. 14–23.) Appellant argues the
Examiner erred in finding Ramamoorthy disclosed this limitation. (Appeal
Br. 14–23.) The Examiner relies on the disclosure in Ramamoorthy of a
system for extracting phone numbers from voicemail messages by detecting
a “trigger word” such as “call me,” which in turn initiates determination of a
phone number following the trigger, and generation of a “macro” allowing a
user to dial the phone number. (Final Act. 2, 19; Ans. 3; Ramamoorthy
Abstr., ¶¶ 5, 33, 35, 39.)
Appellant argues this disclosure does not satisfy the limitation at issue
because
Ramamoorthy merely discloses searching for trigger words
(e.g., “call me,” . . .) and determining a phone number. But
Ramamoorthy does not disclose “extracting from the voice
message a proposed action to be performed by the user,” as
recited in claim 1.
(Appeal Br. 15.) Appellant argues the Examiner disregards the plain
meaning of “action” (“the process of doing” or “performance”), construes
“action” inconsistently with the Specification, improperly maps the same
disclosure to both the “proposed action” and the “action parameter,” and
does not show that any proposed action is extracted from a voicemail
message. (Appeal Br. 14–23.)
None of these arguments are persuasive. “Call me” — i.e., a request
in a voicemail message from the caller asking the recipient to call back — is
certainly an action in the sense of Appellant’s definition: “the process of
doing” or “performance.” Also, the Examiner’s reliance on the above-
described disclosure in Ramamoorthy is manifestly consistent with one of
the examples of the claimed subject matter described in the Specification:
Appeal 2018-002085
Application 14/213,241

7

A recorded voice message is provided from a caller to a
user. . . .
A proposed action to be performed by the user is then
extracted from the voice message. For example, the voicemail
may state “this is John, call me at 650.987.0987 at 9 am
tomorrow.” Here, the action is to call John.
At least one action parameter for undertaking the action
is determined. . . . [T]he at least one action parameter includes
(i) the telephone number of 650.987.0987. . . .
(Spec. ¶¶ 5–7.) Appellant admits that this example in the Specification is
representative of the claimed subject matter. (Appeal Br. 17–18.) Relatedly,
Ramamoorthy states, as an advantage of its approach, “the person receiving
voice mail messages may avoid having to manually write down, type, or
memorize phone numbers left in a voice mail message . . . .” Similarly, the
Specification states, “[The invention] reduces or eliminates the user having
to remember, write down or type in contacts details left by callers in
voicemail messages.” (Ramamoorthy ¶ 16; Spec. ¶ 12.)
Nor does the Examiner improperly map the same disclosure to
separate claim limitations to render limitations superfluous — with respect
to the Ramamoorthy exemplary embodiment that processes, “Please call me
at 555-5555,” the Examiner finds “the action parameter is ‘the phone
number’ and the proposed action is ‘please call me.’” (Ans. 3.)
In addition, Ramamoorthy indeed “extracts” the proposed action from
the voicemail message by virtue of the fact that, in the example the
Examiner relies on, the detection of the occurrence of “call me” triggers the
subsequent detection of the phone number, and further generates a macro
that provides the user with the option to dial the number. (Ans. 3;
Appeal 2018-002085
Application 14/213,241

8

Ramamoorthy ¶¶ 33, 35, 39, 41.) The Specification “extracts” the “call me”
action in the same sense:
[T]he voicemail may state “this is John, call me at
650.987.0987 at 9 am tomorrow.” Here, the action is to call
John.
[T]he at least one action parameter includes (i) the telephone
number of 650.987.0987. . . .
Finally, the user is presented with a prompt to facilitate
undertaking the action using the at least one the parameter. For
example, the user may be given the option to set a reminder to
call John . . . .
(Spec. ¶¶ 6–8.) Appellant argues the fact that, in Ramamoorthy, the user is
presented with additional options, such as saving or deleting the phone
number, as opposed to just dailing the number, somehow undercuts the
Examiner’s finding. (Appeal Br. 21.) But the fact that Ramamoorthy
discloses additional user options is encompassed by the open-ended nature
of the claims — “A method of operating a digital assistant, comprising . . . .”
(E.g., claim 1.) The claims require the user to be presented with a prompt to
facilitate undertaking the extracted action (e.g., a prompt to call the
telephone number), but the claims do not exclude facilitating additional
actions as well.
Lastly, Appellant incorrectly argues that, in Ramamoorthy, “the
macro actions in Ramamoorthy would be generated even if there is no
proposed action in a voice message.” (Appeal Br. 22.) Although some
embodiments disclosed in Ramamoorthy do not require trigger words, such a
“call me,” the Examiner relies on the embodiment that does require them.
(Final Act. 2; Ans. 3; Ramamoorthy ¶ 39.) As explained in Ramamoorthy,
the embodiment using trigger words provides an improvement over other
Appeal 2018-002085
Application 14/213,241

9

embodiments: “Using trigger words may reduce the possibility of mistake
(e.g., spoken numbers that are not phone numbers) . . . .” (Ramamoorthy
¶ 33.)
Accordingly, we sustain the Examiner’s rejection of independent
claims 1, 23, and 28 as anticipated by Ramamoorthy.3 (Appellant raises an
additional argument concerning the obviousness rejection of independent
claim 19, which is discussed further below.)
We also sustain the anticipation rejections of claims 7, 8, and 17; and
the obviousness rejections of claims 4, 6, 16, 26, and 31 over Ramamoorthy
and Cochinwala; of claims 5 and 27 over Ramamoorthy, Cochinwala, and
Macwan; of claim 10 over Ramamoorthy and Trajkovic; of claims 12 and 15
over Ramamoorthy and Macwan; and of claim 13 over Ramamoorthy,
Macwan, and Upadhyay. Appellant’s arguments directed to these rejections
incorporate the above-discussed arguments regarding the independent
claims, and the rejections are not otherwise argued separately with
particularity. (Appeal Br. 22–23.)
Second Issue
Independent claim 19, in addition to requiring extracting a proposed
action from a voice message as discussed above, also requires, “wherein the

3 Appellant also argues the Examiner erred in rejecting claim 23 as
anticipated while relying on Ramamoorthy and additional references.
(Appeal Br. 23.) However, as Appellant concedes, the Examiner relied on
the additional references to support the finding that the use of a computer
readable medium is inherently part of the system of Ramamoorthy. (Appeal
Br. 41; Final Act. 4.) We are not persuaded the Examiner’s inherency
finding is erroneous. MPEP § 2131.01 (9th ed. rev. 08.2017 Jan. 2018)
(permitting reliance on additional references to show that a characteristic not
disclosed in the anticipatory reference is inherent).
Appeal 2018-002085
Application 14/213,241

10

action is to meet at a specified geographic location, and the at least one
action parameter comprises a name or an address of the geographic
location.” The Examiner relies on the combination of Ramamoorthy and
Brown as teaching or suggesting the claimed subject matter, and in
particular relies on the above-discussed disclosure in Ramamoorthy of
extracting a proposed action from a voice message, and determining an
associated action parameter, combined with the disclosure in Brown of
extracting geographic locations for meetings or events from voice messages
that have been converted to text and which contain such phrases as “meeting
is at”, “get together at”, “here at”, “location”, etc. (Final Act. 10–11; Brown
Abstr., ¶¶ 46, 48.)
Appellant argues that neither Ramamoorthy nor Brown teaches any
extracting of proposed actions, including the action of meeting at a specified
geographic location. (Appeal Br. 23–25.) However, as discussed above,
Ramamoorthy does disclose extracting proposed actions, and the Examiner
relies on the ability of extracting the geographic location of a meeting or
event disclosed in Brown, in combination with Ramamoorthy, as rendering
claim 19 obvious. (Ans. 4.) In re Mouttet, 686 F.3d 1322, 1332 (Fed. Cir.
2012) (citing In re Keller, 642 F.2d 413, 425 (CCPA 1981)) (“[T]he test [for
obviousness] is what the combined teachings of the references would have
suggested to those having ordinary skill in the art.” (citation omitted)).
Accordingly, we sustain the Examiner’s rejection of claim 19 as
obvious over Ramamoorthy and Brown.
We also sustain the obviousness rejections of claim 20 over
Ramamoorthy, Brown, and Zhang; and of claim 21 over Ramamoorthy,
Brown, and Gamon. Appellant’s arguments directed to these rejections
Appeal 2018-002085
Application 14/213,241

11

incorporate the above-discussed arguments regarding independent claim 19,
and the rejections are not otherwise argued separately with particularity.
(Appeal Br. 25.)
Third Issue
Dependent claims 2, 24, and 29 additionally require, “converting the
recorded voice message from speech to text; and extracting the action from
the text.” The Examiner relies on the combination of Ramamoorthy and
Miller as teaching or suggesting the claimed subject matter, and in particular
relies on the above-discussed disclosure in Ramamoorthy of extracting a
proposed action from a voice message, and determining an associated action
parameter, combined with the disclosure in Miller of extracting a phone
number from a voicemail message by first converting the recorded speech to
text. (Final Act. 5–6; Ans. 4; Miller Abstr., 5:4–10, 10:8–11.)
Appellant argues the Examiner erred because neither reference
discloses extracting proposed actions from a voicemail message. (Appeal
Br. 25–27.) However, as discussed above, Ramamoorthy does disclose
extracting proposed actions, and the Examiner relies on the explicit
disclosure in Miller of extracting information from a voicemail converted to
text, in combination with Ramamoorthy, as rendering claims 2, 24, and 29
obvious. (Ans. 4.)
Appellant also argues the Examiner failed to explain why the
modification of Ramamoorthy to use the speech-to-text conversion of Miller
would be obvious, and that the Examiner used impermissible hindsight.
(Appeal Br. 27–31.) However, we are not persuaded the Examiner erred in
concluding that the disclosure in Miller of converting speech to text to
facilitate extracting information from voicemail would not be an obvious
Appeal 2018-002085
Application 14/213,241

12

modification of Ramamoorthy, which also extracts information from
voicemail. (Ans. 5.) We note that Ramamoorthy broadly teaches the use of
“voice recognition software,” and that
[r]ecognition of spoken letters and words is known in the art
and one of ordinary skill in the art will also recognize that the
voice mail parser [] may utilize algorithms presently known or
later developed to parse letters, numbers, and words . . . .
(Ramamoorthy ¶ 33.) Miller establishes that voice-to-text conversion was a
well-known approach used in “voice recognition software.” (Ans. 4; see
also Brown ¶ 42.)
Accordingly, we sustain the Examiner’s rejection of claims 2, 24, and
29 as obvious over Ramamoorthy and Miller.
We also sustain the rejections of claims 3, 25, and 30 as obvious over
Ramamoorthy and Miller. Appellant’s arguments directed to these
rejections incorporate the above-discussed arguments regarding claims 2, 24,
and 29, and the rejections are not otherwise argued separately with
particularity. (Appeal Br. 31.)
Fourth Issue
Dependent claim 9 additionally requires, “wherein the at least one
parameter includes a specific time period, and the presenting of the prompt
occurs at or before the specific time period.” The Examiner relies on the
combination of Ramamoorthy and Brown as teaching or suggesting the
claimed subject matter, and in particular relies on the above-discussed
disclosure in Ramamoorthy of extracting a proposed action from a voice
message, and determining an associated action parameter, combined with the
disclosure in Brown of extracting information about meeting times from a
voicemail message, and outputting that information before or after a user
Appeal 2018-002085
Application 14/213,241

13

listens to the recorded voice message. (Final Act. 9–10; Ans. 4; Brown Fig.
7, ¶¶ 48, 51.)
Appellant argues that Brown outputs the extracted meeting time
information dependent solely on the time the user listens to the voice
message, rather than before the meeting time (i.e., “specific time period”) as
claimed. (Appeal Br. 32–33.) Appellant is correct.
Accordingly, we do not sustain the Examiner’s rejection of claim 9 as
obvious over Ramamoorthy and Brown. We also do not sustain the
Examiner’s rejection of claim 22 as obvious over Ramamoorthy, Brown, and
Gamon, which has a similar requirement not met by the references.
Fifth Issue
Dependent claim 11 additionally requires, “wherein the action is to
check on the user’s availability, and the at least one parameter is retrieved
from a calendar of the user.” Appellant correctly points out that the
Examiner failed to address this claim in the Final Action. (Appeal Br. 36–
37.) However, in the Answer, the Examiner rejected claim 11 over the
combination of Ramamoorthy and Gamon, relying on the above-discussed
disclosure in Ramamoorthy of extracting a proposed action from a voice
message, and determining an associated action parameter, combined with the
disclosure in Gamon of extracting calendar items from electronic
communications such as emails, and populating that information into a
user’s calendar application as a calendar entry. (Answer 7; Gamon ¶¶ 4, 24,
25.)
In reply, Appellant argues that nowhere in Gamon is there any
teaching or suggestion that information is retrieved from a user calendar as
Appeal 2018-002085
Application 14/213,241

14

required — rather, the only disclosure is inputting information into a
calendar. (Reply Br. 19–22.) We agree with Appellant.
Accordingly, we do not sustain the Examiner’s rejection of claim 11
as obvious over Ramamoorthy and Gamon.
Sixth Issue
Dependent claim 14 additionally requires, “wherein the prompt offers
to the user an option to store the email address extracted from the recorded
voice message in association with the identified existing contact in the
contact list,” and claim 18 has a similar requirement. The Examiner relies
on the combination of Ramamoorthy and Macwan as teaching or suggesting
the claimed subject matter, and in particular relies on the above-discussed
disclosure in Ramamoorthy of extracting a proposed action from a voice
message, and determining an associated action parameter, combined with the
disclosure in Macwan of retrieving information not supplied in a voicemail
message from a contacts database. (Final Act. 13–15; Macwan ¶ 40.)
In reply, Appellant argues that nowhere in Gamon is there any
teaching or suggestion that email addresses are stored in a contact list as
required — rather, the only disclosure is retrieving information from
contacts. (Appeal Br. 37–39.) We agree with Appellant.
Accordingly, we do not sustain the Examiner’s rejection of claims 14
and 18 as obvious over Ramamoorthy and Macwan.
Seventh Issue
Dependent claims 32–34 each require, “wherein extracting the
proposed action comprises determining the proposed action from a plurality
of proposed actions.” The Examiner finds that Ramamoorthy discloses this
Appeal 2018-002085
Application 14/213,241

15

limitation, based on the disclosure of extracting the action of calling a
telephone number, as discussed above for claim 1. (Final Act. 2–4.)
Appellant argues that the Examiner provides no basis for finding that
Ramamoorthy discloses a “plurality of proposed actions.” (Appeal Br. 39–
41.) We agree with Appellant. Accordingly, we do not sustain the
Examiner’s rejection of claims 32, 33, and 34 as anticipated by
Ramamoorthy.

Appeal 2018-002085
Application 14/213,241

16

CONCLUSION
In summary:

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
Claims
Rejected
35 U.S.C. § Basis Affirmed Reversed
1, 7, 8, 17, 23, 28,
32–34
102 Ramamoorthy 1, 7, 8, 17, 23, 28 32–34
2, 3, 24, 25, 29, 30 103(a) Ramamoorthy and
Miller
2, 3, 24, 25, 29, 30
4, 6, 16, 26, 31 103(a) Ramamoorthy and
Cochinwala
4, 6, 16, 26, 31
5, 27 103(a) Ramamoorthy,
Cochinwala, and
Macwan
5, 27
9, 19 103(a) Ramamoorthy and
Brown
19 9
10 103(a) Ramamoorthy and
Trajkovic
10
11 103(a) Ramamoorthy and
Gamon
11
12, 14, 15, 18 103(a) Ramamoorthy and
Macwan
12, 15 14, 18
13 103(a) Ramamoorthy,
Macwan, and
Upadhyay
13
20 103(a) Ramamoorthy,
Brown, and Zhang
20
21, 22 103(a) Ramamoorthy,
Brown, and Gamon
21 22
Overall Outcome 1–8, 10, 12, 13,
15–17, 19–21, 23–
31
9, 11, 14,
18, 22,
32–34
Appeal 2018-002085
Application 14/213,241

17

AFFIRMED IN PART