Ajita JOHN et al.

Patent Trials and Appeals Board

Apr 26, 2021

2020000026 (P.T.A.B. Apr. 26, 2021)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
12/887,248 09/21/2010 Ajita JOHN 4366-904 2117
48500 7590 04/26/2021
SHERIDAN ROSS P.C.
1560 BROADWAY, SUITE 1200
DENVER, CO 80202
EXAMINER
PEACH, POLINA G
ART UNIT PAPER NUMBER
2165
NOTIFICATION DATE DELIVERY MODE
04/26/2021 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
cjacquet@sheridanross.com
edocket@sheridanross.com
pair_Avaya@firsttofile.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________

Ex parte AJITA JOHN, SHREEHARSH KELKAR, and
DOREE DUNCAN SELIGMANN
___________

Appeal 2020-000026
Application 12/887,248
Technology Center 2100
____________

Before JEAN R. HOMERE, JAMES B. ARPIN, and
PHILLIP A. BENNETT, Administrative Patent Judges.

ARPIN, Administrative Patent Judge.

DECISION ON APPEAL
Appellant1 appeals under 35 U.S.C. § 134(a) from the Examiner’s
decision rejecting claims 1–20. Final Act. 2.2 We have jurisdiction under
35 U.S.C. § 6(b).
We affirm-in-part.

1 “Appellant” refers to “applicant” as defined in 37 C.F.R. § 1.42.
Appellant identifies the real party-in-interest as Avaya Inc. Appeal Br. 2.
2 In this Decision, we refer to Appellant’s Appeal Brief (“Appeal Br.,” filed
July 1, 2019) and Reply Brief (“Reply Br.,” filed September 30, 2019); the
Final Office Action (“Final Act.,” mailed February 1, 2019) and the
Examiner’s Answer (“Ans.,” mailed August 8, 2019); and the Specification
(“Spec.,” filed September 21, 2010). Rather than repeat the Examiner’s
findings and Appellant’s contentions in their entirety, we refer to these
documents.
Appeal 2020-000026
Application 12/887,248

2

STATEMENT OF THE CASE
The claimed methods and systems relate to tags applied to video
segments and, more specifically, to classifying tags into types and to
applying such tags to appropriate segments. See Spec. ¶¶ 1, 3.
As noted above, claims 1–20 are pending. Claims 1, 16, and 19 are
independent. Appeal Br. 15 (claim 1), 17 (claim 16), 18–19 (claim 19)
(Claims App.). Claims 2–15 depend directly or indirectly from claim 1,
claims 17 and 18 depend directly from claim 16, and claim 20 depends
directly from claim 19. Id. at 15–19.
Claims 1 and 16, reproduced below with disputed limitations
emphasized, are illustrative.
1. A method comprising:
receiving, at a processor of a tagging server, a tag
comprising a user-provided text and generated during a
presentation by a media server of a live media event, the live
media event comprising a first segment and a second segment,
the first segment featuring a first speaker and starting at a first
time and ending at a second time later than the first time, the
second segment featuring a second speaker and starting at the
second time and ending at a third time later than the second time;
enhancing the tag, with an automatic tagger, to include a
tag type;
attaching, by the processor, the tag to the second segment
of the live media event based on the tag being received during
the second segment;
analyzing, by the processor, content of the user-provided
text to yield analyzed text content;
determining, by the processor and based on the analysis,
that the analyzed text content does not relate to the second
speaker in the second segment of the live media event, but does
Appeal 2020-000026
Application 12/887,248

3

relate to the first speaker in the first segment of the live media
event; and
adjusting, automatically by the processor and in response
to determining that the analyzed text content relates to the first
speaker rather than the second speaker, the attachment of the tag
from being attached to the second segment to being attached to
the first segment.
Id. at 15 (emphases added).
16. A system comprising:
a tagging server comprising a processor; and
a computer-readable storage device storing instructions
which, when executed by the processor, cause the processor to
perform operations comprising:
receiving a tag comprising a user-provided text and
generated during a presentation of a live media event by a
media server, the live media event comprising a first segment
and a second segment, the first segment featuring a first speaker
and starting at a first time and ending at a second time later than
the first time, the second segment featuring a second speaker
and starting at the second time and ending at a third time later
than the second time;
enhancing the tag, with an automatic tagger, to include a
tag type;
attaching the tag to the second segment of the live media
event based on the tag being received during the second
segment;
analyzing content of the user-provided text to yield
analyzed text content;
comparing the analyzed text content to data about the
first segment and the second segment;
determining, based on the comparison, that the analyzed
text content does not relate to the second speaker in the second
segment of the live media event, but does relate to the first
speaker in the first segment of the live media event; and
Appeal 2020-000026
Application 12/887,248

4

adjusting, automatically and in response to determining
that the analyzed text content relates to the first speaker rather
than the second speaker, the attachment of the tag from being
attached to the second segment to being attached to the first
segment.
Id. at 17 (emphases added). Each of independent claims 16 and 19 recites
limitations corresponding to disputed limitations of claim 1. Id. at 17, 18–
19. Claims 16 and 19 additionally recite the disputed limitation of
“comparing the analyzed text content to data about the first segment and the
second segment,” which is not present in claim 1. Id.
REFERENCES AND REJECTIONS
The Examiner relies upon the following references:
Name3 Reference Issued/Published Filed
Tritschler US 6,424,946 B1 July 23, 2002 Nov. 5, 1999
Kanevsky US 6,434,520 B1 Aug. 13, 2002 Apr. 16, 1999
Lu US 2004/0107100 A1 June 3, 2004 Nov. 29, 2002
Okada US 2004/0263529 A1 Dec. 30, 2004 May 8, 2003
Tsutsui US 2005/0060741 A1 Mar. 17, 2005 Dec. 10, 2003
Finkelstein US 2007/0292106 A1 Dec. 20, 2007 June 15, 2006
Gallagher US 2007/0239683 A1 Oct. 11, 2007 Apr. 7, 2006
Epifania US 2009/0163239 A1 June 25, 2009 Dec. 21, 2007
Badoiu US 2009/0210779 A1 Aug. 20, 2009 Feb. 19, 2008
Chen US 2010/0313113 A1 Dec. 9, 2010 June 5, 2009
Roberts US 2011/0246937 A1 Oct. 6, 2011 Mar. 31, 2010

3 All reference citations are to the first named inventor or author only.
Appeal 2020-000026
Application 12/887,248

5

Nam US 2011/0184729 A1 July 28, 2011 Mar. 29, 20114
Topkara Tag Me While You
Can: Making Online
Recorded Meetings
Shareable and
Searchable, IBM
Research Report,
RC25038 (Wl00S-
057) Computer
Science
Aug. 18, 2010 NA
The Examiner rejects:
1. claims 1, 2, 4, 16, 18, and 19 under 35 U.S.C. § 103(a) as obvious
over the combined teachings of Roberts, Finkelstein, and Lu5
(Final Act. 2–8);
2. claim 3 under 35 U.S.C. § 103(a) as obvious over the combined
teachings of Roberts, Finkelstein, Lu, and Nam (id. at 8–9);
3. claim 5 under 35 U.S.C. § 103(a) as obvious over the combined
teachings of Roberts, Finkelstein, Lu, and Epifania (id. at 9);
4. claim 6 under 35 U.S.C. § 103(a) as obvious over the combined
teachings of Roberts, Finkelstein, Lu, and Gallagher (id. at 9–10);
5. claims 7–13, 15, and 20 under 35 U.S.C. § 103(a) as obvious over
the combined teachings of Roberts, Finkelstein, Lu, and Badoiu
(id. at 10–12);

4 Nam claims priority to Korean Patent Application No. 10-2008-0095330,
filed September 29, 2008.
5 The Examiner withdraws reliance on Topkara’s teachings in this
combination and in the subsequent, related combinations. Ans. 3; see Reply
Br. 2.
Appeal 2020-000026
Application 12/887,248

6

6. claims 14 and 17 under 35 U.S.C. § 103(a) as obvious over the
combined teachings of Roberts, Finkelstein, Lu, and Chen (id. at
13);
7. claims 1, 4, 16, and 19 under 35 U.S.C. § 103(a) as obvious over
the combined teachings of Okada, Tsutsui, and Tritschler (id. at
17).
We review the appealed rejections for error based upon the issues
identified by Appellant,6 and in light of the contentions and evidence
produced thereon. Ex parte Frye, 94 USPQ2d 1072, 1075 (BPAI 2010)
(precedential). The Examiner and Appellant focus their findings and
contentions on claims 1, 16, 19; so do we. See Appeal Br. 7, 10, 11, 13;
Ans. 3, 7, 8, 11; Reply Br. 2, 6, 7, 10. Arguments not made are forfeited.7
Unless otherwise indicated, we adopt the Examiner’s findings in the Final
Office Action and the Answer as our own and add any additional findings of
fact for emphasis. We address the rejections below.

6 In the Reply Brief, Appellant contends the Examiner presents various new
grounds of rejection in the Answer. E.g., Reply Br. 7 (“[T]he Answer relies
on entirely new portions of Okada for the rejection and these new portions
go beyond the scope of what the previously cited portions of Okada teach.
MPEP § 1207.03(a)(I).”). Nevertheless, Appellant did not file a timely
petition seeking review of the Examiner’s alleged failure to designate a
rejection as a new ground and has waived any such contentions. 37 C.F.R.
§ 41.40(a); see MPEP §§ 1002.02(c)(6), 1207.03(b).
7 See In re Google Tech. Holdings LLC, 980 F.3d 858, 863 (Fed. Cir. 2020)
(“We interpret the Patent Office to be arguing that Google’s failure to raise
its lexicography arguments, inadvertent or not, compels a finding of
forfeiture.”); 37 C.F.R. § 41.37(c)(1)(iv) (“Except as provided for in
§§ 41.41, 41.47 and 41.52, any arguments or authorities not included in the
appeal brief will be refused consideration by the Board for purposes of the
present appeal.”).
Appeal 2020-000026
Application 12/887,248

7

ANALYSIS
A. Obviousness over Roberts, Finkelstein, and Lu
1. Independent Claims 1, 16, and 19
As noted above, the Examiner rejects independent claims 1, 16, and
19 as obvious over the combined teachings of Roberts, Finkelstein, and Lu.
Final Act. 2–8; see Ans. 3 (The Examiner no longer relies on the teachings
of Topkara.). In particular, the Examiner finds Roberts teaches or suggests a
majority of the limitations, as recited in these claims, but that Roberts does
not teach or suggest “any speakers at all,” and “Roberts does not explicitly
teach the first segment featuring a first speaker and the second segment
featuring a second speaker.” Final Act. 5. The Examiner finds, “However,
Finkelstein discloses the same in paragraphs ([0027], see determining
‘changes in the person speaking’, [0045]). Note * * * that the incorporated
reference of Lu, fully teaches real-time detecting of speaker change between
two segments (Lu Abstract and Title).” Id. Further, the Examiner finds a
person of ordinary skill in the relevant art would have had reason to combine
the teachings of Finkelstein and Lu with those of Roberts to achieve the
methods of claim 1 and the systems of claims 16 and 19. Id. (citing
Finkelstein ¶ 23).
Appellant contends the Examiner fails to show that Roberts teaches or
suggests the disputed limitations of the independent claims for two reasons.
Appeal Br. 7–10; see Reply Br. 2–6. On this record, Appellant does not
persuade us the Examiner errs in the findings regarding Roberts’s teachings.
First, Appellant contends Roberts does not teach or suggest,
“enhancing the tag, with an automatic tagger, to include a tag type.” Appeal
Br. 7–9. In particular, Appellant contends:
Appeal 2020-000026
Application 12/887,248

8

Examples of tag types in the application as-filed include, for
example, “Mary” (when associated with a specific speaker
during the live media event) and “pension proposal” (when
associated with a specific topic discussed by the speaker in the
live media event). [Spec. ¶ 21.] Roberts discloses a “tag
optimization facility 106” that can, for example, aggregat[e] [a]
tag with one or more other tags based on at least one attribute
common to the tag and the one or more other tags, analyz[e] the
tag compared to the one or more other tags, and perform[] one or
more operations based on the analysis.” Roberts [¶¶ 32–33.] But
Roberts nowhere discloses that the tag optimization facility 106
can enhance a tag comprising a user-provided text to include a
tag type.
Appeal Br. 7–8. Further, even if Roberts teaches “enhancing” a tag,
Appellant contends Roberts describes the user, not an “automatic tagger,” as
recited in the independent claims, as enhancing the tag. Reply Br. 2–3. We
disagree.
Roberts discloses,
a media content tag may be associated with the media content.
Step 504 may also include generating and/or updating the media
content tag before the tag is associated with the media content.
The tag may include any information descriptive of or otherwise
related to the media content, including any of the information
described above.
In certain embodiments, the media content tag may
comprise a user-defined tag including information defined based
on input provided by a user.
Roberts ¶¶ 68–69 (emphases added); see Final Act. 3; Ans. 4. Further,
Roberts explains,
A tag may include any information descriptive of or otherwise
associated with media content, including, without limitation,
information indicating one or more user profiles associated with
the media content, an identifier for the media content (e.g., a
media content program identifier), a segment of the media
Appeal 2020-000026
Application 12/887,248

9

content (e.g., a start point and an end point of a segment of the
media content), and a label for the tag and/or a segment of the
media content specified by the tag.
Roberts ¶ 29 (emphasis added).
Roberts discloses, “Tagging facility 104 may receive the user input
and generate, modify, and/or delete a user-defined tag based on the user
input,” and “the media content tag and the one or more other media content
tags included in the aggregation are analyzed. The analysis may include tag
optimization facility 106 analyzing of one or more attributes of the tags
included in the aggregation of tags.” Id. ¶¶ 30, 93 (emphases added); see id.
¶ 33. According to Roberts,
A common attribute may include any attribute shared by
one or more of the media content tags and that may be used to
aggregate the media content tags. For example, a common
attribute may include, without limitation, a matching media
content program identifier, a matching tag label, a matching
user rating, and matching timestamp data associated with the
one or more media content tags. For instance, in step 1002, one
or more media content tags may be aggregated based on a
matching tag label shared by the media content tags.
Id. ¶ 89 (emphasis added); see Final Act. 3. Thus, we are persuaded the
Examiner shows that user-defined tags may be modified, e.g., enhanced,
with information, including attributes. Such attributes teach or suggest “tag
types.” Ans. 3–4; cf. Spec. ¶ 21.
Roberts further discloses:
User input that may be used to define the media content tag may
be received by system 100 in any suitable way. For example,
system 100 may provide one or more tools configured to allow a
user to provide user input defining a media content tag. The
tools may be provided to the user in any suitable way, including
by inclusion in a GUI or other user interface.
Appeal 2020-000026
Application 12/887,248

10

Roberts ¶ 69 (emphasis added); see id. ¶ 31 (“Additionally or alternatively,
in certain embodiments, tagging facility 104 may be configured to
automatically generate and/or modify a tag without user input.” (emphases
added)). Thus, we also are persuaded the Examiner shows Roberts teaches
or suggests use of an “automatic tagger,” as recited in the independent
claims. Therefore, we are not persuaded of Examiner error by Appellant’s
first reason.
Second, Appellant contends Roberts does not teach or suggest
“attaching, by the processor, the tag to the second segment of the live media
event based on the tag being received during the second segment,”
“determining, by the processor and based on the analysis, that the analyzed
text content does not relate to the second speaker in the second segment of
the live media event, but does relate to the first speaker in the first segment
of the live media event,” and “adjusting, automatically by the processor and
in response to determining that the analyzed text content relates to the first
speaker rather than the second speaker, the attachment of the tag from being
attached to the second segment to being attached to the first segment.”
Appeal Br. 9–10. In particular, Appellant contends:
Rather, in Roberts, the user defines the end point of a segment,
Roberts [¶ 74], the optimization facility matches timestamp data,
[id. ¶ 90], the optimization facility “modif[ies] timestamp data to
effectively merge the two segments into one segment,” [id. ¶ 98],
and the optimization facility “modif[ies] tag data to adjust a start
point or an end point to a different point corresponding to the
preferred start time or end time determined in the analysis” based
on user’s preferences and comments, [id. ¶¶ 100, 102].
Id. at 9. Thus, Appellant contends Roberts is concerned with whether the
segment starts too early or too late, and not with whether the tag identifying
Appeal 2020-000026
Application 12/887,248

11

the speaker is attached to the wrong segment. Id. at 9–10; see Reply Br. 3–
4.
Roberts discloses:
The user may then provide input requesting that the segment be
marked. However, one or more delays may exist between when
presentation of scene commences, when the user decides that the
segment is worth marking, when the user provides input
requesting that the segment be marked, and when the user’s
device actually processes the request and defines a starting point
for the marked segment. Consequently, the starting point of the
segment may not coincide with the beginning of the scene.
Roberts ¶ 3 (emphasis added); see Ans. 5 (Roberts addresses the same
problem as Appellant’s claims); cf. Spec. ¶ 3. Specifically, Roberts
discloses:
To illustrate, a first tag may specify a first segment of media
content (e.g., a first part of an opening monologue of a television
program titled “Conan”) and a second tag may specify a second
segment of media content (e.g., a second part of the opening
monologue of “Conan”).
Roberts ¶ 98. Thus, Roberts teaches or suggests attaching, by a processor, a
tag to a second segment based on the tag received during the second
segment, as recited in the independent claims. Ans. 4–5; see Roberts ¶ 69
(“For example, system 100 may provide one or more tools configured to
allow a user to provide user input defining a media content tag.”).
Although Appellant contends that Roberts does not teach or suggest
the “determining” and “adjusting” limitations (Appeal Br. 10), the Examiner
relies on the combined teachings of Roberts, Finkelstein, and Lu to teach or
suggest these limitations (Ans. 6–7; see Final Act. 5). Appellant cannot
show nonobviousness by attacking references individually when the
Examiner’s rejection is based on the combined teachings of the references.
Appeal 2020-000026
Application 12/887,248

12

See In re Merck & Co., Inc., 800 F.2d 1091, 1097 (Fed. Cir. 1986); In re
Keller, 642 F.2d 413, 426 (CCPA 1981). As the Examiner finds, Finkelstein
discloses, “the metadata associated with the audio data sequence 25A is
extracted, at 49, according to the at least one audio characteristic of the
audio data sequence. In one alternative embodiment, audio characteristics of
the audio data sequence 25A can comprise . . . changes in the person
speaking.” Finkelstein ¶ 27 (emphasis added). Further, Lu discloses, “A
method is provided for real-time speaker change detection and speaker
tracking in a speech signal.” Lu, Abstr. (emphasis added). The Examiner
finds Roberts’s teachings, as modified by those of Finkelstein and Lu, teach
or suggest the “determining” and “adjusting” limitations, as recited in the
independent claims. Ans. 7. We agree with the Examiner, and, thus, we are
not persuaded of Examiner error by Appellant’s second reason.
2. Independent Claims 16 and 19
As noted above, claims 16 and 19 recite “comparing the analyzed text
content to data about the first segment and the second segment.” Appeal
Br. 17, 18–19 (Claims App.). The Examiner finds Roberts teaches or
suggests this limitation. Final Act. 7–8 (citing Roberts ¶¶ 33, 97, 98).
Appellant contends, however, “Paragraph [0033] of Roberts discusses
aggregating the tags. Paragraphs [0097]-[0098] discuss modifying the
media content tags in the aggregated tags, preventing lower quality tags
from being made available for distribution, and modifying timestamp data to
merge two segments into one segment.” Appeal Br. 10–11 (emphases
added); see Reply Br. 6. Thus, Appellant contends, “Roberts does not
discuss or disclose comparing the analyzed text content to data about the
Appeal 2020-000026
Application 12/887,248

13

first segment and the second segment.” Appeal Br. 11. For the reasons
given below, we disagree with Appellant.
Roberts discloses,
timestamp data of one or more tags may be merged together by
modifying tag data representative of timestamps in a tag. To
illustrate, a first tag may specify a first segment of media content
(e.g., a first part of an opening monologue of a television
program titled “Conan”) and a second tag may specify a second
segment of media content (e.g., a second part of the opening
monologue of “Conan”). Tag optimization facility 106 may
determine from the analysis in step 1004 that a merger of the
segments, or at least portions of the segments, may produce a
merged segment that is more likely to produce a quality user
experience when viewed.
Roberts ¶ 98 (emphasis added). Roberts further discloses:
The tag data may be modified in any suitable way that is based
on the analysis performed in step 1004. For example, the
analysis may determine one or more tags that are considered to
be of relatively lower quality than other tags in the aggregation
(e.g., because the tags specify partial segments compared to the
segments of the other tags).
Id. ¶ 97 (emphases added). Based on this description, the Examiner
correctly finds Roberts teaches or suggests comparing user-identified tags
and the media content of segments, to which the tags are assigned. Ans. 7–
8; see also Roberts ¶¶ 102 (“In step 1102, data representative of media
content associated with a media content tag is analyzed. Step 1102 may
include any analysis of media content associated with a media content tag.”),
105 (“In this or a similar manner, at least one of a start point and an end
point of a segment defined by the media content tag may be adjusted to
coincide with a natural break in or other attribute of the media content based
Appeal 2020-000026
Application 12/887,248

14

on the analysis performed in step 1102.”). Therefore, we agree with the
Examiner that Roberts teaches or suggests this limitation.
On this record, we are not persuaded the Examiner errs in rejecting
the independent claims as obvious over the combined teachings of Roberts,
Finkelstein, and Lu. Therefore, we sustain the rejection of those claims.
3. Dependent Claims
As noted above, the Examiner rejects claims 2–15, 17, 18, and 20 as
obvious over the combined teachings of Roberts, Finkelstein, and Lu, alone
or in combination with another reference. Final Act. 2–13. Appellant does
not challenge the obviousness rejections of those dependent claims
separately from its challenge to the rejection of independent claims 1, 16,
and 19. Appeal Br. 11. With the exception of claim 4, we sustain these
rejections.
Claim 4 depends from independent claim 1 via intervening claim 3.
Appeal Br. 15 (Claims. App.). As noted above, the Examiner rejects claim 3
as obvious over the combined teachings of Roberts, Finkelstein, Lu, and
Nam (Final Act. 8–9), but the Examiner rejects claim 4 as obvious over the
combined teachings of only Roberts, Finkelstein, and Lu (id. at 6). The
Examiner finds, “Regarding claim 4, Roberts as modified teaches the
method of claim 3, wherein user-provided text comprises a user reaction to a
portion of the first segment.” Id. at 6 (emphasis added). Nevertheless, the
Examiner also finds, “Regarding claim 3, Roberts as modified does not
explicitly teach, but Nam discloses, wherein the analysis detects that a verb
in the user-provided text is in the past tense ([Nam ¶¶ 107–108, 118]).” Id.
at 9 (emphasis added). These findings appear inconsistent, and the
Examiner fails to explain or otherwise resolve this apparent inconsistency.
Appeal 2020-000026
Application 12/887,248

15

Therefore, on this record, the Examiner errs in rejecting claim 4 as obvious
over Roberts, Finkelstein, and Lu; and we do not sustain this rejection of
claim 4.
B. Obviousness over Okada, Tsutsui, and Tritschler
1. Independent Claims 1, 16, and 19
As noted above, the Examiner rejects claims 1, 16, and 19 as obvious
over the combined teachings of Okada, Tsutsui, and Tritschler. Final
Act. 13–17. In particular, the Examiner finds Okada teaches or suggests a
majority of the limitations, as recited in these claims, but that Okada does
not explicitly teach or suggest “the attachment of the tag from being attached
to the second segment to being attached to the first segment,” and “wherein
the tag does not specify a segment of the live media event to which the user-
provided text applies.” Id. at 16. The Examiner finds, however, that these
limitations are taught or suggested by Tsutsui and Tritschler. Id. at 16–17
(citing Tsutsui ¶¶ 59–62; Tritschler, 3:36–57). Further, the Examiner finds a
person of ordinary skill in the relevant art would have had reason to combine
the teachings of Tsutsui and Tritschler with those of Okada to achieve the
methods of claim 1 and the systems of claims 16 and 19. Id. (citing Tsutsui
¶ 8; Tritschler, 1:48–51).
Appellant contends the Examiner fails to show that Okada teaches or
suggests the disputed limitation of claim 1 for three reasons.8 Reply Br. 7–

8 Appellant also contends the Examiner fails to show where Okada teaches
or suggests a plurality of servers, including a tagging server and a media
server. Appeal Br. 12. However, we are persuaded that the Examiner shows
where Okada teaches or suggests a plurality of servers, and Appellant does
not maintain this contention in the Reply Brief. Ans. 8–9.
Appeal 2020-000026
Application 12/887,248

16

10; see Appeal Br. 11–13. On this record, Appellant does not persuade us
the Examiner errs in the findings regarding Okada’s teachings.
First, Appellant contends Okada does not teach or suggest,
“enhancing the tag, with an automatic tagger, to include a tag type.” Appeal
Br. 12. In particular, Appellant contends,
Okada does not disclose enhancing the tag, with an automatic
tagger, to include a tag type. The Examiner cites to Okada’s
paragraphs [¶¶ 76–79, 138]. [Final Act. 14.] Okada discloses
updating the metadata and assigning keywords to a scene. Okada
[¶¶ 76–79]. Further, Okada does not disclose tags at all.
Appeal Br. 12; see Reply Br. 8.
Nevertheless, Okada discloses assigning “keywords” to media
segments (Okada ¶ 147), and we agree with the Examiner that such
keywords teach or suggest the recited tags (Final Act. 13). Okada also
discloses, “authoring processing may also assign applicable levels for
various types of information such as performers, producer and date, etc.”
Okada ¶ 138 (emphases added); see Final Act. 14. Thus, we are persuaded
that Okada teaches or suggests enhancing a tag to include a tag type.
Moreover, Okada discloses, “As explained using the flow chart in FIG. 5,
when reading a moving image file, the authoring apparatus 100 according to
Embodiment 1 creates a plurality of scenes by automatically dividing the
moving image file into time segments.” Okada ¶ 107 (emphasis added); see
id. ¶¶ 127 (“In the authoring operation with the first keyword, the time
segments of the scene automatically detected as indicated by 901 in the
figure are taken over as indicated by 902 in the figure.” (emphasis added));
see also Okada ¶ 139 (“[T]he authoring processing section 101 according to
Embodiment 1 can be implemented by a general-purpose computer.”).
Therefore, we are persuaded the Examiner shows Okada teaches or suggests
Appeal 2020-000026
Application 12/887,248

17

the “enhancing” limitation as recited in the independent claims, and we are
not persuaded of Examiner error by Appellant’s first reason.
Second, Appellant contends Okada does not teach or suggest,
“analyzing, by the processor, content of the user-provided text to yield
analyzed text content.” Reply Br. 7–8. In particular, Appellant notes the
Examiner’s finding that:
As shown in [Okada’s Figure ]10, the user (or the system)
visually makes a decision that a keyword “shoot” or “goal” are
important keywords and to which scene those keywords should
be adjusted to. This step can be performed automatically or by
a user’s decision – “authoring processing section is provided
with a metadata editing section that associates a keyword and
applicable level with each scene and updates the metadata’
[Okada ¶ 51].
Reply Br. 7 (emphasis added; citing Ans. 8); see Appeal Br. 11–12.
Appellant contends:
Appellant cannot find language in Okada or language describing
Fig. 10 that states that the user or the system identifies a keyword
as important. Further, Appellant can find no disclosure in Okada
describing the system automatically performing this step, and the
Answer does not identify any paragraph in Okada disclosing this
step.
Reply Br. 7–8.
Nevertheless, Okada discloses:
For example, a goal scene in soccer is one of shoot scenes
in which a score is obtained and often has the same start or end
points of a time segment, and therefore if a time segmentation
edited for a shoot is taken over by editing of a goal, the same
editing operation need not be performed over again, which
improves the operation efficiency.
Okada ¶ 116 (emphases added); see Final Act. 14–15. Thus, a “goal” scene
is a subset of “shoot” scenes, that is, “shoot” scenes include scenes in which
Appeal 2020-000026
Application 12/887,248

18

a player scores and those in which the player does not score; and the “shoot”
scenes are analyzed to determine whether or not they are “goal” scenes.
Consequently, we are persuaded that Okada teaches or suggest analyzing
content of the user-provided text (e.g., “shoot” and “goal”) to yield analyzed
text content. Ans. 8. Moreover, Okada discloses that this analysis may be
performed automatically by a processor. Okada ¶¶ 126–128; see Final
Act. 14.
We are persuaded the Examiner shows Okada teaches or suggests the
“attaching” limitation as recited in the independent claims. Therefore, we
are not persuaded of Examiner error by Appellant’s second reason.
Third, Appellant contends neither Tsutsui nor Tritschler teaches or
suggests “determining, by the processor and based on the analysis, that the
analyzed text content does not relate to the second speaker in the second
segment of the live media event, but does relate to the first speaker in the
first segment of the live media event” and “adjusting, automatically by the
processor and in response to determining that the analyzed text content
relates to the first speaker rather than the second speaker, the attachment of
the tag from being attached to the second segment to being attached to the
first segment.” Appeal Br. 12–13; see Reply Br. 9–10 (Appellant contends
neither Okada nor Tsutsui nor Tritschler teaches or suggests the
“determining” or “adjusting” limitations).
In particular, Appellant contends:
Tsutsui discloses using the time metadata created by the internal
clock in the media data audio-visual device to determine the
correct scene after synchronizing the media data and the
metadata based on the time data in the metadata. Tsutsui [¶¶ 59–
62]. Thus, Tsutsui really discloses syncing the clock/timing
devices such that the time in the metadata matches the time in the
Appeal 2020-000026
Application 12/887,248

19

media data, which does not constitute disclosure of the foregoing
limitations. Id.
Tritschler likewise fails to disclose these limitations. The
Examiner cites to Tritschler’s column 3, lines 50-57 for
adjusting, automatically, the attachment of the tag and enhancing
the tag. This portion of Tritschler, however, discloses that “the
present invention will automatically assign the speaker name to
all segments associated with the cluster.” Id.
Appeal Br. 13; Reply Br. 10. Thus, Appellant contends Tsutsui is concerned
with the synchronizing segments, rather than determining whether segments
relate to a first speaker instead of a second speaker, and Tritschler is
concerned with assigning a speaker name to all segments associated with a
cluster. Therefore, Appellant contends that neither Tsutsui nor Tritschler
supplies the limitations that are missing from Okada.
The Examiner finds:
Okada teaches “the first keyword, the time segments of the scene
automatically detected” ([Okada ¶ 127]) and based on the
applicable levels, the corresponding timed for the keywords are
adjusted ([id. ¶¶ 128–129]). Adjusting a time for a keyword
creates a new segment time, as shown in [id. ¶ 128]. First
segment time is 0:15 to 0:30, and a new segment time is 1:00 to
1:30. [Either] one segment can be considered first or second
segment.
Okada also teaches that instead of moving image, the file
can be “a file consisting of only speech”, which can include any
“types of information such as performers”, which can reasonably
correspond to the claimed first and second speakers.
Ans. 10; see Final Act. 14–15 (referencing Notes I and II). Thus, the
Examiner finds Okada teaches or suggests much of the “determining” and
“adjusting” limitations, but the Examiner relies on the teachings of Tsutsui
and Tritschler in combination with those of Okada to fully teach or suggest
these limitations. See Final Act. 14–17.
Appeal 2020-000026
Application 12/887,248

20

Specifically, Tsutsui discloses:
For instance, comments on a specific scene may be displayed
during a scene other than the specific scene. To overcome this
problem, []an initial coarse synchronization may be performed
based on the time data and then a final fine synchronization may
be performed based on the feature amount of an image in the
media data.
Tsutsui ¶ 59 (emphasis added); cf. Okada ¶ 128; Spec. ¶ 3.9 Further, Tsutsui
discloses that, after synchronization, “only the portions corresponding to
those scenes related to a particular player are retrieved from the
corresponding media data and only those retrieved portions are displayed on
the media data displaying portion 231.” Tsutsui ¶ 62. Thus, we are
persuaded Tsutsui teaches or suggests determining whether a tag, such as is
taught by Okada, should be attached to a first segment, instead of a second
segment. See Final Act. 16.
Further, the Examiner finds Tritschler discloses assigning a tag to a
live media event and assigning a speaker to a particular segment. Id. (citing
Tritschler, 1:48–51; 3:36–57); see also Tritschler, 3:12–15 (“The audio-
video source file may be, for example, an audio recording or live feed, for
example, from a broadcast news program.”). We agree with the Examiner.
Although Appellant contends that neither Okada nor Tsutsui nor
Tritschler teaches or suggests the “determining” and “adjusting” limitations
(Appeal Br. 13; Reply Br. 10), the Examiner relies on the combined
teachings of Okada, Tsutsui, and Tritschler to teach or suggest these

9 “Under the correct analysis, any need or problem known in the field of
endeavor at the time of invention and addressed by the patent can provide a
reason for combining the elements in the manner claimed.” KSR Int’l Co. v.
Teleflex Inc., 550 U.S. 398, 420 (2007).
Appeal 2020-000026
Application 12/887,248

21

limitations (Final Act. 14–17). Appellant cannot show nonobviousness by
attacking references individually when the Examiner’s rejection is based on
the combined teachings of the references. Ans. 10 (“It is also noted that
Okada is obviated by the teachings of Tsutsui . . . and Tritschler . . . to show
explicit teaching for the above limitations. However, the appellant does not
argue the combination of references.”); see Merck, 800 F.2d at 1097; Keller,
642 F.2d at 426. The Examiner finds Okada’s teachings, as modified by
those of Tsutsui and Tritschler, teach or suggest the “determining” and
“adjusting” limitations, as recited in the independent claims. We agree with
the Examiner, and, thus, we are not persuaded of Examiner error by
Appellant’s third reason.
2. Independent Claims 1 and 16
As noted above, claim 1 recites “a tag comprising a user-provided text
and generated during a presentation by a media server of a live media event.”
Appeal Br. 15 (Claims App.) (emphasis added). Claim 16 recites a
corresponding limitation. Id. at 17. The Examiner finds Okada and Tsutsui
teach or suggest this limitation. Final Act. 13 (citing Okada ¶ 147 (“in real
time while viewing the picture”)), 16 (citing Tsutsui ¶ 39 (“In addition, the
information processing portion 12 receives TV broadcast programs via a TV
receiver (not shown) and stores the programs in the media data storing
portion 15 as media data.”)); see Ans. 11.
Appellant contends, however, that “in real time” and “live” are not the
same thing. Appeal Br. 13; Reply Br. 10. Moreover, Appellant contends
Okada discloses embodiments in which the media may be recorded and
controls may be used to fast forward or rewind the recorded media. Id. at
13–14 (citing Okada ¶¶ 112, 147, 152, 160, Figs. 4, 10). Thus, Appellant
Appeal 2020-000026
Application 12/887,248

22

contends Okada does not teach or suggest adding tags to a “live media
event,” and neither Tsutsui nor Tritschler teaches or suggests this missing
limitation of claims 1 and 16. Id. at 14; see Reply Br. 10. We disagree with
Appellant.
Okada discloses, “Embodiment 2 of the present invention is enabled
to assign a keyword to a predetermined segment of a picture in real time
while viewing the picture.” Okada ¶ 147 (emphasis added). Further, with
respect to Embodiment 1, Okada discloses:
The authoring apparatus 100 is provided with a moving
image file input section 113 as an apparatus to input a moving
image file to be subjected to authoring by the authoring
processing section 101.
The moving image file input section 113 may be, for
example, a video camera, a storage medium storing a moving
image file such as CD and DVD or an apparatus which
distributes a moving image by means of broadcasting or
communication, etc.
Id. ¶¶ 31–32 (emphasis added). The authoring apparatus is the same for
both embodiments. Id. ¶ 148. Although we agree that “real time” is not
necessarily the same as “live” (see Ans. 11), we agree with the Examiner
that, read in context, Okada’s teaching regarding the tagging “in real time
while viewing the picture” encompasses tagging both live and recorded
media. See In re Preda, 401 F.2d 825, 826 (CCPA 1968) (“[I]t is proper to
take into account not only specific teachings of the reference but also the
inferences which one skilled in the art would reasonably be expected to draw
Appeal 2020-000026
Application 12/887,248

23

therefrom.”).10 Thus, we are persuaded Okada teaches or suggests live
media events. Final Act. 13; Ans. 11.
Further, Tsutsui discloses, “receiv[ing] TV broadcast programs via a
TV receiver.” Tsutsui ¶ 39 (emphasis added). A person of ordinary skill in
the art would have understood that broadcast TV programs encompass both
live (e.g., live sports and news programs) and recorded (e.g., dramas and
comedies) programs. See Tritschler, 11:54–56 (“The engine is both text and
language independent to facilitate live audio indexing of material such as
broadcast news.”). Thus, read in context, Tsutsui’s teaching regarding the
tagging “TV broadcast programs” also encompasses tagging both live and
recorded media. See Preda, 401 F.2d at 826.11 Therefore, we are persuaded

10 Although the Examiner does not rely on Tritschler to teach this limitation,
Appellant contends that Tritschler does not supply this limitation. Appeal
Br. 14 (“Neither Tsutsui nor Tritschler remedies this deficiency.”); but see
Final Act. 16 (“Tritschler discloses wherein the tag does not specify a
segment of the live media event to which the user-provided text applies.”
(emphasis added)). We note, however, that Tritschler discloses that the
media may be live or recorded. Tritschler, 3:12–15, 4:27–30, 11:54–56; see
Okajima v. Bourdeau, 261 F.3d 1350, 1355 (Fed. Cir. 2001).
11 See supra note 10.
Appeal 2020-000026
Application 12/887,248

24

that Tsutsui, alone or in combination with Okada, teaches or suggests
tagging a “live media event,” as recited in claims 1 and 16.
On this record, we are not persuaded the Examiner errs in rejecting
the independent claims as obvious over the combined teachings of Okada,
Tsutsui, and Tritschler. Therefore, we sustain the rejection of those claims.
3. Dependent Claim 4
Appellant does not challenge the obviousness rejection of dependent
claim 4 separately from its challenge to the rejection of independent claim 1.
Appeal Br. 14. Nevertheless, claim 4 depends from independent claim 1 via
intervening claim 3. Appeal Br. 15 (Claims. App.). As noted above, the
Examiner does not reject claim 3 as obvious over the combined teachings of
Okada, Tsutsui, and Tritschler (Final Act. 13–17; Ans. 8), but the Examiner
rejects claim 4 as obvious over that combination (Final Act. 17). The
Examiner finds, “Regarding claim 4, Okada as modified teaches the method
of claim 3, wherein user-provided text comprises a user reaction to a portion
of the first segment (Okada [0129]).” Id. The Examiner, however, does not
explain how the teachings of Okada, Tsutsui, and Tritschler are combined to
achieve the methods of claim 3. Therefore, on this record, the Examiner errs
in rejecting claim 4 as obvious over Okada, Tsutsui, and Tritschler; and we
do not sustain this rejection of claim 4.
DECISIONS
1. The Examiner does not err in rejecting claims 1–3 and 5–20 as
obvious over the combined teachings of Roberts, Finkelstein, and Lu,
alone or in combination with another reference, but errs in rejecting
Appeal 2020-000026
Application 12/887,248

25

claim 4 as obvious over the combined teachings of Roberts,
Finkelstein, and Lu
2. The Examiner does not err in rejecting claims 1, 16, and 19 as obvious
over the combined teachings of Okada, Tsutsui, and Tritschler, but
errs in rejecting claim 4 as obvious over the combined teachings of
Okada, Tsutsui, and Tritschler.
3. Thus, on this record, claims 1–3 and 5–20 are unpatentable, but
claim 4 is not unpatentable.
CONCLUSIONS
We affirm the Examiner’s rejections of claims 1–3 and 5–20 and
reverse the Examiner’s rejections of claim 4.
In summary:
Claim(s)
Rejected
35 U.S.C.
§
Reference(s)/Basis Affirmed Reversed
1, 2, 4, 16,
18, 19
103(a) Roberts, Finkelstein,
Lu
1, 2, 16, 18,
19
4
3 103(a) Roberts, Finkelstein,
Lu, Nam
3
5 103(a) Roberts, Finkelstein,
Lu, Epifania
5
6 103(a) Roberts, Finkelstein,
Lu, Gallagher
6
7–13, 15, 20 103(a) Roberts, Finkelstein,
Lu, Badoiu
7–13, 15, 20
14, 17 103(a) Roberts, Finkelstein,
Lu, Chen
14, 17
1, 4, 16, 19 103(a) Okada, Tsutsui,
Tritschler
1, 16, 19 4
Overall
Outcome
1–3, 5–20 4
Appeal 2020-000026
Application 12/887,248

26

No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).

AFFIRMED-IN-PART