Ex Parte Wu et al

Patent Trial and Appeal Board

Jun 26, 2017

12217037 (P.T.A.B. Jun. 26, 2017)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
12/217,037 06/30/2008 Yongjun Wu 3382-80113-01 7520
26119 7590 06/28/2017
KLARQUIST SPARKMAN LLP
121 S.W. SALMON STREET
SUITE 1600
PORTLAND, OR 97204
EXAMINER
CARTER, RICHARD BRUCE
ART UNIT PAPER NUMBER
2485
NOTIFICATION DATE DELIVERY MODE
06/28/2017 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docketing @klarquist.com
u sdocket @ micro soft .com
AS CChair @klarquist. com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte YONGJUN WU, ERIC S. CHRISTOFFERSEN,
and
NAVEEN THUMPUDI
Appeal 2016-002588
Application 12/217,037
Technology Center 2400
Before CARLA M. KRIVAK, WILLIAM M. FINK, and
DAVID J. CUTITTAII, Administrative Patent Judges.
KRIVAK, Administrative Patent Judge.
DECISION ON APPEAL
Appellants appeal under 35 U.S.C. § 134(a) from a final rejection of
claims 1—15 and 21—34. We have jurisdiction under 35 U.S.C. § 6(b).
We affirm-in-part.
Appeal 2016-002588
Application 12/217,037
STATEMENT OF THE CASE
Appellants’ invention is directed to techniques and tools for “various
aspects of error detection, error recovery and error concealment in video
decoder implementations. These techniques and tools help, for example, to
efficiently detect errors during video decoding, and to recover and conceal
detected errors” (Spec 6:12—15). The techniques and tools employ one or
more “storage media store software” for a computer system to detect and
correct errors in a video bit stream in a multithreaded video decoding system
(Spec. 6:16-19).
Independent claim 1, reproduced below, is exemplary of the subject
matter on appeal.
1. A multi-threaded video decoding system comprising
one or more processors, memory and one or more storage
media having stored thereon software for causing the system to
perform a method for detecting and correcting errors in a video
bit stream, the software comprising:
code for parsing network abstraction layer units
(NALUs) in the video bit stream, including code for:
detecting corrupted NALUs, if any; and
discarding the corrupted NALUs, if any;
code for, during a picture extent discovery (PED) task
with PED sub-stages:
identifying a corrupted picture encoded in the bit
stream, wherein the PED sub-stages include a sub-stage for
initializing structures for decoding the picture and a sub-stage
for entering the initialized but not yet decoded picture in a
decoded picture buffer (DPB) to facilitate management of the
DPB, during decoding of plural pictures in flight at various
stages of multi-threaded decoding, that simulates management
of the DPB as in single-threaded decoding;
determining which of the PED sub-stages, if any,
have been completed for the picture; and
2
Appeal 2016-002588
Application 12/217,037
based on the PED sub-stages, if any, which have
been completed, modifying execution of one or more PED
operations;
code for, during an entropy decoding task:
verifying macroblocks on a per-macroblock basis;
and
terminating the entropy decoding task when
corruptions are discovered; and
code for, during motion compensation and intra­
prediction tasks:
marking corrupted macroblocks as corrupted; and
after discovering corrupted macroblocks,
continuing to decode additional macroblocks; and
code for disabling deblocking across boundaries of
corrupted macroblocks.
REFERENCES and REJECTIONS
The Examiner rejected claims 1 and 15 under 35 U.S.C. § 103(a)
based upon the teachings of Liu (US 2008/0049844 Al, published Feb. 28,
2008), Katsavounidis (US 2012/0147956 Al, published June 14, 2012) and
Jeon (US 7,742,532 B2, issued June 22, 2010).
The Examiner rejected claims 2—14 and 21—34 under 35 U.S.C.
§ 103(a) based upon the teachings of Liu and Katsavounidis.
ANALYSIS
We initially note Appellants group independent claims 1, 2, and 21
together as they argue one limitation found in all three claims (App. Br. 6).
Therefore, we also address these claims together, with claim 1 in particular
being addressed.
Appellants’ main contention is that the combination of Liu,
Katsavounidis, and Jeon does not teach or suggest the claim limitation
3
Appeal 2016-002588
Application 12/217,037
identifying a corrupted picture encoded in the bit stream,
wherein the PED sub-stages include a sub-stage for initializing
structures for decoding the picture and a sub-stage for entering
the initialized but not yet decoded picture in a decoded picture
buffer (DPB) to facilitate management of the DPB, during
decoding of plural pictures in flight at various stages of multi­
threaded decoding, that simulates management of the DPB as in
single-threaded decoding.
Particularly, Appellants contend Liu does not disclose: management of a
DPB (App. Br. 7) or a “sub-stage for entering an initialized but not yet
decoded picture in a DPB” (App. Br. 7—8); Katsavounidis “does not provide
any motivation for using the convention frame buffer 250” for “entering the
initialized but not yet decoded picture in a decoded picture buffer (DPB) to
facilitate management of the DPB, during decoding of plural pictures in
flight at various stages of multi-threaded decoding” (App. Br. 9); and Jeon
does not disclose “code for disabling deblocking across boundaries of
corrupted macroblocks” (App. Br. 10). We do not agree.
We agree with and adopt the Examiner’s findings as our own (Ans. 2—
7; 23—24). First, we note the Examiner also relies on Liu for “entering the
initialized by not yet decoded picture . . .” limitation (Ans. 23—25; Final Act.
3). With respect to Appellants’ argument that Liu does not address
management of a DPB, Liu’s paragraph 58 discloses picture and slice header
information decoded in a previous syntax decoding thread (such as thread
302 in Fig. 3) are stored in a buffer from which the decoder reads the
information, decodes remaining data in each macroblock of the picture, and
reconstructs the macroblock, thus teaching Appellants’ claimed decoded
picture buffer (DPB) (Ans. 23 (citing Liu 158)). That is, Liu slices “each
picture . . . into two or more sections” having slices and macroblocks (see
4
Appeal 2016-002588
Application 12/217,037
Fig. 1, || 27, 29), recursively decodes each section’s macroblocks in a
separate thread (see Figs. 2—3, || 31, 36, 38), and stores the initialized but
not yet decoded picture in the DPB while remaining picture macroblocks
and slices are being decoded (Fig. 8,158). Liu’s Figure 7 shows initializing
the header syntax (step 702) and syntax elements 706, 708, 710, and 712 as
teaching the PED sub-stages, as claimed (Liu | 52, Fig. 7; Ans. 23). We
further note Appellants’ Specification broadly describes an “initialized but
not yet decoded picture” as follows: “[f]or example, the PED module finds
picture boundaries and logs data in a lightweight look-ahead process,
initializing parameters and data structures for pictures encountered in the bit
stream” (Spec. 24:8—12), “[t]he decoder initializes (520) structures for
holding parameters and data for the picture ... at the PED stage, it may
suffice to determine which pictures are going to be decoded and determine
what the values of certain parameters are, without referencing all pixel
data[,alternatively, the decoder initializes other structures for the picture”
(Spec. 24:23—25:2), and “an initialized picture has its picture header and
slice header parameters correctly decoded from the bit stream as part of PED
but sample data and side information are not yet decoded” (Spec. 28:18—20).
Thus, an initialized but not yet decoded picture, as described in Appellants’
Specification, is broad and is similar to Liu’s partially processed picture
sections, slices, and macroblocks in Figures 3 and 7.
Appellants additionally contend that even if Liu’s picture
reconstruction can involve storing “a reconstructed picture in a DPB, such
activity happens after decoding has finished for the picture” (App. Br. 7;
Reply Br. 4). This statement is, however, incorrect. Liu’s Figure 3 and
paragraph 38 recite “Pixel prediction and reconstruction 318 produces
5
Appeal 2016-002588
Application 12/217,037
decoded pixels 319 that include neighbor pixels which may be used as inputs
to the pixel prediction and reconstruction process 318 for a subsequent
macroblock. The de-blocking task group 320 includes a de-blocking stage
322 that produces a decoded picture 324. The decoded picture 324 may
provide neighboring pixels for use in de-blocking a neighboring macroblock.
In addition, decoded pictures 324 may provide reference pixels for pixel
prediction and reconstruction 318 for subsequent macroblocks,” and
decoded picture 324 is a “Decoded Section” of a picture (see Fig. 3, block
324,136). Thus, Liu teaches decoding macroblock by macroblock in
sections, until the entire picture is decoded. That is, Liu’s picture
reconstruction is part of the picture’s decoding process, contrary to
Appellants’ argument that picture reconstruction occurs only “after decoding
has finished for the picture” (App. Br. 7; Reply Br. 4).
Appellants also argue Jeon’s filter deblocking does not relate to
“disabling deblocking ‘across boundaries of corrupted macroblocks’” (App.
Br. 10). However, as the Examiner finds (Ans. 24), Liu handles
corrupted/erroneous macroblocks, for which “decoding and reconstruction
814 may be skipped” (see Liu 1 58 (emphasis added); Liu Fig. 3 (showing
decoding and reconstruction includes de-blocking 322)).
Thus, in light of the broad claim limitations and the Examiner’s
findings, we sustain the Examiner’s rejection of independent claims 1, 2, and
21.
Appellants separately argue dependent claim 3 asserting Liu’s Figures
3 and 7 and paragraphs 34—36 and 52 do not disclose the claimed limitation
“PED sub-stages further include a sub-stage for tracking dependencies for
entry of tasks into a task scheduling data structure” (App. Br. 10-11) and
6
Appeal 2016-002588
Application 12/217,037
Liu’s Table II discloses three main tasks can be performed in parallel but
does not disclose tracking dependencies and a task scheduling data structure
(Reply Br. 5). However, Appellants’ Specification states a decoder tracks
dependencies (Spec. 24:3—13). Further, Appellants’ Specification broadly
defines a task as “a stage plus input data” (Spec. 18, last line in table) and a
task scheduler that “distributes available tasks to different threads according
to one of several available scheduling heuristics. An available task can
process data for a picture, slice or other segment (collection of
macroblocks)” (Spec. 19:3—6). This is similar to Liu’s teachings in Figure 3
(and related discussion, see, e.g., Liu 138 describing recursive macroblock
processing within picture’s sections) and paragraph 63 (see Ans. 24).
Additionally, Appellants recite what Liu discloses and what the claim
recites, but do not state why or how the claimed limitation is different from
Liu’s cited disclosure other than stating Liu does not disclose the claim
language.
Thus, in light of the above findings, we sustain the Examiner’s
rejection of claim 3.
Appellants separately argue claim 4 asserting Liu’s paragraph 52 does
not disclose “tasks for the picture are determined to have entered the task
scheduling data structure and the picture is determined to have entered the
DPB” (App. Br. 11). The Examiner, however, also relies on Liu’s paragraph
63 referencing Figures 3, 7, and Table II (Ans. 25). As discussed supra with
respect to claim 3, we agree with the Examiner that Liu teaches task entry
into a task scheduling data structure. Additionally, Liu teaches a picture
determined to have entered the DPB storage buffer when all picture and slice
header information has been decoded and stored in the buffer (see Liu 1 58
7
Appeal 2016-002588
Application 12/217,037
(describing Figure 8)). As with claim 3, Appellants recite portions of
paragraph 52 without stating how those portions are different from
Appellants’ claim, and do not address the Examiner’s reliance on paragraph
63 and Figures 3 and 5—8 (Ans. 12, 25). Thus, we agree with the Examiner
that Liu, at a minimum, discloses “tasks for the picture are determined to
have entered the task scheduling data structure and the picture is determined
to have entered the DPB” recited in claim 4, and sustain the Examiner’s
rejection of claim 4.
With respect to claim 5, Appellants assert Liu does not disclose “tasks
for the picture are determined to have entered the task scheduling data
structure but the picture is determined not to have entered the DPB” (App.
Br. 12). Claim 5 therefore recites a relationship between entering picture
tasks in the task scheduling data structure and a picture not entering the
DPB. The Examiner cites paragraph 58 of Liu for this limitation (Ans. 25—
26). However, Liu’s paragraph 58 states that “[t]he picture and slice header
information may have been decoded in a previous syntax decoding thread
(e.g., the NAL decoding thread 302 from FIG. 3) and stored in a buffer
[DPB],” but does not describe the claimed “picture is determined not to have
entered the DPB” in connection with a task entering determination, as
claimed (see Liu 1 58 (emphasis added)). On the record before us, we agree
with Appellants that the Examiner has not shown how Liu teaches this
limitation and we therefore do not sustain the Examiner’s rejection of claim
5.
Appellants assert Liu does not disclose structures for a picture are
determined to have not yet been initialized for decoding, and checking the
8
Appeal 2016-002588
Application 12/217,037
picture command queue for commands to execute, as recited in claim 6
(App. Br. 12-13).
The Examiner relies on Liu’s paragraph 57 and Steps 716 and 734
(Fig. 7) for the checking limitation (Ans. 26—27). However, Liu merely
teaches “the mb_err_status is set to ERROR to indicate this is an erroneous
macroblock”—no checking occurs (Liu 157) and Liu’s step 716 only
concerns setting a flag (Ans. 27). On the record before us, we agree with
Appellants that the Examiner has not shown Liu teaches or suggests
checking a picture command queue for commands to execute, as claimed.
We therefore do not sustain the Examiner’s rejection of claim 6.
Appellants assert Liu’s paragraph 52 and Figures 5—8 do not disclose
“structures for the picture are determined to have been initialized but tasks
for the picture are determined not to have entered the task scheduling data
structure,” and “one or more PED operations to handle the error comprise
recycling resources used by the picture”, as recited in claim 7 (App. Br. 13).
Appellants want Liu to recite Appellants’ claim language verbatim.
However, as Appellants’ Specification does not describe “recycling
resources used by the picture” and merely describes a “decoder [that]
recycles the picture” (Spec. 37:24), the Examiner is correct in interpreting
“recycling resources” as recovering, which Liu performs (Liu Fig. 7,1 52;
Ans. 14). As to initializing structures but not entering tasks into the
scheduling data structure, we agree with the Examiner that Liu’s paragraphs
63 and 57 and Figure 7 suggest this limitation (Ans. 27). Thus, we sustain
the Examiner’s rejection of claim 7.
Appellants assert Liu’s paragraphs 40, 43, and 52 do not disclose
“parsing the bit stream and verifying that only network abstraction layer
9
Appeal 2016-002588
Application 12/217,037
units (NALUs) which have proper start codes and which are within a
predetermined maximum size are accepted as valid data” (emphasis added),
as recited in claim 8 (App. Br. 14).
The Examiner relies on paragraphs 40 and 43 of Liu for disclosing
this limitation (Ans. 27). However, the Examiner merely recites the claim
language and these paragraphs of Liu without further explanation. Although
Liu’s Ligure 3, Step 302 discloses NAL decoding (Liu Lig. 3), Ligures 4A—
4B and paragraph 40 disclose picture header syntax decoding (402) and slice
header decoding (404) as part of a NAL decoding module, and Liu’s
paragraph 43 discloses that some decoding and error handling can occur at
the NAL decoding stage, we agree with Appellants Liu does not teach or
suggest verifying that only NALUs with proper start codes and within a
predetermined maximum size are considered valid data, as recited in claim
8. Therefore, on this record, we do not sustain the Examiner’s rejection of
claim 8, and claims 9 and 10 depending therefrom. Additionally, we agree
with Appellants the cited references do not disclose the specific NALU
processing steps recited in claims 9 and 10 (App. Br. 14—16).
With respect to claim 11, Appellants acknowledge Liu’s paragraph 35
discloses entropy decoding, but assert Liu does not disclose specifics as to
how the entropy decoding is implemented and “checking validity of entropy
decoding on a macroblock-by-macroblock basis for each of plural
macroblocks after each macroblock is decoded,” as recited in claim 11 (App.
Br. 16). We agree with Appellants.
The Examiner cites Liu’s paragraph 35 and Ligures 3 and 8 as
disclosing the limitations of claim 11 (Ans. 29). Liu, however, only teaches
entropy decoding, not checking validity of the entropy decoding after the
10
Appeal 2016-002588
Application 12/217,037
macroblock is decoded, as claimed. That is, claim 11 requires the checking
to be done after each macroblock is decoded, while Liu performs general
entropy decoding before macroblocks are decoded (see Liu Fig. 3, step 306).
Thus, on the record before us, we do not sustain the Examiner’s rejection of
claim 11.
Appellants assert Liu’s paragraph 34 and Figure 3 do not teach or
suggest “performing an entropy decoding task utilizing a redundant buffer
comprising an overflow area” and “if a decoded macroblock enters the
overflow area, terminating the entropy decoding task” as recited in claim 12
(App. Br. 17). We agree.
The Examiner, in the Answer, relies on Liu’s paragraphs 35 and 58,
and states “Liu already discloses performing an entropy decoding task,” thus
teaching the claim 12 limitations, without providing argument or reasoning
(Ans. 29-30). As we note above with respect to claim 11, Liu does indeed
teach entropy decoding. However, as with claim 11, Liu does not teach
further specifics of entropy decoding as recited in claim 12, and thus we do
not sustain the Examiner’s rejection of claim 12.
Appellants assert Liu’s paragraph 34 and Figure 3 do not teach or
suggest “performing syntax verifications during an entropy decoding task
and, when syntax verifications indicate a corrupted syntax element,
terminating the entropy decoding task” as recited in claim 13 (App. Br. 17—
18). We agree.
The Examiner again relies on Liu’s paragraphs 34 and 35 and Figure 3
(Ans. 30). As noted above with respect to claim 11, these portions of Liu do
not teach or suggest the further limitations of entropy decoding recited in
11
Appeal 2016-002588
Application 12/217,037
claim 13. Thus, on the record before us, we do not sustain the Examiner’s
rejection of claim 13.
Appellants assert claims 14 and 15 are allowable due to their
dependence from claim 2 (App. Br. 18). As we sustain the Examiner’s
rejection of claim 2, we also sustain the Examiner’s rejection of claims 14
and 15.
Appellants assert claims 22—34 correspond to claims 2—15 and
therefore are allowable for the same reasons as claims 2—15 (id.). As such,
we sustain the Examiner’s rejection of claims 22, 23, 26, 33, and 34, for the
same reasons as discussed with respect to claims 3, 4, 7, 14, and 15, and we
do not sustain the Examiner’s rejection of claims 24, 25, and 27—32 for the
same reasons as discussed with respect to claims 5, 6, and 8—13.
DECISION
The Examiner’s decision rejecting claims 1—4, 7, 14, 15, 21—23, 26,
33, and 34 is affirmed.
The Examiner’s decision rejecting claims 5, 6, 8—13, 24, 25, and 27—
32 is reversed.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv).
AFFIRMED-IN-PART
12