Ex Parte Arimilli et al

Patent Trial and Appeal Board

Oct 30, 2013

10733953 (P.T.A.B. Oct. 30, 2013)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________________

BEFORE THE PATENT TRIAL AND APPEAL BOARD
____________________

Ex parte RAVI KUMAR ARIMILLI, SANJEEV GHAI, and
WARREN EDWARD MAULE
____________________

Appeal 2011-004137
Application 10/733,953
Technology Center 2100
____________________

Before ST. JOHN COURTENAY III, THU A. DANG, and
CARL W. WHITEHEAD, JR., Administrative Patent Judges.

DANG, Administrative Patent Judge.

DECISION ON APPEAL

Appeal 2011-004137
Application 10/733,953

2
I. STATEMENT OF THE CASE
Appellants appeal under 35 U.S.C. § 134(a) from a Final Rejection of
claims 1-6 and 8-24 (App. Br. 2). Claim 7 has been canceled (id.). We have
jurisdiction under 35 U.S.C. § 6(b).
We affirm-in-part.

A. INVENTION
Appellants’ invention is directed to a method and data processing
system having a system memory, a plurality of processing units, and a
memory controller including a memory speculation table that stores
historical information regarding prior memory accesses; wherein, in
response to a memory access request, the memory controller speculatively
initiates access to the system memory based upon data in the memory
speculation table prior to receipt of a coherency message issued by a
processing unit indicating whether or not it is able to process the request and
what proposed data operations it will perform in response to the request
(Spec., ¶¶ [0006] and [0027], Abstract.)

B. ILLUSTRATIVE CLAIM
Claim 1 is exemplary:
1. . A data processing system, comprising:
a system memory;
a plurality of processing cores;
a plurality of a cache memories, each coupled to a
respective one of the plurality of processing cores and to
an interconnect, wherein the plurality of cache memories
temporarily hold cache lines of data identified by
addresses of storage locations in the system memory and
Appeal 2011-004137
Application 10/733,953

3
certain of the plurality of cache memories service
memory access requests received via the interconnect
that target those addresses; and
a memory controller, coupled to said interconnect
and to the system memory, that controls access to the
system memory, said memory controller having a
memory speculation mechanism that indicates whether or
not to perform speculative access to the system memory
based upon historical information regarding whether or
not prior memory accesses were serviced by accessing
the system memory, wherein said memory controller,
responsive to receipt of a memory access request
broadcast to the memory controller and the plurality of
cache memories, said memory access request specifying
a target system memory address:
if speculative access is indicated by the
memory speculation mechanism, speculatively
initiates access to the system memory to service
the memory access request in advance of receipt
by the memory controller of a coherency message
indicating whether or not said memory access
request is to be serviced by the memory controller
accessing said system memory; and
if speculative access is not indicated by the
memory speculation mechanism, initiates non-
speculative access to the system memory to service
the memory access request only in response to the
coherency message indicating that the memory
access request is to be serviced by the memory
controller accessing the system memory.

C. REJECTIONS
The prior art relied upon by the Examiner in rejecting the claims on
appeal is:
Revilla U.S. 5,926,831 July 20, 1999
Appeal 2011-004137
Application 10/733,953

4
Shibayama U.S. 2002/0178349 A1 Nov. 28, 2002
Dice U.S. 2003/0033510 A1 Feb. 13, 2003
Nilsson U.S. 2003/0154351 A1 Aug. 14, 2003
Gharachorloo U.S. 6,697,919 B2 Feb. 24, 2004

Claims 1-4, 6, 8-11, 13-16, and 18 stand rejected under 35 U.S.C.
§ 103(a) as being unpatentable over Gharachorloo in view of Shibayama.
Claims 5, 12, and 17 stand rejected under 35 U.S.C. § 103(a) as being
unpatentable over Gharachorloo in view of Shibayama and Nilsson.
Claims 19, 21, and 23 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over Gharachorloo in view of Shibayama and Dice.
Claims 20, 22, and 24 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over Gharachorloo in view of Shibayama and Revilla.

II. ISSUES
The dispositive issues before us are whether the Examiner has erred in
determining that the combination of Gharachorloo and Shibayama teaches or
would have suggested:
1. A memory controller having “a memory speculation mechanism
that indicates whether or not to perform speculative access to the system
memory based upon historical information regarding whether or not prior
memory accesses were serviced by accessing the system memory” (claim1,
emphasis added);
2. If speculative access is indicated by the memory speculation
mechanism, the memory controller “speculatively initiates access to the
system memory to service the memory access request in advance of receipt
Appeal 2011-004137
Application 10/733,953

5
by the memory controller of a coherency message indicating whether or not
said memory access request is to be serviced by the memory controller
accessing said system memory” (claim 1, emphasis added); and
3. If speculative access is not indicated by the memory speculation
mechanism, the memory controller “initiates non-speculative access to the
system memory to service the memory access request only in response to the
coherency message” (claim 1, emphasis added).

III. FINDINGS OF FACT
The following Findings of Fact (FF) are shown by a preponderance of
the evidence.
The Invention
1. According to Appellants, a processing unit generates a
coherency message after it has reviewed the memory access request and
determined that it is able to process the request and, if so, the proposed data
operations the processing unit will perform in response to the request (Spec.
¶ [0027]).
Gharachorloo
2. Gharachorloo discloses a multiprocessor system 100 including
a plurality of processor cores 106 indirectly coupled to a plurality of
memory controllers 118 which interface directly to a memory bank of
Dynamic Random Access Memory (DRAM) chips; wherein, each processor
core has its own memory and memory controller (col. 4, l. 53 to col. 5, l. 7;
Fig. 1).
3. The system uses thread-level parallelism, such as simultaneous
multi-threading, arising from relatively independent transactions or queries
Appeal 2011-004137
Application 10/733,953

6
initiated by different clients, to hide I/O latency in workloads (col. 1, ll. 31-
34 and col. 2, ll. 21-30).
4. The memory is arranged using a plurality of memory banks
(col. 5, ll. 1-7 and col. 11, ll. 56-61).
Shibayama
5. Shibayama discloses a processor and a multi-processor system
having a data dependence speculation execution function where memory
operation instructions are executed out-of-order by enabling the
microprocessor to execute instructions of a program in an order that is
different from the “program order” (¶ [0062]).
6. The processor includes a speculative execution result history
storage means, speculative execution success/failure prediction means,
instruction execution means, speculation control means, speculative
execution success/failure judgment means, and speculative execution result
history update means ((¶ [0062]). The speculative execution result history
storage means stores history information relating to success and failure
results of past speculative execution of memory operation instructions (id.).
The speculative execution success/failure prediction means predicts whether
a proposed speculative execution of a memory operation instruction will
succeed or fail, by referring to an entry of the speculative execution result
history storage means relative to the memory operation instruction’s target
address (id.).
7. The speculation control means enables the instruction execution
means to execute the memory operation instruction out-of-order when it is
predicted that the speculative execution will be a success; and if the
predication is a determined to be a failure, the speculation control means
Appeal 2011-004137
Application 10/733,953

7
enables the instruction execution means to execute the memory operation
instruction in an in-order manner using non-speculative execution (¶[0062]).
8. The speculative execution success/failure judgment means
judges whether the speculative execution has succeeded or failed by
detecting the dependence relationship between the memory operation
instructions, after which the speculative execution result history update
means updates the history information stored in the speculative execution
result history storage means ((¶ [0062]).

IV. ANALYSIS
Claims 1, 2, 8-11, 13-16, and 18
Appellants contend that “Shibayama’s disclosure is directed to a
processor’s speculative instruction execution . . . rather than a memory
controller’s speculative access to system memory based upon whether or not
prior memory accesses were serviced by accessing the system memory as
claimed” (App. Br. 9). Appellants contend that “Shibayama’s
success/failure indications provide no information ‘regarding whether or not
prior memory accesses were serviced by accessing the system memory’”
(App. Br. 10). Appellants argue that “the combination of cited references
does not disclose . . . the claimed ‘coherency message’” or “speculatively
accessing system memory before a coherency message is received if the
memory speculation mechanism indicates a speculative access and non-
speculatively accessing system memory after a coherency message is
received if the memory speculation mechanism indicates a non-speculative
access” (App. Br. 10-11).
Appeal 2011-004137
Application 10/733,953

8
However, the Examiner finds that Shibayama teaches a “processor
having a function for executing memory operations instructions by means of
speculative execution [] that stores historical information . . . regarding
whether prior memory accesses were serviced by accessing the system
memory” (Ans. 16-17)(citations omitted). The Examiner further finds that
Shibayama teaches “if execution success/failure prediction indicates
speculative access (i.e. success) the memory operation instructions (i.e.,
load/write instructions) are executed in a speculative execution)” and “if
execution success/failure prediction does not indicate speculative access (i.e.
failure) the memory operation instructions (i.e., load/write instructions) are
executed in a non-speculative execution)” (Ans. 18).
We give the claim its broadest reasonable interpretation consistent
with the Specification. See In re Morris, 127 F.3d 1048, 1054 (Fed. Cir.
1997). Claim 1 defines “coherency message” as data received by the
memory controller “indicating whether or not said memory access request is
to be serviced by the memory controller accessing said system memory.”
The Specification discloses that after a processing unit reviewed the memory
access request and determined that it is able to process the request, it
generates the coherency message and, if it is able to process the request,
sends the proposed data operations that it will perform in response to the
request in the coherency message. That is, as set forth in the claim and
Specification, the “coherency message” is data received by the memory
controller that is generated by a processing unit.
We note that “indicating whether or not said memory access request is
to be serviced by the memory controller accessing said system memory”
(claim 1) is the description of the type or content of the data in the
Appeal 2011-004137
Application 10/733,953

9
“coherency message,” however, the type or content of the data does not
change the functionality of or provide an additional function to receipt or
generation of the message. That is, what type or content of data contained in
the message does not limit how the respective message is received or
generated, but is merely non-functionally descriptive of the data. When
descriptive material is not functionally related to the claimed medium, the
descriptive material will not distinguish the invention from the prior art in
terms of patentability. See In re Ngai, 367 F.3d 1336, 1339 (Fed. Cir. 2004)
and In re Gulack, 703 F.2d 1381, 1385 (Fed. Circ. 1983).
That is, we conclude that claim 1 merely requires that the memory
controller: 1) performs a speculative access to system memory prior to
receipt of data relating to the servicing of the memory access request when
the speculation mechanism indicates that a speculative access should be
performed, and 2) performs a non-speculative access to system memory after
receipt of data that affirms that the memory controller should service the
memory access request when speculative access should not be performed.
Gharachorloo is directed to a multiprocessor system including
memory controllers that access a memory bank of DRAM chips (FF 2).
We find Gharachorloo’s multiprocessor system comprises a “memory
controller” (claim 1).
In addition, Shibayama discloses a processor having a data
dependence speculation execution function where memory operation
instructions are executed out-of-order (FF 5). In particular, the processor
includes a speculative execution result history storage means that stores
history information relating to success and failure results of past speculative
execution of memory operation instructions; wherein, a speculative
Appeal 2011-004137
Application 10/733,953

10
execution success/failure prediction means predicts whether a proposed
speculative execution of a memory operation instruction will succeed or fail,
by referring to an entry of the speculative execution result history storage
means relative to the memory operation instruction’s target address (FF 6).
The processor further includes a speculation control means which
enables an instruction execution means to execute the memory operation
instruction out-of-order when it is predicted that the speculative execution
will be a success; and if the predication is a determined to be a failure, the
instruction execution means executes the memory operation instruction in an
in-order manner using non-speculative execution (FF 7). The speculative
execution success/failure judgment means judges whether the speculative
execution has succeeded or failed by detecting dependency relationship
information between memory operation instructions; wherein, the
speculative execution result history update means updates the history
information stored in the speculative execution result history storage means
afterwards (FF 8).
We find that Shibayama’s processor comprises a memory controller
that includes a memory speculation mechanism (a speculative execution
success/failure prediction means and a speculative execution success/failure
judgment means). We also find that Shibayama’s memory controller
(instruction execution means) performs a speculative access to system
memory prior to receipt of any data relating to the servicing of the memory
access request when the speculation mechanism (speculative execution
success/failure prediction means) indicates that a speculative access will be a
success (should be performed), and performs a non-speculative access to
system memory after receipt of data (speculation control means enabling
Appeal 2011-004137
Application 10/733,953

11
control data) that affirms that the memory controller should service the
memory access request when speculative access will fail (should not be
performed).
In view of our claim construction above, we find that the combination
of Gharachorloo and Shibayama at least suggests all of the claimed features
of claim 1.
Accordingly, we find no error in the Examiner’s rejection of claim 1
under 35 U.S.C. § 103(a) over Gharachorloo in view of Shibayama. Further,
independent claims 9 and 14 having similar claim language and claims 2, 8,
10, 11, 13, 15, 16, and 18 (depending from claims 1, 9, and 14) which have
not been argued separately, fall with claim 1.
Claim 3
Appellants contend that “the combination of Gharachorloo and
Shibayama discloses that entries in a speculative execution result history
table are indexed by hashed target addresses of the memory access
instructions, rather than on a per-thread basis” (App. Br. 12) (citation
omitted). However, the Examiner finds that Gharachorloo teaches that a
“directory is in the protocol engine of processor” “that stores a respective
memory access history . . . for each of a plurality of threads executing within
said one or more processing cores” as indicated by the disclosure to
“‘simultaneous multithreading (SMT)’” and “‘instruction-level parallelism
and speculative out-of-order execution’” (Ans. 19).
Gharachorloo discloses a system that uses thread-level parallelism,
such as simultaneous multi-threading (FF 3). We find that Gharachorloo’s
system comprises a plurality of processing cores where “a plurality of
threads [are] execut[ed] within one or more processing cores” (claim 3).
Appeal 2011-004137
Application 10/733,953

12
Thus, we conclude that the combined teaching of Gharachorloo and
Shibayama would at least suggest the features of claim 3.
Accordingly, we find no error in the Examiner’s rejection of claim 3
under 35 U.S.C. § 103(a) over Gharachorloo in view of Shibayama.
Claim 4
Appellants contend that “the combination of Gharachorloo and
Shibayama discloses that entries in a speculative execution result history
table are indexed by hashed target addresses of the memory access
instructions, rather than on a per-bank basis” (App. Br. 12). However, the
Examiner finds that Gharachorloo teaches a system memory that “includes a
plurality of storage locations” “arranged in a plurality of banks” and memory
controller “MC 118 [which] interfaces directly with a memory bank in
memory subsystem 123” (Ans. 19-20).
Gharachorloo discloses a system having a memory that is arranged
using a plurality of memory banks (FF 4). We find that Gharachorloo’s
memory comprises “a plurality of storage locations arranged in a plurality of
banks” (claim 4). Thus, we conclude that the combined teaching of
Gharachorloo and Shibayama would at least suggest all the claim limitations
of claim 4.
Accordingly, we find no error in the Examiner’s rejection of claim 4
under 35 U.S.C. § 103(a) over Gharachorloo in view of Shibayama.
Claim 6
Appellants contend that “Gharachorloo’s disclosure of a conventional
cache coherency protocol fails to disclose or render obvious the speculative
initiation of access to a system memory by a first system memory controller
Appeal 2011-004137
Application 10/733,953

13
based upon historical information recorded by a second memory controller”
as recited in claim 6 (App. Br. 13).
However, the Examiner finds that Gharachorloo teaches a “‘memory
bank;’” “that each ‘processor core has its own memory (Ll cache, L2 cache,
memory bank of DRAM) as well as memory controller;” and “a cache
coherence protocol (CCP) that enables the sharing of memory lines of
information 184 across multiple nodes” (Ans. 20). The Examiner finds
further that “Gharachorloo teaches that memory speculation can be accessed
by another node [such as the] processor core, memory controller” (id.). The
Examiner notes that “a [‘]second memory controller[’] can be any of a
plurality of memory controllers” (Ans. 21).
As noted supra, Gharachorloo discloses a multiprocessor system
including a plurality of processor cores indirectly coupled to a plurality of
memory controllers; wherein, each processor core has its own memory and
memory controller (FF 2). We find that Gharachorloo’s plurality of memory
controllers comprises at least “a first memory controller” and “a second
memory controller” (claim 6).
As noted supra, Shibayama’s processor includes a speculation control
means enables the instruction execution means execute the memory
operation instruction out-of-order when it is predicted that the speculative
execution will be a success using history information stored in the
speculative execution result history storage means (FF 6 and 7). We find
that the processor includes a “memory controller [that] speculatively initiates
access to a first system memory based upon historical information recorded
by [the] memory controller” (claim 6).
Appeal 2011-004137
Application 10/733,953

14
Thus, we conclude that the combined teachings of Gharachorloo and
Shibayama would at least suggest the use of “historical information in a
second memory controller”. Accordingly, we find no error in the
Examiner’s rejection of claim 6 under 35 U.S.C. § 103(a) over Gharachorloo
in view of Shibayama.
Claims 19, 21, and 23
Appellants argue that although “the combination of Gharachorloo,
Shibayama and Dice discloses a system in which speculative instruction
execution is allowed in response to determining ‘at least two processors in
the computerized device do not have a potential to execute instructions that
reference locations within a shared page of memory,’” it “does not disclose,
suggest or render obvious a condition for updating of a memory speculation
mechanism . . . in response to ‘confirmation of correctness of speculative
access to the system memory as indicated by the coherency message’” (App.
Br. 14). However, the Examiner finds that Dice teaches a “system [that]
uses a MESI coherency protocol to update or change a coherency indicator
that indicates if speculative execution is allowed or not (i.e., correct or not))”
(Ans. 21).
As noted supra, Shibayama’s processor a speculative execution
success/failure judgment means which judges whether the speculative
execution has succeeded or failed; wherein, a speculative execution result
history update means updates the history information stored in the
speculative execution result history storage means (FF 8). We find that the
processor comprises a memory controller, responsive to data relating to the
processing of a memory access request that updates the memory speculation
Appeal 2011-004137
Application 10/733,953

15
mechanism in response to confirmation of correctness (success) of
speculative access to the system memory.
In view of our claim construction above with respect to “coherency
message,” we conclude that the combined teachings of Gharachorloo and
Shibayama would at least suggest all the claim limitations of claim 19.
Accordingly, we find no error in the Examiner’s rejection of claim 19 under
35 U.S.C. § 103(a) over Gharachorloo in view of Shibayama and Dice.
Claims 21 and 23 having similar claim limitations (depending from claims 9
and 14, respectively) which have not been argued separately, fall with claim
19.
Claims 20, 22, and 24
Appellants argue that “[t]he combination of cited references, while
disclosing that speculative access to memory should be prevented to avoid
use of incorrect data, clearly does not disclose or render obvious discarding
data associated with a memory access request” (App. Br. 15).
After reviewing the record on appeal, we agree with Appellants.
Though we agree with the Examiner that Revilla teaches that the “memory
controller . . . stop[s] the speculative request and the data associated with the
request is not used” (Ans. 22), we cannot find any teaching in the
Examiner’s recited portion of Revilla teaches or suggests that “the memory
controller, responsive to the coherency message indicating speculative
access to the system memory by the memory controller was incorrect,
discards data associated with the memory access request” as required by
claim 20 (emphasis, added). That is, even though Revilla teaches that the
data is not used, there is no teaching or suggestion of discarding this data.
Appeal 2011-004137
Application 10/733,953

16
Accordingly, we find that Appellants have shown that the Examiner
erred in rejecting claim 20 under 35 U.S.C. § 103(a) over Gharachorloo in
view of Shibayama and Revilla. Claims 22 and 24 having similar claim
limitations (depending from claims 9 and 14, respectively) which have not
been argued separately, fall with claim 20.
Claims 5, 12, and 17
Appellants argue that claims 5, 12, and 17 are patentable over the
cited prior art for the same reasons asserted with respect to claim 1 (App. Br.
7).
As noted supra, however, we find that the combined teachings of
Gharachorloo and Shibayama at least suggest all the features of claim 1.
Therefore, we affirm the Examiner’s rejection of claims 5, 12, and 17 under
35 U.S.C. § 103 (a) over Gharachorloo in view of Shibayama and Nilsson.

V. CONCLUSION AND DECISION
The Examiner’s rejection of claims 20, 22, and 24 under 35 U.S.C.
§ 103(a) is reversed; while the Examiner’s rejection of claims 1-6, 8-19, 21,
and 23 under 35 U.S.C. § 103(a) is affirmed.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED-IN-PART

llw