Ex parte Maguire

Board of Patent Appeals and Interferences

Sep 26, 2001

08560108 (B.P.A.I. Sep. 26, 2001)

Application for patent filed November 17, 1995.1
The copy of claim 6 provided in Appellant's Appendix to2
the Brief contains the typographical errors of using "=" where
"'" is required. In claim 20, line 2, following the word
1
The opinion in support of the decision being entered today
was not written for publication and is not binding precedent
of the Board.
Paper No. 34
UNITED STATES PATENT AND TRADEMARK OFFICE
_____________
BEFORE THE BOARD OF PATENT APPEALS
AND INTERFERENCES
_____________
Ex parte FRANCIS J. MAGUIRE
_____________
Appeal No. 99-2698
Application 08/560,1081
______________
ON BRIEF
_______________
Before HAIRSTON, FLEMING and GROSS, Administrative Patent
Judges.
FLEMING, Administrative Patent Judge.
DECISION ON APPEAL
This is a decision on appeal from the final rejection of
claims 1-11, 13, and 15-21 , all of the claims pending in the2
Appeal No. 1999-2698
Application 08/560,108
"plurality" the word "of" is missing.
Submitted by Appellant in paper No. 25, Amendment D.3
2
present application. Claims 12 and 14 have been canceled.
Dependent claim 22 has not been addressed by the Examiner or3
Appellant after its entry, and, as it has not been rejected it
is not before us.
The invention relates to a system and method for
providing images of real and virtual objects in a head mounted
display (specification, page 1, lines 1-2). A pair of cameras
(figure 1, item numbered 10) are mounted on a viewer's head
(figure 1, item numbered 12) to provide video signals on a
line (figure 1, item numbered 14) to a computer and image
processor (figure 1, item numbered 16). The video signals
contain image information relating to real objects imaged by
the cameras (specification, page 5, line 30 through page 6,
line 2).
The computer and image processor have stored a computer
model of an object space, which may be a room in a house, and
the model may include furniture fixed at various points within
the room (specification, page 6, lines 3-7). Viewer monitors
Appeal No. 1999-2698
Application 08/560,108
3
(figure 1, item numbered 18) provide sensed signals indicative
of viewer movements within the object space to the computer
and image processor. In response to the sensed signals the
computer and image processor provide virtual object selection
signals on a line (figure 1, item numbered 20) to a virtual
object image store (figure 1, item numbered 22) which contains
a plurality of scenarios containing images of one or more
virtual objects. The selected scenario is output on a line
(figure 1, item numbered 24) to the computer and image
processor where images of the virtual objects are integrated
with the images of the real objects provided on line 14. The
integrated image signal is provided on a line (figure 1, item
numbered 26) to a head mounted display (figure 1, item
numbered 28) worn by the viewer in the object space.
Disconcerting time lag between the viewer's actions in an
empty virtual environment filled with virtual objects and the
response of the imaging system (specification, page 2, lines
29-32) is minimized by using images from the head-mounted
camera to be displayed on the head-mounted viewer, and by
using images imitative of virtual objects moving in the object
space which are retrieved from storage and are integrated with
Appeal No. 1999-2698
Application 08/560,108
4
the actual image. This is accomplished by prestoring the
positions and geometric features of the real objects in a
computer spatial model of the space (specification, page 3,
lines 32-36), thus eliminating retrieval of stationary objects
from storage.
Independent claim 1 is reproduced as follows:
1. Apparatus for providing a virtual reality environment
with images of virtual objects provided in response to sensed
actions of a viewer wherein the images of virtual objects lag
after the sensed actions of the viewer, comprising:
a head mounted camera (10) for mounting on a head (12) of
the viewer acting as a cameraman for gathering images of real
objects from a moving perspective of the viewer, said camera
responsive to reflected light from the real objects in an
object space, for providing a real object image signal (14)
indicative of the real objects;
an image store (22), responsive to a selection signal
(20), for providing a prestored virtual object image signal
(24) indicative of moving virtual objects;
a computer and image processor (16) having positions and
geometric features of said real objects prestored in a
computer spatial model of the object space, responsive to a
plurality of monitor signals for providing the selection
signal (20) and responsive to the real object image signal
(14) and the virtual object image signal (24), for providing
an integrated image signal (26) indicative of both said real
objects and said virtual objects;
Appeal No. 1999-2698
Application 08/560,108
5
one or more monitors (18), responsive to one or more
corresponding actions of the viewer acting as a cameraman in
the object space, for providing the plurality of monitor
signals; and
a head mounted display (28) for mounting on the head of
the viewer, responsive to the integrated image signal 26), for
providing integrated images of the object space with real and
virtual objects for viewing by the viewer with the images of
the real objects in registration with the real objects from
the moving point of view of the viewer and for viewing by the
viewer with the images of the virtual objects integrated
therewith, wherein the moving virtual objects move with
respect to the real objects positioned in the spatial model
for interacting with the viewer and whereby the lag of the
images of virtual objects after the sensed actions of the
viewer is reduced.
The Examiner relies on the following references:
Ritchey 5,130,794 Jul. 14,
1992
Ruoff 4,513,317 Apr. 23,
1985
Bajura Michael “Merging Virtual Objects with the Real World”
in: Computer Graphics (1992), pp 203-210.
Ivan Sutherland “A head-mounted three dimensional display”
(1968), pp 757-763.
M. Deering “High Resolution Virtual Reality” Computer Graphics
(1992), pp 195-202.
Claims 1, 4-7, 9, 13 and 19-20 stand rejected under
35 U.S.C. § 103 as being unpatentable over Bajura in view of
Appeal No. 1999-2698
Application 08/560,108
6
Sutherland.
Claims 2-3, 8 and 21 stand rejected under 35 U.S.C. § 103
as being unpatentable over Bajura in view of Sutherland and
Ritchey.
Claims 10-11, 15 and 17-18 stand rejected under 35 U.S.C.
§ 103 as being unpatentable over Bajura in view of Sutherland
and Ruoff.
Claim 16 stands rejected under 35 U.S.C. § 103 as being
unpatentable over Bajura in view of Sutherland and Ritchey and
Ruoff.
Claims 1, 4-7, 9, 13 and 19-20 stand rejected under
35 U.S.C. § 103 as being unpatentable over Bajura in view of
Deering.
Claims 2-3, 8 and 21 stand rejected under 35 U.S.C. § 103
as being unpatentable over Bajura in view of Deering and
Ritchey.
Claims 10-11, 15 and 17-18 stand rejected under 35 U.S.C.
§ 103 as being unpatentable over Bajura in view of Deering
and Ruoff.
Claim 16 stands rejected under 35 U.S.C. § 103 as being
Appeal No. 1999-2698
Application 08/560,108
In the final rejection (paper no. 26) the Examiner4
rejected claims 1-5, 10, 13, 15, 18-19 and 21 under 35 U.S.C.
§ 112, first paragraph, for the specification failing to
support specific limitations of claim 1. As the Examiner
withdrew this rejection at section 14 of the Examiner's Answer
this matter is no longer at issue.
The Brief was received June 4, 1998.5
The Reply Brief was received October 16, 1998. The6
Examiner mailed a letter December 18, 1998 stating that
Appellant's Reply Brief had been entered and considered but no
further response by the Examiner was deemed necessary.
The Examiner's Answer was mailed August 11, 1998. 7
7
unpatentable over Bajura in view of Deering and Ritchey and
Ruoff.4
Rather than reiterate the arguments of Appellants and the
Examiner, reference is made to the Brief , Reply Brief , and5 6
the Examiner's Answer for the respective details thereof. 7
OPINION
We will not sustain the rejections of claims 1-11, 13 and
15-22 under 35 U.S.C. § 103.
The Examiner has failed to set forth a prima facie case.
It is the burden of the Examiner to establish why one having
ordinary skill in the art would have been led to the claimed
invention by the express teachings or suggestions found
Appeal No. 1999-2698
Application 08/560,108
8
in the prior art, or by implications contained in such
teachings or suggestions. In re Sernaker, 702 F.2d 989, 995,
217 USPQ 1, 6 (Fed. Cir. 1983). The Federal Circuit states
that “[t]he mere fact that the prior art may be modified in
the manner suggested by Examiner does not make the
modification obvious unless the prior art suggested the
desirability of the modification.” In re Fritch, 972 F.2d
1260, 1266 n.14, 23 USPQ2d 1780, 1783-84 n.14 (Fed. Cir.
1992), citing In re Gordon, 733 F.2d 900, 902, 221 USPQ 1125,
1127 (Fed. Cir. 1984). It is further established that “[s]uch
a suggestion may come from the nature of the problem to be
solved, leading inventors to look to references relating to
possible solutions to that problem.” Pro-Mold & Tool Co. v.
Great Lakes Plastics, 75 F.3d 1568, 1573, 37 USPQ2d 1626, 1630
(Fed. Cir. 1996) citing In re Rinehart, 531 F.2d 1048, 1054,
189 USPQ 143, 149 (CCPA 1976) (considering the problem to be
solved in a determination of obviousness). The Federal
Circuit reasons in Para-Ordnance Mfg. v. SGS Importers Int’l
Inc., 73 F.3d 1085, 1088-89, 37 USPQ2d 1237, 1239-40 (Fed.
Cir. 1995), that for the determination of obviousness, the
Appeal No. 1999-2698
Application 08/560,108
9
court must answer whether one of ordinary skill in the art who
sets out to solve the problem and who had before him in his
workshop the prior art, would have reasonably expected to use
the solution that is claimed by Appellants. However,
“[o]bviousness may not be established using hindsight or in
view of the teachings or suggestions of the invention.” Para-
Ordnance Mfg. v. SGS Importers Int’l, 73 F.3d at 1087, 37
USPQ2d at 1239, citing W. L. Gore & Assocs., Inc. v. Garlock,
Inc. 721 F.2d 1551, 1553, 220 USPQ 311, 312-13. In addition,
our reviewing court requires the PTO to make specific findings
on a suggestion to combine prior art references. In re
Dembiczak, 175 F.3d 994, 1000-01, 50 USPQ2d 1614, 1617-19
(Fed. Cir. 1999).
We will first consider the rejection of claims 1, 4-7, 9,
13 and 19-20 under 35 U.S.C. § 103 as being unpatentable over
Bajura in view of Sutherland.
On page 6 of the Appeal Brief (hereinafter "Brief"),
Appellant agrees with the Examiner that Bajura shows a head
mounted camera for mounting on the head of a viewer acting as
a cameraman, an image processor for integrating these images
Appeal No. 1999-2698
Application 08/560,108
Brief, page 7; Reply Brief, page 2.8
Brief, page 8.9
10
with sensed images (but of a real fetus with ultrasound, not a
virtual object), one or more monitors responsive to the
actions of the viewer acting as the cameraman, and a head
mounted display responsive to the integrated image signal for
providing integrated images of the object space with the
images of the fetus.
Furthermore, Appellant admits that Bajura suggests a non-8
real-time image, such as a building or addition before
construction, in "other applications" under Section 6.2 on
page 209. This non-real-time image would most likely be
prestored and
be of virtual objects. However, Appellant contends that
Bajura fails to suggest prestoring positions and geometric
features of real objects in a computer model.
Appellant then contends that a person of ordinary skill9
in this art in possession of both Bajura and Sutherland would
not use the prestored images of Sutherland to modify Bajura to
carry
Appeal No. 1999-2698
Application 08/560,108
Page 758, line 9; figures 8-9.10
Brief, pages 9 and 10.11
11
out the claimed invention as “neither reference has to do with
presenting images of actual objects and images of moving
virtual objects in conjunction” (Appellant's emphasis).
Appellant then differentiates Sutherland from Bajura,
pointing out that Sutherland teaches “the use of an optical
see-through display with prestored virtual objects” which
appear to hang in space around the user, while Bajura
discloses “a video see-through display” (Appellant's
emphasis). Sutherland prestores wire frame line drawings and10
has nothing to do with prestoring positions and geometric
features of real objects. Furthermore, Appellant notes that11
although Sutherland teaches the use of a prestored virtual
object, it teaches its use in a stationary way with an
optical-see-through device, and one skilled in this art would
not think of using Sutherland's prestored virtual objects with
the video-see-through application of Bajura.
Appeal No. 1999-2698
Application 08/560,108
Page 209, column 1.12
Brief, pages 17 and 18.13
12
In addition, Appellant asserts that there is no
motivation present in either reference, alone or in
combination, to make such a modification to Bajura based on
Sutherland, and that a person in possession of Bajura would
not look to optical-see-through technology for solving video-
see-through problems evident from these references.
Furthermore, Appellant states that even if one looked to
optical-see-through technology for solving video-see-through
problems, it would merely suggest a prestored image of a
stationary hanging virtual object, and there would be no
prestored positions and geometrical features of real objects
and no integration with moving virtual objects.
In regard to the "Other Applications" of Bajura12
Appellant asserts that there is no need shown or suggested to
replace any total virtual environment and there is no hint to
except moving virtual objects (Appellant's emphasis).
Finally, Appellant argues that the preamble of claim 113
recites the "lag" problem to which Appellant's invention is
Appeal No. 1999-2698
Application 08/560,108
Final rejection, page 3.14
13
directed, and the "whereby" clause in the last paragraph of
claim 1 concludes with the "lag" being reduced, and these
limitations are not disclosed by the applied references.
In the rejection the Examiner admits that Bajura does14
not explicitly disclose that the graphic image signal is a
graphic prestored graphic image signal, and asserts that it is
well known in the art to generate a prestored virtual image
signal from a plurality of prestored virtual image signals as
shown by Sutherland. In addition, the Examiner asserts that
as Bajura suggests that the graphic image signal can be used
to preview buildings on site before construction or visualize
additions to existing architecture, it would have been obvious
to one of ordinary skill in the art to substitute Sutherland's
prestored virtual image signals for Bajura's real-time
generated ultrasound image display.
In reply to Appellant's argument that Bajura does not
show or suggest having positions and geometric features of
real objects prestored in a computer spatial model the
Appeal No. 1999-2698
Application 08/560,108
Answer, page 7.15
14
Examiner asserts that the positions and geometric features of15
the real objects in the computer spatial model have to be
determined in order to display the real objects in the
environment. The Examiner then points to Bajura at lines 6-7
of the first paragraph of section 4.2, section 4.4 and pages
206-207, as
clearly suggesting determining and storing the positions and
geometric features of the real objects in the computer spatial
model.
In response to Appellant's argument directed to the
combination of Sutherland's optical-see-through technology
with Bajura's video-see-through technology, the Examiner notes
that
the rejection only combined the prestoring means of Sutherland
with Bajura's display apparatus.
As to Appellant's argument directed to the time lag
aspects of the invention, the Examiner asserts that when
Bajura is modified as set forth in the rejection, it would
Appeal No. 1999-2698
Application 08/560,108
15
inherently solve this problem because Bajura retrieves only
the virtual world from storage, as it uses real time images
and virtual images.
The Federal Circuit states that "[t]he mere fact that the
prior art may be modified in the manner suggested by the
Examiner does not make the modification obvious unless the
prior art suggested the desirability of the modification." In
re Fritch, 972 F.2d at 1266 n.14, 23 USPQ2d at 1783-84 n.14
(Fed. Cir. 1992), citing In re Gordon, 733 F.2d at 902, 221
USPQ at 1127 (Fed. Cir. 1984). "Obviousness may not be
established using hindsight or in view of the teachings or
suggestions of the inventor." Para-Ordnance, 73 F.3d at 1087,
37 USPQ2d at 1239, citing W. L. Gore & Assocs., 721 F.2d at
1551, 1553, 220 USPQ at 311, 312-13. In addition, our
reviewing court requires the PTO to make specific findings on
a suggestion to combine prior art references. In re
Dembiczak, 175 F.3d 994, 1000-01, 50 USPQ2d 1614, 1617-19
(Fed. Cir. 1999).
Appeal No. 1999-2698
Application 08/560,108
Similar limitations are present in independent claims 616
and 7.
Lines 5-9.17
Lines 6-7 of the first paragraph of section 4.2,18
section 4.4 and pages 206-207
16
As pointed out by our reviewing court, we must first
determine the scope of the claim. "[T]he name of the game is
the claim.” In re Hiniker Co., 150 F.3d 1362, 1369, 47 USPQ2d
1523, 1529 (Fed. Cir. 1998).
Turning first to Appellant's claim 1 , we note that this16
claim calls for "a computer and image processor having17
positions and geometric features of said real objects
prestored in a computer spatial model of the object space . .
.". We agree with Appellant that this is not taught or
suggested by Bajura. We have reviewed the sections of Bajura18
noted by the Examiner and find no such disclosure. The
apposite section of lines 6 and 7 of the first paragraph of
section 4.2 states "Images in the virtual environment are
registered to the real world within the update-rate limit of
the tracking and display system . . . ." Section 4.4 is
directed to calibration of the system and provides for
Appeal No. 1999-2698
Application 08/560,108
Page 758, line 9; figures 8-9.19
17
transducer transformation and camera transformation to
calibrate the test system and notes that the camera
transformation relates the position and orientation of the
head-mounted tracker to the HMD TV camera position,
orientation and field of view. Pages 206 and 207 contain
sections 4.2 and 4.4 and figures 4-6. Contrary to the
Examiner's assertions, these sections of Bajura are devoid of
disclosure of a computer and image processor having positions
and geometric features of said real objects prestored in a
computer spatial model of the object space.
We find that Sutherland prestores wire frame line
drawings and has nothing to do with prestoring positions and19
geometric features of real objects. Although Sutherland
teaches the use of a prestored virtual object, it teaches its
use in a stationary way with an optical-see-through device.
We further find that as neither reference has to do with
presenting images of actual objects in conjunction with images
of moving virtual objects, a person of ordinary skill in this
art in possession of both Bajura and Sutherland would not use
Appeal No. 1999-2698
Application 08/560,108
18
the prestored images of Sutherland to modify Bajura to carry
out the claimed invention.
Therefore, we find that Bajura and Sutherland are
directed to disparate teachings which address different
problems and we find no reason or suggestion in either prior
art reference to enable their combination in this obviousness
analysis.
Furthermore, there is no objective teaching in either Bajura
or Sutherland that would lead one of ordinary skill in this
art to combine the references as proposed by the Examiner.
Therefore, we will not sustain the rejection of claims 1
and 4-7, 9, 13, and 19-20 under 35 U.S.C. § 103 as being
unpatentable over Bajura and Sutherland.
In addition, we will not sustain the following
rejections:
Claims 2-3, 8 and 21 under 35 U.S.C. § 103 as
being unpatentable over Bajura in view of Sutherland
and Ritchey;
Claims 10-11, 15 and 17-18 under 35 U.S.C. § 103
as being unpatentable over Bajura in view of
Sutherland and Ruoff;
Appeal No. 1999-2698
Application 08/560,108
Brief, page 15.20
Brief, page 16.21
19
Claim 16 under 35 U.S.C. § 103 as being
unpatentable over Bajura in view of Sutherland and
Ritchey and Ruoff.
These rejections are all based upon Bajura and Sutherland as
discussed above and the Examiner has only applied the
additional references in these rejections to the specific
limitations added by these dependent claims.
Claims 1, 4-7, 9, 13 and 19-20 are also rejected under
35 U.S.C. § 103 as being unpatentable over Bajura in view of
Deering.
Appellant has adopted the arguments previously made20
pertaining to Bajura to this rejection.
In addition, Appellant asserts that Deering has nothing21
to do with real objects, but is completely directed to virtual
objects, and does not suggest a computer and image processor
having positions and geometric features of real objects
Appeal No. 1999-2698
Application 08/560,108
Brief, page 16.22
Final rejection, page 8.23
20
prestored in a computer spatial model of an object space
within which images of virtual objects are integrated with
images of the real objects acquired from a head mounted
camera.
It is specifically noted by Appellant that Deering deals22
with the image lag problem by using a forward prediction, such
as a linear interpolation or higher order interpolators based
on Kalman filtering. This mode is entirely different from
Appellant's mode of lag reduction which reduces the effect of
latency by concentrating the computational power of the image
processor on the moving virtual objects and substituting real
objects for the stationary parts of the virtual environment
which can be imaged by cameras mounted on the viewer's head.
In the rejection the Examiner admits that Bajura does23
not explicitly disclose that the graphic image signal is a
prestored graphic image signal, and asserts that it is well
known in the art to generate a prestored virtual image signal
Appeal No. 1999-2698
Application 08/560,108
Answer, page 8.24
Similar limitations are present in independent claims 625
and 7.
Lines 5-9.26
21
from a plurality of prestored virtual image signals as shown
by Deering. In addition, the Examiner asserts that as Bajura
suggests that the graphic image signal can be used to preview
buildings on site before construction or visualize additions
to existing architecture, it would have been obvious to one of
ordinary skill in the art to substitute Deering's prestored
virtual image signals for Bajura's real-time generated
ultrasound image display.
The Examiner also states, "The use of Deering in the24
rejection is similar to Sutherland, Deering is not used to
meet all the requirements of the claimed invention, it is used
only to show that a virtual object can be generated from a
prestoring means rather than from a real time generating
means."
Turning again to Appellant's claim 1 , we note that this25
claim calls for "a computer and image processor having26
positions and geometric features of said real objects
Appeal No. 1999-2698
Application 08/560,108
22
prestored in a computer spatial model of the object space . .
. ." We agree with Appellant that this is not taught or
suggested by Bajura, and, as specifically discussed above,
find that it is not disclosed even by those sections of Bajura
cited by the Examiner.
We find that Bajura and Deering are directed to disparate
teachings which address different problems and find no reason
or suggestion in either prior art reference to enable their
combination in this obviousness analysis. Furthermore, there
is no objective teaching in either Bajura or Deering that
would lead one of ordinary skill in this art to combine the
references as proposed by the Examiner.
Therefore, we will not sustain the rejection of claims 1
and 4-7, 9, 13, and 19-20 under 35 U.S.C. § 103 as being
unpatentable over Bajura and Deering.
In addition, we will not sustain the following
rejections:
Claims 2-3, 8 and 21 under 35 U.S.C. § 103 as
being unpatentable over Bajura in view of Deering
Appeal No. 1999-2698
Application 08/560,108
23
and Ritchey;
Claims 10-11, 15 and 17-18 under 35 U.S.C. § 103
as being unpatentable over Bajura in view of Deering
and Ruoff;
Claim 16 under 35 U.S.C. § 103 as being
unpatentable over Bajura in view of Deering and
Ritchey and Ruoff.
These rejections are all based upon Bajura and Deering as
discussed above and the Examiner has only applied the
additional references in these rejections to the specific
limitations added by these dependent claims.
We have not sustained the rejection of claims 1-11, 13
and 15-21 under 35 U.S.C. § 103. Accordingly, the Examiner's
Appeal No. 1999-2698
Application 08/560,108
24
decision is reversed.
REVERSED
KENNETH W. HAIRSTON )
Administrative Patent Judge )
)
)
) BOARD OF
PATENT
MICHAEL R. FLEMING ) APPEALS AND
Administrative Patent Judge )
INTERFERENCES
)
)
)
ANITA PELLMAN GROSS )
Administrative Patent Judge )
Appeal No. 1999-2698
Application 08/560,108
25
MRF:pgg
Francis J. Maguire
Ware Fressola Van Der Sluys & Adolphson
755 Main Street
Monroe, CT 06468