Ex Parte Zhang et al

Patent Trial and Appeal Board

Sep 19, 2017

13835741 (P.T.A.B. Sep. 19, 2017)

United States Patent and Trademark Office
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O.Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
13/835,741 03/15/2013 Wende Zhang P023337-US-NP 1038
72823 7590
Quinn IP Law
21500 Haggerty Road
Suite 300
Northville, MI 48167
09/21/2017 EXAMINER
BOOMER, JEFFREY C
ART UNIT PAPER NUMBER
3669
NOTIFICATION DATE DELIVERY MODE
09/21/2017 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
adomagala@quinniplaw.com
U S Docketing @ quinniplaw .com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte WENDE ZHANG, JINSONG WANG, KENT S. LYBECKER,
JEFFREY S. PIASECKI, JAMES CLEM, CHARLES A. GREEN, RYAN
M. FRAKES, and TRAVIS S. HESTER
Appeal 2016-005272
Application 13/835,7411
Technology Center 3600
Before ST. JOHN COURTENAY III, ERIC S. FRAHM,
and NORMAN H. BEAMER, Administrative Patent Judges.
BEAMER, Administrative Patent Judge.
DECISION ON APPEAL
Appellants appeal under 35 U.S.C. § 134(a) from the Examiner’s final
rejection of claims 1—29. We have jurisdiction over the pending rejected
claims under 35 U.S.C. § 6(b).
We affirm.
1 Appellants identify GM Global Technology Operations LLC, as the real
party in interest. (App. Br. 2.)
Appeal 2016-005272
Application 13/835,741
THE INVENTION
Appellants’ disclosed and claimed invention is directed to displaying
de-warped images on a rearview mirror. (Abstract.)
Claim 1, reproduced below, is illustrative of the claimed subject
matter:
1. A method for displaying a captured image on a rearview
mirror display device comprising the steps of:
capturing a scene by an at least one vision-based imaging
device;
generating a virtual image of the captured scene by a
processor using a camera model, the camera model projecting the
captured image onto a nonplanar imaging surface;
actuating a dynamic rearview mirror display mode to
enable a viewing mode on the rearview mirror display device;
applying a view synthesis technique to the virtual image
by the processor to generate a de-warped image, the view
synthesis technique mapping the virtual image as a function of
the enabled viewing mode to the rearview display device to
generate the de-warped image; and
displaying the de-warped image in the enabled viewing
mode on the rearview mirror display device.
REJECTIONS2
The Examiner rejected claims 1, 24, and 27—29 under 35 U.S.C.
§ 103(a) as being unpatentable over Lee et al. (US 2010/0201816 Al;
published Aug. 12, 2010) and Teo (US 6,064,399; issued May 16, 2000).
(Final Act. 3—6.)
2 The Examiner’s rejection of the claims pursuant to 35 U.S.C. § 112 is
withdrawn. (Ans. 5.)
2
Appeal 2016-005272
Application 13/835,741
The Examiner rejected claims 2, 3, 13, 14, and 16 under 35 U.S.C.
§ 103(a) as being unpatentable over Lee, Teo, and Chu et al. (US
2011/0292233 Al; published Dec. 1, 2011). (Final Act. 6-9.)
The Examiner rejected claims 4 and 12 under 35 U.S.C. § 103(a) as
being unpatentable over Lee, Teo, Chu, and Brown & Lowe, “Automatic
Panoramic Image Stitching using Invariant Features,” Department of
Computer Science, University of British Columbia. (Final Act. 10—11.)
The Examiner rejected claim 5 under 35 U.S.C. § 103(a) as being
unpatentable over Lee, Teo, Chu, and Chen et al. (US 2002/0154812 Al;
published Oct. 24, 2002). (Final Act. 11—12.)
The Examiner rejected claims 6—8, 10, and 11 under 35 U.S.C.
§ 103(a) as being unpatentable over Lee, Teo, Chu, and Schofield et al. (US
2003/0122930 Al; published July 3, 2003). (Final Act. 12-15.)
The Examiner rejected claim 9 under 35 U.S.C. § 103(a) as being
unpatentable over Lee, Teo, Chu, Schofield, and Gehrig et al., “6D Vision
Goes Fisheye for Intersection Assistance,” Canadian Conference on
Computer and Robot Vision {200'S). (Final Act. 15—16.)
The Examiner rejected claim 15 under 35 U.S.C. § 103(a) as being
unpatentable over Lee, Teo, Chu, and Lee et al. (US 2009/0079828 Al;
published Mar. 26, 2009) (hereafter “Lee 2009”). (Final Act. 16.)
The Examiner rejected claims 17—23 under 35 U.S.C. § 103(a) as
being unpatentable over Lee, Teo, and Honda
(http ://houstonhondadealers,com/2011 /PilotPictures/, May 12, 2012.) (Final
Act. 16-20.)
3
Appeal 2016-005272
Application 13/835,741
The Examiner rejected claims 25 and 26 under 35 U.S.C. § 103(a) as
being unpatentable over Lee, Teo, and Mori et al. (US 2006/0215020 Al;
published Sept. 28, 2006). (Final Act. 20—22.)
ISSUES ON APPEAL
Appellants’ arguments in the Appeal Brief present the following
issues.3
First Issue: Whether the Examiner erred in finding the combination
of Lee and Teo teaches or suggests the limitations of independent claim 1.
(App. Br. 10-13.)
Second Issue: Whether the Examiner erred in finding the combination
of Lee and Teo teaches or suggests the additional limitations of dependent
claims 24 and 27—29. (App. Br. 13—15.)
Third Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, and Chu teaches or suggests the additional limitations of
dependent claim 2. (App. Br. 16.)
Fourth Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, Chu, and Brown & Lowe teaches or suggests the additional
limitations of dependent claims 4 and 12. (App. Br. 17—18.)
Fifth Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, Chu, and Chen teaches or suggests the additional limitations of
dependent claim 5. (App. Br. 19.)
3 Rather than reiterate the arguments of Appellants and the findings of the
Examiner, we refer to the Appeal Brief (filed Nov. 12, 2015, “App. Br.”);
the Reply Brief (filed Apr. 21, 2016 “Reply Br.”); the Final Office Action
(mailed June 22, 2015, “Final Act.”); and the Examiner’s Answer (mailed
Mar. 10, 2016, “Ans.”) for the respective details.
4
Appeal 2016-005272
Application 13/835,741
Sixth Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, Chu, and Schofield teaches or suggests the additional
limitations of dependent claim 7. (App. Br. 20.)
Seventh Issue: Whether the Examiner erred in finding the
combination of Lee, Teo, Chu, Schofield, and Gehrig teaches or suggests the
additional limitations of dependent claim 9. (App. Br. 20.)
Eighth Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, Chu, and Lee 2009 teaches or suggests the additional
limitations of dependent claim 15. (App. Br. 20—21.)
Ninth Issue: Whether the Examiner erred in finding the combination
of Lee, Teo, and Honda teaches or suggests the additional limitations of
dependent claims 17, 18, and 23. (App. Br. 21—24.)
ANALYSIS
We have reviewed the Examiner’s rejections in light of Appellants’
arguments that the Examiner errs. We disagree with Appellants’ arguments,
and we adopt as our own (1) the pertinent findings and reasons set forth by
the Examiner in the Action from which this appeal is taken (Final Act. 3—22)
and (2) the corresponding reasons set forth by the Examiner in the
Examiner’s Answer in response to Appellants’ Appeal Brief (Ans. 5—13).
Issue One
In rejecting claim 1, the Examiner relies on the disclosure in Lee of a
multi-display rearview mirror system using multiple electronic imaging
cameras with varying positions and fields of view, and the ability to process
the camera images including cropping and scaling to display the images on
the rearview mirror. (Final Act. 3—5; Lee Figs. 28—30, 60, 109-114.)
5
Appeal 2016-005272
Application 13/835,741
The Examiner further relies on the disclosure in Teo of embedding a planar
image into a panoramic image by translating the planar image onto a curved
surface corresponding to the panoramic image, and de-warping the
composite image onto a viewing plane for display. (Final Act. 4—5; Teo col.
3,11. 1-11, col. 6,11. 28-36, col. 7,11. 42^18.)
In rejecting claim 1, the Examiner interprets “generating a virtual
image of the captured scene by a processor using a camera model, the
camera model projecting the captured image onto a nonplanar imaging
surface,” as referring to a “software-based, mathematical model,” that maps
a “3D scene captured by the cameras . . .onto a 2D surface.” (Final Act. 2,
4.) (Emphasis omitted.) The Examiner elaborates on that interpretation in
the Answer: “The non-planar imaging surface of Claim 1 is read to be a
virtual (i.e. software) imaging surface.” (Ans. 6.) In addition, the Examiner
interprets “view synthesis technique” as “synthesizing or manipulating (a
plurality of) image data such that is displayed in a viewable format.” (Id.)
We agree with these broad but reasonable interpretations of the claim
language.
Appellants argue “Teo does not describe generating a virtual image
of the scene using a camera model where the camera model projects the
captured image onto a nonplanar imaging surface.” (App. Br. 11.) To the
contrary, as the Examiner correctly finds, Teo electronically projects a two-
dimensional planar image onto a curved panoramic virtual image surface,
which satisfies the claim requirement at issue, as construed by the Examiner.
(Ans. 6—7; Teo Figs. 1—3, col. 6,11. 27—36.) Appellants further argue “Lee
and Teo fail to describe applying a view synthesis technique to the virtual
image by the processor to generate a de-warped image where the view
6
Appeal 2016-005272
Application 13/835,741
synthesis technique maps the virtual image as function of the enabled
viewing mode.” (App. Br. 11—12.) Again, however, the Examiner correctly
finds Teo performs the required view synthesis of the composite image,
created electronically by projecting the initial two-dimensional image onto
the virtual curved panoramic surface, wherein the resulting composite image
is then de-warped while being mapped onto the viewing plane to appear
natural. (Ans. 6—7; Teo Fig. 4, col. 7,11. 42-48.) Lee also applies view
synthesis techniques to virtual images, by cropping, expanding, and scaling
the captured images for display on the mirror. (Ans. 6—7; Lee Figs. 31, 32,
1160, 110-111, 118-119.)
In addition, Appellants argue neither Lee nor Teo describe “applying
view synthesis as function of the enabled viewing mode of the rearview
mirror.” Here too, the Examiner correctly finds Lee discloses synthesizing
different views depending on viewing modes — e.g., enhancing the rearward
view “for the special condition of backing out of an alley or narrow
passage.” (Final Act. 5; Lee Fig. 30,1114.) Finally, Appellants argue the
Examiner errs in finding Lee displays the de-warped image on the rearview
mirror. (App. Br. 13.) However, the Examiner finds Lee, “as modified in
[the] above combination [with Teo],” satisfies this claim requirement. (Final
Act. 5.) This, along with Appellants’ other arguments regarding the
Examiner’s rejections, are unpersuasive as attacking the references
individually where, as here, the ground of unpatentability is based upon the
teachings of a combination of references. In re Merck & Co., Inc., 800 F.2d
1091, 1097 (Fed. Cir. 1986). The test for obviousness is whether the
combination of references, taken as a whole, would have suggested the
patentee’s invention to a person having ordinary skill in the art. In re Keller,
7
Appeal 2016-005272
Application 13/835,741
642 F.2d 413, 425 (CCPA 1981). Accordingly, we sustain the Examiner’s
rejection of independent claim 1.
Issues Two through Nine: Dependent Claims 2, 4, 5, 7, 9, 12, 15, 17, 18, 23,
24, and27—29
Claim 2 additionally requires image stitching prior to the camera
modeling step of claim 1, claim 4 further requires identifying and registering
overlapping point pairs as part of image stitching, and claim 5 adds the
requirement that the stitching includes stereo vision processing. (App. Br.
25—26.)4 In rejecting these claims, the Examiner relies, in addition to Lee
and Teo as discussed above, on the disclosure in Chu of image stitching, in
Brown & Lowe of stitching panoramic images by matching portions of
overlapping images, and in Chen of using stereo vision techniques to stitch
images together. (Final Act. 6—8, 10, 12.) For claim 2, Appellants repeat
the argument that Lee fails to disclose camera modeling. (App. Br. 16.)
This argument is unpersuasive for the reasons discussed above. Appellants
also submit, for claim 4, that Brown & Lowe fails to disclose the use of
point pairs for image stitching. (App. Br. 17.) We are not persuaded by this
conclusory argument, that the Examiner errs in finding at least a teaching or
suggestion in this reference of the claim limitations at issue. For claim 5,
Appellants also conclusorily argue Chen fails to disclose the stereo vison
stitching requirements of the claim. (App. Br. 19.) Again, we are not
persuaded the Examiner errs, in relying on the explicit discussion in Chen of
stitching images using stereo vision techniques. (Ans. 10.)
4 The claim language is paraphrased herein for convenience of reference; the
complete limitations as set forth in the record have been fully considered.
8
Appeal 2016-005272
Application 13/835,741
Claim 7 requires image stitching using three narrow field-of-view
(FOV) cameras to generate a 180 degree panoramic scene. (App. Br. 26.)
The Examiner relies on the disclosure in Lee of three cameras of varying
FOV to generate 180 degree panoramas, and on the disclosure in Schofield
of using three narrow FOV cameras to synthesize a rearview mirror image.
(Final Act. 13—14; Lee Figs. 28, 29, 30, Tflf 109, 113—115; Schofield Fig. 1,
44, 46, 48.) Appellants argue Lee does not disclose three narrow FOV
cameras. (App. Br. 20.) This argument is unpersuasive given the
Examiner’s additional reliance on Schofield.
Claim 9 requires three forward facing cameras to create the final
viewed image. (App. Br. 27.) The Examiner relies on the disclosure in
Gehrig of forward facing cameras. (Final Act. 15; Gehrig Fig. 1.)
Appellants argue Gehrig does not disclose image stitching. (App. Br. 20.)
This argument is unpersuasive given the Examiner’s additional reliance, as
discussed above, on Chu for teachings of image stitching.
Claim 12 requires a narrow FOV camera capturing a scene that is a
subset of an image captured by a wide FOV camera, which images are
stitched together. (App. Br. 27.) The Examiner relies on the disclosure in
Lee of use of narrow FOV cameras capturing images that overlap with wide
FOV cameras, together with the image stitching disclosures of Brown &
Lowe discussed above. (Final Act. 10—11; Lee Figs. 28, 29, Tflf 85, 109;
Brown & Lowe Fig. 1, § 3.) Appellants argue the narrow overlapping image
of Lee is not a complete subset of the wide image. (App. Br. 18.) However,
we are not persuaded the Examiner errs in finding the overlapping image
disclosure of Lee at least teaches or suggests the subject matter at issue.
(Ans. 10.) Appellants also repeat the argument that Brown & Lowe does not
9
Appeal 2016-005272
Application 13/835,741
disclose identifying point pairs for images stitching. (App. Br. 10.) That
argument is unpersuasive for the reasons discussed above.
Claim 15 requires a visual icon representing the current view being
captured. (App. Br. 28.) For this limitation, the Examiner relies on the
disclosure in Lee 2009 of displaying a graphic label indicating different
camera views in a vehicle display. (Final Act. 16; Lee 2009 Fig. 3, Tflf 66,
71—72.) Appellants argue Lee 2009 only discloses this indication for a
single camera. (App. Br. 21.) This argument is unpersuasive given the
Examiner’s additional reliance, as discussed above, on the primary Lee
reference for teachings of multiple cameras.
Claim 17 requires the viewing mode “selected from one of a mirror
display mode, a mirror display on with image overlay mode, and a mirror
display on without image overlay mode . . . wherein the mirror display on
with image overlay . . . replicat[es] interior components of the vehicle.”
(App. Br. 28.) Appellants focus on the “interior components of the vehicle”
aspect of the claim, arguing none of the cited references teach or suggest this
requirement. (App. Br. 22.) However, this method claim only requires one
of the three specified modes to be taught or suggested by the cited
references. Appellants’ argument is therefore unpersuasive, as not raising
issue as to two of the three modes.
Claim 18 specifically requires the “mirror display on with image
overlay” mode of claim 17 to include “head rest, rear window trim, and c-
pillars.” (App. Br. 28—29.) Appellants again argue the cited references do
not teach or suggest this feature. (App. Br. 23.) However, given the
dependency of claim 18 on claim 17, the claim still only requires one of the
10
Appeal 2016-005272
Application 13/835,741
three specified modes, and therefore Appellants’ argument is equally
unpersuasive for claim 18.
Claim 23 requires vehicle information to be used to change a camera
pose. (App. Br. 29.) The Examiner relies on the disclosure in Lee of
changing how images are captured in response to different driving
conditions. (Final Act. 20; Lee 1109.) Appellants argue this portion of Lee
does not specifically disclose changing a camera pose. (App. Br. 23—24.)
However, given the overall disclosures of Lee relied on by the Examiner and
discussed here, we are not persuaded the Examiner errs in finding the subject
matter of the claims taught or suggested by Lee. (Ans. 13.)
Claim 24 requires the view synthesis to be “enabled based on a
driving scenario .... [to] generate [] a direction zoom to a region of the
image for enhancing visual awareness . . . .” (App. Br. 29-30.) Claims 27,
28, and 29 additionally require the driving scenario to be based on,
respectively, a turn signal, a steering wheel angle, and vehicle speed. (App.
Br. 30.) The Examiner relies on the disclosure in Lee of changing the
displayed image based on various vehicle conditions including steering
wheel angle (Lee 177), vehicle speed (Lee 1114), and utilization of a turn
signal (Lee 1116). (Final Act. 5—6; see also Lee Fig. 30.) Appellants argue
Lee does not describe dynamic view synthesis of a virtual image. (App. Br.
13—15.) This argument is unpersuasive for the reasons discussed above for
claim 1.
CONCLUSIONS
For the reasons stated above, we sustain the obviousness rejection of
independent claim 1 and dependent claims 24 and 27—29 over Lee and Teo,
11
Appeal 2016-005272
Application 13/835,741
of claim 2 over Lee, Teo, and Chu, of claims 4 and 12 over Lee, Teo, Chu,
and Brown and Lowe, of claim 5 over Lee, Teo, Chu, and Chen, of claim 7
over Lee, Teo, Chu, and Schofield, of claim 9 over Lee, Teo, Chu,
Schofield, and Gehrig, of claim 15 over Lee, Teo, Chu, and Lee 2009, and of
claims 17, 18, and 23 over Lee, Teo, and Honda.
We also sustain the obviousness rejection of claims 3, 13, 14, and 16
over Lee, Teo, and Chu, of claims 6, 8, 10, and 11 over Lee, Teo, Chu, and
Schofield, of claims 19-22 over Lee, Teo, and Honda, and of claims 25 and
26 over Lee, Teo, and Mori, which claims are not argued separately with
particularity. (App. Br. 16, 20, 24.)
DECISION
We affirm the Examiner’s rejections of claims 1—29.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv).
AFFIRMED
12