Alj, Youssef et al.

Patent Trials and Appeals Board

Dec 31, 2019

14372018 - (D) (P.T.A.B. Dec. 31, 2019)

UNITED STATES PATENT AND TRADEMARK OFFICE
UNITED STATES DEPARTMENT OF COMMERCE
United States Patent and Trademark Office
Address: COMMISSIONER FOR PATENTS
P.O. Box 1450
Alexandria, Virginia 22313-1450
www.uspto.gov
APPLICATION NO. FILING DATE FIRST NAMED INVENTOR ATTORNEY DOCKET NO. CONFIRMATION NO.
14/372,018 07/13/2014 Youssef Alj 2011P00137WOUS 8755
147897 7590 12/31/2019
Tutunjian & Bitetto, P.C.
401 Broadhollow Road
Suite 402
Melville, NY 11747
EXAMINER
BADER, ROBERT N.
ART UNIT PAPER NUMBER
2619
NOTIFICATION DATE DELIVERY MODE
12/31/2019 ELECTRONIC
Please find below and/or attached an Office communication concerning this application or proceeding.
The time period for reply, if any, is set in the attached communication.
Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the
following e-mail address(es):
docketing@tb-iplaw.com
PTOL-90A (Rev. 04/07)
UNITED STATES PATENT AND TRADEMARK OFFICE
BEFORE THE PATENT TRIAL AND APPEAL BOARD
Ex parte YOUSSEF ALJ, GUILLAUM BOISSON, PHILIPPE BORDES,
LUCE MORIN, and MURIEL PRESSIGOUT,
Appeal 2019-001565
Application 14/372, 018
Technology Center 2600

BEFORE ROBERT E. NAPPI, KRISTEN L. DROESCH, and
CATHERINE SHIANG, Administrative Patent Judges.

DROESCH, Administrative Patent Judge.

DECISION ON APPEAL
STATEMENT OF THE CASE
Pursuant to 35 U.S.C. § 134(a), Appellant1 appeals from the
Examiner’s decision to reject claims 1, 2, 4–9, and 11–15. We have
jurisdiction under 35 U.S.C. § 6(b).
We AFFIRM.
CLAIMED SUBJECT MATTER
The claims are directed to a method of texturing a mesh associated
with a surface representative of a scene captured in a plurality of images of a

1 We use the word Appellant to refer to “applicant” as defined in 37 C.F.R.
§ 1.42(a). Appellant identifies the real party in interest as Thomson
Licensing DTV. Appeal Br. 4.
Appeal 2019-001565
Application 14/372,018
2
multi-view video stream. See Spec. 1:6–11; Abstract. Claim 1, reproduced
below, is illustrative of the claimed subject matter:
1. A method for texturing a mesh, the mesh being
associated with a surface representative of a scene captured
according to a plurality of points of view, an image of the scene
being associated with each point of view, the method
comprising, for one mesh element of said mesh that is at least
partially visible from at least two first images of the plurality of
images:
a) selecting, by a processor, a first image among said at
least two first images;
b) projecting, by said processor, said mesh element onto
said selected first image and texturing said mesh element with a
first texture information obtained from pixels of said selected
first image, said pixels corresponding to the projection of said
mesh element onto said selected first image;
c) for each first image of said at least two first images,
comparing, by said processor, the first texture information of
said textured mesh element with the texture information
associated with the pixels of said each first image that
corresponds to the projection of said textured mesh element
onto said each first image;
d) computing, by said processor, an error information for
said first texture information according to a result of said
comparing;
e) selecting, by said processor, a first image among said at
least two first images different from the first image selected at
operation a) and reiterating operations b) - e) until each first
image has been selected;
f) selecting, by said processor, the first texture information
minimizing said error information among the plurality of first
texture information.

Appeal 2019-001565
Application 14/372,018
3
REFERENCES
The prior art relied upon by the Examiner is:
Name Reference Date
Debevec et al. Efficient View-Dependent Image-Based
Rendering with Projective Texture-Mapping,
Proceedings of the Eurographics Workshop
Vienna Austria, pp. 105–116.
1998
Lindstrom et al. Image-Driven Simplification, ACM
Transactions on Graphics Vol. 19, No. 3, pp.
204–241
2000
Eckert et al. Shape Refinement for Reconstruction 3D-
Objects Using an Analysis-Synthesis
Approach, 2001 International Conference
ICIP, IEEE Vol. 3, pp. 903–906
2001
Carranza et al. Free-Viewpoint Video of Human Actors,
ACM Transactions on Graphics (TOG) –
Proceedings of ACM SIGGRAPH 2003, Vol.
22, Issue 3, pp. 569–577.
2003
Bogomjakov et al. Free-Viewpoint Video from Depth Cameras,
2006 Proceedings Vision, Modeling, and
Visualization, pp. 89–96.
2006
Iiyama et al. Super-Resolution Texture Mapping from
Multiple Viewpoints, IEEE 2010 International
Conference on Pattern Recognition, pp. 1820–
23
2010
REJECTIONS
Claims 1, 2, 8, 9, and 15 stand rejected under 35 U.S.C.
§ 103(a) as being unpatentable over Iiyama et al. (“Iiyama”), Eckert et al.
(“Eckert”), and Lindstrom et al (“Lindstrom”). Final Act. 2–15.
Claims 4, 5, 11, and 12 stand rejected under 35 U.S.C. § 103(a) as
being unpatentable over Iiyama, Eckert, Lindstrom, and Debevec et al.
(“Debevec”). Final Act. 15–19.
Appeal 2019-001565
Application 14/372,018
4
Claims 6 and 13 stand rejected under 35 U.S.C. § 103(a) as being
unpatentable over Iiyama, Eckert, Lindstrom, and Carranza e al.
(“Carranza”). Final Act. 19–21.
Claims 7 and 14 stand rejected under 35 U.S.C. § 103(a) as being
unpatentable over Iiyama, Eckert, Lindstrom, and Bogomjakov et al.
(“Bogomjakov”). Final Act. 21–23.
OPINION
Rejections of Claims 1, 2, 8, 9, and 15
Appellant argues claims 1, 2 and 15 together as a group. See Appeal
Br. 10–17. Appellant does not present substantive arguments addressing the
limitations of claims 8 and 9 apart from arguments addressing the limitations
of claim 1. See Appeal Br. 18–19. As a result, we group claims 8 and 9
together with claims 1, 2, and 15. Pursuant to our authority under C.F.R.
§ 41.37(c)(1)(iv) (2017), we choose claim 1 as representative of the group.
Appellant disputes the Examiner’s findings that the combination of
Iiyama, Eckert, and Lindstrom teach or suggest the limitations recited in
steps c) through f) of claim 1. See Appeal Br. 12. Specifically, Appellant
contends that Iiyama defines two cost functions Earea (L) and Etex (L), and
argues that cost function Earea (L) does not consider texture information itself
but the size of the projection of the patch. See id. at 14 (citing Iiyama p.
1821, right column, last paragraph through p. 1822, left column, first
paragraph, equation 3, section III, third paragraph). As to Etex (L), Appellant
argues this cost function considers the patch with regard to the neighbor
patches, and not the texture information onto which the patch may be
projected. See id. (citing Iiyama equation 5). Appellant contends that with
Iiyama’s first cost function and second cost function, there is no comparing
Appeal 2019-001565
Application 14/372,018
5
of the first texture information of the textured mesh element with the texture
information associated with the pixels of each first image that corresponds to
the projection of the textured mesh element onto said each first image. See
id. Appellant argues that Eckert fails to cure the deficiencies of Iiyama
because Eckert teaches a shape refinement process based on the local
deformations of a triangular mesh, including moving each triangle of the
mesh into seven different trial positions, and texturing the triangles for each
trial position by using the texture information of only one reference image,
but does not teach or suggest testing of the texture obtained from each image
of the plurality. See id. at 14–15. Appellant also argues that Lindstrom is
completely silent with respect to testing of the texture obtained from each
image of a plurality of images. See id. at 16–17.
Appellant’s arguments are not persuasive of error in the Examiner’s
rejection because the arguments address the teachings of each of Iiyama,
Eckert, and Lindstrom alone. One cannot show non-obviousness by
attacking references individually where the rejections are based on
combinations of references. In re Merck & Co., 800 F.2d 1091, 1097 (Fed.
Cir. 1986); In re Keller, 642 F.2d 413, 426 (CCPA 1981). Contrary to
Appellant’s arguments, the Examiner acknowledges that Iiyama does not
explicitly teach projecting the texture onto the input images to estimate the
error. See Final Act. 7–8 (citing Iiyama, section III, paragraphs 3, 7);
Ans. 4. Instead of relying on Iiyama’s teachings, the Examiner relies on
Eckert for teaching a photo consistency error metric for measuring the
similarity between a synthesized view (i.e., a textured triangle projected into
image planes of all other cameras that observe it), and all of the other input
images which observe the given element of the mesh, as modified by
Lindstrom’s teaching of comparing models using metrics that measure the
Appeal 2019-001565
Application 14/372,018
6
difference between a set of images taken of an original object (reference
camera images) and a set of images taken of a modified model of the object
being optimized (textured mesh elements projected onto reference camera
views). See Final Act. 7–8 (citing Eckert, section 3, paragraph 3), id. at 10
(citing Lindstrom, sections 3, 4.2.3, 4.2, 4.2.1, 4.2.2 ); Ans. 4–5. The
Examiner determined that it would have been obvious to modify Iiyama’s
super-resolution texture mapping system to include Eckert’s photo-
consistency error metric in addition to Iiyama’s patch edge artifact error
metric Etex(i,j) by replacing Etex(i,j) with Ephotex(i, j) = Etex(i,j)
+Ephoto(i) + Ephoto(j), and including every reference camera image, as
suggested by Lindstrom, thereby including the selected reference camera
image being used to texture the mesh element in the set of reference camera
images used for comparison between original reference images and the
textured model. See Final Act. 9, 11–12; Ans. 4–5.
Appellant further argues that the combination of Iiyama and Eckert
would lead one with ordinary skill in the art to select texture information
from one single image to project the textured triangle onto the other images
and to project the thus obtained textured triangle onto all camera images but
the reference image. See Appeal Br. 16. Appellant contends the
combination of Iiyama and Eckert would teach away from the concept of
texturing the triangle with the texture obtained from each and every camera
image of the plurality to test the texture of each camera image. See id.
Appellant’s arguments are not persuasive of Examiner error because
they do not address the teachings of Iiyama, Eckert, and Lindstrom, as
combined by the Examiner. The Examiner does not propose modifying
Iiyama in view of Eckert to use only texture information from one image to
project onto other images. Instead, as explained by the Examiner,
Appeal 2019-001565
Application 14/372,018
7
Lindstrom’s teaching of comparing models using metrics that measure the
difference between a set of images taken of an original object and a set of
images taken of a modified model of the object being optimized, in
combination with Iiyama and Eckert, teaches measuring the photo
consistency between all of the original images and all of the corresponding
synthesized images of the modified model. See Ans. 5; see also Final Act.
10–12 (similar explanation).
For the foregoing reasons, we are not persuaded of error in the
Examiner’s rejections of claims 1, 2, 8, 9, and 15.
Rejections of Claims 4–7 and 11–14
Claims 4–7 depend from claim 1, and claims 11–14 depend from
independent claim 8. For the rejections of these claims, Appellant argues
that each of the additional references (i.e., Debevec, Carranza, and
Bogomjakov) fail to teach or suggest at least the use of first image(s) of the
scene from which the mesh element is only partially visible as is recited in
claims 1 and 8. See Appeal Br. 20–22. Appellant’s arguments are not
persuasive of error in the Examiner’s rejection because they are misplaced.
The Examiner does not rely on Debevec, Carranza, or Bogomjakov for
teaching or suggesting “the mesh being associated with a surface
representative of a scene captured according to a plurality of points of view,
an image of the scene being associated with each point of view . . . for one
mesh element of said mesh that is at least partially visible from at least two
images of the plurality of images,” as recited in claim 1. The Examiner
instead relies on Iiyama for teaching the aforementioned limitations. See
Final Act. 2–3.
Appeal 2019-001565
Application 14/372,018
8
Accordingly, for this reason, we are not persuaded of error in the
Examiner’s rejections of claims 4–7 and 11–14.
CONCLUSION
We affirm the Examiner’s rejections of claims 1, 2, 4–9, and 11–15
under 35 U.S.C. § 103(a).
In summary:
Claims
Rejected
35 U.S.C. § References Affirmed Reversed
1, 2, 8, 9, 15 103(a) Iiyama, Eckert,
Lindstrom
1, 2, 8, 9, 15
4, 5, 11, 12 103(a) Iiyama, Eckert,
Lindstrom, Debevec
4, 5, 11, 12
6, 13 103(a) Iiyama, Eckert,
Lindstrom, Carranza
6, 13
7, 14 103(a) Iiyama, Eckert,
Lindstrom,
Bogomjakovic
7, 14
Overall
Outcome
1, 2, 4–9,
11–15

TIME PERIOD FOR RESPONSE
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a). See 37 C.F.R.
§ 1.136(a)(1)(iv).
AFFIRMED