Ex Parte Paek et al

Board of Patent Appeals and Interferences

Apr 9, 2012

10809172 (B.P.A.I. Apr. 9, 2012)

UNITED STATES PATENT AND TRADEMARK OFFICE
____________

BEFORE THE BOARD OF PATENT APPEALS
AND INTERFERENCES
____________

Ex parte TIMOTHY S. PAEK, SUSAN T. DUMAIS, and
RONALD K. LOGAN
____________

Appeal 2010-001888
Application 10/809,172
Technology Center 2100
____________

Before JOSEPH F. RUGGIERO, JOHN A. JEFFERY, and
ANDREW CALDWELL, Administrative Patent Judges.

CALDWELL, Administrative Patent Judge.

DECISION ON APPEAL

Appeal 2010-001888
Application 10/809,172

2
Appellants appeal under 35 U.S.C. § 134(a) from the Examiner’s
rejection of claims 1-27. We have jurisdiction under 35 U.S.C. § 6(b). We
affirm.

STATEMENT OF THE CASE
Appellants’ invention is a method and system for dynamic
presentation of search result information within a selected area of a display.
The system includes a lens component associated with a portion of a user
interface display that defines an area in which to display information from at
least one search result. A layout component displays a detailed subset of
information within the lens component based upon the search result. See
Abstract. Claim 1 is illustrative (emphasis added):
1. A computer-implemented interface for data presentation embodied
on a computer-readable storage medium, comprising:
a lens component associated with a portion of a user interface display,
the lens component defines an area to display information from at least one
search result; and;
a layout component that displays a detailed subset of information,
comprising textual information, within the area defined by the lens
component based upon the search result, the detailed subset of information is
animated to enlarge in size and to include additional textual information
that is selected from the at least one search result for insertion into the
detailed subset of information based in part on a query associated with the
at least one search result, as compared to the amount of information
displayed for the at least one search result when outside of the area defined
by the lens component.

The Examiner relies on the following as evidence of unpatentability:
Wolton
Montague

US 2004/0030741 A1
US 2005/0168488 A1

Feb. 12, 2004
Aug. 4, 2005
(filed Feb. 3, 2004)
Appeal 2010-001888
Application 10/809,172

3
Szabo US 2007/0156677 A1 July 5, 2007
(filed May 30, 2000)
Stuart Pook et al., Context and Interaction in Zoomable User Interfaces,
PROC. WORKING CONF. ON ADVANCED VISUAL INTERFACES 227 (2000).1
Jonathan C. Roberts & Edward Suvanaphen, Visual Bracketing for Web
Search Result Visualization, PROC. 7TH INT’L CONF. ON INFO.
VISUALIZATION 264 (2003).

THE REJECTIONS2
1. The Examiner rejected claims 1-12, 15-17, and 19-27 under 35
U.S.C. § 103(a) as being unpatentable over Roberts in view of Pook. Ans.
5-14.3
2. The Examiner rejected claim 13 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Wolton. Ans. 14.
3. The Examiner rejected claim 14 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Montague. Ans. 15.

1 A complete copy of the Pook reference was not properly scanned into the
USPTO’s Image File Wrapper (IFW) system. Page 231 of the file copy is
incomplete. Appellants never requested a complete copy of Pook. We will
therefore presume that they received a complete copy and will decide this
appeal accordingly. Rather than remanding the application to the Examiner,
we will enter a complete copy of Pook into IFW with the mailing of this
decision. Accordingly, we have included with this decision a Form 892 and
a complete copy of Pook.
2 The Miscellaneous Communication to Applicant mailed on November 19,
2009, deleted the section of the Answer entitled “35 USC 101 –
Clarifications.” For purposes of this appeal, we will treat as withdrawn any
requirement or rejection made by the Examiner in this section.
3 Throughout this opinion, we refer to (1) the Appeal Brief filed August 6,
2008; (2) the Examiner’s Answer mailed May 1, 2009; and (3) the Reply
Brief filed June 30, 2009.
Appeal 2010-001888
Application 10/809,172

4
4. The Examiner rejected claim 18 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Szabo. Ans. 16.

THE OBVIOUSNESS REJECTION OVER ROBERTS AND POOK
Although Appellants nominally argue independent claims 1, 21, and
25 separately in the Appeal Brief, we will group the claims and treat claim 1
as representative of the group. Appellants state that claims 21 and 25 are
patentable over Roberts and Pook for “at least reasons similar to the reasons
stated herein with regard to independent claim 1.” App. Br. 8-9. Any
additional argument with respect to claims 21 and 25 is either a mere
assertion that Roberts and Pook do not teach what the claims recite or a
discussion of where the claimed subject matter is supported in the
Specification or Figures. The discussion includes text copied from the
Summary of the Claimed Subject Matter. Id. Accordingly, we will select
claim 1 as representative and decide the appeal with respect to the group on
the basis of claim 1 alone.

CONTENTIONS
The Examiner finds that Roberts discloses every feature of claim 1
including a layout component that displays a detailed subset of information,
comprising textual information, within the area defined by the lens
component based upon the search result. Ans. 18-19. The Examiner finds
that Roberts does not disclose that the detailed subset of information is
animated to enlarge in size as compared to the amount of information
associated with the at least one other search result displayed outside the lens
region. The Examiner cites Pook as teaching this feature. Ans. 20.
Appeal 2010-001888
Application 10/809,172

5
Appellants argue that the combination of Roberts and Pook does not
teach a layout component that displays a detailed subset of information,
comprising textual information, within the area defined by the lens
component based upon the search result, the detailed subset of information is
animated to enlarge in size as compared to the amount of information
associated with the at least one other search result displayed outside the lens
region. App. Br. 5-8; Reply Br. 3-6.

ISSUE
Under § 103, has the Examiner erred by finding that Roberts and Pook
collectively would have taught or suggested a layout component that
displays a detailed subset of information, comprising textual information,
within the area defined by the lens component based upon the search result,
the detailed subset of information is animated to enlarge in size as compared
to the amount of information associated with the at least one other search
result displayed outside the lens region?

FINDINGS OF FACT (FF)
1. Roberts discloses a system for web search result visualization. In one
embodiment, the inner part of the visualization contains full information
about a number of search results while the bracketed views contain less
detailed information such as solely the URL. Roberts § 3, ¶ 3. In the center
of Roberts’s Figure 1, a bracketed version of a web search result
visualization is shown. The visualization has two levels of detail. In the
middle is a full detail Search Result Element (SRE) that is bracketed by a
Appeal 2010-001888
Application 10/809,172

6
simplified view containing just Universal Resource Locators (URLs).
Roberts Fig. 1, caption.
2. When describing its bracketing concept and design, Roberts first
explains general detail-and-context techniques. These techniques allow the
user to first focus on features or interesting parts of the visualization while
viewing the surrounding context. Uninteresting information in the context
view may be reduced in detail, distorted, or squashed. Sometimes a
semantic change (e.g., increasing or decreasing the detail in the view)
accompanies the viewing. Roberts § 3, ¶ 3.
3. Roberts describes how the Google API returns a SRE including nine
variables concerning one URL that matches the keywords in a search.
Roberts § 2.1, ¶ 2. Although Roberts does not identify all of the variables
that are part of a Google SRE, Roberts does identify four: title, URL,
paragraph of text, and page size. Id. Roberts also describes other types of
information that could be returned by a search engine like Google. This
information includes text snippets, the search query itself, and the position of
keywords on the page. Roberts § 2.1, ¶ 3.
4. Pook describes a zoomable user interface in which users can zoom (or
enlarge) a section of the view that they find interesting. Pook § 1, ¶ 2.

ANALYSIS
Based on the record before us, we find no error in the Examiner’s
obviousness rejection of representative claim 1. The Examiner finds that
Roberts discloses a layout component that displays a detailed subset of
information, comprising textual information, within the area defined by the
lens component based upon the search result. Ans. 19. Specifically, the
Appeal 2010-001888
Application 10/809,172

7
Examiner finds that Roberts’s Figure 1 describes a web search results page
in which more detailed information is displayed for search results in the
focus view as opposed to search results in the context view. In the context
view, only a URL is shown. FF 1. The Examiner also finds that Roberts
teaches a system in which more detailed information, such as text snippets,
is displayed when a SRE is moved into the focus view. Ans. 19. This
finding is supported by evidence in Roberts that describes additional
information that could be associated with a SRE and returned by a search
engine. FF 3. We therefore agree with the Examiner that Roberts teaches a
layout component that displays a detailed subset of information, comprising
textual information, within the area defined by the lens component based
upon the search result.
The Examiner finds that Roberts does not disclose that the detailed
subset of information is animated to enlarge in size as compared to the
amount of information associated with the at least one other search result
displayed outside the lens region. The Examiner cites Pook as teaching this
feature. Ans. 20-21. Based on our review, we agree that Pook teaches
enlarging a section of a view that a user finds interesting. FF 4. The
Examiner finds that it would be obvious to modify the system of Roberts so
that the text in the focus view is animated to enlarge in size as suggested by
Pook. Id. We agree with the Examiner that this modification would be
obvious. Our conclusion of obviousness is also supported by additional
evidence in the record. Roberts teaches that that some systems combine
visual effects with semantic changes. FF 2.
Before discussing the merits of Appellants’ arguments in their Appeal
Brief, we will first summarize them. Appellants first discuss claim 1 in view
Appeal 2010-001888
Application 10/809,172

8
of the Specification. App. Br. 6. Appellants then repeat the limitation of
claim 1 at issue and assert that Roberts and Pook, either alone or in
combination, do not teach it. Id. This assertion is followed by a discussion
of each reference individually. App. Br. 6-7. Appellants then copy their
discussion of claim 1 from the Summary of the Claimed Subject Matter into
their arguments. App. Br. 7-8. Appellants then conclude with another
discussion of claimed subject matter in view of their Specification. App. Br.
8.
Appellants’ arguments in the Appeal Brief have not persuaded us of
reversible error in the rejection of claim 1 under 35 U.S.C. § 103(a). Where
Appellants discuss the claimed subject matter in view of their Specification,
we fail to see how this discussion identifies an error in the rejection. Where
Appellants provide specific arguments with respect to Pook and Roberts,
Appellants only address the references individually and do not address the
combination set forth by the Examiner. Appellants argue that each reference
individually fails to teach a system where a detailed subset of information is
animated to enlarge in size and to include additional textual information that
is selected from the at least one search result for insertion into the detailed
subset of information based in part on a query associated with the at least
one search result. We agree with Appellants that neither reference alone
teaches both including additional textual information and then animating to
enlarge in size that additional text. The Examiner, however, relies upon the
combination of the two references to teach both features and does not rely
upon a single reference. Since Appellants’ specific arguments with respect
to the individual references do not identify an error in the combination, we
do not find them persuasive. Where Appellants argue the combination, it is
Appeal 2010-001888
Application 10/809,172

9
merely an assertion without any supporting reasoning. Given the evidence
in the record supporting the Examiner’s rejection, we are not persuaded of
error in that rejection based upon Appellants’ mere assertion. Accordingly,
Appellants’ arguments in the Appeal Brief have failed to persuade us of
reversible error.
Appellants’ arguments in the Reply Brief have not persuaded us of
reversible error in the rejection of claim 1 under 35 U.S.C. § 103(a).
Appellants argue that Roberts’s § 3, ¶ 5, teaches a system in which, as
the focus view changes, any additional information is located in a new focus
view window. We fail to see how this paragraph of Roberts teaches what
Appellants assert. Appellants may be referring to Roberts’s Figure 5, which
shows a multiple view variant where the full SRE information for a search
result is displayed in a different window. Roberts’s Figures 1 and 4,
however, show examples where the detailed SRE information is displayed in
the same window.
Appellants argue that Roberts fails to teach a system in which the
detailed subset of information is animated to enlarge in size and to include
additional textual information that is selected from the at least one search
result for insertion into the detailed subset of information. The Examiner,
however, relies upon the combination of Roberts and Pook to teach both
features and does not rely solely upon Roberts. Since Appellants’ argument
only addresses a single reference and does not identify an error in the
combination, we do not find the argument persuasive.
Appellants also argue that Roberts teaches a system in which the
additional information is not located in the focus view. Appellants cite as
support for this argument the fact that each of the SRE’s are stored in a list
Appeal 2010-001888
Application 10/809,172

10
that can be visualized on demand. Reply Br. 5-6 (citing Roberts § 2.1, ¶ 2).
Appellants appear to be arguing that since the SRE information is stored in a
list to be visualized on demand, SRE information is not automatically
displayed when a user changes the focus view. We fail to see persuasive
evidence in Roberts supporting the argument that additional user interaction
is required in order to display SRE information. In fact, we only see
persuasive evidence that no additional interaction is required. Roberts’s
discussion of the sliding window methodology describes the only user
interaction required to display the SRE information -- the user sliding the
focus view. Roberts § 3, ¶ 5. Figure 4 shows a slider bar for changing the
focus view. However, Figure 4 shows no other button or GUI element that
would allow a user to demand the details from a SRE. The weight of the
evidence indicates Appellants are not correctly interpreting the language “on
demand.” The “demand” to display SRE information is the user’s sliding of
the focus view and does not require some additional interaction. We
therefore do not find Appellants’ argument persuasive and we affirm the
rejection of independent claims 1, 20, 21, and 25, and dependent claims 2-
12, 15-17, 19, 22-24, 26, and 27.

THE OBVIOUSNESS REJECTION OVER ROBERTS, POOK, AND WOLTON
Although Appellants nominally argue claim 13 separately in the
Appeal Brief, they merely: (1) incorporate their arguments with respect to
claim 1; (2) provide a statement that Wolton does not cure the deficiencies
identified with respect to claim 1; and (3) provide a general characterization
of Wolton. App. Br. 9-10. In the Reply Brief, Appellants merely argue that
Wolton fails to cure the deficiencies of Roberts and Pook with respect to
Appeal 2010-001888
Application 10/809,172

11
claim 1 and then incorporate their arguments from the Appeal Brief. Reply
Br. 6. Since the Examiner does not rely upon any teaching from Wolton in
the rejection of claim 1 and Appellants have not provided arguments for the
separate patentability of claim 13, we affirm the rejection of claim 13 as
well.

THE OBVIOUSNESS REJECTION OVER ROBERTS, POOK, AND MONTAGUE
Although Appellants nominally argue claim 14 separately in the
Appeal Brief, they merely: (1) incorporate their arguments with respect to
claim 1; (2) provide a statement that Montague does not cure the
deficiencies identified with respect to claim 1; and (3) provide a general
characterization of Montague. App. Br. 10. In the Reply Brief, Appellants
merely argue that Montague fails to cure the deficiencies of Roberts and
Pook with respect to claim 1 and then incorporate their arguments from the
Appeal Brief. Reply Br. 6-7. Since the Examiner does not rely upon any
teaching from Montague in the rejection of claim 1 and Appellants have not
provided arguments for the separate patentability of claim 14, we affirm the
rejection of claim 14 as well.

THE OBVIOUSNESS REJECTION OVER ROBERTS, POOK, AND SZABO
The Examiner finds that the combination of Roberts, Pook, and Szabo
teaches a computer-implemented interface for data presentation embodied
on a computer-readable storage medium in which a dynamic information
view is coordinated with an amount of information to progressively insert
additional information associated with the at least one search result into the
Appeal 2010-001888
Application 10/809,172

12
detailed subset of information according to an amount of time a mouse
hovers over the at least one search result. Ans. 16, 26-27.
Appellants argue that Szabo teaches neither progressively inserting
additional information related to a search result nor progressively inserting
additional information based on the amount of time the mouse hovers over a
search result. App. Br. 10-11.
The issue before us, then, is as follows:

ISSUE
Under § 103, has the Examiner erred in finding that the combination
of Roberts, Pook, and Szabo teaches a computer-implemented interface for
data presentation embodied on a computer-readable storage medium in
which a dynamic information view is coordinated with an amount of
information to progressively insert additional information associated with
the at least one search result into the detailed subset of information
according to an amount of time a mouse hovers over the at least one search
result?

ANALYSIS
Appellants’ arguments in the Appeal Brief have not persuaded us of
reversible error in the rejection of claim 18 under 35 U.S.C. § 103(a).
Where Appellants discuss the claimed subject matter in view of their
Specification, we fail to see how this discussion identifies an error in the
rejection. Where Appellants provide specific arguments with respect to
Szabo, Appellants only address Szabo individually and do not address the
combination set forth by the Examiner. The Examiner relies upon Pook for
Appeal 2010-001888
Application 10/809,172

13
teaching the progressive insertion of information. Ans. 16. The Examiner
relies upon Szabo for teaching the use of the mouse hovering technique to
initiate the progressive insertion of information taught by Pook. Id. Since
Appellants have not provided specific arguments with respect to all of the
references, we are not persuaded of error in the rejection. “[O]ne cannot
show nonobviousness by attacking references individually where . . . the
rejections are based on combinations of references. In re Keller, 642 F.2d
413, 426 (CCPA 1981); In re Merck & Co., Inc., 800 F.2d 1091, 1097 (Fed.
Cir. 1986). Where Appellants assert that the combination of Roberts, Pook,
and Szabo does not teach the limitation at issue, Appellants provide no
supporting rationale. A general allegation of patentability does not convince
us of reversible error in the rejection of claim 18.
Appellants’ arguments in the Reply Brief have not persuaded us of
reversible error in the rejection of claim 18 under 35 U.S.C. § 103(a).
Appellants merely argue that Szabo fails to cure the deficiencies of Roberts
and Pook with respect to claim 1 and then incorporate their arguments from
the Appeal Brief. Reply Br. 7. Given our discussion of claims 1 and 18
above, we are not persuaded that the arguments in the Reply Brief have
identified reversible error.

CONCLUSION
The Examiner’s rejection of claims 1-12, 15-17, and 19-27 under 35
U.S.C. § 103(a) as being unpatentable over Roberts in view of Pook is
affirmed.
The Examiner’s rejection of claim 13 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Wolton is affirmed.
Appeal 2010-001888
Application 10/809,172

14
The Examiner’s rejection of claim 14 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Montague is affirmed.
The Examiner’s rejection of claim 18 under 35 U.S.C. § 103(a) as
unpatentable over Roberts, Pook, and Szabo is affirmed.

ORDER
The Examiner’s decision rejecting claims 1-27 is affirmed.
No time period for taking any subsequent action in connection with
this appeal may be extended under 37 C.F.R. § 1.136(a)(1)(iv).

AFFIRMED

babc

Notice of References Cited

Application/Control No.

10/809,172
Applicant(s)/Patent Under
Reexamination
Timothy S. Paek et al.
Examiner

Kimberly Wilson
Art Unit

2100

Page 1 of 1

U.S. PATENT DOCUMENTS

* Document Number Country Code-Number-Kind Code Date MM-YYYY

Name

Classification

A US-

B US-

C US-

D US-

E US-

F US-

G US-

H US-

I US-

J US-

K US-

L US-

M US-
FOREIGN PATENT DOCUMENTS

* Document Number Country Code-Number-Kind Code Date MM-YYYY

Country

Name

Classification

N

O

P

Q

R

S

T
NON-PATENT DOCUMENTS
* Include as applicable: Author, Title Date, Publisher, Edition or Volume, Pertinent Pages)

U

Stuart Pook et al., Context and Interaction in Zoomable User Interfaces, PROC. WORKING CONF. ON ADVANCED VISUAL
INTERFACES 227 (2000).

V

W

X

*A copy of this reference is not being furnished with this Office action. (See MPEP § 707.05(a).)
Dates in MM-YYYY format are publication dates. Classifications may be US or foreign.

U.S. Patent and Trademark Office
PTO-892 (Rev. 01-2001) Notice of References Cited Part of Paper No.
Delete Last PagelAdd A Page
Context and Interaction in Zoomable User Interfaces
Stuart Pook 1,2,4 Eric Lecol inet I G u y Vaysseix za E m m a n u e l Barillot 2,a
1 I~cole Nationale Sup6rieure des Tg}16communications, CNRS URA 820, 46 rue Barrault, 75013 Paris, France
2 Infobiogen, 7 rue Guy MOquet - BP 8, 94801 Villejuif cedex, France
3 G6n~thon, 1 bis rue de I'lnternationale, 91000 I~vry, France
4 s t u a r t @ a c m . o r g
ABSTRACT
Zoomable User Interfaces (ZUlS) are difficult to use on large in-
formation spaces in part because they provide insufficient context.
Even after a short period of navigation users no longer know where
they are in the information space nor where to find the information
they are looking for. We propose a temporary in-place context aid
that helps users position themselves in zuIs. This context layer is a
transparent view of the context that is drawn over the users' focus of
attention. A second temporary in-place aid is proposed that can be
used to view already visited regions of the information space. This
history layer is an overlapping transparent layer that adds a his-
tory mechanism to ZUlS. We complete these orientation aids with
an additional window, a hierarchy tree, that shows users the struc-
ture of the information space and their current position within it.
Context layers show users their position, history layers show them
how they got there, and hierarchy trees show what information is
available and where it is.
ZUls, especially those that include these new orientation aids, are
difficult to use with standard interaction techniques. They provide a
large number of commands which must be used frequently and on a
changing image. The mouse and its buttons cannot provide a rapid
access to all these commands without new interaction techniques.
We propose a new type of menu, a control menu, that facilitates the
use of zuIs and which we feel can also be useful in other types of
applications.
1. INTRODUCTION
Zoomable User Interfaces (zu]s) are no longer new and their theo-
retical bases [7] and practical applications [1] have been discussed
in various papers. When using a zuI, the users are presented with a
view of an information space. The initial (or top level) view shows
the entire information space at a scale which allows it to fit on the
users' screen. The users can then zoom (or enlarge) a section of
the view that they find interesting. The graphical objects will get
bigger until, as soon as there is enough room on the screen, they
are replaced by other graphical objects showing the underlying in-
formation in more detail. This is called semantic zooming. We
developed a zu! for browsing the HuGeMap [14] database of the
Permission to make digital or hand copie of all or part of this
work for personal or classroom use is granted without fee provi-
ded that copies are not made or distributed for profit or commer-
cial advantage, and that copies bear this notice and the full cita-
tion on the first page. To copy otherwise, to republish, to post on
servers or to redistribute to lists, requires prior specific permis-
sion and/or a fee.
AVI 2000, Palermo, Italy.
© 2000 ACM 1-58113-252-2/00/0005..$5.00
major genetic and physical maps of the human genome. This zu!
was used as a testbed for the new techniques described in this pa-
per. For the purposes of this paper it is sufficient to know that the
top level view in the zu! shows 24 chromosomes, these chromo-
somes each have three genetic maps, and these maps consist of a
large number of genetic markers positionned along an axis. The
first three images on the Colour Plate show what a user would see
in zooming from the top level view (image 1), to a view of the
bands and maps on chromosome 10 (image 2), and then to a view
of the G6n6thon map on this chromosome (image 3).
ZUlS are used to present an information space to users. One of
the reasons that users are unable to successfully use zuis is that the
view of the information space shown to users, or the focus, does not
always contain the context needed by users to position this focus in
the information space. Once users are in this situation they are
disoriented, sometimes to the point of not understanding what they
are looking at and not knowing whether they should pan, zoom, or
dezoom to find what they are looking for. It could be said that they
are ' lost in hyperspace.'
We present two new temporary aids that the system can display at
the users' command if they arrive in this situation. The first, a con-
text layer, allows users to position the focus with respect £o more
global views of the information space. The second, a history layer,
allows users to revisit the route they took through the information
space to arrive at their present position. We also present a third aid,
a hierarchy tree, that is always visible. This aid is a second window
in the zuI and shows at all times the structure of the information
and the users' current position within this structure. Users can also
use this window to change their position in the information space.
zuIs are complex programs with complex user interfaces that are
normally controlled using the mouse, buttons and standard menus.
When navigating in a ZUl, users zoom, dezoom, scroll, create magic
lenses, move and resize magic lenses, move and scrolls portal, etc.
Some of these actions are executed very frequently. A user zooms
until the desired scale has been obtained and scrolls until the object
looked for has been found. The graphical objects used to present
the information space change frequently; a zoom or dezoom can
completely change the objects visible. Making these objects active
and using them to control the zu! is problematic because these ob-
jects change and move too frequently. We propose a new type of
menu, a control menu, that allows users to control zoIs in a consis-
tant and rapid fashion. A control menu can also include up to two
scroll bars; a single interactor can thus control a complex operation.
A zuI incorporating these new techniques can be tested over the
Web at the URL http://www.infobiogen.fr/services/zomit/.
2. CONTEXT LAYER
In zu l s users can only see one view at a time: the focus. Users often
227
cannot understand where the focus fits into the information space
because it only shows a limited region of this space and ignores the
surrounding context. Fisheye views [6] are one way of integrating
the context and focus into a single view. Some of the informa-
tion surrounding the focus is shown following the rule: the greater
the distance of the information from the focus the more interesting
it must be for it to be shown. The Document Lens [15] allows the
user to focus on a part of a document while keeping the surrounding
pages (the context) visible. Three dimensional pliable surfaces [2]
and hyperbolic displays [ 12] relate the screen area given to showing
non-focus information to the distance it is from the focus. The fur-
ther information is from the focus the less interesting it is assumed
to be and thus the smaller it is shown. These methods deform the
information space by eliminating information or by changing the
size and position of the representions of information.
In contrast to the above methods that integrate the context and focus
by deformation, we integrate the context and the focus by drawing
the context (the context layer) over the focus. These two views are
transparent so that they can both been seen at the same time. User
studies [8] have shown that transparent views and overlays are well
accepted by users.
When using a context layer two orthogonal controls are available:
the scale of the view shown in the layer and the relative levels of
transparency of the context layer and the focal view. The scale of
the context layer can be chosen so that the context layer shows any
view between the initial view and the focus. Users can also control
the relative transparencies of the context layer and the focus. This
allows users to concentrate on either the context or the focus by
making the chosen view be drawn solid and the other as transparent
as desired. The interactor used to control the scale and transparency
is described in section 5.
The context layer is positioned so that the region of this layer that
corresponds to the focus is in the centre of the zuI ' s window. This
region is indicated by a rectangle drawn in the centre of the window
that shows the size and position of the focus relative to the view
currently visible in the context layer.
This method of combining the focus and the context avoids the de-
formation of the other methods described above, which often makes
images difficult to recognize and understand. The advantage of de-
formation, that more information can be represented, is maintained
by a very fluid and rapid control of the scale of the context layer.
The users can quickly find the context that they need to identify the
focus. Changing the scale of the context layer changes the posi-
tion and often the form of the objects in the layer. This movement
helps users understand which, objects belong to the context layer
and which to the focus. The context layer is temporary and only
shown during the gesture used to create and control it. This avoids
overloading the screen with context information or allocating valu-
able screen real-estate to context information when it is not needed.
The view of the focus shown in image 3 on the Colour Plate shows
a view that the user might see after having navigated for a while in
the zuI. This view does not contain any clues that would let the
user know what map or chromosome is visible. In this situation the
user can ask the zuI to show the context layer. The context layer
contains the initial view or context (image 1) and is drawn on top
of the focus (image 3) giving image 6. The position of the focus is
indicated on the context layer by a green rectangle. In image 6 this
rectangle covers the text '10q' and tells the user that the focus is
showing the chromosome 10. In image 7 the user has zoomed the
context layer so that it shows the names of the genetic maps (the
focus never changes during the use of the context layer). The green
rectangle showing the position of the focus covers the name of the
Gdn~thon genetic map. The focus is thus showing this map. Using
the context layer the user has been able to position the focus in two
different contexts.
When the context layer is visible the user can choose to concen-
trate on either the focus or the context by changing the relative
transparency of these two views. The relative transparency can be
continuously adjusted from a state where only the focus is visible
to a state where only the context is visible. Image 8 is similar to
image 7 except that the user is now contentrating on the context
and has faded out the focus. The rectangle that show:; the position
of the focus is always visible and the user can see moIe clearly that
the focus is currently showing the Gtnt thon genetic map.
Transparent overview layers [4] are a different type of display that
differs from ours in that: (1) their layer is permanent while ours is
a temporary orientation aid; (2) the transparency level of their layer
cannot be changed by the user; (3) their layer always ,;hows the top
level view of the information space while ours can be: used even if
there is no top level view; and, (4) their layer can be used to move
or modify the objects in the information space while our zuI does
not allow objects to be manipulated in this way.
3. HISTORY LAYER
Context layers allow the user to find the answer to the question
'where am I? ' Another important question is 'how did I get here?'
zuIs need a history so that the user can return to previously visited
regions of the information space and see these regions in relation
to the focus and the top level view. We propose a transparent and
temporary history layer that allows the user to move interactively
along the path taken in the zuI. As with the context layer, the
history layer is temporary so as not to overload the screen and it
disappears when the user releases the mouse button at the end of
the gesture used to create it.
The path taken by the user in the zuI is a sequence of views of
the information space. The first view is the initial (or top level)
view (image 1 on the Colour Plate) and the view on the screen is
the last current view (image 5). All the views (called the historical
views) seen by users are stored in this sequence. Images 2 and 3
are historical views that the user has seen in going from the top
level view to the last current view. The history layer :is drawn over
the top level view (giving image 4) and contains a view that can be
varied by users from the last current view, via the all the historical
views in order, to the initial view (and back again). The user can
thus interactively 'go back in time' and see the evolution of the cur-
rent view in relation to the top level view. The comp~u'ison is done
directly because transparent views are used so as to show the top
level view and the historical view simultaneously. This compari-
son is also aided by the rectangles, drawn in two different colours,
that show the sizes and positions, relative to the top level view, of
the last current view and the historical view. The interactor used to
control the history layer is described in section 5.
As with the context layer, the relative transparency levels of the
history layer and the top level view can be adjusted so users can
concentrate on the history layer or on the top level view. The two
rectangles showing the position of the current view and the last
current view are always drawn solid and are not affected by the
level of transparency.
The current implementation of history overlays requires a top level
view. This may not exist in systems where users can dezoom from
a view of their own files to a view of, potentially and for example,
the whole Intemet. The scale of the top level view could also be so
different from that where the user is currently working that users are
unable to see changes to the positions of the current and historical
views. We are currently investigating whether the system should
228
choose a different view to replace the top level view and whether
(and how) users can control this choice.
4. HIERARCHY TREES
zuIs are often used on hierarchically structured datasets. Creat-
ing an information space for a zu! requires the developer to pro-
vide graphical objects visible in the top level view of the space that
summarize those objects found when users zoom. These objects
will then summarize those objects to be found as users continue to
zoom. A hierarchy is thus created. Objects in the information space
that are not accessible via objects visible in the top level view will
be hard to find by users because they will have no way of knowing
were to zoom to find them. The critical zones technique [10] can
be used to automatically provide the clues that there are objects to
be found by further zooming.
The techniques presented in the previous section help users to un-
derstand the information space from the top level view to the cur-
rent position. However, users do not know what is in other parts
of the information space and in particular what is to be found by
further zooming. The user is also unable to use the hierarchy to
navigate in the information space. Users looking at the details of
an object are unable to rapidly dezoom to see the entire object and
cannot easily move from a sub-object to another sub-object of the
same type.
zuIs are three dimensional spaces [7] and the scale is the verti-
cal dimension. The main view in a zuI shows a horizontal slice
through the information space. We propose a second orthogonal
view, called a hierarchy tree, that is a view of a flattened vertical
slice through the information space. The names of those objects
above the current position of the user in the space are shown. Ob-
jects also have types (an object can be a chromosome, a map, a
sequence, etc). If the information space is highly regular, the entire
hierarchy of types can be displayed in the hierarchy tree, otherwise
just that part of the hierarchy centred on the user's current position.
This hierarchy tree will thus show users the structure of the infor-
mation space, what information is available, where that information
is located and how to find it.
A F M b 0 7 3 x c 1
A F M 1 4 4 y e 9
• . ~ A F M 0 2 5 y b 2
, . ~ A F M b 3 2 1 y f 9
, . ~ A F M 1 5 7 x b 1 2
A F M a 0 5 8 y h 9
A F M a 3 1 l y c l
. - ~ A F M 3 4 3 t d 9
A F M 1 8 3 x h 1 0
ci~romosome
-Arm
~ Chrband
Subchrband
- data
I
- marker
sequence
2 3 4
8 .r~ 10
14 15 16
CHLC v3
Gertmhon s~
Wl/MIT RH
Figure 1: hierarchy tree
Figure 1 shows part of the main view of the information space plus
the hierarchy tree: In this regular information space, the chromo-
somes are visible on the top level view of the information space.
The chromosomes consist of arms, data and maps. This structure
is displayed as soon as the ZUl starts. The user's current position
in the structure is shown in magenta (or light gray); in this example
the user is currently looking at the 'G6n6thon' map on the chromo-
some 9. The structure indicates that if the user continues to zoom
on the map, it will be possible to find the markers' sequences.
4.1 Navigation
This technique offers an efficient method for rapid navigation as
users can use the hierarchy view to navigate in the information
space and to directly access related but currently invisible objects.
If the user clicks on the string 'chromosome' shown in Figure 1,
the zu! will dezoom sufficiently to show all of chromosome 9.
Users can also click on the string 'CHLC v3' (the name of the map
next to the 'G6n6thon' map on the chromosome 9) to move to map
CHLC v3. In this case, the zuI will show the same relative position
on the new map.
4.2 Similar techniques
Excentric Labeling [5] offers a way of identifying objects on the
screen. This technique labels, with ' tool tips' in the main view,
those objects located around the cursor. We propose a different, non
intrusive way, of identifying the object currently under the cursor.
As users move the cursor across the main view the hierarchy tree is
updated to show the type and name of the object under the cursor.
If the cursor leaves the window or is not on an object, the hierarchy
view indicates the lowest level in the hierarchy to which all the
objects in the main view belong. Our method of labelling remains
similar to ' tool tips' in that the user does not have to ask for the
information to appear.
The glBIS system [3] provides a global view of the displayed IBIS
graph structure that is in some ways similar to our hierarchy trees.
Their global view shows the subject of all the nodes in the network
organised by their primary link. As users zoom or pan the local
view of the network, the global view scrolls to show the users' their
current position in the network. The global view can also be used
to navigate in the network. Their global view does however differ
from our hierarchy trees in that it shows all the nodes in the net-
work. This means that at any moment only a small proportion of
its contents are visible and a scroll bar must be used to navigate
within the global view. zuIs typically contain a very large number
of objects and so a global view of all the objects in the information
space would be so big as to be ineffective. As discussed above, our
hierarchy trees are designed to make use of the structure present in
many zuIs and thus show the types of the objects in the informa-
tion space and the names of only that object under the cursor and
its ancestors in the hierarchy.
4.3 Evaluation
To aid the evaluation of the visualization techniques proposed in
this paper we created a modified version of the zu t without the
hierarchy trees. Eight subjects, chosen from our colleagues at Info-
biogen, were taught how to use our zuI. The subjects were asked to
answer 22 multiple choice questions. A training session explained
how to answer these question with and without the hierarchy trees.
This experimental design allowed us to study, using two interfaces
otherwise as similar as possible, whether the hierarchy trees were
of assistance.
Ordering effects were taken into account. Half of the subjects an-
swered their first 11 questions with the hierarchy trees and the other
half of the subjects answered their first 11 questions without the hi-
erarchy trees. Half of the subjects were given the first 11 questions
in the list of 22 to do first while the other half of the subjects did
the second half of the questions first. This lead to four equally sized
groups of subjects.
For each subject we calculated the time taken to answer 11 ques-
tions without the hierarchy trees divided by the time taken to an-
229
swer the other 11 questions with the hierarchy trees. A value greater
than one from this calculation would mean that having the context
aids was an advantage. The mean value was 1.58 with a standard
deviation of 0.54. The high standard deviation was caused by the
lack of familiarity of some of the subjects with zuIs. For these peo-
ple the training session was not long enough and they thus found the
second set of questions easier. In general however the subjects were
faster with the hierarchy trees and were positive in their comments
regarding these aids: in fact those that started with the hierarchy
trees were reluctant to continue the experiment without them. The
other new techniques presented in this paper are currently being
evaluated.
5. A NEW INTERACTOR
Standard menus such as pull-down, pop-up and marking menus
[11] allow actions to be selected. Pop-up and marking menus are
contextual: they are activated at a location chosen by users. The
program can thus adapt the contents of the menu to the objects
found at that position and apply the action chosen to those objects.
Pop-up and marking menus do not allow the operation to be con-
trolled once it has been chosen from the menu (a scroll operation
cannot be completed using just a standard menu) and do not allow
users to supply the parameters required by the operation. An oper-
ation such as a font size change in a word processor often requires
a dialog box to provide the new size. The users must select the op-
eration using the menu and then concentrate on a second interactor.
Once the new size has been entered the dialog box disappears and
the users must reconcentrate on the main task.
Panning in a zuI requires either a dedicated mouse button so that
users can drag the image or two scroll bars. It is not possible to pan
using a standard menu except with difficult to use commands such
as 'move a little to the left.' Zooming is also difficult to perform
with a standard menu as users want to zoom until they reach the
required scale. Standard menus only allow users to zoom by fixed
steps and that only by repeated use of the menu.
5.1 Control menu
We propose a new form of pop-up menu, called a control menu
[13], that can integrate up to two scroll bars or spin-boxes. With
. . . . pan÷
context* (-0 zoom*
m i s c l > ~ r e m o v e I e n sl>
I [ ~ l ~ - n ¢ w le nsl>
Figure 2: the control menu in our ZUI
this menu users can choose an operation and control it or supply
parameters with a single gesture.
A control menu works somewhat like a marking menu. The novice
user presses the mouse button and waits (0.3 seconds) until the
menu appears under the cursor and then moves the cursor in the
direction of the desired operation. The menu disappears and the
operation starts as soon as the cursor has been moved the activa-
tion distance from the centre of the menu. (We have empirically
chosen an activation distance of five times the radius of the circle
in the centre of the menu.) The operation finishes when the user
releases the mouse button. A user that has learnt the position of
the desired operation does not have to wait to see the menu and can
move the cursor immediately. The gesture is otherwise the same.
This user has thus learnt to be an expert and is not distracted by the
now unnecessary menu.
5.2 A control menu integrating a scroll bar
A control menu can be used to modify the scale in a ZOl. This
operation is an example of the integration of a menu and a single
scroll bar. Figure 3 shows the mouse movements during the use of
moUSepressedbutton ~ , / 1. choose zoom>l 2. ;zoom =, I
' w 3. dezoom ~ ~ 1
mouse button
released
I 4. zoom .~!~ final result (dezoom)
Figure 3: zooming with a control menu
a control menu to choose the zoom operation and to then control
the zoom. The user presses the mouse button and :moves it the
activation distance (movement 1) towards the right (as the zoom
operation is on the right of the control menu in Figure 2). This
selects the operation and the cursor changes on the screen. From
this moment until the mouse button is released, mouse movements
to the right (movements 2 and 4) zoom the view and movements to
the left (movement 3) dezoom it. The feedback is immediate: the
view changes as the user moves the mouse. The user releases the
mouse button once the desired scale has been obtained (a dezoom
in this example).
During the zoom operation, the user can undo the current zoom
by moving the mouse up or down a large distance. The user can
then confirm the undo by releasing the mouse button or can undo
the undo by moving the mouse back towards the centre of the dis-
play. It this case the user still has the mouse button pressed and can
continue the zoom.
5.3 A control menu integrating two scroll bars
A control menu can also be used to perform two dimensional pans.
It thus replaces two scroll bars. The pan operation is selected by
pressing the mouse button and by moving the mouse; up (the pan
operation is at the top of the control menu in Figure 2). The view
is dragged by the mouse during the operation. It is not possible to
undo this operation as it is possible to undo a zoom because all the
movements of the cursor already have a meaning.
5.4 Control menus and simple commands
A control menu can contain simple commands that do not have any
parameters. Commands that can be cancelled are executed as soon
as the cursor has been moved the activation distance from where the
mouse button was pressed. As the user still has the mouse button
pressed, a movement in the other direction undoes the operation.
The undo can be undone by moving the cursor back again. The
user releases the button when the desired result has been obtained.
5.5 Marking versus control menus
When an expert uses a marking menu the distance moved during
the selection of an operation is not important. Only the form of
the gesture is important and is analyzed once the user releases the
mouse button or stops moving the cursor (so as to see a menu). A
control menu is different in that the distafice moved by the cursor
is important. The position of the cursor is constantly analyzed and
as soon as the cursor has been moved the activation distance from
where the mouse button was pressed, the operation indicated by the
direction of the movement is started.
230
5.6 Control menus with context and history
layers
A control menu is used to control the history and context layers pre-
sented in sections 2 and 3. The context layer has two parameters:
the scale and the relative transparency. The scale is controlled by
horizontal movements of the cursor and the transparency by ver-
tical movements. The control of the history layer is similar; the
choice of current view is controlled by horizontal movements of
the mouse and the transparency by vertical movements. These pa-
rameters are not integral [9] (a diagonal movement of the cursor
has no simple meaning) and we are currently investigating whether
this poses problems for users.
6. I M P L E M E N T A T I O N
The ideas presented in the previous sections were implemented and
tested in a client/server system designed to be used over the Inter-
net (Figure 4). The client is a Java applet that communicates with
I . I :1 html I [ _roe° r !~-7--1 page [ web TCP/IP
I
connection I ° se ~ server
I client I ; I classes l
I (Java I I " ' I I H u G e M a p
I applet) ~ server application data
user's_machine', I I!brary code t ~ t base
our macl~in~- -" I (C++) (C++) server
Figure 4: client server implementation
a C++ server over a TCP/IP connection. The client communicates
the user's current position to the server and the server responds
with all the objects visible at the position in the information space,
The client stocks these objects and can thus respond rapidly to user
commands; only the arrival and display of new objects is delayed
by the latency of the network connection to the server. The server
remains close to the data base so that it can rapidly read the large
quantities of information required to construct the objects to be sent
to the client. The client and the server library that it communicates
with are standard and do not need to be changed to provide a sys-
tem to visualize a completely different information space. Only the
database code and the code that calls the server library, shown in
bold text in Figure 4, need to be changed.
7. C O N C L U S I O N
We have attempted to address one of the major problems of zuIs:
the lack of context and thus the users' difficulties in understanding
where they are in the information space and where they can find the
information that they are looking for. We have developed new visu-
alization techniques that provide a new way of combining the focus
and context in zuIs. One of our new context aids, hierarchy trees,
is always visible and provides a constant reminder of the user's po-
sition in the information space. The other context aids, context and
history layers, are only visible when the user is lost and requires
orientation help. We will continue to compare these quite different
aids and try to understand when and how the users find them useful.
The interaction complexities that exist in zuIs and especially in
our zuI with its extra context aids lead us to develop a new type of
menu that combines the selection and control of operations. This
new menu has been integrated into our zuI but we believe that it
will be useful in other visualization programs.
We hope that the combination of our new context aids and our new
interaction methods will give users better control over a better in-
terface.
This work was supported by the European Union (contract BIOa-
CT96-0346) and by the CNET (contract 97 754 21).
8. REFERENCES
[1] B. B. Bederson, J. D. Hollan, K. Perlin, J. Meyer, D. Bacon,
and G. Furnas. Pad++: A zoomable graphical sketchpad for
exploring alternate interface physics. J. Vis. Lang. Comput.,
7:3-32, Mar. 1996.
[2] M. S. T. Carpendale, D. J. Cowperthwaite, and F. D. Fracchia.
3-dimensional pliable surfaces: For the effective presentation
of visual information. In UIST '95, pages 217-226, Pittsburgh
PA, USA, Nov. 1995. ACM Press.
[3] J. Conklin and M. L. Begeman. glBIS: A hypertext tool for
exploratory policy discussion. ACM Transactions on Office
Information Systems, 6(4):303-331, Oct. 1988. Selected Pa-
pers from CSCW '88.
[4] D.A. Cox, J. S. Chugh, C. Gutwin, and S. Greenberg. The us-
ability of transparent overview layers. In CHI '98 Summary,
pages 301-302, Los Angeles, CA USA, Apr. 1998. ACM
Press.
[5] J.-D. Fekete and C. Plaisant. Excentric labeling: Dynamic
neighborhood labeling for data visualization. In CHI '99,
pages 512-519, Pittsburgh, PA, USA, May 1999. ACM Press.
[6] G. W. Furnas. Generalized fisheye views. In CHI '86, pages
16-23, Boston MA, USA, Apr. 1986. ACM Press.
[7] G. W. Fumas and B. B. Bederson. Space-scale diagrams: un-
derstanding multiscale interfaces. In CHI '95, pages 234--241,
Denver CO, USA, 1995. ACM Press.
[8] B. L. Harrison, G. Kurtenbach, and K. J. Vicente. An exper-
imental evaluation of transparent user interface tools and in-
formation content. In UIST '95, pages 81-90, Pittsburgh PA,
USA, Nov. 1995. ACM Press.
[9] R. J. K. Jacob and L. E. Sibert. The perceptual structure of
multidimensional input device selection. In CHI '92, pages
211-218, Monterey, CA USA, May 1992. ACM Press.
[10] S. Jul and G. W. Fumas. Critical zones in desert fog: Aids to
multiscale navigation. In UIST '98, pages 97-106, San Fran-
cisco, CA, USA, Nov. 1998. ACM Press.
[11] G. Kurtenbach and W. Buxton. User learning and perfor-
mance with marking menus. In CH1 '94, pages 258-264,
Boston MA, USA, Apr. 1994. ACM Press.
[12] J. Lamping and R. Rao. The hyperbolic browser: A fo-
cus+context technique for visualizing large hierarchies. J, Vis.
Lang. Comput., 7(1):33-35, Mar. 1996.
[13] S. Pook, E. Lecolinet, G. Vaysseix, and E. Barillot. Control
menus: execution and control in a single interactor. In CHI
2000, The Hague, The Netherlands, Apr. 2000. ACM Press.
to appear.
[14] S. Pook, G. Vaysseix, and E. Barillot. Zomit: biological data
visualization and browsing. Bioinformatics, 14(9):807-814,
Nov. 1998.
[15] G. G. Robertson and J. D. Mackinlay. The document lens.
In UIST '93, pages 101-108, Atlanta GA, USA, Nov. 1993.
ACM Press.
231