Designing Hybrid Interactions through an Understanding of the

257 Pages·2007·35.18 MB·English

by  Lucia Terrenghi

Ph.D. Thesis, October 2007
Dissertation
an der Fakultät für Mathematik, Informatik und Statistik
der Ludwig-Maximilians-Universität München
von
Lucia Terrenghi

Designing Hybrid Interactions
through an Understanding
of the Affordances
of Physical and Digital Technologies

Dissertation

an der Fakultät für Mathematik, Informatik und Statistik
der Ludwig-Maximilans-Universität München

von

Lucia Terrenghi

Oktober 2007
Berichterstatter:

Prof. Dr. Andreas Butz
(Ludwig-Maximilians-Universität München, Deutschland)

Dr. Abigail Sellen
(Microsoft Research Cambridge, Großbritannien)

Prof. Dr. Bill Buxton
(Microsoft Research Redmond, USA, und University of Toronto, Kanada)

Tag des Rigorosums: 17 Dezember 2007
Acknowledgments
A relevant part of this thesis argues that the social and physical contexts
affect one’s subjective perception of the surrounding environment. Similar
to that, I feel like I am at the end of a long journey, on which I have visited
several places and met a lot of people who in different ways have made this
trip possible. I have also had a rewarding and diverse set of experiences,
which have affected my personal way of seeing the world. Like on a trip,
whilst you learn about others and the places you visit, you also learn a lot
about yourself. The people I associate with this trip are portrayed in a kind
of virtual photo album, which is alive in my memory and emotions, and
which will accompany me in my future travels.
I deeply thank my advisor Andreas Butz, who made my expedition possi-
bleandgavemeakindofreturntickettogoandexploreanexoticlandcalled
Human-Computer Interaction. He encouraged me, tracked my movements,
and took care that I would not get lost or get into danger, whilst trusting me
with the choices I have taken along the path. A warm thanks goes to Abigail
Sellen for her very generous help: She has given me advice for the route to
take, motivated me to reach further destinations, interpreted my thoughts
and taught me how to express them in the language of academic research.
And like on any important trip, I also met my “guru”, who has inspired and
enlightened my path: I thank Bill Buxton, whose passion for research, and
most of all for life in general, has challenged me and transmitted me the
sake for exploration. I feel extremely fortunate to have benefited from the
diversity of my advisors’ perspectives: They have stimulated my curiosity,
and made my expedition rich of discoveries.
Along the way, I have experienced different situations: like exciting mar-
kets, whose colors, faces, scents and noises stimulated me and my creativity.
I thank the smiles and faces I met in those inspiring places, those people
who shared their time, thoughts, and their perspectives on my path: Abigail
Durrant, Gerhard Fischer, George Fitzmaurice, Richard Harper, Shahram
Izadi, David Kirk, Reinhard Oppermann, Dynal Patel, Thomas Pederson,
Yvonne Rogers, Chia Shen, Ben Shneiderman, Albrecht Schmidt, Marcus
Specht, Alex Taylor, Stuart Taylor, and Ken Wood. Many of these people
have become good friends and our acquaintance has influenced my path in
diverse ways.
I have had the fortune of being hosted in a fruitful environment and
constructive working atmosphere, with competent and supportive colleagues
who accompanied me for much of my trip: I thank Heinrich Hussmann and
my companions of the Media Informatics group Michael Aron, Richard At-
terer, Dominikus Baur, Gregor Broll, Thorsten Bu¨ring, Alexander De Luca,
HeikoDrewes, RainerFink, PaulHolleis, AndreasPleuß, EnricoRukzio, Sara
Streng, Sybille Thomsen, Arnd Vitzthum, Raphael Wimmer, and especially
OtmarHilligesandSebastianBoring, whoseknowledgeablecollaborationwas
verypreciousfortherealizationandelaborationofmanyoftheideasexplored
in this thesis. Likewise, I sincerely thank the students I worked with for
their valuable creative contribution: Manuela Altendorfer, Mark Breisinger,
TorstenFritsche, SebastianKraiker, SebastianKr¨amer, DavidKim,Matthias
Marm, Armin Prosch, and Hendrik Richter. And many thanks go to Amy
Ko, for helping me with the English language.
During my trip, there have also been steep mountains to climb. With-
out the great psychological and emotional support of my friends I wouldn’t
have overcome the difficulties of the ascent. I thank the ones who pushed
me further, sheltered me, and shared my accomplishments: Silvia Aramayo,
Michela Bertero, Jacques Caillaud, Valentina De Tomasi, Ulrike Herb, Lau-
rent Larivi`ere, Magda Marchesotti, Sabrina Marozin, Sara Mazzetto, Anna
Pou, Maddalena Prini, Silvia Raabe, Dagmar Schimanski, M´onica Sol´orzano,
Cristina Tilotta, Carla Valle, Anne Wirth.
A special thanks goes to Timo Ropinski, who was by my side for a large
part of this trip.
And, finally, I thank those very unique and special persons who did not
come along, but always gave me a place to go back to, which I could call
“home”: I thank my brothers Francesco, Marco, Nicola, and my parents.
Most of all I thank my mum, for being always there, whatever happened.
Thanks to the numerous people who contributed to my life and work
during the past years. If their names have not found their way into the
acknowledgements, their trace is somewhere hidden but vivid in the diary of
my trip.
Abstract
Two recent technological advances have extended the diversity of domains
and social contexts of Human-Computer Interaction: the embedding of com-
puting capabilities into physical hand-held objects, and the emergence of
large interactive surfaces, such as tabletops and wall boards. Both interac-
tive surfaces and small computational devices usually allow for direct and
space-multiplex input, i.e., for the spatial coincidence of physical action and
digital output, in multiple points simultaneously. Such a powerful combina-
tion opens novel opportunities for the design of what are considered as hybrid
interactions in this work.
This thesis explores the affordances of physical interaction as resources
for interface design of such hybrid interactions. The hybrid systems that
are elaborated in this work are envisioned to support specific social and
physical contexts, such as collaborative cooking in a domestic kitchen, or
collaborative creativity in a design process. In particular, different aspects
of physicality characteristic of those specific domains are explored, with the
aim of promoting skill transfer across domains.
First, different approaches to the design of space-multiplex, function-
specific interfaces are considered and investigated. Such design approaches
build on related work on Graspable User Interfaces and extend the design
space to direct touch interfaces such as touch-sensitive surfaces, in different
sizes and orientations (i.e., tablets, interactive tabletops, and walls).
These approaches are instantiated in the design of several experience pro-
totypes: These are evaluated in different settings to assess the contextual im-
plications of integrating aspects of physicality in the design of the interface.
Suchimplicationsareobservedbothatthepragmaticlevelofinteraction(i.e.,
patterns of users’ behaviors on first contact with the interface), as well as on
user’ subjective response. The results indicate that the context of interaction
affects the perception of the affordances of the system, and that some quali-
ties of physicality such as the 3D space of manipulation and relative haptic
feedback can affect the feeling of engagement and control. Building on these
findings, two controlled studies are conducted to observe more systematically
the implications of integrating some of the qualities of physical interaction
into the design of hybrid ones. The results indicate that, despite the fact
that several aspects of physical interaction are mimicked in the interface, the
interaction with digital media is quite different, which suggests the existence
of mental models and expectations resulting from previous experience with
the WIMP paradigm on the desktop PC.
Kurzzusammenfassung
Zwei aktuelle technologische Entwicklungen haben dazu gefu¨hrt, dass inter-
aktive Computersysteme in v¨ollig neuen sozialen und physikalischen Situa-
tionen eingesetzt werden k¨onnen. Diese Technologien sind in Alltagsobjekte
eingebettete Computer und interaktive Ober߬achen. Beide erlauben kom-
plexe Eingaben (z.B. mit mehreren Fingern und H¨anden) und zwar direkt,
alsoamgleichenOrtwiediezugeh¨origenAusgaben(spacemultiplexing), was
neue M¨oglichkeiten fu¨r die Gestaltung hybrider interaktiver Systeme schafft.
Die vorliegende Arbeit untersucht den Designraum dieser Systeme.
Den Ausgangspunkt fu¨r die Gestaltung hybrider Benutzerschnittstellen
bildet dabei das Konzept der “affordances” physikalischer Interaktionsob-
jekte. DiehierentwickeltenBeispielsystemeunterstu¨tzenverschiedenesoziale
und physikalische Situationen, wie z.B. das gemeinsame Kochen in einer
h¨auslichenKu¨che,oderdiegemeinsameIdeenfindungineinemDesignprozess.
Dabei werden Metaphern eingesetzt, um verschiedene physikalische Aspekte
derjeweiligenInteraktionssituationzuvermitteln. HierdurchkannderNutzer
bereits erlernte F¨ahigkeiten auf die neuen Systeme u¨bertragen.
Die Arbeit untersucht verschiedene Methoden, solche direkte und an-
wendungsspezifische Benutzerschnittstellen zu konzipieren. Dabei baut sie
auf verwandten Arbeiten im Bereich anfassbarer Schnittstellen (graspable
user interfaces) auf und erweitert den Designraum auf direkt beru¨hrbare
(direct touch) Schnittstellen unter Verwendung interaktiver Ober߬achen in
verschiedenen Gr¨oßen und Orientierungen, wie z.B. TabletPCs, interaktiver
Tische und W¨ande.
Die verschiedenen Methoden wurden dann dazu eingesetzt, eine Reihe
von Prototypsystemen zu bauen, die jeweils bestimmte Benutzungserfahrun-
gen vermitteln (experience prototypes). Dabei wurde immer untersucht,
welchen Einfluss die Integration physikalischer Aspekte in der jeweiligen Sit-
uation, sowohl auf pragmatischer als auch auf emotionaler Ebene, hat. Eine
wesentliche Beobachtung ist, dass die Anwendungssituation maßgeblichen
Einfluss auf die Wahrnehmung der jeweiligen “affordances” hat, und dass
einige physikalische Qualit¨aten, wie z.B. haptisches Feedback und echte Drei-
dimensionalit¨at insbesondere die emotionale Reaktion der Benutzer beein-
flussen und dazu fu¨hren, dass diese sich st¨arker an der Interaktion beteiligt
fu¨hlen. Auf Basis dieser Beobachtungen wurden zwei kontrollierte Benutzer-
studien durchgefu¨hrt, die den Einfluss physikalischer Interaktionselemente in
hybriden Schnittstellen systematisch untersuchen.
Die Ergebnisse dieser Studien belegen, dass es nicht ausreicht, physikalis-
che Aspekte in hybriden Schnittstellen nachzubilden, sondern dass die Inter-
aktion mit digitalen Medien oft durch bereits vorhandene mentale Modelle
aus der PC-Welt beeinflusst wird, die demnach fu¨r den Entwurf hybrider
Benutzerschnittstellen genauso wichtig sind.