New dragging paradigms
Introduction
Dragging and dropping shows limitations under certain conditions (unreachable target, large displays, ...).
This page presents some solutions proposed so far to address drag-and-drop limitations.
Demos
- A prototype implementing the "instrumental interaction model for drag-and-drop and its extensions" in a multi-computer environment (use RMI):
PoIP.2006-07-21.zip (needs java 5.0). - A prototype implementing the "instrumental interaction model for drag-and-drop and its extensions" on a single computer:
directManipulation.2005-11-03.zip (needs java 5.0). - Here is a video presenting the differents interaction techniques discuted in this page :
hascoet-collomb-throwing_models.avi (18.1 Mo) - The screenshots presented in this page have been taken with this demo :
dragging.demo.2005-03-11.zip (for Win32 - if you're not using win XP, then you must install
gdi+ to run this demo). - The demo presented for Interactive experiment on "Speed and accuracy in throwing models" at HCI 2004 is also available :
hci2004collomb.dragging.demo.zip (needs java 1.4).
Drag-and-drop
Publication
Evolution du drag-and-drop : du modèle d'interaction classique aux surfaces multi-supports Mountaz Hacoët, Maxime Collomb et Renaud Blanch. Revue I³ (information, Interaction, Intelligence), 4(2), 2004.
Ressources
For readers of i3, here are the diagrams of drag-and-drop operations :
- MacOS X/Carbon diagram UML sequence diagram
- X-window/Motif diagram UML sequence diagram
- Windows/OLE diagram UML sequence diagram
- Java/Swing diagram UML sequence diagram
You can also test some toy prototypes of implementations of drag-and-drop on MacOS X/Carbon, X-window/Motif, Windows/OLE, Java/Swing.
Description
Drag-and-drop is commonly used in actual systems :
- first, the user presses the mouse button on the source icon,
- then, he drags the cursor to reach the target,
- and he releases the mouse button.
Pick-and-drop [Rekimoto, 1997]
Reference:
the pick-and-drop homepage
« Pick-and-Drop is an extended concept of the commonly used drag-and-drop. With this technique, a user picks up an object on one computer display with a stylus, then drop it on a (possibly different) computer display. For example, a user can select or create a text on one's own PDA and pick-and-drop it at the desired location on the whiteboard. From the implementation point of view, the data is transferred through the network, but from the user-interface point of view, this technique allows a user to pick up digital data as if it were a physical object. »
See it in action: video
Using pick-and-drop on a single surface with a simple mouse :
- first, the user click on the source,
- then, he moves the cursor to reach the target,
- and he click on the target.
Drag-and-pop [Baudisch, 2003]
Reference:
the drag-and-pop homepage
« Drag-and-pop and drag-and-pick are interaction techniques designed for users of pen- and touch-operated display systems. They provide users with access to screen content that would otherwise be impossible or hard to reach, e.g., because it is located behind a bezel or far away from the user. Drag-and-pop is an extension of traditional drag-and-drop. As the user starts dragging an icon towards some target icon, drag-and-pop responds by temporarily moving potential target icons towards the user's current cursor location, thereby allowing the user to interact with these icons using comparably small hand movements. Drag-and-Pick extends the drag-and-pop interaction style such that it allows activating icons, e.g., to open folders or launch applications. »
See it in action: video
- the user begins a drag-and-drop operation,
- after a little move, possible targets (of compatible type, in the directin of the move) pops near the user cursor
- then, the user terminates his drag-and-drop operation using the moved targets (or the original ones, if he wants to),
- after the user has dropped the icon, popped targets desapear.
Stitching [Hinckley, 2004]
Reference:
Ken Hinckley homepage
« Stitching is a new interaction technique that allows users to combine pen-operated mobile devices with wireless networking by using pen gestures that span multiple displays. To stitch, a user starts moving the pen on one screen, crosses over the bezel, and finishes the stroke on the screen of a nearby device. Properties of each portion of the pen stroke are observed by the participating devices, synchronized via wireless network communication, and recognized as a unitary act performed by one user, thus binding together the devices. »
Drag-and-throw / push-and-throw
[collomb-hascoët, 2004]
Publications
Speed and accuracy in throwing models. Collomb, M., Hascoët, M. (2004). HCI'04,
Leeds, UK, British HCI Group.
Throwing models for large displays.
Hascoët, M. (2003). HCI'03, Bath, UK, British HCI Group.
Common description
We introduce new interaction models for throwing objects. These models aim at reducing error rates significantly while preserving reasonable time performances. They share a common design strategy based principally on (a) feedback strategies, (b) metaphors, and (c) the explicit definition of object trajectories. They differ mainly on the model used to compute associated trajectories resulting in different possibly useful trajectories.
Three types of visual feedback have been developed :
- Trajectory feedback: this consists of displaying the trajectory of the object to be thrown as a line between his actual location and the location where it will be thrown if the mouse button or pen is released. It also enables "trajectory snapping", i.e. automatically adjusting the trajectory when it is near a potential target such as a folder icon.
- Target feedback: this is the endpoint of the trajectory.
- Take-off feedback: this is the area in the source display that maps to the complete destination display. This feedback helps users understand where they can move the pointer or pen.
Drag-and-throw
Drag-and-throw uses the archery metaphor: user has to move the cursor in the oppposite direction of the wanted icon movment.
- user first initiates a drag,
- then, the visual clues appear (take-off area, trajectory),
- user adjusts his throw keeping the cursor in the take-off area,
- user ends the drag and the object is thrown on the selected target.
Push-and-throw
Push-and-throw uses the pantograph metaphor: user's movement is amplifed and is not reversed as with drag-and-throw
- user first initiates a drag,
- then, the visual clues appear (take-off area, trajectory),
- user adjusts his throw keeping the cursor in the take-off area,
- user ends the drag and the object is thrown on the selected target.
Push-and-pop
[collomb-baudisch-hascoët-Lee, 2005]
Publications
Improving drag-and-drop on wall-size displays Collomb, M., Hascoet, M., Baudisch, P., and Lee, B. Proceedings of Graphics Interface 2005. Victoria, BC, May 2005.
Description
See it in action: video 1 video 2
Push-and-pop is based on push-and-throw and drag-andpop, it combines the strengths of both techniques. In the shown example, the user is dragging an html document into IEXPLORE. The interaction proceeds as follows.
- The user starts dragging the word document icon.
- As a response, the system surrounds the pointer with a miniature representing the whole desktop — the takeoff area — here containing the icons of the word and internet explorer applications, a folder, and the recycle bin.
- The user drags the html document icon over IEXPLORE, which responds by highlighting itself with a rectangular frame.
- The users lets go of the html document icon and the html document is dropped on IEXPLORE. The take off area disappears.
In case users need to rearrange icons on the desktop, they can switch push-and-pop temporarily into a push-and-throw mode. Users invoke this functionality by moving the pointer back to the location of invocation, marked with a black circle in the figure.
Accelerated push-and-throw
[collomb-baudisch-hascoët-Lee, 2005]
Publications
Improving drag-and-drop on wall-size displays Collomb, M., Hascoet, M., Baudisch, P., and Lee, B. Proceedings of Graphics Interface 2005. Victoria, BC, May 2005.
Description
One of the main limitations of the original push-and-throw is its lack of precision due to the size reduction that occurs when mapping the desktop to the takeoff-area. To address this issue, nonlinear acceleration, as it is common with indirect input devices, is introduced in push-and-throw.
In accelerated push-and-throw, moving the pointer slowly results in a much slower motion of the dragged icon, helping users acquire small targets. In addition, the acceleration factor is reduced when the dragged icon is close to a target (similar to semantic pointing). Accelerated push-and-throw also allows clutching, i.e., lifting and repositioning the pen/finger within a drag interaction. This allows users to reach very distant targets.
With acceleration, there is no more immediate correspondence between physical pointer location and the location of the dragged icon. As a consequence, the technique does not have a clearly defined take-off area anymore and we cannot provide a preview of it.
See it in action: video
