tinmith - mobile outdoor augmented reality Glove UI
Tinmith Glove-Based User Interface

 
Share

tinmith gloves 2006

This page describes the design of hand-worn gloves for interacting with mobile outdoor augmented reality systems. Most existing systems rely on more traditional 2D input devices such as mice and keyboards. Since augmented reality information is typically registered in 3D to the environment, user input devices need to be designed that are capable of supporting the more complex operations possible.

Our Tinmith 2006 outdoor augmented reality system uses these gloves, a novel menu-based user interface, and our action and construction at a distance techniques to implement a modelling system capable of building shapes of buildings, trees, automobiles, and other outdoor structures. These gloves were developed by Wayne Piekarski and Ross Smith.


glove hardware

We use metallic thread and adhesive fabric to add conduction sensing to a standard set of leather gloves. Since the metallic thread is sewn into the seams in the gloves, it is protected from harsh treatment and is more reliable than externally mounted wires. A small low-powered MSP430F1232 microcontroller senses finger presses and consumes negligible power. A Promi-SD202 Bluetooth controller makes the gloves fully wireless, making them easy to wear and with no cables to interfere with operation. ARToolKit with a randomised thresholding algorithm is used for reasonably reliable 3D tracking of fiducial markers on the thumbs, and uses the existing video camera on the helmet.


menu controls

Menu commands map to the hands as shown in the first image, with each finger corresponding to a menu item. The user picks the menu item by simply pressing that finger against the menu. There is no need to reach out and pull down menus or select widgets, which would require accurate tracking that is not available outdoors.

With the ability to enter commands, there is also a need to perform various 3D transform operations. When an object is selected, it can be translated, scaled, and rotated, using one or both of the user's hands. The pictures below show a tree that is being manipulated by the gloves.


Share


user performing 3D modelling task outdoors using wireless gloves and tinmith system 2006


example showing an AR user viewing various 3D virtual objects overlaid onto the physical world


user manipulating a 3D virtual object using the gloves


user performing a real-time carving operation to create a model overlaid on top of a physical building


ARQuake game running outdoors, showing virtual monsters overlaid onto the physical world


a tree created using the surface of revolution technique overlaid onto the physical world


Demonstration of the HOG table, showing how an indoor user's hand can be captured and shown to an outdoor user with a wireless network


Rear-view of the current 2006 tinmith system


example showing how the user interface links up to the fingers on the user's gloves

user performing 3D modelling task outdoors using wireless gloves and tinmith system 2006


example showing an AR user viewing various 3D virtual objects overlaid onto the physical world


user manipulating a 3D virtual object using the gloves


user performing a real-time carving operation to create a model overlaid on top of a physical building


ARQuake game running outdoors, showing virtual monsters overlaid onto the physical world


a tree created using the surface of revolution technique overlaid onto the physical world


Demonstration of the HOG table, showing how an indoor user's hand can be captured and shown to an outdoor user with a wireless network


Rear-view of the current 2006 tinmith system


example showing how the user interface links up to the fingers on the user's gloves

Contact Wayne Piekarski via email wayne AT tinmith.net for more information

Last Updated 2017