Apple may unveil new multi-touch gesture language

Posted by Dennis Sellers Jul 3, 2008 at 5:53am

An Apple patent (number 20080163130) for gesture learning has appeared at the US Patent & Trademark Office. It promises innovative features for the iPod, iPhone and perhaps other devices.

The patent describes how Apple may go about teaching you a whole new multi-touch gesture language, consisting of hundreds of words.According to Apple systems and methods that incorporate various techniques for teaching gestures to a user of a multi-touch sensitive device are disclosed. Such techniques can include presenting visible feedback of gestures, such as animated motion trails and/or hand motions, along with affirmative feedback for correctly performed gestures and negative feedback for incorrectly performed gestures. Such techniques can be expanded to provide training exercises that present tasks requiring a particular gesture or sequence of gestures to be performed. These training exercises can take the form of games or other engaging activities to encourage use.

Here’s Apple’s background and summary of the invention: “Many attempts have been made over the years to improve the way users interact with computers. In the beginning, cards or tapes with punched holes were used for user input. Punch cards gave way to terminals with alphanumeric keyboards and text displays, which evolved into the modern keyboard, mouse, and graphical-display based graphical user interfaces. Many expect that the use of multi-finger, touch-sensitive user interfaces ‘multi-touch interfaces’), such as those described in the references incorporated above, will become widely adopted for interacting with computers and other electronic devices, allowing computer input to become even more straightforward and intuitive.

“Users of these multi-touch interfaces may make use of hand and finger gestures to interact with their computers in ways that a conventional mouse and keyboard cannot easily achieve. A multi-touch gesture can be as simple as using one or two fingers to trace out a particular trajectory or pattern, or as intricate as using all the fingers of both hands in a complex sequence of movements reminiscent of American Sign Language. Each motion of hands and fingers, whether complex or not, conveys a specific meaning or action that is acted upon by the computer or electronic device at the behest of the user. The number of multi-touch gestures can be quite large because of the wide range of possible motions by fingers and hands. It is conceivable that an entirely new gesture language might evolve that would allow users to convey complex meaning and commands to computers and electronic devices by moving their hands and fingers in particular patterns.

“Techniques for teaching these gestures and gesture languages to new users are needed. Techniques that have been proposed include playback of motion trails indicating the gesture, animated hands performing the gesture, and various graphical depictions of gestures and their meanings. Each of these techniques suffers from one or more deficiencies.

“The present invention can relate, for example, to a method for teaching gestures. The method can include presenting a display having two display areas, one of which can be a multi-touch monitor window. The multi-touch monitor window can be used to display interactive feedback to a user indicating what gesture the user is performing. The multi-touch monitor window can be overlaid on the first display area or can be a separate window, such as a side-by-side arrangement. If the multi-touch monitor window is overlaid, it can incorporate transparency or translucency that allows the display area behind it to also be perceived.

“The interactive feedback can take a variety of forms. For example, the interactive feedback can comprise an animated hand together with one or more motion indicators. The animated hand can be a line drawings, a three-dimensional rendering, a translucent shadow of a hand, or other representation. The motion indicators can include motion trails or other representations, which can also be color-coded. The interactive feedback may also be superimposed with an animated display of a correctly performed gesture so that the user can see the difference, if any, between the gesture he performs and the idealized gesture.

“The first display area can be used in conjunction with a variety of application programs. For example, the first display area can be used with a utility, entertainment, or communication application, in which case the feedback provided by the multi-touch monitor window serves as reinforcing feedback. As another example, the first display area can be used in conjunction with a gesture learning application.

“The gesture learning application can take a variety of forms. In some embodiments, the gesture learning application can be a game. In other embodiments, it can be an application that presents gestures to be performed to the user in the form of various repetitive drills. The gesture learning application can detect the gesture or gestures performed by the user in response to the presented gesture or gestures and provide feedback indicating whether the gesture or gestures were performed correctly.

“The present invention can also relate, for example, to a graphical user interface for a gesture learning application. The graphical user interface can include a main window including an indication of a gesture or sequence of gestures to be performed using a multi-touch interface and a multi-touch monitor window including an interactive feedback mechanism that indicates a gesture or gestures actually performed by the user. To facilitate learning, a sequence of gestures may be presented. These gestures can be arranged according to particular chords, motions, or sequences thereof.

“The indication of a gesture or gestures to be performed can take a variety of forms, including an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture to be performed, a textual description of a command corresponding to the gesture to be performed, and combinations thereof. Similarly, the indication of a gesture or gestures actually performed can also take a variety of forms, including an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture currently being performed, a textual description of a command corresponding to the gesture currently being performed, a positive feedback indicator, a negative feedback indicator, and combinations thereof. The positive and negative feedback indicators could also be in an audible form.

“In other embodiments of the invention, computer systems including one or more applications or graphical user interfaces as described above are provided. The computer systems can take the form of a desktop computer, notebook computer, tablet computer, handheld computer, personal digital assistant, media player, mobile telephone, or the like.”

The inventor is Wayne Carl Westerman. The graphic below illustrates a prior art display of gesture motion trails.

Advanced Search

Article Information

Comment on this Article Print this Article Email this Article Digg This

Contributor

Recent Articles