Macsimum News - Apple Patent: virtual input device placement on a touch screen user interface

Apple Patent: virtual input device placement on a touch screen user interface

Posted by Dennis Sellers Apple ico Feb 16, 2006 at 8:04am

On Feb. 16 the US Patent & Trademark Office revealed an Apple patent titled “Virtual input device placement on a touch screen user interface,” originally filed on Sept. 15, 2005.

Patent FIG. 1-1 is a block diagram of an exemplary computer system 50, in accordance with one embodiment of the present invention. The computer system 50 may correspond to a personal computer system, such as a desktops, laptops, tablets or handheld computer. The computer system may also correspond to a computing device, such as a cell phone, PDA, dedicated media player, consumer electronic device, and the like.

Apple’s abstract

A display is generated on a touch screen of a computer. The display includes an application display, associated with an application executing on the computer, and a virtual input device display for a user to provide input to the application executing on the computer via the touch screen. In response to a virtual input device initiation event, initial characteristics of the virtual input device display are determined. Based on characteristics of the application display and the characteristics of the virtual input device display, initial characteristics of a composite display image are determined including the application display and the virtual input device display. The composite image is caused to be displayed on the touch screen.

Description of the related art

A touch screen is a type of display screen that has a touch-sensitive transparent panel covering the screen, or can otherwise recognize touch input on the screen. Typically, the touch screen display is housed within the same housing as computer circuitry including processing circuitry operating under program control. When using a touch screen to provide input to an application executing on a computer, a user makes a selection on the display screen by pointing directly to graphical user interface (GUI) objects displayed on the screen (usually with a stylus or a finger).

A collection of GUI objects displayed on a touch screen may be considered a virtual input device. In some examples, the virtual input device is a virtual keyboard. Similar to a conventional external keyboard that is not so closely associated with a display screen, the virtual keyboard includes a plurality of keys (“virtual keys”). Activation of a particular virtual key (or combination of virtual keys) generates a signal (or signals) that is provided as input to an application executing on the computer.

External keyboards and other external input devices, by their nature (i.e., being external), do not cover the display output of an application. On the other hand, virtual input devices, by virtue of being displayed on the same display screen that is being used to display output of executing applications, may cover the display output of such applications.

What is desired is methodology to intelligently display a virtual input device on a touch screen to enhance the usability of the virtual input device and the touch screen-based computer.

Apple’s summary

This summary is not intended to be all-inclusive. Other aspects will become apparent from the following detailed description taken in conjunction with the accompanying drawings, as well as from the appended claims.

About Figure 1

The exemplary computer system 50 shown in FIG. 1-1 includes a processor 56 configured to execute instructions and to carry out operations associated with the computer system 50. For example, using instructions retrieved for example from memory, the processor 56 may control the reception and manipulation of input and output data between components of the computing system 50. The processor 56 can be implemented on a single-chip, multiple chips or multiple electrical components. For example, various architectures can be used for the processor 56, including dedicated or embedded processor, single purpose processor, controller, ASIC, and so forth.

In most cases, the processor 56 together with an operating system operates to execute computer code and produce and use data. Operating systems are generally well known and will not be described in greater detail. By way of example, the operating system may correspond to Mac OS X, OS/2, DOS, UNIX, Linux, Palm OS, and the like.

The operating system can also be a special purpose operating system, such as may be used for limited purpose appliance-type computing devices. The operating system, other computer code and data may reside within a memory block 58 that is operatively coupled to the processor 56. Memory block 58 generally provides a place to store computer code and data that are used by the computer system 50. By way of example, the memory block 58 may include Read-Only Memory (ROM), Random-Access Memory (RAM), hard disk drive and/or the like. The information could also reside on a removable storage medium and loaded or installed onto the computer system 50 when needed. Removable storage mediums include, for example, CD-ROM, PC-CARD, memory card, floppy disk, magnetic tape, and a network component.

By way of example, the touch-sensing device may correspond to a touchpad or a touch screen. In many cases, the touch-sensing device recognizes touches, as well as the position and magnitude of touches on a touch sensitive surface. The touch sensing means reports the touches to the processor 56 and the processor 56 interprets the touches in accordance with its programming. For example, the processor 56 may initiate a task in accordance with a particular touch. A dedicated processor can be used to process touches locally and reduce demand for the main processor of the computer system. The touch sensing device may be based on sensing technologies including but not limited to capacitive sensing, resistive sensing, surface acoustic wave sensing, pressure sensing, optical sensing, and/or the like. Furthermore, the touch sensing means may be based on single point sensing or multipoint sensing. Single point sensing is capable of only distinguishing a single touch, while multipoint sensing is capable of distinguishing multiple touches that occur at the same time.

The input device 70 may be a touch screen that is positioned over or in front of the display 68. The touch screen 70 may be integrated with the display device 68 or it may be a separate component. The touch screen 70 has several advantages over other input technologies such as touchpads, mice, etc. For one, the touch screen 70 is positioned in front of the display 68 and therefore the user can manipulate the GUI 69 directly. For example, the user can simply place their finger over an object to be controlled. In touch pads, there is no one-to-one relationship such as this. With touchpads, the touchpad is placed away from the display typically in a different plane. For example, the display is typically located in a vertical plane and the touchpad is typically located in a horizontal plane. In addition to being a touch screen, the input device 70 can be a multipoint input device. Multipoint input devices have advantages over conventional singlepoint devices in that they can distinguish more than one object (finger). Singlepoint devices are simply incapable of distinguishing multiple objects. By way of example, a multipoint touch screen, which can be used herein, is shown and described in greater detail in copending and commonly assigned U.S. patent application Ser. No. 10/840,862, which is hereby incorporated herein by reference.

The computer system 50 also includes capabilities for coupling to one or more I/O devices 80. By way of example, the I/O devices 80 may correspond to keyboards, printers, scanners, cameras, speakers, and/or the like. The I/O devices 80 may be integrated with the computer system 50 or they may be separate components (e.g., peripheral devices). In some cases, the I/O devices 80 may be connected to the computer system 50 through wired connections (e.g., cables/ports). In other cases, the I/O devices 80 may be connected to the computer system 80 through wireless connections. By way of example, the data link may correspond to PS/2, USB, IR, and RF, Bluetooth or the like.

About Figures 3-1 & 4

In accordance with an example, illustrated in FIG. 3, a resulting composite display is such that the application display (e.g., illustrated in FIG. 2) is substantially unchanged except, however that the virtual input display is overlaid on top of a portion, but not all, of the application display. In accordance with another example, illustrated in FIG. 3-1, a resulting composite display is such that the application display (e.g., illustrated in FIG. 2) is substantially unchanged except, however, that the application display is “slid up” and the virtual input device is displayed in the portion of the touch screen vacated by the “slid up” application display.

The display processing 106 accounts for the indicated characteristics of the application display to determine the location of the virtual input device display in the composite display on the touch screen. For example, the display processing 106 may determine characteristics of the composite display such that significant portions of the application display, such as an input field associated with the application display (and the virtual input device), are not covered by the virtual keyboard display.

That is, an input field of an application display is typically determined to be significant because it may represent the portion of the application with which the user is interacting via the virtual input device. However, other portions of the application display may be determined to be significant. For example, a portion of the application display that is directly affected by input via the virtual input device may be determined to be significant. In some examples, there may not even be an input field of the application display.

What is determined to be significant may be a function of a particular application and/or application display, or may be a function of characteristics of applications in general. In some situations, portions of the application display other than the input field may be relatively significant so as to warrant not being covered in the composite display by the virtual input device display. The relative significance may be context-dependent. For example, the relative significance may be dependent on a particular mode in which the application is operating.

In accordance with some examples, rather than the application display being substantially unchanged (such as is illustrated in FIG. 3 and FIG. 3-1, the display processing 106 determines characteristics of the composite display such that, while substantially all the information on the application display remains visible within the composite display, the application display is modified in the composite display to accommodate the virtual input device display. In some examples, the display processing 106 determines characteristics of the composite display such that the spatial aspect of the application display is adjusted to provide room on the composite display for the virtual input device while minimizing or eliminating the amount of information on the application display that would otherwise be obscured on the composite display by the virtual input device display. FIG. 4 illustrates one example where all portions of the application display are substantially equally compressed on the composite display, in one orientation.

About Figures 5 & 6

Patent FIG. 5 illustrates another example, where less than all portions of the application display are compressed on the composite display. In other examples, portions of the application display are expanded on the composite display where, for example, these portions of the application display are significant with respect to the virtual input device.

In some examples, which portion or portions of the application display are compressed on the composite display is based on the characteristics of the application display. For example, some portions of the application display determined to be of greater significance may not be compressed, whereas other portions of the application display determined to be of lesser significance may be compressed. In some examples, the amount by which a particular portion of the application display is compressed is based on the relative significance of that portion of the application display. Different portions of the application display may be compressed (or expanded) by different amounts in the composite display, including no change in spatial aspect.

In yet other examples, characteristics of the virtual input device on the composite display may be user configurable, as a preset condition and/or the characteristics of the virtual input device display can be dynamically configured. As an example of dynamic configuration, the user may change the position of the virtual input device display in the composite display by touching a portion of the virtual keyboard display and “dragging” the virtual input device display to a desired portion of the composite display.

In other examples, an example of which is illustrated in FIG. 6, an indication of the input may appear in a portion 604 of the display associated with the virtual input device 602, but not directly associated with the application display. Input may then be transferred to the application (directly, to be acted upon by the application, or to an input field 608 associated with the application display) either automatically or on command of the user. In accordance with one example, automatic transfer occurs upon input via the virtual input device 602 of “n” characters; where “n” may be a user-configurable setting. In accordance with another example, automatic transfer occurs every “m” seconds or other units of time, where “m” may be a user-configurable setting.

In some examples, the virtual input device display 602 includes a visual indicator 606 associated with the virtual input device 602 and the input field 608 of the application display. Referring to the example display 600 in FIG. 6, the virtual input device display 602 includes the visual indicator arrow 606, which points from the virtual input device display 602 to a corresponding input field 606 of the application display. The visual indicator 606 is not limited to being a pointer. As another example, the visual indicator 606 may be the input field 608 of the application field being highlighted.

NOTICE

Macsimum News presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application and/or grant is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent applications and/or grants should be read in its entirety for further details.

neo@macsimumnews.com

Leave a comment ⇒
Please post the article topic & comment in our forums. No registration required.

Article Information

Comment on this Article Print this Article Email this Article Digg This

Contributor

Dennis Sellers

Dennis has been a newspaper editor/reporter (seven years) and teacher (seven years). He has over 10,000 magazine, newspaper and online articles to his credit. He has also covered the Mac and tech industries for over a decade for such online publications as MacCentral, MacMinute and now MacsimumNews.