Mike Elgan’s article The Mouse is Dead certainly struck a chord regarding the ubiquitous mouse we all love. Yes, this fine and beautifully simple creature has stood the test of time because it has consistently adapted from its humble beginnings in 1963 to become the undisputed and unchallenged king of user interfaces. Over time it has lost its tail (become wireless), dropped the funky rubber ball we all enjoyed cleaning from dust and crumbs, and morphed into a sleek ergonomic wonder with a chrome finish and flashing lights.

However, father time is beginning to take its toll on our dear friend. Is only one XY coordinate sufficient in a new age of computing with complex software applications running on hyper speed processors, or is this new age of computing demanding multiple and simultaneous XY coordinates? I would guess the latter, with reservations.

Our treasured mouse is a tool that very accurately navigates (locates XY), activates (right click), and/or commands (left click). The multi-touch technologies that may unseat our cherished mouse must accomplish these tasks at least as well, and that’s the rub. Commands are tasks well suited for gestures, but navigation and activation is a challenge. Low resolution capacitive multi-touch track pads do not measure up and this results in less than acceptable user satisfaction. Pushing your cursor around your screen with a track pad is bad enough, but when finally locating your target XY the lifting and re-touching to activate can and will shift the XY, increasing user frustration. Unless you can provide the accuracy (resolution) to navigate precisely with your track pad, and activate without lifting and retouching, you have not achieved what his majesty, the mouse, does effortlessly. Any heir to the throne must share common traits: simplicity (idiot proof), cost (cheap), reliability (forever faithful), and most importantly, accuracy (always on point).

Ironically, it is the same folks who popularized our beloved mouse, Apple Computer, now most prominently are leading the charge for its demise with multi-touch track pads and gesture recognition patent filings. No question Apple has the resource and expertise to bring a usurper to market, but can they deliver the most critical element; accuracy. This accuracy is a function of resolution, and I have yet to see any touch sensor offering from Apple or any other, except SiMa, with the ability to match the ‘always on point’ feature our esteemed monarch delivers…”THE KING IS DEAD. LONG LIVE THE KING!”

- Phil

Taking multi-touch and gesture recognition (MT/GR) into the auto may have some very cool user and manufacturer benefits. This article posted by Marie Freebody at Optics.Org gives some very good insight on this topic in her discussion of Osram Opto Semi’s auto solution.

Taking advantage of MT/GR use in the automotive environment can increase safety, enhance the pleasure of motoring and provide significant opportunities to auto manufacturers for part and weight reduction.

Any technology that reduces driver distraction can and will save lives and prevent injuries. By providing MT/GR technology to the standard GPS display available in most cars virtually all control functions of the auto’s subsystems (i.e. audio, climate control, seat adjustment, windows, etc.) are managed by gestures on the display without the need to look at the display. This eliminates the need to visually locate a given control knob or toggle switch located in various areas of the dashboard.

The driver experience is enhanced by the elimination of analog knobs and switch controls. By providing audio feedback to gesture input on the display the driver and auto can intuitively communicate precise control settings.

By eliminating the need for the analog controls and their associated wiring assemblies the auto manufacturing is dramatically simplified with fewer parts and increased reliability.

-Phil

I finally get around to comment on the announcement of this report…

In early May, DisplaySearch announced its upcoming report “Touch Panel Market Analysis“. The announcement highlights some interesting data points:

• Touch panel market to grow to $3.3 billion and 660 million units by 2015
• Mobile phones will account for 34% of all touch panel shipments and 21% of revenues
• Shipments of touch enabled mobile phones will reach 223 million units by 2015
• Resistive touch solutions will retain 86%
• Resistive revenue share will drop from 78% in 2007 to 64% in 2015

The industry is currently investing heavily in R&D to develop new touch technologies, so I’m a bit surprised to see the lion’s share of both shipments and revenues for the next 7 years to be resistive touch. Could it be that DisplaySearch expects resistive multi-touch to stave off competing technologies?

 I’m cautious about reports that so obviously uses approximate figures ($3.3 billion / 660 million units = $5/unit), but I do interpret it as a validation point that Sima is on to something big and that we’ve only seen the beginning of multi-touch and gesture recognition in mobile and portable devices.

- Marten

There is much talk about multi-touch for both Apple’s OS X and Microsoft’s Windows. While multi-touch screens certainly offers users an improved user experience, it just isn’t feasible to use a touch screen in all situations. Sure, there are applications where a multi-touch screen makes sense, such as gaming, kiosks, etc., but in the most common situations such as at a desk at work or at home, it would be a rather terrible user experience to have to move between a keyboard, mouse and the touch screen.

This doesn’t mean multi-touch is only reserved for certain applications. The opposite - multi-touch will completely change how we interact with computers. The mouse as a pointing device will be replaced by devices with functionality similar to the touch pads integrated in laptops.

At Sima, we’re envisioning a device that is similar to a mouse pad in look and feel. It’s made out of a flexible material that allows it to be placed on non-flat surfaces. The functionality of the pad would include:

• Controlling the mouse by moving a finger on the pad
• Replace right/left mouse clicks by either tapping a second finger, or applying pressure to the pad with the “mouse finger”
• Gesture recognition (zoom in/out, scroll vertically/horizontally, etc.)
• Use any stylus to input handwriting

We believe this will be the natural peripheral input device for Macs and PCs. This device makes it possible to upgrade any [modern] PC to Windows 7 and utilize the integrated multi-touch functionality. 

This unique type of device can be built today using Sima’s resistive multi-touch technology

A key element of effective gesture input is the ability to navigate the touch surface and activate the touch sensor WITHOUT lifting the finger(s). This multi-force capability has a couple of significant user benefits:

Improved accuracy — Lifting the finger(s) and retouching the touch sensor i no longer required. Without this capability, lifting the finger(s) and retouching the touch sensor can and will cause miss hits especially with standard low resolution touch sensors. Instead, simply applying more pressure with the finger(s) to activate the sensor provides a high level of input accuracy to applications such as gaming and CAD where accuracy is mission critical.

Relative positioning — The touch sensor places controls based on touch locations. The gesture interface is no longer static but becomes intuitive because no matter where you touch the sensor it intuitively ‘knows’ what you are doing. For example, let’s look at an application we are familiar with, key board strokes.

When typing on a keyboard, we rest our fingers on the ‘home row’ and strike keys with our fingers relative to ‘home row’ often without looking at the keyboard. The key locations are fixed while our fingers move relative to the ‘home row’. “Relative positioning” allows the user to touch ANYWHERE on the sensor with eight fingers in line horizontally and intuitively recognize this gesture as keyboard entry. Another example is the number pad. Three fingers down anywhere horizontally and the sensor ‘knows’ this is a number entry.

In a desktop environment one can easily see the benefits of “relative positioning” by eliminating the back and forth from keypad to mouse and back to keyboard. In other words; one finger mouse, three fingers number pad, eight fingers keyboard and so on. These are only a couple of examples of potentially hundreds of defined gesture input scenarios. 

Based on customer inquiries, we believe this functionality will become standard in a range of applications including automotive, mobile applications and gaming.