It was revealed a little over a year ago that Microsoft was exploring a Google-Glass like product for recording live events via their "Life Streaming" technology that included a software app called "Lifecam Director." Last week the US Patent and Trademark Office published a patent application from Microsoft relating to smart glasses that resemble more traditional styled glasses. Microsoft's patent pending invention focuses on multi-touch interactions with various sensors built-into different parts of their glasses be it the side arms (as noted in our cover graphic), or the front frame or the Nose Bridge area and so forth. Depending on the application that you're running, there'll be differing areas of the glasses frame that you'll interact with so as to control functionality. For instance, touching the side frame of your glasses could raise or lower the volume on your TV. Microsoft also envisions using biometric sensors extensively starting with user ID through to warning you that you're running a fever. With Google likely to take the lead in the area of smart glasses later this year, Microsoft has to hustle to get this to market sometime in 2015. It's a known fact that Microsoft's new CEO Satya Nadella has pointed to the company working with a variety of sensors for future devices and today's report clearly shows you that their sensor driven glasses could be a great way for Nadella to build on his vision.
Microsoft's Patent Background
As computer and video technology evolves, users need to interface with computers in convenient ways. One way is with head-worn augmented reality glasses, in which the user sees displayed content rendered on the glasses.
However, in scenarios with head-worn augmented reality glasses (continuous use scenarios), interactivity is constrained by the inability to have an input device continuously available in the user's hand. While it is possible to use hand gestures or voice commands in those situations, this necessitates cameras and/or microphones, and may not be appropriate in many situations such as libraries where speech and even silent gestures may be distracting to others.
Microsoft Invents Multi-Touch Interactive Eyewear
Various aspects of Microsoft's invention describe making the frames or other areas of glasses or a similar eyewear device touch-sensitive for use in interfacing with a computer system.
Any or all of the surface area of the frames, and/or the glass surface, may be touch-sensitive for touch interactions, such as by combining capacitive and/or resistive touch sensors molded directly into the frames. Further, dedicated areas on the eyewear such as the location behind the ears, or the spot on the frame between the lenses may be used for additional functionality, (e.g., detecting whether the user is currently wearing the glasses). Biometric sensors may be built into the frames for sensing different usage states.
Also described are various interaction techniques that provide for numerous types of possible interactions. One or more of these techniques may be combined with input from other input devices such as wireless remotes, three-dimensional (3D) pointers, mobile phones, and so forth, to provide for further or modified interaction. The input received via the eyewear may be used and/or combined with other input data, such as obtained via eye-tracking and/or head-tracking, to modify and/or provide additional input information.
Microsoft notes that while glasses are used as a prime device example for multi-touch sensors, any other type of eyewear may benefit from the technology such as corrective lenses, 3D (stereoscopic) glasses, safety glasses, sunglasses, reading glasses, glasses with active shutters, goggles and so forth.
Microsoft additionally notes that the eyewear may also be used or combined with input data, such as obtained from head or eye-tracking.
Microsoft further notes further into the patent that their invention could apply to devices beyond glasses such as a tablet, smartphone, computer, TV, and 3D pointers.
Where it gets a little strange is when Microsoft describes their invention applying to "an inter-cranial/ocular and/or other device built into the body." Obviously this invention could be used in future areas of law enforcement (think Robocop - ha!), the military or medical applications.
According to Microsoft, the sensor types could include capacitive sensors, resistive (potentiometer) sensors, piezo-electric-based, optical, light-based (e.g., infrared), reflective (emitter/detector)-based and/or any other type of sensor that detects contact and/or proximity (e.g., hover) of a finger or other suitable pointing mechanism.
The entire surface area of the frames and/or lens, or something less than the entire area, may be touch sensitive. The sensors may be embedded into the frame and/or lens material, or a coating applied to the frame and/or glasses as noted below in patent FIG. 2B under feature #229.
Microsoft further notes that the application may determine a suitable sensor type, e.g., a waterproof sensor, a sensor that detects interaction at a relatively larger distance (e.g., reflective at near the user's arm's length), and so on.
In Microsoft's patent FIG. 3B noted below we're able to see sensor 338 configured to sense actuation via contact/proximity or a one-dimensional swipe. A single swipe direction may be the only swipe direction allowed for interaction, or two swipe directions may be allowed, each with the same meaning or independent meanings with respect to user intent. For example, a user viewing video content may swipe right-to-left on the sensor 338 for rewind, left-to-right for fast forward, or tap the sensor 338 for play; (note that a tap need not require physical contact, e.g., a finger that is briefly in close proximity to a proximity-type sensor and then removed may actuate the sensor as a "tap" without ever making physical contact). As another example, zoom-in or zoom-out with a camera-type application may be controlled via swipe interaction.
In Microsoft's patent FIG. 4B noted above we're able to see how two sensors 444 and 445 may be used together with single hand interaction. Contact/close proximity at the top sensor 444 may have one meaning, at the bottom sensor 445 another meaning, or both actuated together have a third meaning. Two handed operations can be achieved by two fingers from the right hand interacting with sensors #444 and #445 while the left hand is interacting with sensors #446 and 447.
Timing mechanisms and the like, such as implemented in the sensor handling logic may be used to allow the user to be slightly inexact in actuating the sensors together. Double-taps and triple-taps or the like may have yet another set of meanings. Swipes also may be used with such a pair of sensors.
Biometrics & Health Apps
Microsoft notes that various types of biometric and/or other sensing may be implemented in the design so that the frames could automatically "know" whether they are being worn on the head or not by sensing the glasses touching (i.e., skin contact) around the areas of the nose and ears.
Temperature sensing is one way to possibly detect whether the eyewear is being worn, particularly if compared to ambient temperature measured elsewhere. Capacitive sensing can also detect deformations in the nearby skin, which, for example, may be used to detect certain types of facial expressions such as raising or furrowing eyebrows.
Sensing of whether the eyewear is being currently worn or not may be used for turning off the displays in the eyewear when the eyewear are not in use, and automatically turning the system back up when in use; note that some logic or circuitry may be built into the eyewear to act upon such a state change. Such locations also may be used as points that sense touch-sensitive interaction.
Moreover, touch/proximity/other sensors embedded in the eyewear also may be used for collecting biometric data that may be used for purposes beyond knowing whether the glasses are currently being worn or not. For example, one or more sensors may sense galvanic skin response, provide electrocardiogram (EKG) signals, detect heart rate and so forth, such as by looking at two or more different sensors (e.g., electrodes) on different sides of the head (e.g., behind the wearer's ears).
Moreover, the touch-sensitive glasses may be combined with other sensors also built into the glasses, such as sensors for eye tracking, cameras, inertial sensors, jaw tracking, temperature sensors, heart-rate monitors, galvanic skin response, EEG, EMG, EKG, and the like. Such additional sensors may provide, for example, an always-on input-output solution in health monitoring, as well as with 3D input (e.g., when combined with eye gaze monitoring, inertial sensors, or cameras).
When combined with cameras/inertial sensors/microphones/gaze tracking and/or the like, these glasses may be used to enable gaze-directed visual search or augmented reality videos, for example.
Microsoft also gives you the impression that different users may be able to use these glasses at home and that by employing biometrics the glasses will know what data to show and what files can be opened depending on your fingerprint that is automatically scanned as you pick up the glasses and place them on your head.
Eyewear for Controlling your TV
Another use for sensor rich eyewear is watching television. Using head and eye tracking along with various sensors on the eyewear a user will be able to easily control volume or interact with the TV to change channels or to assist a user to work through a series of menus quickly to find shows, movies, music and so forth. Speech, hand gestures are another option that could be integrated into these next generation glasses.
Microsoft's patent application which was made public last week by the US Patent and Trademark Office, was originally filed back in Q4 2012. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
To give you an idea that the Glasses that Microsoft envisions could become a reality, look at the photo below of a pair of smart glasses that already exists is China. The smart glasses were invented by a Chinese student that were designed to help him cheat on his final exams. The inventiveness of the student should have landed him 100% mark for a practical invention. Yet I highly doubt that's the reward he recieved from his school.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Mobile presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Posting Comments: Patently Mobile reserves the right to post, dismiss or edit any comments.
Comments