Microsoft introduced a possible Oculus Rift killer in January called the HoloLens. Whether they can get that to market for Xbox games in a timely manner is another matter. Microsoft updated and discussed their HoloLens device last month during their BUILD developer conference. Now a patent application has been discovered by Patently Mobile that illustrates how Microsoft is trying to bring a part of their HoloLens experience and technology to future versions of their Surface devices without the need of glasses. The invention reveals a unique light source that could be set up on a Microsoft Surface device that would be able to generate a holographic layer that floats just over the entire surface of the keyboard and allow for a secondary user interface to be visible. In one example, the floating interface would be that of a virtual Xbox gaming controller.
Microsoft Invents: Holographic Interaction Device
Computing has over the years gone from Keyboard and mouse to multitouch gestures on touchscreen devices. The next phase coming to computer devices could be holographic interaction, claims Microsoft.
In one or more implementations, an input device is configured to include a holographic recording mechanism that may be utilized to indicate available functionality of the computing device through use of a hologram, such as to indicate gestures that may be supported by the device.
For example, a holographic recording mechanism may be disposed over an input device, such as a thin film secured over one or more keys of a pressure sensitive keyboard. A light source may then be positioned (e.g., a laser included as part of the computing device) such that light emitted by the source on the holographic recording mechanism causes a hologram to be displayed above the input device.
The hologram may be configured to indicate gestures that are available for recognition by the computing device, such as part of a near-field user interface (NUI, also referred to as a natural user interface) that does not involve contact with a surface through detection using one or more sensors, such as an image sensor, long range capacitive sensor, and so on.
The hologram, for instance, may be configured as a display of a navigation control. Movement of one or more fingers of a user's hand in relation the displayed hologram may then be recognized as gestures involving navigation. In this way, a user may be readily informed as to a range of gestures that are available for interaction with a computing device.
In patent FIG. 1 below we're able to see a holographic device in the form of a Surface Tablet or notebook-like device. The input/output module #108 is representative of functionality relating to processing of inputs and rendering of outputs of the computing device #102.
In the illustrated example, the input device #104 is configured as having an input portion that includes a keyboard having controls that include a QWERTY arrangement of physical keys and track pad although other arrangements of controls are also contemplated. Other configurations are also contemplated, such as a display that also includes touchscreen functionality (e.g., a capacitive grid of sensors), and so on. Other non-conventional configurations are also contemplated, such as a game controller, configuration to mimic a musical instrument, and so forth. The keyboard could also be a virtual one.
Thus, the input device and controls incorporated by the input device 104 may assume a variety of different configurations to support a variety of different functionality.
The gesture module #116 noted in the graphic above may be configured to receive one or more inputs via touch interaction with a hardware device, such as a keys of the input device as shown in FIG. 1. Touch interaction may involve pressing a button, moving a joystick, movement across a track pad, use of a touch screen of the display device 110 (e.g., detection of a finger of a user's hand or a stylus), and so on.
Recognition of the touch inputs may be leveraged by the gesture module to interact with a user interface output by the computing device such as to interact with a game, an application, browse the internet, change one or more settings of the computing device and so forth.
In Microsoft's patent FIG. 3 noted above (in the combined figure 1 & 3), we're able to see a depiction of an example of a hologram #302 hovering over the keyboard area.
The hologram 302 is displayed as a control that is configured to support navigation through the user interface displayed by the display device #110 of the computing device. A variety of touch sensitive input mechanisms may be supported by the input device (e.g., the keys of the keyboard) with a secondary level of user interaction supported "floating above" a surface of the input device.
As illustrated in the patent figure above, the device's sensor detects a position of a tip of a user's finger and initiates an operation associated with a virtual "button" of the hologram #302 when the user's fingertip enters the hologram volume associated with the button. The hologram volume is noted below in patent FIG. 2.
Microsoft's patent FIG. 2 depicts a system 200 in an example implementation showing a side view of the computing device 102 and input device 104 of FIG. 1 as outputting a hologram and detecting gestures that are made consistent with the hologram.
In this example, the computing device includes one or more sensors #202 that are configured to support interaction with a near-field user interface. For example, the one or more sensors may be leveraged by the gesture module #116 to identify gestures and cause operations to be performed that correspond to the gestures, and so on. The inputs may be detected by the sensors for processing by the gesture module in a variety of different ways.
For example, the gesture module may be configured to receive one or more inputs via touch interaction with a hardware device, such as a keys of the input device as shown in FIG. 1. Touch interaction may involve pressing a button, moving a joystick, movement across a track pad, use of a touch screen of the display device 110 (e.g., detection of a finger of a user's hand or a stylus), and so on.
Recognition of the touch inputs may be leveraged by the gesture module to interact with a user interface output by the computing device, such as to interact with a game, an application, browse the internet, change one or more settings of the computing device and so forth.
A camera, for instance, may be configured to include multiple lenses so that different perspectives may be captured and thus determine depth.
The gesture module, for instance, may be configured to perform skeletal mapping along with feature extraction of particular points of a human body (e.g., 48 skeletal points) to track one or more users (e.g., four users simultaneously) to perform motion analysis.
A variety of different types of gestures may be recognized, such a gestures that are recognized from a single type of input (e.g., a motion gesture) as well as gestures involving multiple types of inputs, e.g., a motion gesture and an object gesture made using an object such as a stylus.
The Holographic Recording Mechanism
The holographic recording mechanism (#206) noted in patent FIG. 2 above, may also be configured in a variety of ways. For example, the holographic recording mechanism in the illustrated instance is configured as a film (e.g., a transparent medium including a silver halide photographic emulsion or photopolymer layer) that is disposed over an input portion of the input device, e.g., over one or more of the keys, track pad, and other parts of the input portion that do not include controls as illustrated in FIG. 1.
The holographic recording mechanism is configured to record the phase of an optical field of an original object in an image scene. For example, the holographic recording mechanism is configured as a recording of information regarding a light field as captured from an original scene scattered in a range of directions. Thus, the holographic recording mechanism may support a three-dimensional output that is viewable by a user. The hologram may be configured as a "floating graphic" of the object captured by the holographic recording mechanism.
For example, the light source #208 may be configured as an LED or laser diode to provide illumination that is used to reconstruct the hologram. A green laser, for instance, may be used to record a hologram as part of the holographic recording mechanism and a green LED may be used as the light source to reconstruct the hologram. The light source is situated above an interaction plane #212 that is used to display the hologram in this example.
Microsoft's patent FIG. 5 noted below depicts a pressure sensitive key #500 as including a holographic recording mechanism having multiple layers. In this example, the holographic recording mechanism includes a plurality of holographic recording layers #502 and #504. Each of these layers are configured to support output of different holograms in response to different wavelengths of light.
Microsoft's patent FIG. 6 noted above depicts an example of a system #600 in which multiple holograms are displayable based on alterations made to one or more light sources. In this example, holograms are displayed of an Xbox game controller #602 and radial dial #604 that are usable to indicate supported interactions. The holograms may be displayed "above" the input device (keyboard) as previously described.
A variety of different functionality may be supported by the holograms. For instance, depending upon the recording geometry, the hologram may have a limited field of view. If the hologram cannot be seen from the side, it can also be used as a security device, e.g., as an invisible floating keypad, for entering a PIN or gesture password where a person standing next to the user cannot see the "keys" being pressed, and so on. This was also covered in our weekend report titled "Microsoft Invents Computer Display "Privacy Mode" Viewing."
Additionally, a hologram may be used to create the effect of a user interface that "pops up". If the hologram has underneath it a display showing the icon that is on the hologram, and the reconstruction source is initially turned off, the icon can be animated to make it look like it is flipping upwards as the hologram is turned on.
A variety of different techniques may be leverage this functionality previously described. For example, situations may be encountered in which physical touching of a surface is to be avoided. For instance, if you're cooking and your hands are mucky or you're gardening and your gloves are dirty or working on your car with greasy hands, you won't have to touch the surface of the device, just the hologram interface on your Surface tablet/notebook to get your work down, control your device or type a note.
Microsoft filed their patent application back in Q4 2013. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Mobile presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Posting Comments: Patently Mobile reserves the right to post, dismiss or edit any comments.