A new Microsoft patent application that was published by the U.S. Patent & Trademark Office published last week presents a vision of delivering a full wall interactive 3D display. Beyond the one illustration presented in our cover graphic showing the display being used as part of a future Xbox gaming display system, Microsoft doesn't reveal what other industries that could possibly take advantage of such large displays that could be manipulated by a user's touch, in-air hand gestures or a stylus. Microsoft's patent filing is mainly an overview of the technical aspects of this next generation display with very little overview of future markets or applications.
Overview of Microsoft's Tangible 3D Light Display
Diving right into the invention, Microsoft notes that patent FIG. 1 noted below shows an example environment including a tangible three-dimensional display #102. As shown in FIG. 1, three-dimensional display is outputting a three-dimensional image and is receiving input from a user. Three-dimensional display may include a plurality of light emitters moveable in three dimensions. The plurality of light emitters may be tangible and create a display with high touch affordance, in that a user may experience the display through the sense of touch and may interact with the display with via touch.
For example, the user may touch and/or move the light emitters in order to enter input to the display, expose one or more underlying layers of the display, or otherwise interact with the display. In one example, a user may perform a "dive-in" gesture in which the user reaches into the display, past some number of emitters or levels of emitters, to interact with a next set of emitters.
Microsoft's patent FIG. 2 illustrated below, schematically shows a system including an example three-dimensional display; FIG. 3 illustrates a cross-sectional partial view of an example three-dimensional display comprising a plurality of optical fibers [up to millions of fibers]; FIG. 8 is a flow chart illustrating a method for outputting an image via a three-dimensional display.
More specifically, Microsoft notes that the display device invention includes individual discontinuous light-emitting elements (referred to as light emitters) that may be touched, moved, or otherwise manipulated by a user. Each light emitter may be movable in three dimensions throughout a given range of motion.
In one example, the plurality of moveable light emitters may comprise a plurality of optical fibers. In other examples, the plurality of moveable light emitters may comprise light emitting diodes (LEDs) or other suitable pinpoint light element coupled to a tube, wire, or other moveable structure.
The 3D Display's Capture, Location, Tracking, Mapping & Image Engines
In order to maintain a desired image, even as the light emitters are moving, a capture engine (noted below in FIG. 9) may be used to capture location information usable to determine the physical location in real, three-dimensional space of each light emitter. In one example, the capture engine may include a camera to capture light output by each light emitter.
A location engine may use the captured location information to map the current location of each light emitter. A mapping engine may compute a mapping between the discrete light-outputting locations of the image to be displayed (e.g., pixels) and the locations of the emitters, in a way that may enable the emitters to collectively reproduce the original image (albeit in a potentially sparse fashion).
Then, an image engine may use the mapping to drive the output light for each emitter. Further, user interaction with the light emitters may be tracked and the light emitters may be driven to output light based on the user interaction, and/or based on incidental physical interactions between emitters, other external forces such as wind that may move the emitters, and/or actuators that may be causing the emitters to move in a random or predetermined pattern individually, collectively, or in controlled or random subsets.
A tracking engine may be present to track a position, shape, and/or orientation of one or more user input implements, such as a hand of a user, a stylus, and/or other objects. By knowing where a hand is located, and identifying a touch event, the content of emitters that do not move and/or that do move may be affected. For example, emitters may become brighter as a hand is approaching them, even if their location is not changed.
The tracking engine may include and/or communicate with a tracking camera in some examples. In other examples, the tracking engine may include depth sensors, touch sensors, or other suitable mechanisms for capturing the tracking information. Further, in some examples the location camera and tracking camera may be the same camera.
Additionally, the three-dimensional display is also configured to recognize touch input from a user's finger(s), an object such as a pen, or any other suitable touch implement. In one example, in order to detect a position of the touch implement in three dimensions, the user tracking camera may capture image information usable by the computing system to track a position of the touch implement relative to the light emitters. For example, the tracking camera may be a depth camera configured to acquire depth information and send the depth information to the computing system.
Natural User Interface
And lastly, Microsoft briefly touches on their input subsystem that may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller.
Microsoft notes that "In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for light emitter location mapping, touch implement tracking, machine vision, and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity" – a topic that we covered in a 2012 Sony Gaming patent report which pointed to the emerging Emotiv system.
Microsoft's overview patent application, such as the one revealed last week, is usually over time accompanied by supporting inventions that will eventually describe other parts of a much larger project. In a future filing we're bound to read about Microsoft's future markets that could take advantage of this technology or how this light-display may work with their HoloLens project or a VR/AR headset like the new Project Alloy Heads- Up-Display (HUD) that was introduced at Intel's IDF event this month.
For now, Microsoft has provided us with a general overview of a next generation Tangible 3D Light Display that could be as big as full wall and is highly interactive and immersive. In 2012 Patently Apple covered a Microsoft patent covering an immersive gaming system that turned the walls of a home into the landscape found in a video game. Microsoft's latest invention truly extends that thinking to using a laser lighting system in conjunction with interactive capabilities via touch, air-gestures and beyond. In the end, this is a patent application and therefore the timing of such a product to market is unknown at this time.
The U.S. Patent Office published this invention last week which was filed in February 2015.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Mobile presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Posting Comments: Patently Mobile reserves the right to post, dismiss or edit any comments.