Patently Apple posted a report on Wednesday titled "Samsung's President of Mobile Communications has confirmed that a Smart Speaker is in fact coming to Market." We first reported on this possibility back in early July. Samsung had also been granted design patents for a possible future smart speaker that we covered in a previous report. Then yesterday a utility patent from Samsung came to light that revealed a new radical smart speaker system in detail. The system includes a master unit designed to communicate with a series of secondary units spread throughout the home. The interactive smart speaker includes a video projector that's in sync with its digital assistant called 'Bixby.' The system is designed in a certain way to resemble the head and face of a robot which enables the device to physically move when spoken to as well as redirect video responses or the playing of video content unto a nearby wall with its built-in projector.
Samsung's Patent Background
In their patent background, Samsung begins by stating that electronic devices are becoming increasingly multi-functional. Often, functions that were once served by two or three separate devices are now combined into a single multi-functional device. For example, while a fax machine, a scanner, and a copier used to require three separate machines that each only performed one function, a single machine today can serve all three purposes. Such multi-functionality minimizes clutter and simplifies life.
One of the areas where it is difficult to consolidate a plurality of devices into a single device is the area of display devices. Generally, a display device has a specific fixed display size and can only display images on that fixed-size display.
For example, a digital clock has a small display, and a user has to stand close to the clock to read the visual content. It is difficult to consolidate a large screen function like a television with a small-screen device like a tablet into a single multi-functional device and achieve significant space/size reduction.
A single device that can fulfill both large-screen functions and small-screen function is desired.
Samsung's First Multimodal Projection System
Samsung's invention pertains to a device comprising a shell that has an internal projection screen and an opening, and a projector emitting a projection beam from inside the shell. There is an optical guide adjusting a position of the projector to aim the projection beam at one internal projection screen and one opening.
To get the overview of the invention I'll jump ahead to patent FIG. 18 noted below to establish what Samsung's system is in the big picture acting as an in-home smarthome hub device.
Samsung patent FIG. 18 depicts an exemplary diagram of connecting the central device to multiple secondary devices throughout the home. Samsung illustrates that the kitchen, bathroom, kids room, study and bedroom are considered secondary devices with the one in the living room being the master device.
The central device may be wirelessly connected to each secondary device via a Wi-Fi connection or a Bluetooth connection. The secondary device includes a video camera, a microphone array, and a speaker.
For example, a video camera of the secondary device captures and detects a broken window. The secondary device would then send the image of the broken window to the central device in the living room that could then transmit the image to the main user's mobile device(s) to notify them of a possible home break in.
According to another embodiment, the central device provides feedback in response to a voice input by a user to establish that it is engaged for human voice interaction. The feedback includes one or more visual feedback, audio feedback, and movement feedback.
For example, when a user provides a trigger voice command such as "Hello," the central device may tilt its head shell (illustrated further below) to one side to simulate listening, display wide open eyes on a UI on the head shell as a listening facial expression, and provide a voice feedback.
The movement feedback may include raising the head shell and turning the head shell in the direction of the sound source. According to one embodiment, the central device includes a 4-degree of freedom (DOF) mechanical structure design.
Samsung's patent FIG. 6 illustrated below depicts an exemplary device with FIGS. 8A and 8B depicting the rotational range of the device's shelled-head.
More specifically, Samsung notes that patent FIG. 6 depicts the exemplary interactive device #10 which is shown to include a head #11 and a body #12. The head includes a head shell #13 and the internal projector screen #80 that includes a user interface (UI).
The sensor #30, which is a camera in this particular embodiment, is positioned behind and on the inside of the internal projector screen. The microphone #150 is positioned to the side of the internal projection screen to resemble 'ears.' In this particular embodiment, the speaker #160 is positioned at the bottom of the body.
The Interactive Device
Samsung notes in their patent filing that their multi-modal display system is configured to project visual content on a large surface or a small screen display. One set of projection system is used to display visual content (e.g., images, video) on different manners to satisfy user's need under various situations.
In one embodiment, when a user is close to the device, content is displayed on a small internal screen of the device. When the user is far from the device, content is displayed on a larger external surface via projection, e.g. on a wall, tabletop, or floor. The switch between the small-display mode and the large-display mode may be manual or automatic.
In Samsung's patent FIG. 1 noted above we're able to see a basic overview of the interactive device (#10) that includes a processor #20, a projector #30, an optical guide assembly #40, a sensor/camera #50, and a distance measurement unit #60 connected to one another. The sensor may be any sensor capable of proximity sensing, including but not limited to a camera, an infrared sensor, or a laser sensor.
The optical guide assembly receives instructions from the processor and is able to move the direction of the projection beam generated by the projector. The distance measurement unit measures a distance between the device and an external projection surface.
Samsung's patent FIG. 5 below illustrates a flowchart depicting how the central device decides between the two display-modes.
More specifically, Samsung states that the process of patent FIG. 5 may be repeated periodically (e.g., at every preset time interval) such that the display mode can be dynamically adjusted in response to the changes in the environment, e.g. as the user moves, room darkens, etc.
In addition, depending on the embodiment, the processor may dynamically switch the display mode in the middle of showing content. For example, even if a user began watching a movie on the small screen because it was too bright when the movie started, the display may automatically switch to wall projection after the room becomes less bright. This is done by the sensor that continually monitors the ambient light level. Similarly, the sensor may continually monitor the distance between the user and the projector and dynamically switch display modes to suit the distance.
The display mode may be changed manually by a user. In one embodiment, a user can press a button or spin a knob to change the display mode. While the processor uses sensor data and selects a particular display mode, the user may be able to override the processor decision.
When displaying the visual content in large-screen mode, the device may use the distance measurement unit (e.g., a laser tape measure) to determine how far the intended projection surface is from the device and automatically adjust the focal length such that the displayed content will be in focus regardless of where it is projected. The measurement unit may be attached to the projector and move with the projector.
Where there is an optimal distance range or distance limit between the projector and the projection surface, the device may be able to move itself (e.g., using wheels or rollers attached to the base) to the optimal position to display the image.
Furthermore, by using the sensor, the device can find a flat, blank surface for the large-screen display mode. This will avoid visual content from being displayed on an artwork or any other surface that would not be suitable.
In Samsung's patent FIG. 10 noted below we're able to see an exemplary block diagram of the system architecture; FIG. 11 depicts an exemplary block diagram of NLU engine architecture; and FIG. 12 depicts an exemplary block diagram of hardware architecture of the device.
It should be noted that back in January Patently Mobile posted a report titled "A Microsoft Patent Reveals a Highly Advanced Home Automation Device that's Miles ahead of Amazon's Echo" which also illustrated a future smart home speaker system using a projection system as noted in the Microsoft patent figure below.
And lastly, back in May Patently Mobile posted a report titled "Samsung Invents a Smartphone-Like Device with a 360 Degree Camera and Projector that Creates a Virtual Keyboard," illustrating that Samsung's engineering teams definitely have projection systems on their minds of late.
Samsung filed their latest patent application for a smart speaker back in Q4 2016. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Mobile presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Posting Comments: Patently Mobile reserves the right to post, dismiss or edit any comments. Those using abusive language or behavior will result in being blacklisted on Disqus.
Comments