Earlier this month the U.S. Patent & Trademark Office published a patent application from Microsoft that revealed their work on a future Surface tablet that will work with an all-new digital pen-brush that will provide users/artists with the ability to deform the pen-brush head to deliver varying stroke thickness and special effects. The pen-brush will work with a future Surface tablet that will support a new infrared sensing system. Microsoft's deformable pen-brush is likely to work with a future version of their Fresh Paint app.
Microsoft's Patent Background
Modern computer graphics have allowed for some truly artistic creations on digital platforms and have provided a large gallery of paint functions and features. However, the painting experience includes more than just visual results, but also requires physical manipulation of brushes in the actual painting process. The viscosity, friction and the bending force associated with the physical manipulation may provide rich information to users, varying with different input possibilities/forms.
Microsoft further notes that their "Digital painting software may provide a user with the ability to simulate various visual effects based on different selectable stroke types (e.g., charcoal pencil, oil painting, watercolor, etc.). A stylus may assist the user in drawing with high accuracy, and pressure sensing may be available. However, digital painting software may not support changes in brush stroke, as a stylus or fingertip cannot change its shape, compared to different types and strokes of real painting brushes."
Microsoft's Deformable Pen-Brush for Digital Artists
Microsoft's invention introduces a selection of concepts relating to a digital painting application that supports a deformable pen-brush for artists. The deformable tip is adjustable to define a shape tip configuration (e.g., an oil brush tip configuration). In some implementations, the deformable tip may be more rigid in a jammed tip configuration in which solid particles in a membrane are jammed together using a vacuum pump, for example.
Some implementations provide techniques and arrangements for illuminating at least a portion of the deformable tip in a color corresponding to a color that is selected to be used in an application (e.g., a digital painting application, a game, a notes application, etc.).
The new deformable shape-changing tip will simulate a real paint brushes so as to provide users with a superior digital drawing experience. In some implementations, the new pen-brush will introduce a new controllable "jamming" structure that will provide different levels of stiffness to the tip.
When the tip is soft and deformable, a user may be able to easily shape the tip and define a unique tip pattern. In some cases, a vacuum pump may be used to fix the tip shape customized by the user when the tip is deformable. For example, solid particles within a membrane may be "jammed" together such that the particles behave in a solid-like manner, similar to coffee particles under vacuum in a coffee bag. The degree of hardening or jamming may be variable, depending on the desired hardness of the tip for a given application.
Microsoft's patent FIG. 1 illustrates a computing device 100 (such as a future Surface Tablet) that may be configured to present an exemplary user interface #102 associated with an electronic painting application.
A First Tip Configuration: Oil Brush
An input device (e.g., a stylus) may include a deformable tip #104 that may allow a user to customize a tip pattern for a particular painting application (e.g., for oil brush painting, Chinese brush painting, etc.). For example, FIG. 1 illustrates a first tip configuration #106, a second tip configuration #108, and a third tip configuration #110. However, numerous other tip configurations are possible that may allow the user to create various brush stroke sizes, shapes, and/or patterns.
In some cases, the deformable tip may be attached or otherwise connected to a stylus #112 and may include a jamming structure (see e.g., FIG. 2 further below) to adjust a tip stiffness.
When the deformable tip remains deformable (see e.g., the unjammed tip configuration 202 in FIG. 2), the user may shape the deformable tip in order to customize a tip pattern for the desired painting application.
As an illustrative non-limiting example, after the user has shaped the deformable tip, a vacuum pump may be used to fix or hold the customized deformed shape of the deformable tip in a state to maintain the desired tip pattern (see e.g., the jammed tip configuration 204 in FIG. 2). The state may be rigid, semi-rigid, flexible or a more tightly packed, but still deformable shape.
Referring to FIG. 1 again, the first tip configuration #106 may be associated with an oil brush tip configuration to allow the user to shape a stroke pattern to mimic a brush stroke pattern that may be possible when painting with an oil brush on paper or canvas.
An oil brush allows a painter to create wide or narrow strokes depending on a contact area of the brush and a movement angle of the brush. For example, an oil brush may have a "wide" side that may be used to create wide strokes and a "narrow" side to create narrow strokes.
Microsoft's patent FIG. 1 also illustrates a first oil brush tip shape #114(1) that corresponds to a view of a wide side of the deformable tip (see e.g., the first oil brush tip configuration top view 306 in FIG. 3 further below).
A second oil brush tip shape #114(2) corresponds to a narrow side of the deformable tip 104 (see e.g., the first oil brush tip configuration side view #304 in FIG. 3). When the user orients the stylus such that the first oil brush tip shape #114(1) defines a contact area, a wide stroke pattern #120 may be created. When the user orients the stylus such that the second oil brush tip shape #114(2) defines a contact area, a narrow stroke pattern #122 may be created.
Future Surface Tablet with Infrared Sensing System
In some implementations, a next generation Surface Tablet may use an infrared sensing system to provide access to raw image data of the tip contact area on a screen. For example, infrared cameras may be used to sense objects, hand gestures, touch input, or a combination thereof.
For each detected touch point, this next generation Surface Tablet may automatically draw a line through these points. The line may be constructed by many closely placed circles (or other shapes) filled with a particular color. Therefore, the width of the line may correspond to the diameter of the circles. The diameter may vary with the size of the touch area. In some cases, a touch area may be defined as an ellipse, and an average of a length of a major axis and a length of the minor axis may be used to define the diameter of a circle.
A Second Tip Configuration: Chinese Brush
The second tip configuration #108 illustrated in FIG. 1 may be associated with a "Chinese brush" tip configuration to allow the user to mimic a stroke pattern that may be possible when painting with a Chinese brush on paper or canvas.
A Chinese brush may include a brush tip that may be flexible like a hair brush. A Chinese brush allows a painter to create various brush stroke patterns depending on an amount of pressure applied by the user and an angle of movement of the brush tip. For example, a Chinese brush may have an elliptical shape, with a brush diameter increasing from an initial diameter at a base of the brush before decreasing to form a substantially pointed brush tip.
Microsoft's patent FIG. 1 also illustrates that a variable stroke pattern #124 that depends on a deformity of the brush tip that results from the user applying a particular amount of pressure and moving the tip 104 a particular angle.
In the second tip configuration, the deformable tip may remain in a deformable state (see e.g., FIG. 6 further below). As the deformable tip remains deformable, the user may be provided with tactile feedback that simulates Chinese brush painting.
In the example of FIG. 1, the user has applied pressure to the deformable tip and moved the pen-brush in a semi-circular motion downward and to the left with respect to an initial contact point. Thus, the Chinese brush tip shape #116 is shown in FIG. 1 with the narrow tip of the Chinese brush oriented toward the initial contact point and the brush deformed at an angle associated with an amount of pressure applied by the user when moving the pen-brush in the semi-circular motion.
In the second tip configuration, various other tip shape deformities may be possible depending on the orientation of the pen-brush, a stroke angle, an amount of pressure applied by the user, and based on a stiffness (i.e., how tightly packed) of the deformable tip.
Further, while not illustrated in FIG. 1, in some implementations, in the Chinese brush configuration, "airbrushes" may be used to display a spray of ink by painting pixels in a circular region around the contact area of the tip (e.g., as shown at 1404 in FIG. 14 – bottom patent figure).
As the Chinese brush tip shape includes a substantially pointed tip, other shapes may be created. For example, when the pen-brush is substantially perpendicular to a plane defined by the user interface, a substantially circular shape may be created, with a size of the circle determined based on an amount of pressure applied by the user. To illustrate, when the user applies a first amount of pressure, a small circular shape #126 may be created. When the user applies more pressure, a larger circular shape #128 may be created.
A Third Tip Configuration: Crayon Tip
Microsoft then describes the third tip configuration #110 illustrated in FIG. 1 that corresponds to a crayon tip shape #118, in which the deformable tip may be jammed but not deformed. In the third tip configuration, the stroke pattern may include a fixed shape pattern (e.g., a fixed circle diameter).
In the illustrative user interface of FIG. 1, a color palette #134 may include a plurality of selectable colors. In FIG. 1, the color palette includes a selectable red (R) color #136, a selectable green (G) color #138, and a selectable blue (B) color #140 (shown as different patterns in FIG. 1 for illustrative purposes only). Of course alternative or additional colors may be included in the color palette.
Further, the selectable colors may be mixed. To illustrate, a user may select the red color, the green color, the blue color or a combination thereof, and the corresponding color may represent a combined RGB value. Further, it will be appreciated that other alternative methods of selecting a color may be provided.
In the illustrative example of FIG. 1, the user interface also includes a selectable paint brush icon #142 and a selectable pencil/crayon icon #144. In some cases, selection of the paint brush icon may result in the color palette being displayed. Further, while not shown in FIG. 1, selection of the pencil/crayon icon may result in a plurality of selectable colored crayons being displayed instead of the color palette.
The Pen-Brush's Jamming Mechanism
Next, Microsoft reviews "jamming" in patent FIG. 2 noted below which represents a "jamming" mechanism that employs a vacuum to fix the deformable tip in a particular configuration. However, alternative methods of controlling the stiffness and shape of the deformable tip may also be used. For instance, an electromagnet may be used to fix particles or beads of ferromagnetic material, a robotic tip could be employed that is deformable, etc.
Microsoft notes that the term "jamming" refers to a mechanism by which a material may transition between a liquid-like state and a solid-like state with little or no change in volume.
Jamming is commonly experienced in products such as vacuum packed coffee which has a stiff "brick-like" state. When the package is punctured, air enters the package, the confining vacuum is released, and the coffee particles behave in a liquid-like manner.
Jamming describes a situation when granular media exhibits a yield stress, such that forces can be distributed through chains of grains as if each chain were a rigid object. The effective "solid" phase is achieved when the vacuum results in the density of particles exceeding a threshold. Further, the tip configurations are reversible repeatedly, and a jamming condition is continuously variable to any desired level of hardness by adjusting vacuum pressure. As jamming is limited to physical changes without any chemical reagent, it may be considered safe and environmentally friendly.
In FIG. 2, particular components of the pen-brush and associated deformable tip of FIG. 1 are illustrated, according to some implementations. The components illustrated in FIG. 2 are for illustrative purposes only and numerous other implementations may include alternative, additional, or fewer components to effect a transition from an unjammed tip configuration #202 to a jammed tip configuration #204.
In the example of FIG. 2, the deformable shape tip includes a substantially non-porous membrane #206 (e.g., elastic silicone rubber among other alternatives) that is filled with a plurality of solid particles #208 (e.g., crystal, glass, ceramic or plastic balls, among other alternatives).
When the solid particles are in a loosely packed state, the deformable shape tip may be soft and deformable. A vacuum pump #210 may remove air (or other gas) from within the membrane to jam the particles into a solid-like, rigid state.
The vacuum pump may remove air via tubing #212 (e.g., flexible tubing), with a first valve #214 actuatable to allow air (e.g., at atmospheric pressure) into the tubing and a second valve #216 that is actuatable to be closed after the vacuum pump has removed the air from inside the membrane. That is, the vacuum pump may power on to remove the air, remove the air, and then power off.
In the unjammed tip configuration, the first valve remains open to air (e.g., at atmospheric pressure) in order to allow the plurality of particles to remain in a liquid-like state such that the deformable tip remains flexible. In the example of FIG. 2, a switch #218 or other user actuatable control may be used to transition the deformable shape tip between the unjammed tip configuration and the jammed tip configuration, and vice versa.
Microsoft's patent FIG. 3 noted above illustrates an example of a tip shape that is deformed and jammed after deformation; FIG. 6 below illustrates an example of a tip shape that remains deformable.
Microsoft's patent FIG. 14 below illustrates an example in which a deformable shape tip may allow the user to stylize an appearance of a path.
In the example of FIG. 14, the various stylized brush patterns include a first brush pattern #1402, a second brush pattern #1404, a third brush pattern #1406, a fourth brush pattern #1408, and a fifth brush pattern #1410. However, various other stylized brush patterns may be possible. Various user-defined characteristics of a brush pattern, such as size, shape and color may be saved by this future Surface Tablet. The saved brush pattern may be used to evenly fill the brush path with a stylized brush pattern along a length of the drawing path.
Microsoft filed their patent application back March 2016. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patently Mobile presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Posting Comments: Patently Mobile reserves the right to post, dismiss or edit any comments.