Become a virtual Bob Ross or Monet with Apple's new technology.
6 Facebook x.com Reddit
If you've ever wanted to unleash your inner Bob Ross, you might soon be able to create your own lucky breaks in Apple Vision Pro.
We recently reported on rumors that the Apple Pencil was being tested for augmented and virtual reality, but it looks like the company is hoping to eventually ditch the stylus entirely.
Light emitters built into the stylus will allow the augmented reality system to track with high precision.
Apple's proposed technology is based on two patents that allow users to draw or paint in augmented reality. Users will be able to draw on a virtual canvas. However, this technology also allows for the creation of art in the real world, where the artwork can be superimposed on real-life elements.
Want to add a hat and birds to the real world? Put on your headset and start drawing.
The first patent application, “Stylus-based input system for a head-mounted device,” accurately describes what the rumors suggested: Apple is working on using a stylus for interaction. with the virtual world.
While the Apple Pencil seems like a natural choice as an input device, the patent describes a new device with a light emitter (likely infrared) that allows the headset to track the position and orientation of the device.
The second patent, “Drawing Methods in a Computer Reality Environment,” goes even further.
This illustration shows Apple's plan to track a user's hand. Bring your fingers together as if you were holding a brush, and one of them will appear on top of your hand. Switch tools and Apple Vision Pro changes the display of what you're holding in your hands.
You don't even need a brush to create art using Apple technology.
Although not mentioned in the app, adding gestures for different media would be easily. You can virtually spray paint using gestures by holding the can and pressing a button, or work the paint with your fingers by extending your fingers.
Apple wants to create a virtual simulation of an artist's workflow with unique color selection gestures. With this technology, you can select and mix colors by flipping one hand over and “dipping” a virtual brush into virtual paint. This experience would simulate artists holding a palette of paint in one hand and a brush in the other.
Drawing in 3D space is not unique. Anyone with a headset like Quest can use Tilt Brush or other art programs to create virtual works of art.
However, these programs use controllers and are awkward at best. Pressing a button isn't a natural way to draw or draw, and controllers get in the way of that.
Create works of art literally and virtually.
Apple's patent suggests that in addition to virtual painting, audio feedback could allow the user to hear brush strokes or pencil lines on paper . Immersive experiences like these could revolutionize art creation in AR/VR.
Audio feedback can also be used for virtual art activities. Put on a headset, listen to soothing music, and follow the drawing instructor's prompts. The result could be a drawing lesson in a headset.
We can imagine this combined with Apple's other AR/VR patents, which would allow multiple users to view and interact with the same virtual objects, even if they are in different locations. For example, you can take a class with a real instructor and paint with him on the same virtual canvas.
As with all patent applications, there is no guarantee that these technologies will appear any time soon or even ever. But with rumors of Apple testing the Apple Pencil in the Apple Vision Pro, it's safe to say that some of these technologies will be available very soon, perhaps as early as WWDC 2024.
Apple Patent for Using a Stylus for input is credited to Killian Moore, the lead applicant for numerous AR/VR patents filed by Apple. The AR/VR drawing patent was created by Edwin Iskander, senior manager of Apple VisionOS.
Follow AppleInsider on Google News.
