I know this is just a developer beta, and if anyone from Apple asks, my first impressions of Visual Intelligence will be purely in the context of its future potential. But I have to say that having tried it, I am excited about that future.
The actual Apple Intelligence part is currently relatively limited. In most cases, you have a choice between asking ChatGPT to describe what you see to you, or searching Google …
Apple Intelligence's Proprietary Parts
As far as we can tell, Apple Intelligence does a few things directly:
- Summarizing text seen in a scene
- Getting information about businesses from Apple Maps
- Recognizing the date and time in text and opening a calendar for that slot
But for object recognition, this feature currently appears to rely exclusively on ChatGPT and Google. In my limited experience, Google has impressed me the most.
Image Recognition Using Google and ChatGPT
I tend to do a fair amount of research on most non-trivial purchases before I click the button. I read reviews, ask friends for advice, and generally interrogate Google to the nth degree.
So if I see something used somewhere and want to check it out, I want details: telling me I'm looking at a compact bean-to-cup coffee maker or something similar wouldn't be quite right.
So how does Visual Intelligence v0.1 perform? I tested it on a few things around my house.
First, I had the Sage Combi Wave 3-in-1, a combination microwave, oven, and air fryer. It's sold in the US under the Breville brand. Visual Intelligence identified it immediately (first match, top right). It also showed a competing product, but all the other popular results were correct.
Next up, my Sage Barista Touch coffee maker. VI was a little less sure about this one, suggesting it could be either that or the much more expensive Oracle Touch model from the same brand, but it's not hard to figure out which of the two is correct. (Although he does seem to add a third model to the mix; this is a listing from a website with a mislabeled photo.)
Aarke Carbonater II sparkling water? No problem.
He had a little more trouble with the Reidel O Wine Tumbler, but that's forgivable since there are plenty of copycats out there. He mostly only showed originals and straight copies.
Moving into the living room, he identified it as an Eames Lounge Chair, and since it's a millimeter-perfect copy, I'll forgive him for not being able to tell the difference between it and the real thing.
I have tried to apply this method to some original works by little-known (but not unknown) artists, but without success. However, when it came to the fingerprint, he had no problem identifying it.
With my Naim Mu-so, he assumed it was a second-generation unit rather than a first, but given that they are visually identical and almost every reference online is to that model, that makes sense.
I tried it on a bunch of other gear in my house:
- Original HomePod
- Echo Dot
- Philips Hue Switch
- Netatmo Smart Doorbell
He was right about all of that.
This is a great start, and there's much more to come
I was first introduced to AI-powered image recognition technology when I tried the Ray-Ban Meta glasses, although it took some careful wording and/or follow-up questions to get the most out of them.
Visual intelligence already feels better than this, and it’s the pace of development that excites me the most. With several companies competing in this space, I think we’ll see a lot of progress in a very short time.
Of course, there will be the occasional bug, as with all AI technology, but the very nature of this technology is that it learns from every interaction.
I don’t think it will be long before we can do things like point an iPhone at a random product we see in a store or on the street and instantly get an average rating and decent review summary using AI. Usually Apple tells us that the company can't wait to see what we do with its new technology, but in this case it's the other way around.
If you have access (in my case it took a couple of weeks), what are your first impressions of Visual Intelligence? Please share your thoughts in the comments, and watch Fernando's video below.
Photo: 9to5Mac
