Project BlackPearl will significantly improve the Mail app with machine learning and AI
0 Facebook x.com Reddit
Apple's Mail app is getting a major AI upgrade with Project BlackPearl. Here's how iOS 18 will make your life better and your workday easier.
People familiar with the matter told AppleInsider exclusively that the Mail app will get major improvements thanks to Apple's new AI initiative. In previews of the company's upcoming operating systems expected at WWDC, search results in the Mail app include information from contacts, as well as locations and locally stored documents.
This means the Mail app will be able to take advantage of Apple's Ajax LLM to produce more diverse search results. It will also allow so-called “smart replies” to be sent to emails generated by Apple's artificial intelligence software.
Apple Smart Replies will make email communication easier and save knowledgeable users a significant amount of time. This feature will be especially useful for business professionals, customer service representatives, or large companies that regularly deal with large volumes of email.
As detailed in our report on Project Greymatter, Siri and Spotlight will also use artificial intelligence software. Siri will greatly improve its understanding of context and will be able to take into account locations, calendar events, and objects such as people and companies.
Mail, Messages and Siri will also get a new Smart Reply feature, which will give users the ability to reply with an instant AI-generated reply to an email or message. Siri will be able to use Apple's artificial intelligence to create responses based on user requests.
A customer service representative tasked with solving problems for various users will be able to respond to emails much faster with Apple Smart Replies. Administrative assistants can use this feature to make appointments or send business-related requests.
Previews of Apple's upcoming operating systems also include a new system-wide recording tools environment. People familiar with the new platform told AppleInsider that it could allow text editing through Siri.
While the exact implementation of this new structure remains to be seen, there are signs that it will make it easier to customize the tone of user-generated text in emails and other places. Users could instruct Siri to change the text by adding a phrase like, “Make this text more professional,” so their message or email would be taken seriously.
This means Siri can analyze emails and make specific suggestions or generate responses to better achieve the desired effect for the user. The virtual assistant can offer suggestions that will change the overall tone of the user's response or text, making it more concise, friendly, or professional.
The BlackPearl project will enable email categorization using machine learning
The Mail app is expected to receive significant improvements using machine learning as part of the BlackPearl project. These improvements will allow you to categorize emails based on the text they contain.
Image courtesy of Apple
By analyzing the text content of an email, machine learning software will be able to identify urgent emails and classify email senders. Also. People familiar with the matter told us that the software can classify emails into several predefined categories:
- Commerce
- News
- Other
- Stocks
- Social Media
- Transactions
>
Apple could include these categories in the app's user interface, although it's possible that this new feature was created to help Siri identify different types of email. The iOS 18 and macOS 15 version of Siri will also be able to summarize emails and email threads, among other things.
What else will Siri be able to summarize ?
A new and improved version of Siri is expected to debut at WWDC. Apple's virtual assistant will be able to summarize a wide variety of texts, and we now have some specific information about the types of text involved.
Apple's AI is capable of summarizing web pages, similar to Arc Browser
AppleInsider recently obtained detailed information about the instructions and hints used in the development of Apple's generative algorithm. AI software. The instructions created by Apple provide significant insight into the company's plans in the field of artificial intelligence.
Generative AI software in iOS 18 and macOS 15 will be able to summarize the following types of text, according to people familiar with the matter:
- Emails and email threads
- Messages
- Notifications and notification stacks
- Notes
- Publications
- Texts and news articles provided by users
Internally, Apple tested several levels summarizing:
- Topic-level summaries—consisting of three topic words that represent key concepts in the text
- 10- or 20-word summaries that use sections rather than complete sentences
- Paragraph-level summary is a short three-sentence paragraph of 60 words or less.
Apple has designed several types of summaries to address specific use cases and user interface elements. , because certain summaries require more context and specific instructions to achieve the desired result.
More about the various abstraction levels and text types
As explained in our exclusive report on Apple's Ajax LLM, a large language model is capable of identifying keywords and sentences in text. This means it can identify units of language in a note, message, notification or web page, and create short summaries of selected keywords.
Apple Ajax LLM is capable of performing topic-level summarization entirely on the device.
This simple keyword selection is the basis for topic-level software. summary. At this level of generalization, Apple's AI is asked to extract three topic words from the input text that represent core concepts.
While Ajax LLM can handle topic-level summaries, advanced summarization and more complex tasks appear to require cloud processing. Apple may have recently partnered with OpenAI to provide advanced AI features, as indicated in a recent report.
In addition to topic summaries, Apple has also experimented with 10- to 20-word summaries. With this type of summary, Apple’s generative AI software is instructed to use sentences rather than full sentences and to generate a result of no more than 10 or 20 words.
Paragraph-level summarization requires Apple’s AI software to create a short three-sentence paragraph of no more than 60 words. Specific instructions are given depending on the type of text the software is intended to summarize.
How does Apple instruct and guide its artificial intelligence?
While Apple's approach to AI and LLM does not include a chatbot, the instructions used to train the software are somewhat reminiscent of ChatGPT-style prompts. Apple has reportedly entered into a licensing agreement with OpenAI, meaning we could see OpenAI technology in new Apple operating systems.
Apple created ChatGPT-style suggestions for its AI
The company's instructions begin by explicitly telling the AI to take on the role of expert when summarizing a certain type of text . The artificial intelligence is told to maintain its role or its responses will not be considered useful or helpful, indicating that the overall goal is to help end users of Apple's operating systems.
When summarizing a notification stack, the AI should focus on the common theme of all notifications in the specified stack. Likewise, when analyzing and creating a summary of user-supplied text, such as an article, the software is encouraged to focus on the main topic of the text in its entirety.
For end-user-provided texts, AI is encouraged to refrain from using unnecessary prefaces and omit minor details. To create a summary of the latest messages or notification stacks, the software must pay special attention to key information—the sender and recipient names, as well as the time and date.
Essentially, all this means that Apple's virtual assistant, Siri, will become the main recipient of updates to the LLM-based information summarization system. Thanks to generative AI and integration with core system apps, Siri will be able to do more than ever before.
What else will happen Can Siri do this?
Siri is also expected to get significantly improved functionality within the app, according to people familiar with the matter. This means that Apple's virtual assistant will be able to perform possibly complex tasks across a variety of system apps.
Siri will be able to perform internal tasks in apps such as Photos, Messages, and Notes.
For example, in the Photos app “Users will be able to enhance individual photos, open certain types of images, play back memories, and set their device's wallpaper using Apple's virtual assistant.
Users can say things like “Improve this photo” or “Show me photos from last year” to make these actions easier. Siri will likely be able to display photos from the user's library rather than immediately searching for images online.
Similar Siri features are expected to be available in other system apps, such as Messages and the built-in Notes app. The virtual assistant will be able to move individual notes, open and analyze documents, and send images or files through the Messages app.
The default Photos app will also receive some minor UI changes as some interface elements will be reorganized or renamed. However, the design change itself is nothing more than a stylistic choice, as other AI-powered editing features are the main upgrade we can expect.
Create and edit images using Apple's artificial intelligence software
Apple's artificial intelligence software is also capable of generating images in internal test environments. We now have more details about the different types of images the program can create.
Apple's built-in Photos app will get a new Cleanup tool, as well as some changes to the user interface.
The company's operating system previews include an entirely new imaging framework, according to people familiar with the matter. It can be used to create different types of images using artificial intelligence.
The new image creation system uses the following image types and styles:
- Animation
- Emoji
- Illustration
- Line Art
- Sketch
Since Apple has developed its own internal art creation app, as well as a new platform for it, it is likely that the company will make this feature available to end users in one form or another. One report indicated that AI-generated emojis are in development, and our results support this claim.
AI image editing is also in the works as Apple looks to replace the existing retouching tool with a new feature called Cleanup. The Cleanup option will allow users to remove objects from photos using generative artificial intelligence.
However, it is worth emphasizing that the information comes from people familiar with pre-release versions of Apple operating systems and test environments. Apple has been known to cancel, delay, or rename new OS features before release, so there's always a chance that features mentioned here won't make it to the general public.
Apple is set to unveil its next-generation operating systems such as iOS 18, iPadOS 18 and macOS 15 at its annual Worldwide Developers Conference (WWDC) on June 10. At this event, we'll likely see an updated Calculator, along with updated versions of the built-in Calendar, Notes, and Safari apps.
Follow AppleInsider on Google News.
