Image created using Image Creator from Microsoft Designer
1 Facebook x.com Reddit
Apple in collaboration with researchers from the University California has released its own AI-powered image editing model that anyone can try.
The model is called MGIE, which stands for MLLM-Guided Image Editing and allows users to edit images based on natural language instructions. It uses multimodal large language models (MLLMs) that integrate different types of information, such as text, photographs, and video, to understand and generate human language.
Although MLLM research has shown promise in understanding and creating images, it has not yet been widely adopted.
MGIE was presented in a paper at the International Conference on Representation Learning 2024. The paper shows how MGIE can improve automatic metrics and human evaluation while maintaining the efficiency of competitive inference.
MGIE uses MLLM to obtain expressive instructions based on user input. Venture Beat offers an example: a user might tell MGIE to “make the sky bluer,” and MGIE interprets this as “increase the saturation of the sky area by 20%.”
How MGIE interprets natural language | Image credit: Apple, University of California, Santa Barbara
Then, once the instructions are received from the natural language input, it generates an edited version of the image. It can be used to create a wide range of edits, from adjusting colors to creating or removing parts of an image.
MGIE is open source and available on GitHub for everyone. The GitHub page allows users to get code, data, and pre-trained models.
Image creation speed will vary significantly depending on hardware performance. In addition, such models are incredibly demanding on RAM.
Apple has been working behind the scenes on its features using artificial intelligence for some time now. In January, beta code for iOS 17.4 suggested that Apple was working on a new version of Siri powered by artificial intelligence.
Apple shares have fallen sharply recently, in part because analysts have vocally said the company is lagging behind Meta, Google and Microsoft in adopting generative artificial intelligence. It's not clear why this wasn't a problem when it first arose not with a mobile phone, tablet, smartwatch or virtual reality headset, but with generative artificial intelligence.
Perhaps to dissuade these analysts, Apple CEO Tim Cook recently said that Apple is spending “an enormous amount of time and effort” on artificial intelligence features that will be announced in the coming months. He also noted that Apple is working on generative artificial intelligence internally.
