Given the success of ChatGPT, The introduction of Apple’s Ferret marks a new phase in the AI race between tech giants with a growing interest in language model.
What do we know about Ferret?
In October 2023, Apple teamed up Cornell University’s researchers to develop an open-source Multimodal Large Language Model (LLM) and discreetly released it an on GitHub, an online software development platform. A research paper detailing the new model was also released on the university’s webpage.
Ferret employs a unique strategy of interacting with visual content by integrating computer vision and natural language processing. It can not only recognize object and areas within an image but link textual concepts to the visual elements and subsequently use the knowledge it gathered to spark insightful textual discussions. It claims to be versatile, accepting a variety of region inputs from points, bounding boxes, and free-form shapes.
To ensure robustness of the model, its researchers also introduce GRIT, an extensive “refer and-ground instruction tuning dataset”. It comprises 1.1 million diverse samples and intricate spatial knowledge along with 95,000 challenging negative examples, fortifying the model's resilience.
When compared with OpenAI’s GPT-4V, a multimodal model that allows a user to upload an image and have conversations on the model, researchers claim it surpasses certain benchmarks. While GPT-4V can be more knowledgeable in general question and answering Ferret shines with its detailed comprehension, promising more accurate analysis.
Why does it matter?
Ferret may be seamlessly integrated into our iPhones and MacBooks through iOS and macOS softwares to help Apple users with various tasks. Imagine enhanced Siri’s capabilities that can answer more specific queries as well as the ability to recognize and sort out photos or even create images and texts, all at our fingertips.
However, since Ferret is still in the development phase, it could still be a while before any official announcement on the project is made.
That said, Apple is taking a significant stride in the unveiling of its own AI model, signalling its dedication to pursue more advancements in the AI field, more specifically on large-language models. Apple's decision to license Ferret under a non-commercial open-source license speaks volumes, enabling wide collaboration while promoting transparency, and potentially expediting development.
With Google’s recent unveiling of Gemini, its very own conversation AI model, it seems that Apple is determined to stay ahead of the AI race with Ferret, with users reaping the benefits of having more efficient and comprehensive systems to help with our day-to-day tasks.
