Apple Releases Open Source AI Models That Run On-Device

Apple today released several open source Extended Language Models (LLMs) designed to run on-device rather than through cloud servers. Called OpenELM (Open-source Efficient Language Models), LLMs are available on the Hugging Face Hub, an AI code-sharing community.

As outlined in a white paper (PDF), there are eight OpenELM models in total, four of which have been pre-trained using the CoreNet library, and four models optimized for instructions. Apple uses a layered scaling strategy aimed at improving accuracy and efficiency.

Apple has provided code, training logs and multiple versions rather than the final trained model, and the researchers behind the project hope this will lead to faster progress and “more reliable results” in the field of AI in natural language.

OpenELM, a state-of-the-art open language model. OpenELM uses a layer-wise scaling strategy to efficiently allocate parameters within each layer of the transformer model, leading to improved accuracy. For example, with a parameter budget of approximately one billion parameters, OpenELM shows a 2.36% improvement in accuracy over OLMo while requiring 2x fewer pre-training tokens.

Departing from previous practices that only provide model weights and inference code, and pre-train on private datasets, our version includes the full framework for training and evaluating the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training. training configurations.

Apple says it releases OpenELM models to “empower and enrich the open research community” with cutting-edge language models. Sharing open source models gives researchers a way to study risks and biases in data and models. Developers and businesses can use the templates as is or make changes.

Open sharing of information has become an important tool for Apple to recruit top engineers, scientists, and experts because it provides the opportunity to publish research papers that would not normally have been able to be published in the part of Apple’s secret policies.

Apple hasn’t yet brought these kinds of AI features to its devices, but iOS 18 is expected to include a number of new AI features, and rumors suggest Apple plans to run its large language models on the device for privacy purposes.