Core ML
Also: CoreML, Apple Core ML
Published
Definition
Core ML is Apple’s framework for running machine learning models inside an app, on the device. It uses the CPU, the GPU and the Neural Engine, and needs no network connection to make a prediction.
Core ML has been part of Apple’s systems since 2017, in macOS 10.13 and iOS 11. An app hands it a model and some input, such as audio, an image or text, and it returns the model’s answer. Where the arithmetic happens is Core ML’s decision.
You do not install or open Core ML. It is part of macOS. You meet the name in an app’s description, in a model’s file name, or when a download says it contains Core ML weights.
What it does
Apple’s description: Core ML provides a unified representation for all models, and an app uses it to make predictions, and to train or fine-tune models, all on a person’s device.
On performance, Apple says Core ML optimizes on-device performance by leveraging the CPU, GPU and Neural Engine while minimizing its memory footprint and power consumption. The Neural Engine is the part of Apple’s chips built for this kind of arithmetic, and Core ML is the usual way for an app to reach it.
Why on-device matters
Apple makes the argument itself: running a model strictly on a person’s device removes any need for a network connection, which helps keep a person’s data private and the app responsive.
For speech, that means the audio does not have to leave the Mac to become text. Whether a particular app keeps it there is up to the app, but the framework makes it possible.
The file types
Core ML models come in three forms, and the extension tells you which.
- `.mlmodel`: the original model definition file.
- `.mlpackage`: the newer package format, which keeps the weights separate from the program. It is supported from Xcode 13.
- `.mlmodelc`: a compiled model, which is what an app loads. A folder with this ending inside an app or its downloads is a Core ML model ready to run.
Where the models come from
Most models are trained with other tools and then converted. Apple publishes Core ML Tools for that, to convert models from training libraries such as PyTorch and TensorFlow into the Core ML format.
The same tools can compress a model on the way in, by quantization or by palettization, which makes it smaller and often faster on the Neural Engine.
Core ML and MLX
Apple describes the two differently. Core ML is for integrating models into apps. MLX is an open-source array framework for experimenting with, training and fine-tuning generative models on Apple silicon. Apple’s documentation says MLX runs on the CPU and the GPU, while Core ML can also use the Neural Engine.
Apple’s pages now also point developers who work with the newest model architectures to a separate framework, Core AI. Core ML remains the long-standing way to run a model in an app.
Questions people ask.
What is Core ML used for?
Running machine learning models inside apps on Apple devices: recognizing speech, classifying images, reading text and similar tasks, without sending the data to a server.
Does Core ML need the internet?
No. Apple says that running a model strictly on the device removes any need for a network connection.
Does Core ML use the Neural Engine?
Yes. Apple says Core ML leverages the CPU, GPU and Neural Engine.
What is an .mlmodelc file?
A compiled Core ML model, the form an app loads and runs.
What is the difference between Core ML and MLX?
Apple describes Core ML as the way to integrate models into apps, and MLX as an array framework for experimenting with and fine-tuning generative models on Apple silicon. Core ML can use the Neural Engine; MLX runs on the CPU and GPU.
Is Core ML on Intel Macs?
Yes. Core ML is part of macOS from version 10.13. The Neural Engine is only in Apple silicon, so on an Intel Mac it uses the CPU and GPU.
