Foundation Models: running Apple's on-device LLM in your iOS app

Hi everyone, have you ever wanted an AI feature in your iOS app without sending user data to a server? This article covers Apple’s Foundation Models framework, which runs a language model on the iPhone itself.

By the end of this article, you will know what Apple shipped, how to call the on-device model in five lines of Swift, how to get typed Swift values back with guided generation, and when to fall back to a cloud model instead.

The model is already on your user’s iPhone, so you need no API key, no server, and no internet connection. For two years, adding an AI feature meant sending the user’s data to someone’s GPU and hoping. With Foundation Models, a capable language model ships with the OS. Calling it costs you nothing and leaks nothing.

I research retrieval and language models for a living, so I went in doubtful of the marketing. It is smaller than a frontier cloud model. For a large class of everyday app tasks, that does not matter. Here is what it is, how to call it, and where its limits are.

A private model, on the device, for free

The framework exposes the same on-device model that powers Apple Intelligence. It has about three billion parameters and is tuned to run on the Neural Engine. For app developers, three things stand out: it runs offline, it sends nothing off the device, and there is no per-token bill. You no longer have to choose between “add an AI feature” and “take on a cloud dependency plus a privacy review.”

LanguageModelSession, a prompt, a response

The entry point is small on purpose. You create a session and ask it something:

import FoundationModels

let session = LanguageModelSession()
let response = try await session.respond(
    to: "Summarise this note in one sentence: \(noteText)"
)
print(response.content)

That is the whole “hello world.” Before you rely on it, check availability. The model is not present on every device or in every region. When it is missing, fall back without breaking the feature:

switch SystemLanguageModel.default.availability {
case .available:
    // use the model
case .unavailable(let reason):
    // fall back to a non-AI path; never block the feature
}

Make the model return typed Swift, not a wall of text

Free text is a pain to work with. You end up writing fragile JSON parsers and hoping the model closes its braces. The key idea in Foundation Models is guided generation. You mark a Swift type with @Generable. The framework then forces the model to produce exactly that structure, decoded into your type.

@Generable
struct Expense {
    @Guide(description: "Merchant or store name")
    var merchant: String

    @Guide(description: "Total amount in Rupiah, digits only")
    var amount: Int

    @Guide(description: "Spending category")
    var category: Category
}

let expense = try await session.respond(
    to: "Extract the expense from: \(receiptText)",
    generating: Expense.self
).content   // -> a real Expense, no JSON parsing

Small, private, offline: the sweet spot

The model’s limits fit neatly with things real apps need. This is most true where sending user data to a server is a privacy or cost problem:

• Offline note summaries: shorten a long note on the device, no network round-trip.
• Transaction categorisation in a finance app: label “GoFood”, “PLN”, “Tokopedia” into categories without sending a user’s spending history anywhere.
• POS receipt descriptions: turn a list of items into a tidy line, generated locally during a busy checkout.

Best for: short, well-scoped tasks that run often and care about latency and privacy. Watch out: it is an English-biased model. For Bahasa Indonesia, test real prompts before you ship, and keep a non-AI fallback.

It is a small model, and that’s the deal

A three-billion-parameter model is not a frontier model. Pretending otherwise will burn you. It is weaker at multi-step reasoning. It has a limited context window. As noted above, it leans English. Treat it as a fast, free, private function for simple language tasks. Knowing which tasks are simple is the whole skill.

On-device by default, cloud when it’s earned

You do not have to pick one for the whole app. Route per task. Handle the fast, private, simple cases on-device. Fall back to a cloud model only for the hard ones. Most requests never leave the phone. That keeps average latency, cost, and privacy exposure low, and the cloud bill small.

On-device or cloud for this task?

Recap

The goal was to show you what Apple’s on-device model is, how to call it, and when to use it. The main points:

• Foundation Models gives your app a roughly 3-billion-parameter model that runs on the iPhone. It is offline, private, and free.
• A LanguageModelSession plus respond(to:) is the whole hello world. Always check SystemLanguageModel.default.availability first and keep a non-AI fallback.
• Guided generation with @Generable makes the model return a typed Swift value, not text you have to parse.
• It is a small model. It is weaker at multi-step reasoning, has a limited context window, and leans English. Test Bahasa Indonesia prompts before you ship.
• Use a hybrid setup: on-device by default, cloud only when a task needs deep reasoning or a large context.

What makes Foundation Models worth your time is the price and the privacy, more than the model’s size. “Free, local, and good enough” beats “excellent but metered and remote” for many of the AI features apps actually ship. Start there, and reach for the cloud only when a task earns it.

I hope this writing helps you. Thanks for reading.