Foundation Models: running Apple's on-device LLM in your iOS app

No API key. No server. No internet. The model is already on your user’s iPhone. After two years of every AI feature meaning “send the user’s data to someone’s GPU and hope,” Apple’s Foundation Models framework inverts the arrangement: a capable language model ships with the OS, and calling it costs you nothing and leaks nothing.

I research retrieval and language models for a living, so I went in skeptical of the marketing. The honest summary: it is smaller than a frontier cloud model, and for a large class of everyday app tasks that does not matter at all. Here’s what it is, how to call it, and where its edges are.

What Apple actually shipped

A private model, on the device, for free

The framework exposes the same on-device model that powers Apple Intelligence, on the order of three billion parameters, optimised to run on the Neural Engine. Three properties make it interesting for app developers: it runs offline, it sends nothing off the device, and there is no per-token bill. You are no longer choosing between “add an AI feature” and “take on a cloud dependency plus a privacy review.”

Your first request in five lines

`LanguageModelSession`, a prompt, a response

The entry point is deliberately small. You create a session and ask it something:

import FoundationModels

let session = LanguageModelSession()
let response = try await session.respond(
    to: "Summarise this note in one sentence: \(noteText)"
)
print(response.content)

That’s the whole “hello world.” Before relying on it, check availability (the model isn’t present on every device or in every region) and degrade gracefully when it isn’t:

switch SystemLanguageModel.default.availability {
case .available:
    // use the model
case .unavailable(let reason):
    // fall back to a non-AI path; never block the feature
}

The feature that makes it practical: guided generation

Make the model return typed Swift, not a wall of text

Free text is a pain to consume: you end up writing brittle JSON parsers and praying the model closes its braces. Foundation Models’ key idea is guided generation: annotate a Swift type with @Generable, and the framework constrains the model to produce exactly that structure, decoded into your type.

@Generable
struct Expense {
    @Guide(description: "Merchant or store name")
    var merchant: String

    @Guide(description: "Total amount in Rupiah, digits only")
    var amount: Int

    @Guide(description: "Spending category")
    var category: Category
}

let expense = try await session.respond(
    to: "Extract the expense from: \(receiptText)",
    generating: Expense.self
).content   // -> a real Expense, no JSON parsing

Guided generation versus free text: a @Generable type constrains the model to a known shape, so you get a decoded Swift value instead of a string you have to parse and validate by hand.

Real uses for Indonesian apps

Small, private, offline: the sweet spot

The model’s constraints map neatly onto things real apps need, especially where sending user data to a server is a privacy or cost problem:

Offline note summaries: condense a long note on the device, no network round-trip.
Transaction categorisation in a finance app: label “GoFood”, “PLN”, “Tokopedia” into categories without shipping a user’s spending history anywhere.
POS receipt descriptions: turn a list of items into a tidy line, generated locally during a busy checkout.

Two inference paths for the same feature. On-device keeps the prompt and response on the phone; the cloud path leaves the device, adds latency and cost, and needs a privacy review. Pick per task, not per app.

Best for: short, well-scoped, latency- and privacy-sensitive tasks that run often. Watch out: it is an English-biased model; for Bahasa Indonesia, test real prompts before you ship, and keep a non-AI fallback.

Honest limits

It is a small model, and that’s the deal

A three-billion-parameter model is not a frontier model, and pretending otherwise will burn you. It is weaker at multi-step reasoning, has a limited context window, and, as above, leans English. The right mental model is not “a worse GPT” but “a fast, free, private function for simple language tasks.” Knowing which tasks are simple is the whole skill.

Dimension	On-device (Foundation Models)	Cloud LLM (API)
Latency	Low, no network	Network round-trip
Cost	Free	Per token
Privacy	Data stays on device	Leaves the device
Works offline	Yes	No
Reasoning / context	Limited	Strong, large context

The practical pattern: hybrid

On-device by default, cloud when it’s earned

You don’t have to choose globally. Route per task: handle the fast, private, simple cases on-device, and fall back to a cloud model only for the hard ones. Most requests never leave the phone, which keeps the average latency, cost, and privacy footprint low, and the cloud bill small.

Hybrid routing: a request hits a router that checks whether the task is simple and private; if so it stays on-device, otherwise it escalates to the cloud. The common case never leaves the phone.

On-device or cloud for this task?

Routing a single feature

Simple · private · offline-friendlycommon case→ On-device

Deep reasoning or large contexthard case→ Cloud LLM

Mixed workloadroute per request→ Hybrid, on-device first

The framing I’d leave you with: the most interesting thing about Foundation Models isn’t the model’s size, it’s the price and the privacy. “Free, local, and good enough” beats “excellent but metered and remote” for a surprising number of the AI features apps actually ship. Start there, and reach for the cloud only when a task earns it.