# Add a local LLM without making your app depend on it

> Add optional Ollama-generated summaries to a Swift app while preserving deterministic fallback data, short health checks, failure backoff, caching, and testability.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-08-16 | Topics: [Swift](https://flaviocopes.com/tags/swift/) | Canonical: https://flaviocopes.com/optional-local-llm-swift-app/

AI can improve an application without becoming a requirement.

The application should still work when the model is missing, slow, or broken.

I used a local Ollama model to turn factual activity records into one short daily sentence. The app tracks project work and shows a daily summary. Without AI, that summary is a plain factual line: "3 commits across 2 projects". With AI, it becomes a readable sentence. The original records remain the product. The generated sentence is a convenience.

This post walks through the architecture I ended up with. None of it is specific to my app. It applies any time you want a local model to add polish without adding a dependency.

## Why local, and why optional

A local model through [Ollama](https://ollama.com) gives you three things: no API keys, no per-token costs, and no user data leaving the machine. For a personal productivity app, that last one matters most. The activity records never touch a server.

But local also means you control nothing about the environment. The user may never install Ollama. They may have it installed but not running. They may have pulled a different model. They may be on a machine where a 4B model takes 40 seconds to respond.

If any of those situations breaks your app, you didn't add a feature. You added a dependency.

So the design goal is: every screen must render correctly with the model completely absent, and the code path for "no AI" must be the ordinary one, not an error branch.

## Start with a small protocol

Keep the application independent from Ollama:

~~~swift
protocol NarrativeEngine: Sendable {
    func isAvailable() async -> Bool
    func write(system: String, prompt: String) async throws -> String
}
~~~

The rest of the app asks for a narrative. It does not know which local server produces it. If I switch from Ollama to Apple's on-device models later, one conforming type changes and nothing else.

This also makes tests easy. A fake engine can return fixed text or throw an error:

~~~swift
struct FixedEngine: NarrativeEngine {
    let text: String

    func isAvailable() async -> Bool { true }

    func write(system: String, prompt: String) async throws -> String {
        text
    }
}
~~~

With this, I can test the whole narrative pipeline without a model installed, including on CI.

## Check availability quickly

Ollama serves an HTTP API on `127.0.0.1:11434`. Its tags endpoint lists installed models, and it answers instantly, which makes it a good health check:

~~~swift
func isAvailable() async -> Bool {
    var request = URLRequest(
        url: host.appending(path: "api/tags")
    )
    request.timeoutInterval = 2

    guard let (_, response) = try? await session.data(for: request) else {
        return false
    }

    return (response as? HTTPURLResponse)?.statusCode == 200
}
~~~

Two seconds is enough for a service expected on `127.0.0.1`. If nothing is listening, the connection is refused in milliseconds and we return `false` immediately.

Do not make the interface wait 90 seconds to discover that the user never installed Ollama. That's what happens if you skip the health check and let the generation request itself time out.

You could go further and parse the response to confirm the specific model is pulled. I decided against it: if the model is missing, generation fails fast with a clear error, and the failure path below handles it the same way.

## Give generation a different timeout

Generation is allowed to take longer:

~~~swift
var request = URLRequest(
    url: host.appending(path: "api/generate")
)
request.httpMethod = "POST"
request.timeoutInterval = 90
request.setValue(
    "application/json",
    forHTTPHeaderField: "Content-Type"
)
~~~

Health checks and real work have different expectations. Do not use one timeout for both. A 2-second generation timeout kills every request on a slow machine. A 90-second health check freezes the UI for users without Ollama.

The request body asks for a low-temperature, non-streaming response, because I need one short stable sentence, not a creative stream:

~~~json
{
  "model": "gemma3:4b",
  "system": "Write one factual sentence.",
  "prompt": "Summarize these recorded project updates: ...",
  "stream": false,
  "options": {
    "temperature": 0.2
  }
}
~~~

Low temperature matters here. At the default temperature the same records produce a different sentence every launch, and the UI feels unstable. At `0.2` the output is close to deterministic.

`stream: false` means Ollama replies with a single JSON object once generation finishes. Decoding is one small struct:

~~~swift
struct GenerateResponse: Decodable {
    let response: String
}

func write(system: String, prompt: String) async throws -> String {
    // build the request as above
    let (data, _) = try await session.data(for: request)
    let decoded = try JSONDecoder().decode(
        GenerateResponse.self,
        from: data
    )
    return decoded.response
        .trimmingCharacters(in: .whitespacesAndNewlines)
}
~~~

The trim is not optional. Small models love leading newlines.

## Return nil when enhancement is unavailable

The orchestration layer can make failure ordinary:

~~~swift
func narrative(for summary: Summary) async -> String? {
    guard await engineIsReachable() else {
        return nil
    }

    do {
        return try await engine.write(
            system: systemPrompt,
            prompt: prompt(for: summary)
        )
    } catch {
        markUnavailable()
        return nil
    }
}
~~~

Notice the return type. Not a `Result`, not a thrown error the view must catch. Just `String?`. The view receives either:

- generated prose
- `nil`, and shows the factual summary it already had

~~~swift
Text(narrative ?? summary.factualLine)
~~~

There is no broken screen, no error alert, no "AI unavailable" banner. The user who never installed Ollama never learns the feature exists. That's the correct experience for an optional enhancement.

## Back off after failure

After a failed health check, later requests should not keep retrying Ollama. If the user opens a screen with twenty rows and the server is down, that's twenty health checks, each burning its timeout.

~~~swift
private var unavailableSince: Date?
private let retryDelay: TimeInterval = 60

private func engineIsReachable() async -> Bool {
    if let unavailableSince,
       Date().timeIntervalSince(unavailableSince) < retryDelay {
        return false
    }

    guard await engine.isAvailable() else {
        unavailableSince = Date()
        return false
    }

    unavailableSince = nil
    return true
}
~~~

This is a small circuit breaker for requests arriving after the failure was recorded.

After one failure, the app leaves the local service alone for a minute. If the user starts Ollama in the meantime, the next check after the minute succeeds and narratives quietly appear.

An actor can still be re-entered while `isAvailable()` is suspended, so several concurrent calls may each run their own health check before the first one records the failure. If you need to coalesce simultaneous health checks too, keep one shared in-flight availability task and have every caller await it.

## Generate lazily and cache the results

Do not summarize every historical record at launch. A 4B model producing twenty summaries serially will peg the machine for a minute, for text the user may never scroll to.

Ask for text when a row becomes visible:

~~~swift
.task(id: summary.signature) {
    await narratives.load(summary, on: day)
}
~~~

SwiftUI's `.task(id:)` runs when the row appears and cancels when it disappears. The `id` is a **signature** of the underlying facts — a hash of the records that feed the summary. That signature is also the cache key:

~~~swift
private var cache: [Signature: String] = [:]

func load(_ summary: Summary, on day: Date) async {
    if cache[summary.signature] != nil { return }

    if let text = await narrative(for: summary) {
        cache[summary.signature] = text
    }
}
~~~

Keying the cache by content signature instead of by date gives you invalidation for free. New records change the signature, the old cached sentence no longer matches, and a fresh one gets generated. Unchanged days never hit the model again.

When a row disappears, cancellation can prevent a stale UI update. Propagate cancellation into the narrator's pending task if you also want to stop the underlying Ollama request instead of letting it finish for nobody.

This keeps startup fast and avoids generating text the user never sees.

## Keep facts outside the model

The model should rephrase known records.

Do not ask it to infer:

- whether a task is finished
- how much progress was made
- which project is most important
- work not present in the event log

Derive those facts in code, and put them in the prompt already computed:

~~~text
Summarize these recorded project updates:
- notes-app: 3 commits, last one at 17:40
- blog: 1 post drafted
~~~

The model turns that into a sentence. It cannot get the numbers wrong, because it isn't producing them — it's decorating them. If it hallucinates anyway, the damage is a slightly odd sentence next to correct data, not a wrong number presented as truth.

The model's absence should reduce polish, not correctness.

That is the boundary I want for optional AI in local software.