Every recent flagship launch has included a segment on artificial intelligence running locally, usually illustrated with a photo edit or a summarised inbox. The demonstrations are real. What they rarely make clear is which part of the work happened on the device, which part travelled to a server, and what that division means for the person holding the phone.

What running locally actually means

A model is a set of numerical weights plus the code to run them. Running it on-device means those weights are stored in the phone's memory and executed by its processor — typically a dedicated neural accelerator alongside the CPU and GPU. Nothing needs to leave the handset for that specific operation. Running it in the cloud means the input is transmitted to a remote server, processed there, and the result returned.

The distinction that matters

Ask of any AI feature: does this specific request leave the device? Vendors increasingly ship hybrid systems where simple requests stay local and complex ones are routed out, often without a visible indicator. The answer is in the privacy documentation, not the marketing page.

"On-device" is a claim about where computation happens, not a promise about where your data ends up.
THRYV Tech Desk

Three genuine consequences

1. Latency

Local execution removes network round-trips. For interactive work — live transcription, keyboard prediction, camera processing — that difference is the feature. It is also why these are the categories where on-device models appeared first.

2. Data exposure

If a request never leaves the handset, it cannot be retained by a provider or accessed under a legal request to that provider. This is a real privacy improvement, but it is conditional: it holds only for the requests that actually stay local, and only until the feature falls back to a server.

3. Capability and battery

Local models are constrained by memory and thermal limits, so they are smaller and generally less capable than hosted equivalents at the same task. Sustained local inference also draws meaningful power. Both constraints are physical and will not be marketed away by a software update.

How to evaluate the claim on a spec sheet

  • Find the vendor's privacy or AI documentation and identify which features are described as processed on-device versus in the cloud.
  • Check whether there is a user-visible setting to disable cloud fallback, and what stops working when you do.
  • Look for a stated data-retention period for any request that is sent to a server, including whether it is used for training.
  • Treat generic phrases such as 'privacy-first AI' as marketing until the documentation names a specific processing location.

THRYV has not benchmarked individual handsets for this article; no device is scored or recommended here. When our testing process for on-device inference is complete, results will be published as a review with the methodology attached.