Gemma 4 moves agent workflows onto edge devices

What Google's Gemma 4 and LiteRT-LM release means for tool-using agents on phones, laptops, browsers, and small edge devices.

Google announced Gemma 4 on April 2, 2026. The release focuses on agents that run on the user's device, not only on chat.

That distinction matters. A local chatbot produces text. A local agent must plan steps, use tools, process device data, and return a predictable result. It must also work within strict limits for memory, power, and response time.

Google says Gemma 4 supports multi-step planning, autonomous actions, offline code generation, audio and visual input, and more than 140 languages without specialized fine-tuning. I have not independently benchmarked these claims. This post examines the product and deployment details in Google's release.

Quick take

Gemma 4 makes local agents more practical because the release combines a small model, constrained output, tool calling, and deployment support for several device classes. Product teams still need to test reliability on their own tasks and hardware.

Agent Skills demonstrates the workflow

Google added Agent Skills to the Google AI Edge Gallery for iOS and Android. A skill gives the model a focused capability that it can use in a multi-step workflow.

Google demonstrates skills that:

Query a source such as Wikipedia.
Convert speech, text, or video into summaries and flashcards.
Convert data into charts.
Call other local models for speech, images, or music.
Complete a multi-step task through one conversation.

These examples show the difference between model capability and orchestration. The model does not only generate an answer. It selects a capability, supplies input, reads the result, and continues the task.

Tool use also creates failure modes. The application must constrain tool selection, validate arguments, limit authority, and handle partial failure. A demo can tolerate an occasional wrong call. A product cannot.

LiteRT-LM handles deployment constraints

LiteRT-LM is Google's deployment layer for generative models on mobile, desktop, web, and edge hardware. The Gemma 4 announcement describes three features that are useful for agents.

First, LiteRT-LM supports 2-bit and 4-bit weights. Google reports that Gemma 4 E2B uses less than 1.5 GB of memory on some devices.

Second, constrained decoding limits output to a required structure. This feature can improve tool-call reliability because the application does not need to recover a schema from arbitrary text.

Third, dynamic context lengths let one model run across CPUs and GPUs with different memory limits. Gemma 4 supports a context window of up to 128K tokens, but the usable context still depends on the device and workload.

Google reports two useful edge measurements

Google reports that LiteRT-LM processes 4,000 input tokens across two skills in less than three seconds. On a Raspberry Pi 5 CPU, Google reports 133 tokens per second for prompt prefill and 7.6 tokens per second for output decoding.

These measurements come from Google, not from my own test. They also measure different phases:

Prefill processes the input context.
Decode generates the output one token at a time.

The decode rate gives a better indication of how quickly a user sees generated text. Prefill matters when an agent must read a large prompt or tool result before it acts.

One stack supports several device classes

Google lists support for:

Android and iOS on CPUs and GPUs.
Windows, Linux, and macOS, including Metal on macOS.
Browsers through WebGPU.
Raspberry Pi 5 and Qualcomm Dragonwing IQ8 hardware.

The litert-lm command-line interface runs on Linux, macOS, and Raspberry Pi. It also supports the tool calling used by Agent Skills.

This platform range can reduce deployment differences, but it does not remove them. Mobile thermal limits, browser memory, desktop GPUs, and embedded CPUs behave differently. Teams still need device-specific tests for latency, memory, battery use, and output quality.

Good uses are narrow and measurable

Many local tasks do not require a frontier model. A smaller model can be useful when the task has a clear input, limited authority, and a result that the application can validate.

Examples include:

Classifying a message on the device.
Converting speech into a structured action.
Summarizing a local note.
Reading a local file and selecting an approved operation.
Running a routine when the network is unavailable.

Local execution can reduce network delay and per-token cost. It can also keep data on the device. These benefits apply only if the complete workflow remains local. An application that sends tool results or telemetry to a remote service has a different privacy boundary.

Local agents still need guardrails

Gemma 4 does not remove the main engineering limits of local agents.

Smaller models can fail on complex reasoning. Tool calls need validation and authorization. Long context consumes memory. Sustained inference can reduce performance when a device reaches its thermal limit.

I would use a hybrid routing policy:

Keep routine, private, and latency-sensitive work on the device.
Send difficult tasks to a hosted model only when the user and policy allow it.
Validate both local and hosted tool calls before the application performs an action.

This policy gives each model a defined role. It also makes cost, privacy, and failure behavior easier to measure.

What I would test before release

Before I ship a Gemma 4 agent, I would measure:

Tool selection accuracy on representative user requests.
Argument validity before and after constrained decoding.
End-to-end latency, including tool execution.
Peak memory and sustained thermal behavior on each supported device.
Recovery when a tool times out or returns incomplete data.
Quality loss when the workflow uses a smaller context window.
Whether any data leaves the device through tools, logs, or analytics.

The release provides the components for an on-device agent. These tests determine whether the agent is reliable enough for a product.