Free · No sign-up · Open source · iOS · Core ML · Apple Neural Engine

Large diffusion models, entirely on your iPhone.

MobileDiffuser runs FLUX.2 Klein and a distilled SD3 Medium on the Neural Engine — text encoder, denoiser and VAE, no server, no GPU fallback. On an iPhone 15 Pro: SD3 Medium at 512 px in 9.4 s and 1024 px in 31 s, FLUX.2 Klein at 1024 px in 33 s with mixed-resolution sampling, in about 200 MB of app memory.

View the code 🤗 Core ML models ▶ Demo

100% free. No account, no sign-up, no API key — the app and the model downloads are open to everyone, and your prompts never leave the phone.

Wenwu Tang · Dong Wang · Olga Saukh

9.4s

SD3 Medium · 512 px
4 steps, warm, iPhone 15 Pro

31s

SD3 Medium · 1024 px
4 steps, tiled VAE included

33s

FLUX.2 Klein · 1024 px
mixed-resolution, 77 s at full resolution

185MB

app memory with FLUX.2 Klein
(full resolution; ~220 MB mixed)

How it works

A 2B-parameter transformer, fed to the Neural Engine one block at a time.

The ANE compiles static graphs and refuses ones that are too large. MobileDiffuser splits the MMDiT into small Core ML stages, quantizes the weights to INT8, and streams them through the Neural Engine — each stage is loaded, run and released, so the app never holds the whole model.

inputPrompt77 tokens × 2

ANECLIP-L + CLIP-G6-bit palettized

ANE · × 4 steps Split MMDiT

Stage 0 embeddings + 24 single-block stages · INT8 · query-chunked attention

ANEVAE decoder3 × 3 feathered tiles at 1024

◧

Distilled, no CFG

2-step and 4-step DMD students of SD3 Medium with a flow-matching schedule (shift 3). Guidance is baked in, so every step is a single forward pass.

▤

Split & streamed

Conditioning, embeddings and transformer blocks are separate compiled models. Compiled plans are cached by the system, so reloading a stage costs ~25 ms.

◎

ANE-first, verified on device

.cpuAndNeuralEngine only. A silent CPU fallback is treated as a failure — placement is checked per-op on the phone with MLComputePlan.

Getting to 1024 × 1024

One transformer block ran out of memory. The weights were only 85 MB.

Converting the 512 recipe at a 128 × 128 latent got the app killed on a single block. A small on-device probe showed why: with the stock attention op and 4,250 tokens, the ANE compiler gives up, Core ML quietly moves every op to the CPU, and the plan balloons to ~4 GB.

01
Exact attention in query chunks
Queries are split into 16 chunks; each computes softmax(QKᵀ)V against all keys. No approximation — and the block drops to 190 ms, 100% ANE, 64 MB.
02
One block per Core ML stage
Four blocks per stage stalled the on-device compiler for 20+ minutes. One block compiles in ~3 s; the full model is 25 stages, compiled once and cached.
03
Tiled VAE with feathered overlap
The 512 decoder handles the 128 × 128 latent as 3 × 3 overlapping tiles blended with linear ramps. No seams, no new VAE.

1024 by 1024 image of a rain-soaked neon city street with a figure in a glowing transparent raincoat — The same three techniques carry over to FLUX.2 Klein. This 1024 × 1024 image was denoised on the iPhone 15 Pro’s Neural Engine (4 steps); the latent was decoded on a Mac.

One 1024 px block on iPhone 15 Pro

ANEProbe · iOS 18.5

Stock SDPA

CPU fallback → 4 GB → killed

Query chunks × 8

217 ms

Query chunks × 16

190 ms

Block without attention

27 ms

Attention is ~86% of a 1024 block. A Mac reported the stock-SDPA model as fully ANE-placed — placement does not transfer between devices.

FLUX.2 Klein, on device

A 4B image model and its 4B language model. All of it on the Neural Engine.

FLUX.2 Klein reads prompts with Qwen3-4B, a language model as large as the denoiser itself. In MobileDiffuser the text encoder, the transformer and the VAE are 37 Core ML stages that run on the ANE one at a time — the app stays under 200 MB.

01
Qwen3, cut to what Klein uses
The prompt embedding is the hidden states after layers 9, 18 and 27, so only 27 of 36 layers are converted — nine INT8 stages at 23 ms each, 256 tokens instead of 512.
02
A language model in half precision
One token carries a 16,000-magnitude activation whose square overflows fp16. The norm divides by the largest value first, and the MLP is rescaled around its last layer; without that the ANE returns inf.
03
One block per stage, again
5 double-stream and 20 single-stream blocks (dim 3072) with exact query-chunked attention, then a tiled FLUX.2 VAE. Every op is placed on the ANE.

Three prompts, a fox, a neon OPEN 24 HOURS sign and a bowl of ramen, generated by PyTorch in the top row and by the Core ML stages in the bottom row; the pairs are nearly identical — Top: PyTorch reference. Bottom: the INT8 Core ML stages on the Neural Engine, same noise. LPIPS 0.10 / 0.20 / 0.13 — the neon lettering survives quantization.

FLUX.2 Klein · 1024 px · 4 steps in the app

iPhone 15 Pro · iOS 18.5 · warm

Text encoder (Qwen3)

2.3 s

Denoiser, 4 steps

69.9 s

VAE, 3 × 3 tiles

4.7 s

About 77 s per image and ~185 MB of app memory at full resolution. The first run also compiles the 37 stages on the phone (about 3 minutes, once).

Mixed-resolution sampling: compose at 512, finish at 1024.

A 1024 step costs about 17.5 s on the phone; a 512 step about 2.7 s. The early steps only decide the composition, so the app runs them at 512, upsamples the clean-image estimate, adds noise back, and spends a single step at 1024 on detail. No retraining and no new model — both stage sets are the same weights.

ANE · steps 1–3512 px stages1,024 image tokens · ~2.7 s per step

CPUUpsample ×2, re-noisebicubic in VAE latent space, σ = 0.77

ANE · step 41024 px stages4,096 image tokens · ~17.5 s

ANEVAE decoder3 × 3 tiles

Three prompts generated at full 1024 resolution in the top row and with mixed-resolution sampling in the bottom row; both rows are sharp and the neon sign reads OPEN 24 HOURS in both — Top: all four steps at 1024. Bottom: three steps at 512, one at 1024. Different noise, so the compositions differ; sharpness and lettering hold. Rendered on a Mac with the same Core ML stages.

FLUX.2 Klein · 1024 px in the app

iPhone 15 Pro · warm

Full resolution, 4 steps

77 s

Mixed: 3 × 512 + 1 × 1024

33 s

Text 1.7 s, denoising 26.5 s, VAE 4.7 s. Checked by eye on three prompts, not a benchmark; a seed does not reproduce the full-resolution image. The first mixed run compiles the 512 stages once.

ReSeed, on device

Few-step models forget how to be different. A 72k-parameter bias reminds them.

Distilled students map different noise seeds to nearly the same image. ReSeed adds a learned per-channel bias to every transformer block output during the first sampler step only. Because each block is already its own Core ML stage, the bias is added between stages in Swift — same weights, no reload, and about 0.2% extra time.

“A portrait photo of a young woman in a park, natural light” · seeds 1–4 · 512 px

Without ReSeed, four seeds give almost the same frontal portrait.

＋

Between-stage bias

Biases for blocks 0–22 are added to stage outputs at step 1; the last block’s bias is a small extra input of the final stage. Steps 2–4 are the unchanged student.

≡

One set of weights

α = 0 is the plain 4-step model; α = 0.5 / 1.0 / 1.5 scale the adapter. Switching never reloads the pipeline.

✓

Checked against PyTorch

The Core ML chain reproduces the ReSeed effect — a 39% change of the first-step output — within the usual fp16 / INT8 error. On the iPhone the bias add takes 18 ms at 512 px and about 63 ms at 1024 px, first step only.

Performance

Measured on an iPhone 15 Pro, warm, end to end.

512 × 512SD3 Medium · 4 steps

9.4 s

1024 × 1024SD3 Medium · 4 steps

31 s

1024 × 1024FLUX.2 Klein · 4 steps

77 s

1024 × 1024FLUX.2 Klein · mixed-resolution

33 s

Text encodersDenoiser (4 steps)VAE decode

Model	Resolution	Text	Denoiser	VAE	Total	Peak app memory
SD3-M 2-step	512	—	—	—	~5.6 s	~86 MB
SD3-M 4-step + ReSeed	512	0.7 s	8.2 s	0.4 s	~9.4 s	~58 MB
SD3-M 4-step	1024	0.7 s	25.2 s	5.4 s	~31 s	~146 MB
FLUX.2 Klein 4-step	1024	2.3 s	69.9 s	4.7 s	~77 s	~185 MB
FLUX.2 Klein mixed 512 → 1024	1024	1.7 s	26.5 s	4.7 s	~33 s	~220 MB

iOS 18.5, ANE-first. The first run after install also compiles the stages on device, once (about 45 s for SD3 Medium at 1024, about 3 minutes for FLUX.2 Klein); cached afterwards.

Negative results, kept

What did not make it faster.

Everything below was measured on the phone. Rewriting the same attention maths does not help; removing computation without retraining breaks a 4-step student.

W8A8 activations199 ms vs 207 ms
ANE-native (B,C,1,S) conv layout249 ms · slower
Per-head split einsum192 ms · same
Chunks aligned to image/text boundary194 ms · same
All 25 stages residentkilled after 24 loads
2 × 2 KV pooling, all blocks2.9× faster · LPIPS 0.82

Grid comparing exact attention with key-value pooled attention variants; pooled variants look foggy or blocky — Training-free KV downsampling on the 4-step model. Columns: exact · 2×2 all · 4×4 all · 2×2 first 12 · 2×2 last 12 · with mass correction (3). Fast on the ANE, but the image falls apart — cheaper attention has to be trained in.

Samples

Generated with Core ML on the Neural Engine.

Four cyberpunk images: a neon street in the rain, a portrait with a cybernetic eye, a neon motorcycle in a tunnel, and a giant mech in a night market — FLUX.2 Klein (4B), 4 steps, 1024 × 1024, from the INT8 Core ML stages on the Neural Engine. The same stages run in the iPhone app; these four were rendered on a Mac.

▶ Watch the demo

Settings panel downloading Core ML resources inside the app — Models download inside the app from Hugging Face — no Python, no CLI.

Get started

Build it, download the models in-app, generate.

git clone -b MCL https://github.com/TWWinde/MobileDiffuser.git
cd MobileDiffuser
open MobileDiffuser.xcodeproj
# set your signing team, run on a physical iPhone,
# then Settings → download a model

DeviceiPhone 15 Pro or newer recommended
OS / toolsiOS 18.2+, Xcode 16.2+
Models~2.9 GB per SD3 Medium folder, 7.5 GB for FLUX.2 Klein, downloaded in-app
ReproduceConversion, probe and validation tools in tools/; notes in docs/ANE_1024.md, docs/FLUX2_KLEIN.md, docs/RESEED.md

Codegithub.com/TWWinde/MobileDiffuser↗ Core ML resourcesWenwu2000/MobileDiffuser-SD3-medium↗ Write-upHow 1024 px got onto the Neural Engine↗ Write-upFLUX.2 Klein on the Neural Engine↗ RelatedReSeed project page↗

Cite

If MobileDiffuser is useful in your work

@misc{tang2025mobilediffuser,
  author       = {Wenwu Tang and Dong Wang and Olga Saukh},
  title        = {MobileDiffuser: On-device Stable Diffusion 3 Medium on iPhone with Core ML},
  year         = {2025},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/TWWinde/MobileDiffuser}}
}