You can run Gemma 4 locally on a computer with integrated graphics. The important decisions are not simply “CPU or GPU.” Choose a quantized model that fits in memory, keep the context only as large as the task requires, and accept the speed-versus-capability tradeoff.
The Direct Answer
Start with the classic LM Studio download, select an official or trusted Gemma 4 GGUF variant, and let the application estimate whether it fits. On a 16GB system, E4B Q4 is the safer first model. Try 12B QAT Q4 only when LM Studio says it can load and there is enough memory left for the operating system, context, and attached images or documents.
Use thinking mode selectively. Keep LM Studio's defaults first, measure a real task, then adjust one setting at a time. The creator spent hours changing low-level options and saw little benefit compared with simply disabling thinking for easy work.
Watch the Full Gemma 4 Tutorial
Source and credit: The embedded tutorial and performance tests are by ExplainingComputers. The measurements below describe that one machine and configuration, not a universal benchmark. The current product documentation and implementation guidance are added by JQ AI SYSTEMS.
Download the Right Application: LM Studio or Bionic?
The LM Studio download page now presents LM Studio Bionic first and the classic LM Studio application second. The video uses the classic application: it provides model discovery, per-model settings, a chat interface, and programmable local APIs.
LM Studio describes Bionic as a separate agent application for coding and document work. Bionic can run local models, connect to another device through LM Link, or use cloud models. Choose the classic application when following this tutorial or when you need direct control over model loading, context, GPU offload, and the local server.
What Fits on Modest Hardware?
Google currently lists five Gemma 4 sizes: E2B, E4B, 12B, 31B, and 26B A4B. Smaller or lower-precision models require less memory and usually generate faster, while larger models generally offer more capability. The file size is only part of the memory requirement: the runtime, context or KV cache, vision inputs, and other applications also consume memory.
| Starting point | Why choose it | What to watch |
|---|---|---|
| E2B or E4B Q4 | Better fit for laptops, mini PCs, and first experiments. | Lower capability may show up on uncommon images, nuanced instructions, or complex code. |
| 12B QAT Q4 | More capability while remaining practical on some 16GB systems. | Leave headroom for Windows, LM Studio, context, and multimodal processing. |
| 31B or 26B A4B | Higher-capability local or workstation experiments. | Not sensible defaults for the modest 16GB integrated-graphics machine demonstrated. |
Google's official Gemma 4 documentation recommends QAT Q4 GGUF variants for llama.cpp and LM Studio. In QAT, lower-precision behavior is simulated during training, allowing the model to compensate for quantization. PTQ instead compresses a trained model afterward and may lose more quality.
Install LM Studio and Load Gemma 4
- Download the classic LM Studio application from the official download page. It appears below Bionic.
- Open Discover and search for
Gemma 4. Prefer Google's official QAT collection or a clearly identified, trusted GGUF publisher. - Open the available-downloads list and compare the model size, quantization, and LM Studio's fit warning.
- For the demonstrated 16GB computer, the creator downloaded a 7.15GB Gemma 4 12B QAT 4-bit file. E4B Q4 is a more conservative first download.
- Choose Use in New Chat, let the model load, and begin with a small, non-sensitive prompt.
- Enable Developer Mode if you want persistent timing and throughput statistics or need to verify detected hardware.
Thinking Mode Is the Largest Easy Speed Lever
With default thinking enabled, the creator's first 12B request spent about 18 seconds reasoning and reported 5.46 tokens per second. Disabling thinking made a simple conversational request begin answering almost immediately. That does not make the model intrinsically faster; it avoids generating an additional reasoning trace for a task that does not need one.
- Thinking off: simple rewriting, translation, short summaries, extraction, and routine formatting.
- Thinking on: difficult coding, planning, multi-step reasoning, and agentic work where the quality gain is measurable.
- Evaluate both: compare accepted results, time, and memory on ten representative examples instead of choosing by feel.
The Settings That Actually Matter
Context length
The demonstrated model loaded with an 8,192-token context. Reducing it to 4,096 saves memory and can improve responsiveness; increasing it allows larger documents but consumes more memory and can reduce performance. The 3,900-word chapter test used the 8,192-token setting and occupied 65.9% of the available context.
CPU thread pool
The creator increased the thread pool from four to six, matching the 5600G's physical cores, but saw only a small change within normal run-to-run variation. Do not assume every logical thread improves generation. Measure before keeping the change.
GPU and KV-cache offload
The 12B QAT model could not be partially offloaded in the creator's setup. A partial E4B Q_K_M offload to the integrated GPU provided no practical advantage, likely because CPU and GPU shared the same system memory and introduced another bottleneck. Offloading the KV cache with 2GB allocated to the integrated GPU did help. Current LM Studio builds can estimate memory and automatically choose GPU usage, so defaults are the correct baseline.
What the Creator Actually Tested
| Task | Result on 12B QAT | Useful lesson |
|---|---|---|
| Offline prose | Generated about 100 words on Southern Italy while the computer was disconnected. | Once downloaded, model inference can run without an internet connection. |
| Translation | Translated the answer into French. | Offline multilingual work is possible, but a fluent reviewer should verify important output. |
| Image understanding | Identified both a giraffe and a less-common okapi. | Multimodal capability is useful, but two images are a demonstration, not an accuracy benchmark. |
| Document summary | Reduced a 3,900-word book chapter to roughly 200 words. | Watch context use and compare summaries with the source before relying on them. |
| Python coding | Produced a working BMI calculator in about 1 minute 50 seconds. | Run and review generated code; one successful small program does not establish broader reliability. |
LM Studio can extract text from PDFs and documents for chat even when the model itself natively expects text and images. That convenience belongs to the application layer, so file support and model modality should not be treated as the same claim.
E4B Versus 12B: Faster Is Not Always Better
On the same machine, E4B produced about 11.75 tokens per second versus roughly 6 for 12B. It still recognized the giraffe but failed to identify the okapi, describing the image as an abstract texture. The larger model was slower and succeeded.
This is the central hardware lesson: select the smallest model that reliably completes your task. A fast failure is not useful, and a larger model that swaps memory or stalls may also be the wrong choice. Build a compact evaluation set containing common inputs, difficult edge cases, and the errors you cannot accept.
Practical Starter Settings
| Control | Begin with | Change it when |
|---|---|---|
| Model | E4B QAT Q4 GGUF | Move to 12B only if quality is insufficient and memory permits. |
| Context | 4K to 8K | Increase only when the actual input does not fit. |
| Thinking | Off for easy tasks | Enable for complex reasoning, coding, or agentic work. |
| Threads | LM Studio default | Test physical-core count and retain it only if measured performance improves. |
| GPU offload | Automatic | Adjust only after checking LM Studio's memory estimate and hardware detection. |
| Evaluation | Ten representative examples | Add every meaningful failure to the test set. |
LM Studio's current Windows requirements recommend at least 16GB RAM and 4GB dedicated VRAM, but the tutorial demonstrates that CPU and integrated-graphics inference can still be useful. Treat “recommended” as a performance target and the creator's test as evidence of feasibility, not a compatibility promise for every laptop.
Related Lesson: What Changes With Agentic AI?
The local chat workflow above produces answers. Agentic systems add tools, actions, files, and longer task loops, which increases both capability and risk. This companion ExplainingComputers episode provides the broader context.
Video Chapters
- 0:00 - Titles and introduction
- 0:56 - Choosing your model
- 4:25 - LM Studio
- 8:51 - Speed and application
- 13:55 - Images and text files
- 17:20 - Coding
- 18:57 - Gemma 4 E4B
- 21:31 - Wrap-up
Sources and Links
- ExplainingComputers: Running Gemma 4 Local AI
- ExplainingComputers YouTube channel
- LM Studio downloads: Bionic and classic LM Studio
- LM Studio: Introducing Bionic
- LM Studio: Get started
- LM Studio: System requirements
- LM Studio: Offline operation
- LM Studio: Per-model defaults
- Google AI for Developers: Gemma 4 overview and QAT guidance
- Google: Gemma 4 model card
- ExplainingComputers: Explaining Agentic AI