Local AI Explained: Four Ways to Run a Model on Your Computer

Running AI locally means the model's inference happens on your computer, using a downloaded model and a program that loads it. In his walkthrough, Tech With Tim explains the vocabulary and demonstrates four ways to do it: LM Studio, Ollama, Docker Model Runner and Python. The best first step is not the largest model you can find. It is a small model you can actually run, on one task you know how to judge.

Watch the Walkthrough

Disclosure: The video includes a sponsored MindsHub Cowork segment at 09:43, as well as creator promotions. The setup routes below are based on the demonstration and checked against the projects' own documentation; this article is not sponsored by those tools. We did not benchmark Tim's computer or reproduce his performance results.

Five Building Blocks, Without the Jargon

Tim's explanation is useful because it separates the model from the program running it. A few distinctions make the purchase and download decisions clearer:

LM Studio's current requirements recommend 16 GB system memory for its supported desktop platforms; some smaller models may work with less. That is a product recommendation, not a guarantee that any specific model will fit. Start with a small text model, check the download size and license, and leave memory headroom for your operating system. A tiny model is a setup test, not a promise of frontier-model quality.

Which Route Fits Your Workflow?

1. LM Studio: The Graphical First Run

1. Check the supported operating systems and hardware, then download the desktop app from LM Studio.
2. Open Discover, select a small instruction/chat model, inspect its size and quantization, and download it. LM Studio's getting-started guide shows the current interface.
3. Open Chat, load that downloaded model, and ask one short question with an answer you can verify. If the load fails, choose a smaller variant or lower context setting before changing hardware.

Once the model is downloaded, LM Studio says core local chat and its local server can work offline. That statement does not cover optional MCP servers, network tools or an external app you later connect.

2. Ollama: A Local Model From the Terminal

1. Install Ollama for your operating system from its official download page.
2. For a lightweight smoke test, open a terminal and run ollama run gemma3:1b. Ollama's model card lists this particular quantized download at about 806 MB. Runtime memory will be higher, and a 1B model may give weak answers.
3. Ask a simple question, exit the chat, and inspect installed models with ollama list. When you need a programmatic interface, use Ollama's documented local API rather than assuming a chat subscription is involved.

The official quickstart now also offers cloud models and says to sign in for those. To test local AI, select a downloaded local model. The default service is on localhost:11434; do not expose it to other devices or the public internet without authentication and network controls. Ollama's FAQ documents the service and ollama ps for checking CPU/GPU placement.

3. Docker Model Runner: For an Existing Container Workflow

1. Install Docker Desktop, then in Settings > AI enable Docker Model Runner. Follow the current Docker guide for your platform and GPU. Linux Docker Engine has a separate plugin path in the same guide.
2. In the Models tab, pull a small model and use its local chat, or follow Docker's example: docker model pull ai/smollm2:360M-Q4_K_M and docker model run ai/smollm2:360M-Q4_K_M.
3. Only after that local test, connect a Compose application. Keep host-side TCP support disabled unless a local application needs it, and configure accepted origins and network access deliberately.

Docker's example is deliberately tiny and useful for validating the route, not evaluating model quality. Model Runner's supported backends and GPU paths vary by platform, so a Docker installation alone does not guarantee fast inference.

4. Python: Load a GGUF Model Directly

Tim's final demonstration loads a local model through a Python inference binding. This is for readers who want to control code and model loading, not the fastest path to a first chat. The llama-cpp-python project documents Python and compiler requirements, prebuilt wheels and platform-specific GPU options.

1. Create a fresh Python environment, then install llama-cpp-python following the project's instructions for your CPU or GPU. A plain pip install may compile native code; use a supported wheel or compiler setup if it fails.
2. Download a small instruction-tuned GGUF from a model publisher you trust. Read its license and chat-template guidance, and put it in a local models directory. A random GGUF from a search result is not a good starting point.
3. Load that local path and generate one response. Adjust the context size and GPU offload only after a basic run works.

from llama_cpp import Llama

model = Llama(model_path="models/your-model.gguf", n_ctx=2048)
reply = model.create_chat_completion(
    messages=[{"role": "user", "content": "Explain local AI in one sentence."}]
)
print(reply["choices"][0]["message"]["content"])

Replace the filename with the model you downloaded. This code invokes local inference directly. Calling Ollama's API from Python is useful too, but then Ollama is still the inference server; Python is only its client. GPU acceleration in llama-cpp-python may need a different install than the CPU example.

How to Check That Your First Run Is Really Local

After downloading and loading a model, turn off network access briefly and repeat a simple prompt. If it still responds, that is a useful offline check for this specific path. It does not prove an agent with search, browser, plugins or remote logging will keep future tasks on the device. Check the selected model and endpoint, not just where the app's window appears. Also inspect performance: a 1B model may fit easily but fail the task, while a larger model may run slowly with CPU offload. Measure quality, response time, memory pressure and electricity or hardware cost before replacing a cloud workflow.

Where MindsHub Cowork Fits

MindsHub Cowork is the video's sponsor and an agentic workspace, not a fifth way to execute model weights. Its source repository is public, and the product can connect to different models. Depending on the model and deployment you choose, work may use local inference or a hosted service. Its pricing page lists a fair-use free tier for its Air model and metered use for other models. Check the selected route and connector permissions before calling an agent workflow free, private or offline. You do not need MindsHub for any of the four first-run tutorials above.

Video Chapters

Understand the pieces

Choose a route

Source Note and Limits

Checked 2 November 2026. Tim's video and the supplied transcript are the source for his explanations, four demonstrations and sponsor segment. We checked setup and offline statements against LM Studio's guide, Ollama's quickstart, Docker's Model Runner guide and llama-cpp-python's README. The decision table, starter test and privacy qualifications are our analysis. We did not install all four tools on the same hardware, benchmark Tim's results, or audit MindsHub's hosted infrastructure. Model availability, package commands, licenses, pricing and hardware support can change; check each linked project before installing.