You don't need to memorize any of this
Local AI comes with its own vocabulary, and it can feel intimidating at first. This glossary explains the terms you'll come across on this site, in app interfaces, and in model names — in plain language, with simple examples. Bookmark this page and come back whenever you hit a word you don't recognize.
Tip: you don't need to understand every term before you start. Install LM Studio, download a small model, and look things up here only when you're curious.
Hardware & Performance
6 terms- VRAM
- The memory built into your graphics card (GPU). It's the single most important spec for running larger local AI models — more VRAM lets you load bigger models and get faster responses.
- RAM (System Memory)
- The regular memory in your computer, separate from VRAM. If a model doesn't fit in VRAM, some tools can use RAM instead, but it's usually much slower.
- GPU (Graphics Processing Unit)
- Originally built for games and video editing, GPUs are also extremely good at the math behind AI models. A dedicated GPU makes local AI dramatically faster.
- CPU (Central Processing Unit)
- Your computer's main processor. Local AI can run on the CPU alone, but it's typically much slower than using a GPU, especially for larger models.
- Unified Memory
- On Apple Silicon Macs (M1–M4), RAM and VRAM aren't separate — the whole system shares one pool of memory. This is why Macs with high RAM can run surprisingly large models.
- Tokens per second (t/s)
- A common way to measure how fast a model generates text. Higher numbers mean faster responses — useful when comparing models or hardware, but not something to obsess over as a beginner.
Model Files & Formats
6 terms- GGUF
- A popular file format for local AI models, used by tools like LM Studio and Ollama. If a model file ends in
.gguf, it's built to run efficiently on regular computers. - Quantization
- A technique that shrinks a model's file size and memory needs by reducing the precision of its internal numbers. It trades a small amount of accuracy for a big gain in speed and compatibility.
- Q4 / Q6 / Q8
- Common quantization levels you'll see in model names. Lower numbers (Q4) are smaller and faster but slightly less precise; higher numbers (Q8) are closer to full quality but need more memory.
- Parameters (e.g. 7B, 13B)
- The "B" in a model's name (like Llama 3 8B) stands for billions of parameters — roughly, the number of internal values the model learned during training. More parameters usually means a more capable but heavier model.
- Context Window
- How much text a model can "see" and remember at once, including your prompt and its own previous responses. A longer context window lets you work with longer documents and conversations.
- Safetensors
- Another common format for storing model weights, considered safer than older formats because it can't contain hidden executable code.
How Local AI Works
9 terms- LLM (Large Language Model)
- The general term for the type of AI model used in tools like ChatGPT, LM Studio, and Ollama. It's trained on large amounts of text to predict and generate language.
- Inference
- The process of actually running a model to get a response, as opposed to training it. When your local AI answers a question, it's performing inference.
- Prompt
- The text you type or send to the AI — a question, an instruction, or a piece of text you want it to work with.
- System Prompt
- A special instruction, often hidden from the main chat, that tells the AI how to behave overall — for example, "You are a helpful, concise assistant."
- Token
- The small chunks of text a model actually processes — often close to a word or part of a word. Both your prompt and the AI's response are measured in tokens.
- Temperature
- A setting that controls how random or predictable a model's responses are. Low temperature gives more focused, repeatable answers; high temperature gives more varied, creative ones.
- Fine-tuning
- Further training an existing model on a specific dataset to specialize it for a particular task or style. Most beginners use ready-made models without ever needing to fine-tune one.
- RAG (Retrieval-Augmented Generation)
- A technique where the AI searches your own documents or data first, then uses that information to answer more accurately. It's how many "chat with your documents" tools work.
- Embedding
- A way of converting text into a list of numbers that represents its meaning, so a computer can compare how similar two pieces of text are. Embeddings power semantic search tools.
Software & Tools
7 terms- LM Studio
- A free, beginner-friendly desktop app for downloading and chatting with local AI models through a visual interface, without needing to type commands.
- Ollama
- A popular tool for running local AI models, often used by developers to connect models to other apps and scripts. It relies more on typed commands than LM Studio.
- Jan
- A free, open-source alternative to LM Studio, offering a similar chat-style interface for running local models.
- Hugging Face
- A website that hosts thousands of open AI models, including most of the ones you'll download through LM Studio, Ollama, or Jan.
- API
- A way for one piece of software to talk to another. Local AI tools like LM Studio and Ollama often expose a local API so other apps on your computer can use the model too.
- Open-weight model
- A model whose trained weights are published for anyone to download and run. This is what makes local AI possible — proprietary cloud models don't offer this.
- Model Card
- A page (usually on Hugging Face) describing a model's size, intended use, and limitations — worth a quick look before downloading a new model.
Popular Model Families
5 terms- Llama
- A well-known family of open-weight models originally released by Meta, widely used as a base for other local AI models.
- Qwen
- A family of open-weight models developed by Alibaba, popular for its range of sizes and strong performance relative to size.
- Gemma
- A family of lightweight open-weight models released by Google, designed to run well on regular consumer hardware.
- DeepSeek
- A family of open-weight models known for strong reasoning and coding performance relative to their size.
- Mistral
- A family of open-weight models from the French AI company Mistral AI, known for efficient, capable smaller models.
Ready to put these terms into practice?
The best way to make this vocabulary stick is to see it in action. Install LM Studio, download a small model, and check the VRAM guide and model recommendations to pick your first one with confidence.
See how to get started