Artificial intelligence has moved from academic abstraction to everyday tool in the span of a few years, yet the question of what it actually is still trips up engineers, hobbyists, and executives alike. The short answer: modern AI is pattern recognition at scale, built on statistical models that learn from data rather than explicit rules. The longer answer — including what hardware you need, how the math works, and what "AI" means in practice for makers building local-model pipelines in 2026 — follows below.
What Artificial Intelligence Actually Is
AI, in its contemporary form, describes systems that learn statistical relationships from large datasets and use those relationships to make predictions or generate outputs. The dominant paradigm in 2026 is deep learning: multi-layered artificial neural networks trained on text, images, code, audio, or structured data.
The transformer architecture — introduced in the 2017 paper "Attention Is All You Need" by Vaswani et al. at Google Brain, published on arXiv — underpins virtually every large language model (LLM) released since 2020, from GPT-class models to open-weight alternatives like Llama, Mistral, and Qwen. Transformers use a mechanism called self-attention to weigh relationships between tokens in a sequence, enabling a model to understand context across long passages rather than just adjacent words.
What makes 2026 AI different from the rule-based expert systems of the 1980s or the narrow convolutional networks of the 2010s is scale and generality. Per the Stanford HAI AI Index Report, compute used in frontier model training roughly doubled year-over-year through the mid-2020s. The resulting models exhibit emergent capabilities — abilities not explicitly trained for — that have made them useful across domains from protein structure prediction to code generation.
Machine Learning vs. Deep Learning vs. "AI"
These terms are frequently conflated:
| Term | Scope | Example |
|---|---|---|
| Artificial Intelligence | Broad umbrella for machine-driven cognition | Any chatbot, self-driving system, or image classifier |
| Machine Learning | AI subset: learns from data rather than rules | Random forests, gradient boosting, SVMs |
| Deep Learning | ML subset: uses multi-layer neural networks | LLMs, diffusion image models, speech recognition |
| Generative AI | Deep learning that produces new content | ChatGPT, Stable Diffusion, GitHub Copilot |
For makers, "AI" in practice usually means one of three things: running an LLM locally (inference), fine-tuning a model on custom data, or building edge pipelines that classify sensor data in real time.
How AI Math Differs from Traditional Computing
Traditional software follows deterministic logic: if X then Y. AI systems follow probabilistic inference: given input X, the model produces output Y with some confidence score, based on billions of learned parameters.
The underlying operation is matrix multiplication — stacking inputs as vectors and multiplying through learned weight matrices layer by layer. This is why GPUs dominate AI workloads: they execute thousands of parallel multiply-accumulate operations simultaneously, the same parallel math that shades a 3D game frame, just applied to neural network activations instead of vertices.
Why FP16 and BF16 Matter
Full 32-bit floating-point (FP32) operations consume more memory and run more slowly than 16-bit alternatives. Modern AI frameworks — PyTorch, JAX, TensorFlow — default to mixed-precision computation: FP16 or BF16 for most arithmetic, FP32 for gradient accumulation where numerical precision is critical. Per TensorFlow's mixed-precision documentation, 16-bit computation can roughly halve memory consumption and increase throughput on hardware with native FP16 support, without meaningful accuracy loss for most model families.
For inference — running a trained model rather than training it — INT8 and INT4 quantization go further. A 70B-parameter model's memory footprint can shrink by 4–8× compared to FP16, fitting it onto consumer VRAM. Tools like llama.cpp, GGUF quantization format, and HuggingFace's bitsandbytes library have made this accessible to anyone with a capable GPU.
AI Hardware for Makers: What You Actually Need
AI hardware requirements depend entirely on the workload. Three maker-relevant tiers define the landscape:
Tier 1 — Local Inference (Running Models)
Running a 7B-parameter LLM at full FP16 precision requires roughly 14 GB of VRAM (7 billion parameters × 2 bytes each). At INT4 quantization, that shrinks to approximately 4–5 GB, putting capable local models within reach of mid-range consumer GPUs. Image-generation pipelines (Stable Diffusion, FLUX) need 6–12 GB depending on resolution and model architecture.
Hardware that makes local AI practical in this tier:
- 8–12 GB VRAM GPUs: Comfortable for quantized 7B LLMs and 512px image generation
- 16–24 GB VRAM GPUs: Unquantized 13B models, faster image generation at 1024px and above
- SBCs with dedicated NPU co-processors (Raspberry Pi 5 + Hailo-8, NVIDIA Jetson Orin Nano): Real-time edge inference for computer vision and audio classification
For context on what's achievable at the constrained end of maker hardware in an entirely different domain, see the SpecPicks guide on Ethernet and WiFi routing on a Pi Pico 2W — a reminder that maker AI spans everything from sub-$10 microcontrollers to multi-GPU workstations.
Tier 2 — Fine-Tuning
Fine-tuning a pre-trained model on custom data requires more VRAM than inference, because the system must store not just model weights but optimizer states and gradients. Parameter-efficient fine-tuning (PEFT) methods like LoRA and QLoRA reduce this substantially — QLoRA can fine-tune a 7B model on a consumer 24 GB GPU by quantizing the frozen base while training low-rank adapters in FP16.
For serious fine-tuning workloads, community benchmarks on forums like r/LocalLLaMA consistently point to 24+ GB VRAM as the practical entry point, with multi-GPU configurations required for models above 30B parameters.
The SpecPicks feature on doubling RTX 3070 VRAM via hardware hacks explores one community-driven approach to stretching available memory — a niche but instructive example of how makers push hardware past stock specifications.
Tier 3 — Data-Center Accelerators
For workloads at the scale of LLM pre-training from scratch, hardware shifts to specialized server-grade accelerators:
| Accelerator | Architecture | Memory | Primary Target |
|---|---|---|---|
| AMD Instinct MI300X | CDNA 3 | 192 GB HBM3 | LLM training, large-batch inference serving |
| NVIDIA H100 SXM5 | Hopper | 80 GB HBM3 | LLM training, scientific AI |
| NVIDIA B200 | Blackwell | 192 GB HBM3e | Frontier model training |
These are server-grade cards costing tens of thousands of dollars — outside typical maker budgets but essential context for understanding what training GPT-scale models actually requires. AMD's ROCm software stack has matured substantially, and MLCommons' MLPerf Inference benchmarks now include Instinct submissions across multiple task categories. The SpecPicks comparison of AMD Ryzen AI Halo vs NVIDIA DGX Spark vs RTX 3060 grounds these scales against more accessible hardware options.
Practical AI Workloads for Makers
Local LLM Inference
Running open-weight models locally has become the default entry point for maker AI projects. The toolchain is mature: ollama abstracts model management, llama.cpp runs GGUF-format models across CPU, GPU, or hybrid setups, and HuggingFace's model hub provides tested hardware configurations per model. A GPU with 16 GB VRAM can serve a capable coding assistant, document Q&A pipeline, or conversational interface for home automation.
The economics of local versus cloud inference have shifted substantially. The SpecPicks piece on DeepSeek V4's task cost and what it means for local AI rigs covers the crossover point in detail — very high call volumes favor on-premise hardware, while sporadic use often favors API access.
Image Generation
Diffusion models (Stable Diffusion, FLUX) translate text prompts into images through iterative noise removal. VRAM determines maximum resolution, batch size, and generation speed. Community measurements from Automatic1111 and ComfyUI forums indicate that 1024×1024 FLUX.1-dev generation runs in the range of 20–40 seconds on a 16 GB consumer GPU, with faster times on cards with higher memory bandwidth and tensor throughput. These figures vary by scheduler, step count, and quantization applied.
Edge AI and Embedded Inference
Edge AI runs classification or detection models directly on devices at the sensor — no cloud required, no round-trip latency. Hailo-8 and Google Coral TPU are the dominant co-processors for this tier; Raspberry Pi 5's PCIe slot enables Hailo-8 M.2 integration for efficient on-device neural-network inference. ONNX and TensorFlow Lite serve as cross-platform inference runtimes for these pipelines.
For a sense of how far embedded hardware hacking has come, the SpecPicks feature on Linux booting on a Sega Genesis is a useful — if extreme — proof of concept for running real software stacks on constrained hardware.
Code Generation and Agentic Pipelines
The most practical maker AI workflow in 2026 is agentic coding: a model plans tasks, writes code, executes it, reads output, and iterates. Tools like Claude Code and GitHub Copilot's workspace mode have demonstrated this at production scale. The SpecPicks article on Claude Code and Fable 5 porting Command & Conquer to iOS shows what agentic coding looks like at the hobbyist-to-professional boundary, and what local hardware setup enables it.
Benchmarking AI Performance
If you are evaluating hardware for an AI workload, the metrics that actually matter differ from gaming GPU comparisons:
| Metric | What It Measures | Why It Matters |
|---|---|---|
| FP16 TFLOPS | Peak 16-bit math throughput | Throughput ceiling for training and batched inference |
| Memory bandwidth (GB/s) | Weight transfer speed to compute units | Often the real bottleneck for LLM token generation |
| VRAM (GB) | Total on-device memory | Sets maximum model size without CPU offloading |
| Tokens per second (tok/s) | LLM generation speed at a given batch size | Practical inference quality-of-life metric |
| TFLOPs/Watt | Compute efficiency | Operating cost for always-on workloads |
MLPerf, maintained by MLCommons, is the standard benchmark suite for AI hardware. Results are published across Inference and Training tracks for data-center and edge hardware, with all participating vendors' submissions publicly available. For consumer GPU comparisons, community benchmarks tracking llama.cpp tokens per second across GPU models are more practical than vendor-reported peak FLOPS — the gap between rated peaks and real-world inference throughput is significant.
A key caveat: TFLOPS figures are theoretical peaks under ideal conditions. Memory bandwidth, software stack maturity (CUDA vs. ROCm vs. Metal), and quantization format all affect real-world numbers. Vendor TFLOPS should be treated as a ceiling, not a performance projection.
Choosing Hardware by Workload
| Workload | Minimum VRAM | Recommended |
|---|---|---|
| Quantized 7B LLM (INT4) | 4–6 GB | 8 GB |
| Unquantized 13B LLM (FP16) | ~26 GB | 32 GB |
| Stable Diffusion 1.5 | 4 GB | 8 GB |
| FLUX.1-dev at 1024px | 12 GB | 16 GB+ |
| Fine-tune 7B via QLoRA | 10 GB | 24 GB |
| Edge vision (co-processor) | N/A | PCIe x1 slot |
What "AI" Means for the Maker Community in 2026
The practical upshot is that makers in 2026 have access to capabilities that would have required a research institution five years ago. Open-weight models like Llama 3.1, Qwen 2.5, and Mistral NeMo run on commodity GPUs with sub-second response latency. Diffusion models generate useful design assets. Voice pipelines combining transcription, language models, and text-to-speech fit on a single small-form-factor system.
The Hackaday Europe 2026 retro PC build is a useful reminder that maker culture has always been about pushing capability onto constrained or unexpected hardware — AI is the current frontier, but the ethos of building and understanding from first principles is unchanged.
Understanding what AI is also matters because the decisions it enables are increasingly consequential. As Sony moves toward ending physical media production — covered in detail here — AI-driven content delivery and recommendation systems are central to that transition's economics. The infrastructure beneath everyday consumer software is increasingly AI-dependent, which makes the question "what is AI?" more than academic curiosity for anyone building the next layer of tools.
FAQs
Q: Do I need a dedicated GPU to run AI locally?
Not always. Quantized models in GGUF INT4/INT8 format run on CPU-only machines via llama.cpp, though throughput is much lower than GPU inference. Community measurements indicate a modern laptop CPU generates roughly 5–15 tokens per second with a quantized 7B model; a mid-range discrete GPU reaches 40–100+ tokens per second. For image generation, a GPU is effectively required for practical iteration speed.
Q: What is the difference between AI training and AI inference?
Training builds a model by adjusting billions of parameters across millions of examples — a GPU-intensive process measured in hours to months. Inference runs a finished model on new inputs — faster, cheaper, and what most maker projects involve. Most hobbyists run inference on pre-trained open-weight models rather than training from scratch.
Q: How much does running AI locally cost versus using an API?
Hardware amortized over two to three years typically beats API costs at high call volumes. At low volumes — a few hundred queries per day — cloud APIs (many with free tiers) are often cheaper. The crossover depends on the model tier and API pricing, and has shifted considerably as API prices have dropped through 2024–2026.
Q: What is an LLM?
A Large Language Model is a transformer-based neural network trained on large corpora of text to predict the next token in a sequence. By doing this across trillions of tokens, the model learns grammar, facts, reasoning patterns, and coding conventions — enabling it to generate coherent, contextually appropriate responses.
Q: Is running AI on a Raspberry Pi viable?
Viability depends on the task. Raspberry Pi 5 paired with a dedicated neural processing co-processor handles real-time object detection efficiently. Running a quantized 7B LLM on the Pi 5's CPU alone yields roughly 1–3 tokens per second — technically functional, practically slow for interactive use. Specialized edge AI hardware or a small x86 system with a discrete GPU is more practical for LLM inference at the edge.
Q: What is the difference between AI and machine learning?
Machine learning is a subset of artificial intelligence — all ML is AI, but not all AI is ML. Traditional rule-based expert systems qualify as AI in the broad sense without any machine learning. In contemporary usage, "AI" almost always refers to machine learning systems, particularly deep learning, because these dominate the current technology wave.
Citations and sources
- https://arxiv.org/abs/1706.03762 — Vaswani et al., "Attention Is All You Need" (2017): the foundational transformer architecture paper
- https://hai.stanford.edu/ai-index-report — Stanford HAI AI Index Report, tracking AI compute and capability trends annually
- https://mlcommons.org/benchmarks/ — MLCommons MLPerf benchmark results for AI hardware across training and inference tracks
- https://www.tensorflow.org/guide/mixed_precision — TensorFlow documentation on mixed-precision (FP16/BF16) training methodology
- https://huggingface.co/docs/transformers/index — HuggingFace Transformers documentation and community model cards with hardware configurations
- https://github.com/ggerganov/llama.cpp — llama.cpp: CPU/GPU LLM inference via GGUF quantization, with community benchmark results
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.