When your phone slows down because too many apps are open, it lags, gets warm, and everything takes twice as long to load. Machine learning has the same problem, just at a much bigger scale. The algorithms and models we’ve covered in this series all need physical computing power to run, and the type of hardware you use matters more than most people expect.
This is Part 4 of our Foundations series. In Part 1, we explored data. In Part 2, models. In Part 3, algorithms. Now we’re looking at the physical infrastructure that makes it all work.
The Basics
Your computer has four main components that matter for ML:
Memory (RAM) is your working space. It holds whatever you’re actively processing. More memory means you can work with larger datasets without your machine choking.
Storage (Hard Drive/SSD) is where data lives when you’re not using it. Bigger but slower to access than memory.
CPU (Central Processing Unit) is the main processor. It’s fast and versatile, but it handles tasks one at a time (or a few at a time). Think of it as one very skilled worker.
GPU (Graphics Processing Unit) was originally built for rendering video game graphics, but it turns out to be perfect for machine learning. Where a CPU is one skilled worker, a GPU is hundreds of less-skilled workers who can all work simultaneously. ML involves massive amounts of simple, repetitive math, which is exactly what GPUs are built for.
That’s the important distinction. CPUs are fast at complex tasks. GPUs are fast at doing thousands of simple tasks in parallel. Machine learning needs the second kind of speed.
The Speed Problem
When I first started working with machine learning in the late 90s, I was running simple neural networks on my desktop computer. The project was an image classification problem for breast cancer tumor detection, trying to train a model to distinguish between malignant and benign growths. Writing most of the code from scratch, dealing with hardware that was never designed for this kind of work. I left a training job running overnight and came back the next morning to find it had barely made progress.
A few days later, I got access to a server at work with more memory and a faster processor. Same code, same data. It finished in under an hour.
A Real Example
On a recent image recognition project, we wanted to train a model to spot defects in manufacturing parts using photos.
On a regular computer: loading 100 images took a couple of seconds, but processing them took 3 minutes. When we tried 1,000 images, the program crashed. Out of memory.
On a machine with a GPU and more RAM: we loaded 10,000 images at once and processed them in 30 seconds. Same code, same algorithm. Different hardware. That gap is why hardware matters.
Software Frameworks
Good hardware alone isn’t enough. You also need software that knows how to use it efficiently. Python frameworks like TensorFlow and PyTorch handle this for you. They manage how data flows through memory, how calculations get distributed across GPU cores, and how to avoid common performance bottlenecks.
You don’t need to understand the internals of these frameworks to use them. But knowing they exist and what they do helps you make better decisions about your ML setup.
Getting Started
You don’t need expensive hardware to start learning ML.
A regular laptop handles small projects fine. Free cloud services like Google Colab give you access to GPUs when you need more power. You pay nothing for casual use and only scale up when your projects demand it.
You need serious hardware when you’re processing large image or video datasets, training models from scratch (rather than using pre-trained ones), working with large text datasets, or need results fast. For everything else, start with what you have.
The Bigger Picture
Throughout this series, we’ve seen how the four foundations work together:
Data provides the raw material. Models capture the patterns. Algorithms find those patterns. And hardware makes the whole thing run at a speed and scale that’s actually useful.
These pieces depend on each other. A brilliant algorithm with bad data produces garbage. Perfect data with insufficient hardware just sits there. The value is in understanding how they connect, not in mastering any one piece alone.
