Part 1: From Scratch to Systems
.
This machine learning series will be a real ride. It’s an interactive journey where I’ll be sharing and raising lots of questions while building real-world AI systems. My goal is to make this deeply engaging, drive home the right questions, and bridge the gap between AI theory and engineering reality.
Note: There will be follow-up videos complementing this series, so keep an eye out for video links as they are released!
Let’s start at the very beginning.
Differentiating the Field: Who Does What?
There is massive confusion in the tech industry right now regarding job titles. To understand what a Machine Learning Engineer (MLE) actually does, we have to look at how they fit alongside Data Scientists and AI Software Engineers.
1. The Machine Learning Engineer (MLE)
An MLE is a specialised software engineer responsible for researching, building, scaling, and deploying machine learning models. They sit firmly at the intersection of software engineering, data science, and infrastructure.
Their core responsibilities include:
Model Development: designing and training algorithms and deep neural networks.
Data Preparation: cleaning, organising, and transforming large datasets.
Production Deployment & MLOps: deploying models to the cloud and continuously monitoring their performance.
System Architecture: building the core application infrastructure so these models can be used reliably at scale.
2. The Data Scientist (DS)
A Data Scientist is primarily focused on research, statistics, and business analysis. While an MLE is focused on specialised software engineering and scalability, a Data Scientist spends their time exploring raw data, testing hypotheses, and building prototype models to prove a business concept works.
3. The AI Software Engineer
An AI Software Engineer focuses on integration and utility. They treat machine learning models as powerful building blocks. Their primary job is to build the application frontends, backends, and user experiences that consume AI capabilities (often via external APIs), rather than training the core models from scratch.
The Core Blueprint: What You Need to Know
Most successful ML Engineers start as traditional software engineers who specialise, or are builders who learn the entire technical stack from the ground up. If you want to excel as an MLE, these are the core requirements:
Programming Languages: Python is the non-negotiable standard of the discipline. However, depending on the organisation and performance needs, some MLEs are also highly skilled in Java or C++.
ML Frameworks: you must be comfortable navigating production-grade libraries like PyTorch, TensorFlow, Keras, and scikit-learn.
Mathematics & Statistics: This is the area where many engineers fall short. While you might not be quizzed on complex proofs in every interview, understanding linear algebra, calculus, and probability is vital for debugging model behaviour.
Infrastructure & Tooling: You need to be comfortable with cloud computing platforms (GCP, AWS, etc.). Version control (Git/GitHub) is essential for managing your work, and Docker is effectively mandatory for containerising applications.
Understanding the Data
Before a system can learn, we have to understand the raw material. Data is broadly classified into two categories:
Structured Data: highly organised data that easily fits into rows and columns (e.g., CSV files, SQL databases).
Unstructured Data: complex data that does not have a pre-defined structure (e.g., images, audio, and natural language text).
The mechanics, architectures, and training pipelines required for unstructured data are entirely different from those used for structured data.
The Machine Learning Engineering Pipeline
Here is the bird’s eye view of how a model evolves from raw data into a living system:
Data ➔ Clean & Transform ➔ Analyse ➔ Baseline vs. Custom Model ➔ Train & Test ➔ Containerise & Deploy ➔ Expose as API
1. Ingestion & Analysis
Gathering Data: highly dependent on your organisation’s size. Sometimes this is handled by Data Engineers, but an MLE must know how to pull and ingest data independently.
Cleaning and Organising: raw data is messy. Here, data is cleaned, structured, and transformed into features. We will dive incredibly deep into this layer later in this series.
Exploratory Data Analysis (EDA): an MLE must thoroughly analyse data distributions and shapes before moving to the modelling phase.
2. The Modelling Strategy (Baseline vs. Custom)
When designing a system, you never just train a model blindly. You start with a baseline model.
A baseline can be an existing state-of-the-art architecture or a previously developed model. The ultimate goal of training a custom model is to prove it can outperform your baseline. If your complex custom model performs worse than a simple baseline, it shouldn’t go into production.
3. Training, Testing, and the “Skew” Bug
Training is heavily dependent on the data type and requires a deep understanding of concepts like epochs, loss functions, and regularisation techniques such as dropout (which help prevent overfitting).
Once trained, we must rigorously evaluate the model. Beyond traditional software bugs, MLEs face unique production challenges like training-serving skew.
What is training-serving skew?
This occurs when the data a model encounters in production (real life) is significantly different from the data it was trained on. This causes performance to plummet and can lead to the model producing unreliable or low-quality predictions. An MLE’s job is to ensure the training data is adequate and representative to minimise this skew.
You must also master evaluation metrics to track success, including the confusion matrix (precision, recall, accuracy, F1-score) and mAP (mean average precision) for spatial / object-detection systems.
4. Conversion to APIs & Building Systems
A model sitting on your local machine is useless. An MLE transforms a trained model into a lightweight, usable form by converting it into an API (using frameworks like FastAPI).
This makes the model highly accessible, allowing other software engineers or IoT infrastructures to seamlessly integrate its AI capabilities into a broader system.
5. Production Testing (Ensuring Utility)
Our job isn’t done just because the system runs. It must be reliable, secure, and useful. A system builder must implement a strict testing hierarchy to protect against vulnerabilities and low-standard features:
Unit Testing: testing individual components and functions in isolation.
Integration Testing: ensuring the data pipeline, API, and model interact perfectly.
Smoke Testing: rapidly verifying that the core infrastructure doesn’t crash upon deployment.
Testing isn’t just the icing on the cake; it is the foundational requirement for safety, reliability, and engineering confidence.
6. The Core Paradigms: From Linear Regression to Clustering
As we dive deeper into modelling throughout this series, we will break down the core learning paradigms that power these systems. You can’t build advanced architectures without mastering the fundamentals:
Supervised Learning: training models on labelled data (where the system knows the correct answers during training). This includes foundational algorithms like linear regression for predicting continuous values, all the way up to complex neural networks.
Unsupervised Learning: giving the model raw, unlabelled data and letting it discover hidden structures on its own. A prime example of this is clustering, where the system automatically groups similar data points without human intervention.
What’s Next?
Throughout this series, we are going to cover all of these concepts in depth. We will write the code, sketch the architectures, look at how these systems impact modern workplaces, and discuss how to position these skills to land an elite MLE role.
Buckle up, this is going to be an incredible build.
