Machine Learning

Here is the truth about machine learning: it is curve fitting with very good bookkeeping. You choose a family of functions with adjustable knobs, show it examples, and nudge the knobs until its outputs match the examples. Everything from a spam filter to a large language model is that idea, scaled up.

What it is, in plain words

Ordinary programming: you write the rules, the computer applies them to data. Machine learning: you give the computer data and the answers, and it finds the rules.

The three big families:

Supervised learning – examples come with labels (this email is spam, this house sold for X). The model learns to predict the label. Regression predicts numbers; classification predicts categories.
Unsupervised learning – no labels. Find structure: clusters, the main directions of variation (PCA), unusual points.
Reinforcement learning – an agent acts, gets rewards, and learns a policy that earns more reward.

Since 2017 one architecture, the transformer, has taken over language, images, audio and code. Today’s large language models are transformers trained first on huge amounts of text to predict the next token (self-supervised learning), then tuned with human and AI feedback. But underneath they use exactly the same ingredients as the list below: data as vectors, a model, a loss, and optimisation.

Why it matters

It powers search ranking, recommendations, translation, speech recognition, fraud detection, medical imaging and the AI assistants everyone now uses.
It is statistics, not magic. A model only knows the patterns in its training data, including the biases and mistakes. Understanding the basics is how you judge its claims.
The mathematics is learnable. The outline below (linear algebra, calculus, probability, optimisation) is the whole foundation; nothing else is hidden.

An everyday analogy: learning to throw darts

Your first throws go everywhere. After each throw you see how far off you were (the loss) and adjust your arm a little in the direction that would have helped (the gradient). After hundreds of throws you are hitting the board. You never wrote down the physics of darts. Your arm learned a function from “where I aim” to “where it lands”. That is gradient descent.

The core equation

Linear regression is the “hello world” of the field. Given inputs $x$ and targets $y$ , find weights $w$ and bias $b$ so that $\hat{y} = w x + b$ is close to $y$ . “Close” means minimising the mean squared error:

L(w, b) = \frac{1}{n} \sum_{i=1}^{n} \left( w x_i + b - y_i \right)^2

Gradient descent repeats a small step downhill, with learning rate $\eta$ :

w \leftarrow w - \eta \frac{\partial L}{\partial w}, \qquad b \leftarrow b - \eta \frac{\partial L}{\partial b}

A neural network is the same thing with many layers of weights and a non-linear function between them.

A hands-on example

Gradient descent from scratch with NumPy, then the same with scikit-learn:

import numpy as np

rng = np.random.default_rng(0)
x = rng.uniform(0, 10, 100)
y = 3 * x + 2 + rng.normal(0, 1, 100)       # true w=3, b=2, plus noise

w, b, eta = 0.0, 0.0, 0.01
for step in range(2000):
    err = w * x + b - y
    w -= eta * 2 * np.mean(err * x)          # dL/dw
    b -= eta * 2 * np.mean(err)              # dL/db
print(f"learned w={w:.2f}, b={b:.2f}")      # close to 3 and 2

from sklearn.linear_model import LinearRegression
m = LinearRegression().fit(x.reshape(-1, 1), y)
print(m.coef_, m.intercept_)

Install with pip install numpy scikit-learn in a virtual environment (python -m venv .venv). For deep learning the standard library in 2026 is PyTorch, with JAX as the main alternative. You can follow the same loop by hand even in PicoLisp with fixed-point numbers, which is a good way to prove to yourself there is no magic.

Common misconceptions

“More data always fixes it.” More bad or biased data makes a confident wrong model. Quality and coverage matter more than size.
“High accuracy means a good model.” If 99% of emails are not spam, a model that always says “not spam” is 99% accurate and useless. Look at precision, recall and the confusion matrix, and always test on data the model has not seen.
“The model understands.” It has learned statistical patterns. Sometimes those patterns are deep and useful; it can still be confidently wrong.
“ML means neural networks.” For tabular business data, gradient-boosted trees (XGBoost, LightGBM) still often win. Start simple.
“You need a GPU to start.” Everything on this page runs on a laptop CPU in seconds.

Connections

Python, NumPy, SciPy, IPython, Python spaCy for language.
Linear Algebra, Vector Spaces, Linear Operators and Matrices, Inner Product Spaces, Tensors.
Algorithms, Data Structures and Algorithms, Julia.

Machine learning in a Nutshell:

Data: Raw Environment
Model: Maps input –> Output
Learning: Optimizes Models by finding better parameters to models (patterns).
Predictor: A MLA that is a system that makes predictions
We Represent Data as Vectors
Choose a Model
Optimize using numerical methods

The mathematics needed:

Linear Algebra
- Systems of Equations
  - Matrices
  - Vector Spaces
  - Linear Mappings
  - Affine Spaces
Analytic Geometry
Matrix Decomposition
Vector Calculus
Probability Distributions
Continuous Optimisation
Models vs Data
Linear Regression
Dimensionality
Principal Component Analysis
Density Estimation (Gaussian Mixture Models
Classification with Support Vector Machines