Image Classification
Fundamentals

Image classification is one of the fundamental tasks in computer vision. Given an image, a model learns to predict which category or categories it belongs to.

What Is Image Classification?

In image classification, a machine learning model receives an image as input and produces a class prediction as output. The model learns this relationship by studying many labeled examples during training.

Given an image, predict the class or classes that describe it.

For example, a model trained on images of cats and dogs may receive a new image and produce probabilities such as Cat: 0.97 and Dog: 0.03. The class with the highest probability becomes the prediction.

Types of Image Classification

# Multi-class classification
Cat:   0.05
Dog:   0.87  ← predicted class
Bird:  0.04
Horse: 0.04

# Multi-label classification
Dog:   0.96  ✓
Grass: 0.89  ✓
Car:   0.12  ✗

How Does a Computer Understand an Image?

A computer does not see an image as a human does. An image is represented as numerical pixel data. An RGB image contains three channels: red, green, and blue.

A typical image classification model may receive an input with the shape 224 × 224 × 3. During training, images are commonly resized, normalized, and converted into tensors.

# Typical image tensor shape
Batch × Channels × Height × Width

32 × 3 × 224 × 224

The Image Classification Pipeline

A typical classification system follows a sequence of steps:

1. Collect and Organize Data

Images are usually organized into classes. A common dataset structure looks like this:

dataset/
├── train/
│   ├── cats/
│   └── dogs/
├── validation/
│   ├── cats/
│   └── dogs/
└── test/
    ├── cats/
    └── dogs/

2. Preprocess Images

Common operations include resizing, normalization, tensor conversion, and optionally data augmentation.

Convolutional Neural Networks (CNNs)

CNNs are one of the most important architectures in computer vision. They learn visual features progressively, from simple patterns to complex structures.

Convolution

A convolutional filter moves across an image and detects patterns. Different filters can learn horizontal edges, vertical edges, curves, textures, and other visual structures.

Pooling

Pooling reduces the spatial size of feature maps. This decreases computation while retaining important information.

Fully Connected Classifier

After extracting visual features, the final layers convert those features into class probabilities.

ResNet and Residual Connections

As neural networks become deeper, training them can become difficult. ResNet introduced residual connections, also called skip connections, to help information and gradients flow through very deep networks.

Popular architectures include ResNet-18, ResNet-34, ResNet-50, and ResNet-101.

Vision Transformers (ViT)

Vision Transformers approach image understanding differently. Instead of processing the entire image as one continuous grid, the image is divided into smaller patches. These patches are treated as tokens and processed using self-attention.

A key advantage of attention is that the model can learn relationships between different parts of an image, even when they are far apart spatially.

The Training Loop

A model learns through repetition. It makes predictions, measures its error, computes gradients, and updates its parameters.

for images, labels in dataloader:
    predictions = model(images)
    loss = criterion(predictions, labels)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

Loss Functions

The loss function measures how different the model's prediction is from the correct label. The training process tries to minimize this error.

Cross-Entropy Loss

Cross-entropy is one of the most common loss functions for multi-class image classification. It penalizes confident incorrect predictions more heavily than uncertain predictions.

How Do We Evaluate a Classifier?

Accuracy is a simple starting point, but real-world evaluation often requires multiple metrics.

Accuracy =
Correct Predictions
───────────────────
Total Predictions

Confusion Matrix

                 Predicted
                 Cat    Dog
Actual  Cat       90     10
        Dog        5     95

Classification vs Other Computer Vision Tasks

A Simple PyTorch Classifier

The following example shows the basic structure of a small convolutional classifier.

import torch.nn as nn

class Classifier(nn.Module):

    def __init__(self, num_classes):
        super().__init__()

        self.model = nn.Sequential(
            nn.Conv2d(3, 32, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),

            nn.Conv2d(32, 64, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),

            nn.Flatten(),
            nn.Linear(64 * 56 * 56, num_classes)
        )

    def forward(self, x):
        return self.model(x)

Key Takeaways