Convolutional Neural Networks (CNN)

Convolutional neural networks (CNNs) ruled computer vision for a decade — and their ideas (locality, weight sharing, feature hierarchy) are still required knowledge, even if today you train transformers.

Intuition: why a fully connected net is bad for images

Take a modest 224×224×3 image and a fully connected layer with 1000 neurons: 224·224·3·1000 ≈ 150 million weights in one layer. Beyond the cost, the network has to separately learn “a cat ear in the top-left corner” and “the same ear a bit to the right”: a fully connected layer has no translation invariance, every pixel is an independent feature to it.

Convolution fixes both problems with one trick: instead of connecting every neuron to every pixel, we take a small kernel (filter) — say 3×3 — and slide it across the whole image. At each position we compute the dot product of the pixel window and the kernel — one number. After all positions we get a feature map: a “response map” of the detector at every patch of the image.

How it works

Key properties and parameters:

• Weight sharing: the same kernel is used at every position. Instead of millions of weights — 9 numbers, and the “vertical edge” feature is detected anywhere in the image.
• Channels and filters: the input has several channels (RGB — three), so the kernel is really 3×3×C_in. A layer has many filters (32, 64, 128…), each looking for its own feature and producing its own map — the output has C_out channels.
• Stride — the sliding step: stride 2 halves the map.
• Padding — a border of zeros around the edges: without it the map shrinks (16×16 with a 3×3 kernel → 14×14); with “same” padding the size is preserved.
• Pooling — aggregating neighboring values (usually max over a 2×2 window): shrinks the map and adds robustness to small shifts.

Feature hierarchy and a typical architecture

The magic of a CNN is in a stack of convolutional layers. The first layer sees 3×3 windows and learns the simplest features: edges, brightness gradients. The second layer looks at the first layer's maps — its receptive field is wider, and it assembles edges into textures and corners. Further up — object parts (an eye, a wheel), and at the top layers — whole objects. This hierarchy edges → textures → parts → objects is not programmed: it emerges during training, and you can actually see it when visualizing filters.

A typical classifier architecture:

# [conv 3×3 → ReLU → conv 3×3 → ReLU → maxpool 2×2] × N
# then flatten → fully connected layer → softmax
x = pool(relu(conv2(relu(conv1(img)))))   # the block repeats, more channels each time
logits = fc(flatten(x))                   # classify from the compressed representation

With depth the spatial size of the maps falls (pooling/stride) and the number of channels grows: the network trades “where” for “what”.

CNNs today

In image classification, vision transformers (ViT) have pushed CNNs aside: they cut the image into patches and process them with attention, winning on large datasets. But it is too early to bury convolutions: they are still strong with small data (locality and weight sharing are built-in knowledge about the nature of images — an inductive bias), they live in mobile and realtime systems, and understanding receptive field, stride and weight sharing is asked in interviews regardless of what is fashionable now.

All interview questions in this section: Neural Networks — interview questions