ModelRefs / Convolutional Neural Networks — Tutorial
Convolutional Neural Networks — Tutorial
Filters, feature maps, and pooling — how CNNs see images and why they still matter. Covers The convolution operation, Pooling and the network architecture.
Overview
Filters, feature maps, and pooling — how CNNs see images and why they still matter
Level: Advanced. Estimated reading time: 45 minutes.
The convolution operation
A convolution slides a small filter (kernel) across the input, computing a dot product at each position. A 3×3 filter on a 28×28 image produces a 26×26 feature map (assuming no padding). Each position in the output captures whether the filter's pattern is present at that location.
Key properties: - Parameter sharing: the same filter weights are used at every position. A 3×3 filter has 9 weights regardless of image size. Compare to a fully-connected layer where each input pixel has its own weight — 784×128 = 100k parameters for a 28×28 image. - Translation equivariance: if a cat moves in the image, the activation map shifts accordingly but the detection still fires. - Local connectivity: each output neuron depends only on a small region (receptive field) of the input. Deeper layers have larger effective receptive fields.
Typical conv layer: Conv2d(in_channels, out_channels, kernel_size=3, padding=1). padding=1 keeps spatial dimensions the same.
Pooling and the network architecture
Pooling reduces spatial dimensions. Max pooling takes the maximum value in each 2×2 region, halving height and width while keeping the strongest activations. Average pooling takes the mean.
A standard CNN architecture: [Conv → ReLU → Conv → ReLU → MaxPool] × N → Flatten → Dense → Softmax.
Early layers detect edges and textures. Middle layers detect shapes and parts. Late layers detect objects and scenes. This hierarchy of features — from simple to complex — is why deep CNNs generalise so well.
Modern architectures go beyond this basic pattern: - Batch normalisation between conv and activation - Skip connections (ResNets) that add the input directly to the output: out = F(x) + x. These allow training networks 100+ layers deep without vanishing gradients. - Depthwise separable convolutions (MobileNet) that are 8–9× cheaper than standard convolutions
Practical CNN training tips
Data augmentation is essential. On small datasets, CNNs overfit quickly. Standard augmentations: horizontal flip, random crop, colour jitter, rotation ±15°. In PyTorch: torchvision.transforms. In Keras: keras.layers.RandomFlip, RandomRotation.
Input normalisation: subtract dataset mean and divide by std per channel. For ImageNet-pretrained models, use ImageNet's mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225].
Batch size: larger batches (128–256) are common for vision. Use a learning rate that scales with batch size (linear scaling rule: if batch doubles, double the learning rate).
Transfer learning: almost always better than training from scratch for less than 100k images. ResNet50, EfficientNet, and ViT checkpoints pretrained on ImageNet are available in torchvision.models and keras.applications.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Convolutional Neural Networks — Tutorial.