Convolutional Networks Exploit the Shape of Images

Convolution writes one claim about images into the architecture: useful patterns are local, and they repeat across space. That claim is the prior. The note on hidden layers called architecture a bet about the structure of the data. Convolution is that bet written into the wiring.

A ConvNet refuses to flatten the image for as long as it can

A fully connected network treats an image like a long spreadsheet row. Width, height, and channels collapse at the door. Flattening keeps every pixel value and loses the arrangement.

A ConvNet holds on to the grid until the classifier. Nearby pixels are related. The same edge detector is useful in many locations. Convolution puts both facts into the architecture rather than making the weights find them again.

Classic stacks repeat conv, nonlinearity, and pooling, then hand the result to a classifier. Later stacks changed the filters, the depth, and the connections. The constraint stayed the same. A network must keep spatial structure while it turns pixels into class evidence.

A local receptive field is the whole first assumption

A convolutional neuron looks at one small region of the input. That region is its receptive field. A 3 by 3 filter asks a small question. A 5 by 5 filter asks a larger one and pays more parameters for the answer.

Early visual evidence is local. Edges, corners, colors, and textures do not need every pixel at once. Full connectivity in the first layer buys connections the image cannot justify.

Stack the layers and the region one unit sees grows with depth. The receptive field at the last conv layer is something you compute.

Weight sharing is where the efficiency comes from

The same filter slides across the image, so one learned detector runs at every position. A horizontal edge that matters in the top-left probably matters in the bottom-right, and one set of weights covers both.

The note on linear classifiers priced the other arrangement. One row of the weight matrix is one class template. That template has to average every pose, background, and lighting the class shows up in. Weight sharing is the opposite trade. One detector gets reused across many positions, so position stops eating capacity.

Without sharing, a first conv layer explodes into millions of parameters. With sharing, you learn one filter and apply it everywhere.

The output size formula decides whether the layers fit

Stride sets how far the filter moves each step, and a larger stride shrinks the output. Padding adds border values so filters fit at the edges and the spatial size survives the layer.

With input width W, filter size F, padding P, and stride S, one line gives the output width.

output_size = (W - F + 2 * P) / S + 1

A non-integer result means the layer does not fit cleanly. The same arithmetic decides memory and how fast spatial information shrinks. Track it per layer or architecture design becomes guesswork.

The backpropagation note called the boring bugs shapes, caches, and missing averages. This is the same discipline moved up from one tensor to a whole stack. A layer that does not fit is a shape bug you can catch before any data moves.

Pooling discards detail on purpose, so name the trade

Max pooling keeps the strongest activation in a region and drops the rest. It cuts compute and buys tolerance to small shifts.

The image classification note listed what a vision system must survive: pose, lighting, small translation, occlusion, and clutter. Convolution and pooling buy the small translation. Everything else on that list, plus class imbalance, still lands on the data and the loss. Locality says nothing about a cat photographed from above.

im2col is the systems compromise that made convolution fast

A convolution is many dot products between filters and local image patches. im2col unfolds those patches into columns, reshapes the filters into rows, and turns the layer into one matrix multiply.

The cost is duplicated input values, because the same pixel lands in several columns. The return is that the implementation runs on optimized matrix multiply kernels. Budget the duplicate copies next to the activations, since both fall out of the same layer arithmetic. This compromise, more memory for a faster kernel, appears everywhere in deep learning.

The Builder Test

Take the stack you plan to train and do the arithmetic by hand before you run it.

• Compute output width and height at every layer, and stop at the first non-integer result.
• Say what one unit in the last conv layer sees in the original image.
• Name what pooling discarded on the way there, and name the task that needs it back.

A blank answer on the third line means the pooling layer arrived by habit. Change it or defend it before you spend a training run on it.

What Carries

The prior is the advantage. A ConvNet starts with locality and reuse already assumed, and it spends its capacity on which filters are worth keeping.

A prior that does not match the data is a cost you already paid. Locality and reuse hold on natural images. On data whose useful pattern is global, or tied to one fixed position, the constraint returns nothing.

A matching prior still lets a network learn the wrong evidence. The next job is to look at what it learned, one image at a time.