The Old Habits Survive Because the Failure Modes Survive

The field moved from AlexNet to ResNets to ConvNeXt to ViTs, and the training habits did not become optional. Old ConvNet advice still works because the failures it prevents did not change. Bad data, wrong normalization, unstable learning rates, leakage, weak baselines, and uninspected errors still break strong models.

Most practical ConvNet work is preparation. You clean the old machinery well enough that the model gets a fair chance.

Augmentation states which changes must not change the label

An augmentation is an invariance claim written in code. Random crops, flips, color jitter, and scale changes each assert that some transformation leaves the label alone. The classification note named what a vision system must stay invariant to: pose, lighting, small translation, occlusion, and clutter.

The constraint is taste. Augmentation must not change the task. Flip a street-sign dataset blindly and you can change the answer. Turn medical scans casually and you teach the model a lie.

Normalization must be identical between train and test except for stochastic augmentation

Subtract means. Scale channels. Run the same pipeline at test time, with the random transforms switched off. A model trained under one input distribution and served under another must improvise. Nothing crashes when the two pipelines disagree, so the bug shows up only as a worse number.

Stacks of small filters beat one large filter

Stacks of small filters build large receptive fields with fewer parameters and more nonlinearities than one giant filter. That is why repeated 3 by 3 convolutions became a durable pattern. The motif matters when you pick a backbone, because it sets parameter count at a fixed receptive field.

The learning rate is a schedule that changes during training

Warmups, step decay, cosine schedules, and careful restarts express one instinct. The correct step size at the start is the wrong step size later. An unstable rate looks like a bad architecture from the outside. Watch the curve before you blame the model.

A dull recipe beats a clever mess

Start from a known backbone and match preprocessing to it. Use pretrained weights when a good one exists. Train a simple head and keep the shapes honest. Get a validation baseline that behaves sanely before you tune anything.

Stack tricks after that. Augmentation, schedules, weight decay, label smoothing, mixup, and stronger pretraining can all help. Each one is easier to reason about once the baseline holds still.

The one-change rule is the same discipline the training note defined, one level further along the pipeline. Change augmentation, optimizer, batch size, and architecture at once, and you can win the run while learning nothing. That trade is fine for a desperate leaderboard run. It is bad for understanding.

Log every run. Save the config beside its checkpoint, track data versions, inspect calibration, and test on corruptions or distribution slices. Those checks guard the same split the classification note sealed. A dataset that changes under a run makes every comparison meaningless.

If you cannot name the dominant error mode, changing the architecture is theater

Accuracy reports the scoreboard. Error analysis decides the next intervention. Look at false positives, false negatives, confusion pairs, rare classes, lighting failures, background shortcuts, and corrupted inputs.

One error mode usually dominates. A newer model does not fix it until the boring checks pass.

The Builder Test

Name the dominant error mode from your last run in one sentence, with the slice it lives in. Without that sentence, run error analysis before you touch the architecture. The sentence is the permission slip for the change.

What Carries

Dull work buys the right to test an interesting architecture. Discipline decides whether a result is real. The last two notes ask what result you were trying to produce.