Vision Is Representation Under Pressure
Every method here answers one question: what representation survives an image nobody arranged for the model. Pose, lighting, scale, and occlusion decide the answer.
Deep Learning for Computer Vision.
Pixels to representations, and what breaks them
Notes
Sorted in orderImage Classification Is a Promise About Generalization
Image classification is a promise that the rule holds on the next image. The split is the only instrument that tests the promise, and it leaks in two ways.
One Template Per Class Is the Whole Limit
A linear classifier throws away the training set and keeps a weight matrix. The price of that compression is one averaged template per class.
Optimization Is How Models Pay for Being Wrong
Optimization is repeated local correction. The loss curve is the instrument that names which correction is broken, and it speaks before final accuracy does.
Backpropagation Is Blame Accounting
Backpropagation is the chain rule kept as bookkeeping. Almost every bug in it is a wrong shape, a stale cache, or a missing batch average.
Neural Networks Are Learned Feature Machines
A hidden layer is a learned change of coordinates, and its only defense is that it makes the final comparison easier. Remove the nonlinearity and the stack collapses into one linear map.
Most Training Failures Are Setup Failures
Most failures that look like model failures are setup failures. Data scale, initialization, normalization, regularization, and the loss must agree with each other.
Training Neural Nets Without Lying to Yourself
Training is an inspection loop. A network that cannot memorize ten examples has not earned a larger dataset, and no optimizer repairs bad labels or leakage.
At Two Dimensions Nothing Hides
A spiral in two dimensions is the cheapest place to watch a linear model fail and a hidden layer fix it, and at that size no part of the loop can hide.
Convolutional Networks Exploit the Shape of Images
Convolution writes one claim about images into the architecture: useful patterns are local, and they repeat across space. The prior is the advantage, and a wrong prior is a cost already paid.
Visualizing ConvNets Is Debugging the Representation
A visualization earns its place only when it changes a decision. Its job is to catch the model using the wrong evidence, and the test you run afterward is the only proof it produced anything.
Transfer Learning Is Buying a Head Start
Two numbers set the plan: how much labeled data you have, and how far it sits from the pretraining set. Where you freeze and what you fine-tune follow.
RNNs Remember by Carrying State
An RNN carries one compressed summary forward, and that compression is both the mechanism and the ceiling. Gates buy the state a straighter path through time.
Attention Routes Context Instead of Compressing It
Queries ask, keys advertise, values answer. A captioning decoder builds a different context vector for every word instead of reading one compressed summary.
Transformers Turn Attention Into an Architecture
A vision transformer trades convolution's locality and weight sharing for learned routing between patches, and the training recipe pays the difference.
Generative Models Learn the Shape of the Data
Autoregressive models buy likelihood and pay in sampling speed. VAEs organize the latent space and blur. GANs sharpen and collapse. Name the metric first.
NeRF Learns a Scene as a Function
NeRF stores a scene as a function you query, and posed photographs are enough supervision because rendering is differentiable.
Adversarial Examples Expose the Gap Between Scores and Sight
The gradients that trained the model are the gradients that break it. FGSM takes one step, PGD takes many, and a defense means nothing until you name the attacker.
The Old Habits Survive Because the Failure Modes Survive
Old ConvNet advice still works because the failures it prevents did not change: bad data, wrong normalization, unstable learning rates, leakage, weak baselines, and uninspected errors.
Exact Likelihood Has a Price in Order and Speed
An autoregressive image model buys an exact likelihood and pays in the order it commits to and the sampling speed it gives up. The mask is what keeps the contract honest.
Name the Job Before You Name the Family
Generative modeling is a set of contracts, and the contract you name chooses the model family. Name the metric, then name the failure it hides.