Environmental Impact Is a Design Constraint

Inference can outspend the training run that produced the model. The training bill closes when the run ends. The serving bill reopens on every call, and it keeps reopening for as long as people use the model.

Models run on physical infrastructure that spends energy, materials, water, land, and attention. Once a model serves billions of tokens, the serving side of that bill passes the training side. Compute is part of the product, so its cost belongs in design review.

Many teams raise environmental impact as an apology after the benchmark lands. By then model size, the serving path, and the growth in demand are already fixed. The question belongs earlier, while the design is still soft. What did this capability cost, and was the cost worth it?

Compute Has Two Lives

Training emissions are easy to narrate because the run is visible. It starts, it ends, and someone can put a number in a paper. Inference is harder to see and easier to leave out. It spends power on every call, and the calls do not stop.

The training number is a fact about the past. The serving number is a forecast about demand, and it moves every time the product gets more users.

After that crossing, an argument about training efficiency is an argument about the smaller half. The footprint becomes a research question and a product question at once. The product side owns the traffic, and traffic is the term that grows.

The Life Cycle Is Wider Than The Run

Life cycle assessment widens the boundary past the training job. Hardware manufacturing, data center energy, cooling, grid carbon intensity, development experiments, inference traffic, and hardware retirement all sit inside it. Leave one out and the total is a guess.

Energy use depends on hardware efficiency, utilization, run duration, power draw, and cooling overhead. Emissions depend on the carbon intensity of the grid that powered the work. The same job on a different grid emits a different amount. Idle capacity spends power while it waits for work that has not arrived.

Embodied carbon counts from manufacturing to disposal, and the manufacturing half is spent before the first token exists. The runs that failed count too. Hyperparameter sweeps and repeated experiments spend real power, and they rarely reach the headline figure.

The parallelism note put its cost under one rule: nothing is free once it has to move. Energy is that rule in another unit. Power moves in and leaves as heat, and cooling overhead pays to remove it.

Report Compute, Energy, Location, Carbon Intensity, And Method

Treat emissions as a product metric. Estimate them before training, record them during the run, and report them after deployment. A team that cannot state its energy and carbon assumptions cannot price the trade it is making.

A usable report names six things: compute, energy, location, carbon intensity, hardware, and the estimation method. Location is there because the grid sets the carbon in every unit of energy. Method is there because all six numbers are estimates. Publish the method beside the total and a reader can check the assumption.

Efficiency Can Raise Total Use

A cheaper token invites more tokens. Efficiency lowers the cost of one call and can raise the total, because demand expands into whatever the price allows. The saving is real only when total work goes down.

The scaling-laws note weighed a marginal parameter against the loss it bought. Price that same marginal gain in energy and some of those trades stop clearing. Traffic decides how often the cost repeats, so the answer moves as the product grows.

Design Choices Move The Footprint

The alternatives are the ordinary ones: smaller models, distillation, retrieval, routing, caching, quantization, batching, and task-specific systems. Each one removes work the product never needed. The greenest token is the one you did not need to generate.

Two of those arrived earlier in this unit. Routing and retrieval come from the selective-architectures note. Waking a few experts per token cuts the compute each token spends. Moving knowledge into an index keeps it out of the weights.

The adaptation note added the rule that fits here: take the smallest rung that clears the reliability bar. Both notes argued for the lower-footprint option before energy entered the argument.

A team that measures picks model size, caching, batching, distillation, and deployment differently from a team that guesses.

The Builder Test

Draw the life cycle of one system you run: training, evaluation, deployment, repeated inference, hardware, and retirement.

• Split the compute into training experiments, failed runs, and serving traffic.
• Mark the stages where you can state a number today, and the stages where you are guessing.
• Find the design change that removes the most repeated work, and price it in energy.

The pile that grows is the one to design against. The stages you are guessing at are the ones to instrument first.

What Carries

Carry the full bill. The greenest system avoided computation nobody needed. Useful work per unit of cost is how that shows up in a review.

Take one deployment claim and trace it from the corpus to the carbon. If it survives the whole trip, the claim is worth defending.