MLOps & Infrastructure
-
Batch Inference Is the Cheapest Optimisation Most Teams Skip
Work that does not need an immediate answer costs substantially less when submitted asynchronously, and much of a typical workload…
Read More » -
Your GPU Is Idle: Why Inference Throughput Is Memory-Bound
Language model inference rarely saturates GPU compute. It saturates memory bandwidth, and understanding that changes every optimisation decision you make…
Read More » -
Feature Stores Solve Training-Serving Skew, Not Data Access
Adopting one to organise features is overkill. Adopting one because training and serving compute features differently is addressing the real…
Read More » -
Deploy a New Model the Same Way You Deploy New Code
Swapping the production model in one step means every user meets the new version simultaneously, with no chance to catch…
Read More » -
Model Monitoring Needs the Signals That Arrive Before Labels Do
Accuracy requires ground truth that may take weeks to arrive. Input distribution and prediction distribution are available immediately.
Read More » -
One Job Per GPU Wastes Most of the GPU
Exclusive allocation is simple and leaves accelerators idle between batches, during data loading, and whenever a job is smaller than…
Read More » -
Model Debugging Requires Knowing Which Data Produced It
A model that regressed is impossible to investigate if the dataset it trained on has since been overwritten in place.
Read More » -
Nobody Can Reproduce Your Model — And That Is an MLOps Failure
If your best model exists only in a notebook on someone's laptop, you do not have a model — you…
Read More » -
Untracked Experiments Are Not Experiments
A model that beat the baseline but cannot be reproduced provided no information, because you cannot build on a result…
Read More » -
Dynamic Batching Is Why Two Identical Requests Get Different Latency
A request arriving alone is served immediately. The same request arriving during a full batch window waits for the batch…
Read More »