NVIDIA introduced the Global Performance Tuner, a feature for automatic performance tuning released in TensorRT 11.1, in the first of a two-part technical series. The post frames engine building as a compilation problem: TensorRT makes interdependent scheduling decisions about graph-IR transformations, fusion, implementation selection, memory placement and quantization, and different choices preserve model semantics while producing different inference performance. Historically those choices were encoded as heuristics derived from internal benchmark models, which the authors describe as producing heuristic sprawl, entangling performance with accuracy, and transferring poorly to unseen models and GPUs. The tuner instead generates a set of candidate engine configurations subject to an accuracy tolerance, then selects the fastest by measuring complete engines end to end. The article names two sources of suboptimality it is meant to address: heuristics that filter tactics before they can be evaluated, and local kernel profiling that diverges from whole-graph execution, citing adaptive grid sizing on Blackwell GPUs as an example.
Our takeAutotuning shifts cost from runtime to build time: candidate engines have to be compiled and measured, so the payoff depends on how often a model is rebuilt and how long a build can be allowed to run. Two details matter more than the chef analogy. The accuracy constraint is defined against chosen reference inputs and a tolerance, so the guarantee is only as good as that test set. And the post concedes that per-kernel timings disagree with whole-engine behaviour, which is precisely why end-to-end measurement becomes the selection criterion. Whether tuned artefacts survive driver or GPU changes