Review Compute

Compute is the processing capacity and associated work performed by computer hardware to train, evaluate, or run models and other software.

AI compute depends on hardware, memory, networks, software efficiency, energy, time, and availability. A raw chip count does not describe the usable capacity of the whole system.

Compute affects cost, latency, model development, deployment location, environmental load, and who can build or operate certain systems.

Training a large model uses many accelerators for weeks, while each deployed request consumes a smaller amount of inference compute.

Compute is not a single fixed measure of intelligence or capability, and more hardware does not compensate for every data, design, or evaluation problem.

Compute is the processing work and capacity behind the model.

* [[gpu-tpu|GPU and TPU]] * [[inference|Inference]] * [[scaling-laws|Scaling Laws]] * [[operational-slack|Operational Slack]] * [[layered-sovereignty|Layered Sovereignty]]

* [U.S. Department of Energy, “Supercomputing”](https://www.energy.gov/topics/supercomputing) — official explanation of high-performance computing as multiple computer systems working in parallel on work that less powerful systems cannot perform. * [Roy Schwartz and colleagues, “Green AI”](https://arxiv.org/abs/1907.10597) — connects machine-learning computation to financial cost, access, and energy use, and argues that efficiency should be evaluated alongside accuracy.

* Connect processing demand to specialized hardware in [[gpu-tpu|GPU and TPU]].