Distributed AI training coordinates data, parameters, activations, gradients, and optimizer state across many accelerators so ...