Interconnect Choices for Multi-Node Inference: Ethernet vs RoCE vs InfiniBand
Measured performance of three interconnect options for tensor-parallel inference on a two-node 8-GPU cluster, and when InfiniBand is worth the premium.
clusters · networking · tensor-parallel