JUPITER

1 stories

Every JUPITER story collected by The AI Daily, 1 so far, newest first, refreshed hourly.

Related topics

Domyn and PyTorch train 1T MoE on 384 GH200 GPUs

Domyn, working with TorchTitan and PyTorch maintainers, trained a ~1T-parameter MoE on 384 GH200 GPUs in the JUPITER supercomputer at ~1,300 tokens/s per GPU, a 14x gain over their starting point. They used 384-way FSDP plus 4-way expert parallelism kept inside nodes, skipping pipeline parallelism to keep iteration fast.

r/MachineLearning ·
That is everything