NVIDIA and IIT Madras Release 2.8T-Token Indic Pretraining Dataset
NVIDIA and IIT Madras' AI4Bharat lab released Nemotron-pretraining-Indic-v1, about 2.8 trillion tokens covering 22 constitutionally recognized Indian languages, plus an open GPU-accelerated data generation pipeline with prompts, LLM workflows and quality checks that developers can reuse to produce more data.