Bias Analysis and Debiasing for IndicLLMSuite Datasets

Investigate potential biases present in the Sangraha and IndicAlign datasets used for training Indic large language models and develop debiasing techniques to mitigate these biases across the 22 Indian languages.

Background

The datasets in IndicLLMSuite are compiled from public web, PDF, and speech sources, as well as synthetic machine-translated and LLM-generated content. Such sources may embed societal and cultural biases that can propagate to models trained on them.

The authors explicitly state that assessing these biases and devising practical debiasing methods remains to be done, leaving open the comprehensive characterization of biases and the evaluation of mitigation strategies tailored to diverse Indic languages.

References

We leave the analysis on potential biases and debiasing techniques for future work.

— IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages  (2403.06350 - Khan et al., 2024) in Ethics Statement (Section)