Leveraging Large Language Models for Effective Data Augmentation in Imbalanced Rumor and Fake-News Detection

Document Type : Research Article

Author

Department of Computer Engineering, Faculty of Engineering, Ardakan University, Ardakan, Yazd, Iran.

Abstract

Rumor detection on social media remains challenging due to rapidly spreading information and the prevalence of misleading or unverified content. A major obstacle is the imbalance in available datasets, where minority classes, such as verified rumors, often represent less than 30% of samples. This imbalance biases classifiers toward majority classes and limits accurate misinformation detection. To address this, we propose a multi-model data augmentation framework leveraging four large language models; DeepSeek-R1-Distill-Qwen-1.5B, Gemma-2-2B, GPT-4o-mini, and LLaMA-3.2-3B fine-tuned on minority-class samples from the PHEME and MediaEval datasets. The generated synthetic tweets undergo comprehensive multi-dimensional quality validation, including GPT-2 perplexity for fluency, VADER for sentiment alignment, LDA for thematic coherence, and MAUVE and Self-BLEU for semantic similarity and diversity, confirming high fidelity to original data. When incorporated into a DistilBERT-CNN hybrid classifier, the augmented datasets significantly improve performance, increasing F1-scores from 0.8767 to 0.9806 on PHEME and from 0.8074 to 0.9013 on MediaEval. Our approach substantially outperforms conventional augmentation techniques (Random Oversampling, Easy Data Augmentation, and Back Translation), with statistical significance validated through paired t-tests and large effect sizes. These findings establish LLM-based augmentation as a robust, scalable, and practical solution for addressing class imbalance in rumor and fake-news detection.

Keywords

Main Subjects

References

[1] N. Difonzo and P. Bordia. Rumor Psychology: Social and Organizational Approaches. American Psychological Association. 2007. [DOI ]
[2] G.W. Allport and Leo Postman. The Psychology of Rumor. Psychological Review. 1947.
[3] J. Ma and W. Gao and P. Mitra and S. Kwon and B.J. Jansen and K.-F. Wong and M. Cha. Rumor Detection on Twitter. Proceedings of the 25th International Joint Conference on Artificial Intelligence. 3818--3824, 2016.
[4] Castillo, Carlos and Mendoza, Marcelo and Poblete, Barbara. Information Credibility on Twitter. Proceedings of the 20th International Conference on World Wide Web (WWW '11). 675--684, 2011. [DOI ]
[5] Kwon, Sejeong and Cha, Meeyoung and Jung, Kyomin and Chen, Wei and Wang, Yajun. Prominent Features of Rumor Propagation in Online Social Media. Proceedings of the IEEE International Conference on Data Mining (ICDM). 1103--1108, 2013. [DOI ]
[6] Thakur, Hardeep Kaur and Gupta, Aditi and Bhardwaj, Anshika and Verma, Dinesh. Rumor Detection on Twitter Using a Supervised Machine Learning Framework. International Journal of Information Retrieval Research. 8(3): 1--13, 2018. [DOI ]
[7] Wang, Shihang and Li, Zhen and Wang, Yumeng and Zhang, Qiang. Machine Learning Methods to Predict Social Media Disaster Rumor Refuters. International Journal of Environmental Research and Public Health. 16(8): 1452, 2019. [DOI ]
[8] He, Xiaoting and Tuerhong, Gulila and Wushouer, Mairidan and Xin, Dong. Rumors Detection Based on Lifelong Machine Learning. IEEE Access. 10: 25605--25620, 2022. [DOI ]
[9] N. Ruchansky and S. Seo and Y. Liu. CSI: A hybrid deep model for fake news detection. Proceedings of the International Conference on Information and Knowledge Management. 797--806, 2017. [DOI ]
[10] Z. Jin and J. Cao and H. Guo and Y. Zhang and J. Luo. Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs. Proceedings of the 25th ACM International Conference on Multimedia. 795--816, 2017.
[11] Y. Peng and J. Wang. Rumor detection based on attention cnn and time series of context information. Futur. Internet. 13: 2021. [DOI ]
[12] Sachin Kumar and R. Asthana and S. Upadhyay and N. Upreti and M. Akbar. Fake news detection using deep learning models: A novel approach. Trans Emerging Telecommun Technol. 2019. [DOI ]
[13] N. Rani and P. Das and A.K. Bhardwaj. A hybrid deep learning model based on CNN-BiLSTM for rumor detection. Proceedings of the 6th International Conference on Communications and Electronics Systems. 1423--1427, 2021. [DOI ]
[14] D. Choi and H. Oh and S. Chun and T. Kwon and J. Han. Preventing rumor spread with deep learning. Expert Systems Applications. 197: 116688, 2022. [DOI ]
[15] J. Ma and J. Li and W. Gao and Y. Yang and K.F. Wong. Improving Rumor Detection by Promoting Information Campaigns with Transformer-Based Generative Adversarial Learning. IEEE Transactions on Knowledge and Data Engineering. 35: 2657--2670, 2023. [DOI ]
[16] Q. Chang and X. Li and Z. Duan. A novel approach for rumor detection in social platforms: Memory-augmented transformer with graph convolutional networks. Knowledge-Based Systems. 292: 111625, 2024. [DOI ]
[17] N.V. Chawla and K.W. Bowyer and L.O. Hall and W.P. Kegelmeyer. SMOTE: Synthetic Minority Over-sampling Technique | Journal of Artificial Intelligence Research. Journal of Artificial Intelligence Research. 16: 321--357, 2002.
[18] R. Akter and M.M. Rahman and A. Debnath and N. Samrat and M. Roman and A.N. Khan and N. Alvi and M. Raihan. Enhancing Rumor Detection of Twitter Tweets Using a Deep Learning Approach. Proceedings of the 2024 15th International Conference on Computer Communications and Network Technology. 2024. [DOI ]
[19] Z. Zojaji and B. Tork Ladani. Adaptive cost-sensitive stance classification model for rumor detection in social networks. Social Network Analysis and Mining. 12: 1--17, 2022. [DOI ]
[20] J. Lai and X. Yang and W. Luo and L. Zhou and L. Li and Y. Wang and X. Shi. RumorLLM: A Rumor Large Language Model-Based Fake-News-Detection Data-Augmentation Approach. Applied Sciences. 14: 3532, 2024. [DOI ]
[21] X. Chen and D. Zhu and D. Lin and D. Cao. Rumor knowledge embedding based data augmentation for imbalanced rumor detection. Information Sciences. 580: 352--370, 2021. [DOI ]
[22] M. Askarizade. Enhancing rumor detection with data augmentation and generative pre-trained transformer. Expert Systems Applications. 262: 125649, 2025. [DOI ]
[23] DeepSeek-AI et al.. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv. 1--22, 2025. [DOI ]
[24] M. Riviere and S. Pathak and P.G. Sessa and C. Hardin and S. Bhupatiraju and L. Hussenot and T. Mesnard and B. Shahriari and etc.. Gemma 2: Improving Open Language Models at a Practical Size. arXiv preprint. 2024. [DOI ]
[25] Alec Radford and Jeffrey Wu and Rewon Child and David Luan and Ilya Sutskever and Dario Amodei. Language Models are Unsupervised Multitask Learners Alec. OpenAI Blog. 1: 2019. [DOI ]
[26] A. Dubey and A. Jauhri and A. Pandey and A. Kadian and A. Al-Dahle and etc.. The llama 3 herd of models. ArXiv preprint. 2024. [DOI ]
[27] Victor Sanh and Lysandre Debut and Julien Chaumond and others. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. ArXiv preprint. 2024. [DOI ]
[28] A. Zubiaga and M. Liakata and R. Procter and G. W. S. Hoi and P. Tolmie. Analysing how people orient to and spread rumours in social media by looking at conversational threads. PloS one. 11: 2016.
[29] Achiam, Josh and Adler, Steven and Agarwal, Sandhini and Ahmad, Lama and Akkaya, Ilge and Aleman, Florencia Leoni and Almeida, Diogo and etc.. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774. https://arxiv.org/abs/2303.08774. 2023.
[30] Boididou, Christina and Papadopoulos, Symeon and Dang-Nguyen, Duc-Tien and Boato, Giulia and Riegler, Michael and Kompatsiaris, Yannis. Verifying multimedia use at MediaEval 2015. MediaEval 2015 Workshop. 2015.
[31] Chalehchaleh, Razieh and Farahbakhsh, Reza and Crespi, Noel. Addressing data scarcity in multilingual fake news detection: an LLM-based dataset augmentation approach. Social Network Analysis and Mining. 15: 2025. [DOI ]
[32] Tong, Zhao and Gu, Yimeng and Liu, Huidong and Liu, Qiang and Wu, Shu and Shi, Haichao and Zhang, Xiao-Yu. Generate First, Then Sample: Enhancing Fake News Detection with LLM-Augmented Reinforced Sampling. International Journal of Information Retrieval Research. 1: 24276–24290, 2025. [DOI ]
[33] Tao, Jiran and Cheng, Wang and Binyan, Jiang. LLM-Enhanced Rumor Detection via Virtual Node Induced Edge Prediction.. arXiv preprint. arXiv:2602.13279. 2026.
[34] Zhou, Xiaojie and Ning, Tong and Deguang, Wang and Fengqi, Li. From latent structures to explicit reasoning: Synergizing clustering and LLM for explainable and robust rumor detection.. Information Processing and Management. 63: 2026. [DOI ]
[35] Wan, Herun and Wu, Jiaying and Luo, Minnan and Zeng, Zhi and Su, Zhixiong. Truth over tricks: Measuring and mitigating shortcut learning in misinformation detection. Advances in Neural Information Processing Systems. 38: 141018--141060, 2026. [DOI ]
[36] Luo, Ziyu and Li, Wei and Huang, Haoping and Liu, Kecheng and Gao, Min. SharP: Soft and hard prompt-guided augmentation with LLM for low resource fake news detection. Expert Systems with Applications. 131178, Elsevier. 2026. [DOI ]
[37] Askarizade, Mojgan and Davoodijam, Ensieh and Ladani, Behrouz Tork. Advancing rumor detection in social media: A quality-stratified systematic review from CNNs to LLM-GNN hybrids. Array. 100842, Elsevier. 2026. [DOI ]
[38] Pillutla, Krishna and Swayamdipta, Swabha and Zellers, Rowan and Thickstun, John and Welleck, Sean and Choi, Yejin and Harchaoui, Zaid. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers. NeurIPS. 2021.
[39] Hutto, Clayton and Gilbert, Eric. Vader: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the international AAAI conference on web and social media. 8(1): 216--225, 2014. [DOI ]

Files

History

  • Receive Date: 27 April 2026
  • Revise Date: 15 September 2026
  • Accept Date: 19 October 2026

Share

How to cite

Statistics