Aspect-Based Sentiment Analysis in Persian Using a Fine-Tuned LLaMA3 Model

Document Type : Research Article

Authors

Department of Computer Engineering, Jam Faculty of Engineering, Persian Gulf University, Bushehr, Iran.

10.22108/jcs.2026.145865.1175

Abstract

Aspect-Based Sentiment Analysis (ABSA) is less developed for Persian than for English, particularly for extracting complete aspect-category-opinion-sentiment structures rather than classifying sentiment for predefined aspects. This paper presents a generative Persian ABSA framework based on fine-tuned LLaMA3-family instruction models. Given a Persian review, the model generates a JSON array of quadruples containing aspect terms, categories, opinion spans, and sentiment labels.
The training data are drawn from a GPT-3.5-assisted Persian ABSA corpus whose annotations were reviewed and filtered by human annotators; we therefore treat it as GPT-assisted/human-validated rather than purely human-authored. To reduce circular evaluation, we also construct an independent human-only test set from raw Persian reviews. Three annotators manually annotated 240 reviews from scratch, and a two-of-three consensus procedure produced 555 gold aspect-level labels from 239 reviews.
We evaluate zero-shot LLaMA3, fine-tuned LLaMA3 models, and Dorna-LLaMA3-8B-Instruct as a Persian-oriented generative baseline under the same schema, parser, and metrics. On the human-only test set, zero-shot LLaMA3 obtains 0.1426 aspect F1 and 0.1185 aspect-sentiment F1. Fine-tuned LLaMA3-CG improves the five-seed mean scores to 0.5477 0.0229 and 0.4394, 0.0212, while Dorna-CG reaches 0.5809, 0.0123 and 0.4897, 0.0194. Adding ParsiNLU substantially improves in-domain ParsiNLU performance but reduces human-only performance, revealing a domain and annotation-style trade-off.

Keywords

Main Subjects

References

[1] Bo Pang and Lillian Lee. Opinion Mining and Sentiment Analysis. Foundations and Trends® in Information Retrieval. http://dx.doi.org/10.1561/1500000011. 2(1–2): 1-135, 2008. [DOI ]
[2] Gao, Zhengjie and Feng, Ao and Song, Xinyu and Wu, Xi. Target-Dependent Sentiment Classification With BERT. IEEE Access. 7: 154290-154299, 2019. [DOI ]
[3] Margarita Rodríguez-Ibánez and Antonio Casánez-Ventura and Félix Castejón-Mateos and Pedro-Manuel Cuenca-Jiménez. A review on sentiment analysis from social media platforms. Expert Systems with Applications. https://www.sciencedirect.com/science/article/pii/S0957417423003639. 223: 119862, 2023. [DOI ]
[4] Mohammad, Saif M. and Zhu, Xiaodan. Sentiment Analysis of Social Media Texts. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts. https://aclanthology.org/D14-2001. Association for Computational Linguistics. 2014.
[5] Dashtipour, Kia and Gogate, Mandar and Adeel, Ahsan and Ieracitano, Cosimo and Larijani, Hadi and Hussain, Amir. Exploiting Deep Learning for Persian Sentiment Analysis. Advances in Brain Inspired Cognitive Systems. 597--604, Springer International Publishing. 2018. [DOI ]
[6] Vakili, Yazdan Zandiye and Fallah, Avisa and Zakeri, Soodabeh. Enhancing Sentiment Analysis of Persian Tweets: A Transformer-Based Approach. 2024 10th International Conference on Web Research (ICWR). 226-230, 2024. [DOI ]
[7] Tianyu Zhao and Ling-ang Meng and Dawei Song. Multimodal Aspect-Based Sentiment Analysis: A survey of tasks, methods, challenges and future directions. Information Fusion. https://www.sciencedirect.com/science/article/pii/S1566253524003300. 112: 102552, 2024. [DOI ]
[8] Part AI. Dorna-Llama3-8B-Instruct. https://huggingface.co/PartAI/Dorna-Llama3-8B-Instruct. Accessed: 2026-06-27. 2024. [DOI ]
[9] Llama Team, AI at Meta. The Llama 3 Herd of Models. https://arxiv.org/abs/2407.21783. 2024. [DOI ]
[10] Farsi, Farhan and Aghababaloo, Farnaz and Shariati Motlagh, Shahriar and Ghofrani, Parsa and SadraeiJavaheri, MohammadAli and etc.. MELAC: Massive Evaluation of Large Language Models with Alignment of Culture in Persian Language. https://arxiv.org/abs/2508.00673. 2025. [DOI ]
[11] Ariai, Farid and Tayefeh Mahmoudi, Maryam and Moeini, Ali. Enhancing Aspect-based Sentiment Analysis with {ParsBERT} in Persian Language. Journal of AI and Data Mining. 12(1): 1--14, 2024. [DOI ]
[12] Xue, Linting and Constant, Noah and Roberts, Adam and Kale, Mihir and Al-Rfou, Rami and Siddhant, Aditya and Barua, Aditya and Raffel, Colin. m{T}5: A Massively Multilingual Pre-trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://aclanthology.org/2021.naacl-main.41/. 483--498, Association for Computational Linguistics. 2021. [DOI ]
[13] Tang, Duyu and Qin, Bing and Feng, Xiaocheng and Liu, Ting. Effective {LSTM}s for Target-Dependent Sentiment Classification. https://arxiv.org/abs/1512.01100. 2015. [DOI ]
[14] Vazan, Milad and Razmara, Jafar. Jointly Modeling Aspect and Polarity for Aspect-based Sentiment Analysis in Persian Reviews. arXiv preprint. https://arxiv.org/abs/2109.07680. arXiv:2109.07680: 2021. [DOI ]
[15] Ghiasvand Mohammadkhani and Niloofar Ranjbar and Saeedeh Momtazi. E2TP: Element to tuple prompting improves aspect sentiment tuple prediction. Neural Networks. https://www.sciencedirect.com/science/article/pii/S0893608025007270. 191: 107847, 2025. [DOI ]
[16] An, Wenbin and Tian, Feng and Chen, Ping and Zheng, Qinghua. Aspect-Based Sentiment Analysis With Heterogeneous Graph Neural Network. IEEE Transactions on Computational Social Systems. 10(1): 403-412, 2023. [DOI ]
[17] Chen, Yinuo and Zhao, Jing and Gao, Haoran. Adversarial Training Enhanced Multi-Channel Graph Convolutional Network for Aspect Sentiment Triplet Extraction. Proceedings of the 2023 6th International Conference on Signal Processing and Machine Learning. https://doi.org/10.1145/3614008.3614054. 301–307, Association for Computing Machinery. 2023. [DOI ]
[18] Jafarian, Hamoon and Taghavi, Amir Hossein and Javaheri, Alireza and Rawassizadeh, Reza. Exploiting BERT to Improve Aspect-Based Sentiment Analysis Performance on Persian Language. 2021 7th International Conference on Web Research (ICWR). 5-8, 2021. [DOI ]
[19] Shangipour ataei, Taha and Darvishi, Kamyar and Javdan, Soroush and Minaei-Bidgoli, Behrouz and Eetemadi, Sauleh. Pars-{ABSA}: a Manually Annotated Aspect-based Sentiment Analysis Benchmark on {F}arsi Product Reviews. Proceedings of the Thirteenth Language Resources and Evaluation Conference. https://aclanthology.org/2022.lrec-1.763. 7056--7060, European Language Resources Association. 2022.
[20] Farahani, Mehrdad and Gharachorloo, Mohammad and Farahani, Marzieh and Manthouri, Mohammad. ParsBERT: Transformer-based Model for Persian Language Understanding. Neural Processing Letters. https://doi.org/10.1007/s11063-021-10528-4. 53(6): 3831-3847, 2021. [DOI ]
[21] Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and etc.. LLaMA: Open and Efficient Foundation Language Models. https://arxiv.org/abs/2302.13971. 2023. [DOI ]
[22] OpenAI. GPT-3.5: The Latest in Language Models. OpenAI Blog. https://www.openai.com/blog/gpt-3-5-the-latest-in-language-models. 2023.
[23] Zheng, Yaowei and Zhang, Richong and Zhang, Junhao and Ye, Yanhan and Luo, Zheyan. Llama Factory: Unified Efficient Fine-Tuning of 100+ Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). https://aclanthology.org/2024.acl-demos.38/. 400--410, Association for Computational Linguistics. 2024. [DOI ]
[24] Jingwei Xu and Junyu Lai and Yunpeng Huang. MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models. https://arxiv.org/abs/2405.13053. 2024. [DOI ]
[25] Luca Massaron. Fine-tune Llama 2 for sentiment analysis, Kaggle. https://www.kaggle.com/code/lucamassaron/fine-tune-llama-2-for-sentiment-analysis.
[26] Furquan. furquan/llama2-sentiment-prompt-tuned. https://huggingface.co/furquan/llama2-sentiment-prompt-tuned.
[27] Hugging Face. SetFitABSA: Few-Shot Aspect Based Sentiment Analysis using SetFit. https://huggingface.co/models?pipeline_tag=absa.
[28] Khashabi, Daniel and Cohan, Arman and Shakeri, Siamak and Hosseini, Pedram and Pezeshkpour, Pouya and Alikhani, Malihe and etc. ParsiNLU: A Suite of Language Understanding Challenges for Persian. Transactions of the Association for Computational Linguistics. 9: 1147--1162, 2021. [DOI ]
[29] Tang, Duyu and Qin, Bing and Feng, Xiaocheng and Liu, Ting. Effective LSTMs for Target-Dependent Sentiment Classification. Proceedings of {COLING} 2016, the 26th International Conference on Computational Linguistics: Technical Papers. https://aclanthology.org/C16-1311. 3298--3307, The COLING 2016 Organizing Committee. 2016.
[30] vsmid, Jakub and Priban, Pavel and Kral, Pavel. LLaMA-Based Models for Aspect-Based Sentiment Analysis. https://aclanthology.org/2024.wassa-1.6/. 63--70, Association for Computational Linguistics. 2024. [DOI ]

Files

History

  • Receive Date: 12 July 2025
  • Revise Date: 28 June 2026
  • Accept Date: 22 July 2026

Share

How to cite

Statistics