Development of a CGAN-Based Method for Aspect Level Text Generation: Encouragement and Punishment Factors in the Aspect Knowledge

Document Type : Research Article

Authors

1 Department of Computer Engineering, Shahreza Campus, University of Isfahan, Isfahan, Iran.

2 Department of Artificial Intelligence, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran.

Abstract

Text mining systems may benefit from the use of automated text generation, especially when dealing with limited datasets and linguistic resources. Most successful text generation approaches are generic rather than aspect-specific, resulting in relatively inaccurate and similar sentences in different aspects. The present study proposes a solution to this problem by extracting aspect knowledge from relevant topics and creating the correct phrase based on the Conditional Generative Adversarial Network (CGAN) for each aspect. The proposed method produces sentences using an auxiliary dataset that cannot be distinguished from genuine sentences by the discriminator. In order to generate an auxiliary dataset, aspect-based information from datasets related to the target concept is extracted. To further improve the accuracy, the generator is encouraged or punished depending on the similarity with the training corpus. Two datasets in English and Persian are used to evaluate the performance of the proposed text generation method. The results show that adding similar aspects to the auxiliary dataset improves the quality of the generated sentences. In addition, encouragement leads to more accurate sentences, while punishment leads to more varied sentences.

Keywords

Main Subjects

References

[1] W. Yu, Ch. Zhu, Z. Li, Zh. Hu, Q. Wang, H. Ji, and M. Jiang. A survey of knowledge-enhanced text generation. ACM Computing Surveys, 54:1--38, 1 2022. [ bib | DOI | http ]
[2] M. Bayer, MA. Kaufhold, B. Buchhold, M. Keller, J. Dallmeyer, and Ch. Reuter. Data augmentation in natural language processing: a novel text generation approach for long and short text classifiers. International Journal of Machine Learning and Cybernetics, 14:135--150, 1 2023. [ bib | DOI ]
[3] Sh. Prabhumoye, A. W. Black, and R. Salakhutdinov. Exploring controllable text generation techniques. pages 1--14. International Committee on Computational Linguistics, 2020. [ bib | DOI | http ]
[4] A. BELZ. Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. Natural Language Engineering, 14:431--455, 10 2008. [ bib | DOI | http ]
[5] A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi. The curious case of neural text degeneration. CEUR Workshop Proceedings, 2540, 2019. [ bib | http ]
[6] W. Kool, H. van Hoof, and M. Welling. Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replacement. 3 2019. [ bib | http ]
[7] S. Zarrie, H. Voigt, and S. Schüz. Decoding methods in neural language generation: A survey. Information, 12:355, 8 2021. [ bib | DOI | http ]
[8] H. Hayashi. Neural aspect-based text generation, dissertation, carnegie mellon university. Technical report, Dissertation, Carnegie Mellon University, 2021. [ bib ]
[9] P. Sun, L. Wu, K. Zhang, Y. Su, and M. Wang. An unsupervised aspect-aware recommendation model with explanation text generation. ACM Transactions on Information Systems, 40:1--29, 7 2022. [ bib | DOI | http ]
[10] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, Sh. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. 6 2014. [ bib | http ]
[11] M. Mirza and S. Osindero. Conditional generative adversarial nets. 11 2014. [ bib | http ]
[12] K. Wang and X. Wan. Sentigan: Generating sentimental texts via mixture adversarial networks. pages 4446--4452. International Joint Conferences on Artificial Intelligence Organization, 7 2018. [ bib | DOI | http ]
[13] A. Aggarwal, M. Mittal, and G. Battineni. Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights, 1:100004, 4 2021. [ bib | DOI | http ]
[14] M. R. Pavan Kumar and P. Jayagopal. Generative adversarial networks: A survey on applications and challenges. International Journal of Multimedia Information Retrieval, 10:1--24, 3 2021. [ bib | DOI | http ]
[15] T. Iqbal and Sh. Qureshi. The survey: Text generation models in deep learning. Journal of King Saud University - Computer and Information Sciences, 34:2515--2528, 4 2020. [ bib | DOI | http ]
[16] D. Saxena and J. Cao. Generative adversarial networks (gans): Challenges, solutions, and future directions. ACM Computing Surveys, 54:1--42, 6 2021. [ bib | DOI | http ]
[17] B. Ghosh, I. K. Dutta, M. Totaro, and M. Bayoumi. A survey on the progression and performance of generative adversarial networks. pages 1--8. IEEE, 7 2020. [ bib | DOI | http ]
[18] G. H. de Rosa and J. P. Papa. A survey on text generation using generative adversarial networks. Pattern Recognition, 119:108098, 11 2021. [ bib | DOI | http ]
[19] T. Zhao, G. Li, Y. Song, Y. Wang, Y. Chen, and J. Yang. A multi-scenario text generation method based on meta reinforcement learning. Pattern Recognition Letters, 165:47--54, 1 2023. [ bib | DOI ]
[20] D. M. Blei, A. Y. Ng, and M. I Jordan. Latent dirichlet allocation. Journal of machine learning research, 3:993--1022, 2003. [ bib | http ]
[21] M. Shams and A. Baraani-Dastjerdi. Enriched lda (elda): Combination of latent dirichlet allocation with word co-occurrence analysis for aspect extraction. Expert Systems with Applications, 80:136--146, 9 2017. [ bib | DOI | http ]
[22] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9:1735--1780, 11 1997. [ bib | DOI | http ]
[23] M. Shams, N. Khoshavi, and A. Baraani-Dastjerdi. Lisa: Language-independent method for aspect-based sentiment analysis. IEEE Access, 8:31034--31044, 2020. [ bib | DOI | http ]
[24] Zh. Chen and B. Liu. Topic modeling using topics from many domains, lifelong learning and big data. volume 32, pages 703--711. Proceedings of the 31st International Conference on Machine Learning - ICML '14, 2014. [ bib ]
[25] J. Devlin, M. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. pages 4171--4186. Association for Computational Linguistics, 10 2019. [ bib | DOI | http ]
[26] M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri. Parsbert: Transformer-based model for persian language understanding. Neural Processing Letters, 53:3831--3847, 12 2021. [ bib | DOI | http ]

Files

History

  • Receive Date: 05 October 2022
  • Revise Date: 08 March 2023
  • Accept Date: 03 April 2023

Share

How to cite

Statistics