Systematic Literature Review: Machine Learning Methods in Emotion Classification in Textual Data
DOI:
https://doi.org/10.32736/sisfokom.v12i3.1787Keywords:
systematic literature review, klasifikasi emosi, data tekstualAbstract
Emotions are a person's response to an event. Emotions can be expressed verbally or nonverbally. Over time people can express their emotions through social media. Considering that emotion is a reflection of society's response, it is important to classify emotions in society to find out the community's response as information for consideration in decision-making. This study is aimed to identify and analyze the datasets, methods, and evaluation metrics that are being used in the classification of emotional texts in textual data from research data from 2013 to 2022. Based on the inclusion and exclusion design in selecting literature, a total of 50 kinds of literature were used in extracting and synthesizing data. Analysis of the data shows that out of 50 pieces of literature, there are 36 works of literature that use public datasets while 14 kinds of literature use private datasets. In the method of developing models for classifying, the SVM and Naive Bayes models are the most popular among the other models. In evaluating the model, the F-measure or F1-score metric is the most widely used metric compared to other metrics. There are three main contributions identified in this study, namely methods, models, and evaluationReferences
A. Dzedzickis, A. Kaklauskas, and V. Bucinskas, “Human Emotion Recognition: Review of Sensors and Methods,” Sensors, vol. 20, no. 3, 2020
L. Schoneveld, A. Othmani, and H. Abdelkawy, “Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition,” Pattern Recognition Letters, vol. 46, pp. 1-7, 2021
F. A. Acheampong, C. Wenyu, and H. Nunoo‐Mensah, “Text‐based emotion detection: Advances, challenges, and opportunities,” Engineering Reports, vol. 2, no. 7, 2020
Oberländer, L.A.M. and Klinger, R., “An analysis of annotated corpora for emotion classification in text,” in Proceedings of the 27th International Conference on Computational Linguistics, USA, 2018
P. Nandwani and R. Verma, “A review on sentiment analysis and emotion detection from text,” Social Network Analysis and Mining, vol. 11, no. 1, 2021
[
S. Zad, M. Heidari, J. H. J. Jones, and O. Uzuner, “Emotion Detection of Textual Data: An Interdisciplinary Survey,” 2021 IEEE World AI IoT Congress (AIIoT), USA. 2021
Keele, S, “Guidelines for performing systematic literature reviews in software engineering”, In: Technical report, Ver. 2.3 EBSE Technical Report, UK, 2007
V. Torres-Carrion, C. S. Gonzalez-Gonzalez, S. Aciar, and G. Rodriguez-Morales, “Methodology for systematic literature review applied to engineering and education,” 2018 IEEE Global Engineering Education Conference (EDUCON), Spain, 2018
Wahono RS, “A systematic literature review of software defect prediction”, Journal of software engineering, vol. 1, no. 1, pp. 1-6, 2015
Xiao, Y. and Watson, M., “Guidance on conducting a systematic literature review”. Journal of planning education and research, vol. 39, no. 1, pp. 93-112, 2019
T. Parvin, O. Sharif, and M. M. Hoque, “Multi-class Textual Emotion Categorization using Ensemble of Convolutional and Recurrent Neural Network,” SN Computer Science, vol. 3, no. 1, 2021
K. Kowsari, K. Jafari Meimandi, M. Heidarysafa, S. Mendu, L. Barnes, and D. Brown, “Text Classification Algorithms: A Survey,” Information, vol. 10, no. 4, p. 150, 2019
Y. Wang, R. Khardon, and P. Protopapas, “Nonparametric Bayesian estimation of periodic light curves,” The Astrophysical Journal, vol. 756, no. 1, pp. 67–67, 2012
T. Rabeya, S. Ferdous, H. S. Ali and N. R. Chakraborty, "A survey on emotion detection: A lexicon based backtracking approach for detecting emotion from Bengali text," 2017 20th International Conference of Computer and Information Technology (ICCIT), Dhaka, Bangladesh, 2017, pp. 1-7, doi: 10.1109/ICCITECHN.2017.8281855.
Ardiada, I.M.D., Sudarma, M. and Giriantari, D., 2019. “Text Mining pada Sosial Media untuk Mendeteksi Emosi Pengguna Menggunakan Metode Support Vector Machine dan K-Nearest Neighbour”. Maj. Ilm. Teknol. Elektro, vol. 18, no. 1, 2019
Fudholi, D.H., “Klasifikasi Emosi pada Teks dengan Menggunakan Metode Deep Learning”. Syntax Literate; Jurnal Ilmiah Indonesia, vol. 6, no. 1, pp.546-553, 2021
Osinga, D., Deep learning cookbook: practical recipes to get started quickly, Sebastopol: O'Reilly Media, Inc., 2018
Devlin, J., Chang, M.W., Lee, K. and Toutanova, K., “Bert: Pre-training of deep bidirectional transformers for language understanding”, arXiv (Cornell University), 2018
M. Grandini, Enrico Bagli, and Giorgio Visani, “Metrics for Multi-Class Classification: an Overview,” arXiv (Cornell University), 2020
Batbaatar E, Li M, Ryu KH. Semantic-emotion neural network for emotion recognition from text. IEEE access. 2019 Aug 12;7:111866-78.
Sailunaz K, Alhajj R. Emotion and sentiment analysis from Twitter text. Journal of Computational Science. 2019 Sep 1;36:101003.
Kratzwald B, Ilić S, Kraus M, Feuerriegel S, Prendinger H. Deep learning for affective computing: Text-based emotion recognition in decision support. Decision Support Systems. 2018 Nov 1;115:24-35.
Chatterjee A, Gupta U, Chinnakotla MK, Srikanth R, Galley M, Agrawal P. Understanding emotions in text using deep learning and big data. Computers in Human Behavior. 2019 Apr 1;93:309-17.
Li W, Xu H. Text-based emotion classification using emotion cause extraction. Expert Systems with Applications. 2014 Mar 1;41(4):1742-9.
Perikos I, Hatzilygeroudis I. Recognizing emotions in text using ensemble of classifiers. Engineering Applications of Artificial Intelligence. 2016 May 1;51:191-201.
Kiritchenko S, Zhu X, Mohammad SM. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research. 2014 Aug 20;50:723-62.
Deshpande M, Rao V. Depression detection using emotion artificial intelligence. In2017 international conference on intelligent sustainable systems (iciss) 2017 Dec 7 (pp. 858-862). IEEE.
M. Karna, D. S. Juliet and R. C. Joy, "Deep learning based Text Emotion Recognition for Chatbot applications," 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184), Tirunelveli, India, 2020, pp. 988-993, doi: 10.1109/ICOEI48184.2020.9142879.
S. Chawla and M. Mehrotra, "An Ensemble-Classifier Based Approach for Multiclass Emotion Classification of Short Text," 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India, 2018, pp. 768-774, doi: 10.1109/ICRITO.2018.8748757.
M. -H. Su, C. -H. Wu, K. -Y. Huang and Q. -B. Hong, "LSTM-based Text Emotion Recognition Using Semantic and Emotional Word Vectors," 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia), Beijing, China, 2018, pp. 1-6, doi: 10.1109/ACIIAsia.2018.8470378.
T. Patil and S. Patil, "Automatic generation of emotions for social networking websites using text mining," 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), Tiruchengode, India, 2013, pp. 1-6, doi: 10.1109/ICCCNT.2013.6726704.
U. Rashid, M. W. Iqbal, M. A. Sikandar, M. Q. Raiz, M. R. Naqvi and S. K. Shahzad, "Emotion Detection of Contextual Text using Deep learning," 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), Istanbul, Turkey, 2020, pp. 1-5, doi: 10.1109/ISMSIT50672.2020.9255279.
J. De Silva and P. S. Haddela, "A term weighting method for identifying emotions from text content," 2013 IEEE 8th International Conference on Industrial and Information Systems, Peradeniya, Sri Lanka, 2013, pp. 381-386, doi: 10.1109/ICIInfS.2013.6732014.
H. Al-Omari, M. A. Abdullah and S. Shaikh, "EmoDet2: Emotion Detection in English Textual Dialogue using BERT and BiLSTM Models," 2020 11th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan, 2020, pp. 226-232, doi: 10.1109/ICICS49469.2020.239539.
K. P. -Q. Nguyen and K. Van Nguyen, "Exploiting Vietnamese Social Media Characteristics for Textual Emotion Recognition in Vietnamese," 2020 International Conference on Asian Language Processing (IALP), Kuala Lumpur, Malaysia, 2020, pp. 276-281, doi: 10.1109/IALP51396.2020.9310495.
R. Majid and H. A. Santoso, "Conversations Sentiment and Intent Categorization Using Context RNN for Emotion Recognition," 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2021, pp. 46-50, doi: 10.1109/ICACCS51430.2021.9441740.
H. A. Ruposh and M. M. Hoque, "A Computational Approach of Recognizing Emotion from Bengali Texts," 2019 5th International Conference on Advances in Electrical Engineering (ICAEE), Dhaka, Bangladesh, 2019, pp. 570-574, doi: 10.1109/ICAEE48663.2019.8975417.
H. Al Huzali and S. Ananiadou, "Improving Textual Emotion Recognition Based on Intra- and Inter-Class Variation," in IEEE Transactions on Affective Computing, doi: 10.1109/TAFFC.2021.3104720.
R. J. Hasudungan and M. L. Kodhra, "Detecting Emotion on Indonesian Online Chat Text Using Text Sequential Labeling," 2018 International Symposium on Advanced Intelligent Informatics (SAIN), Yogyakarta, Indonesia, 2018, pp. 167-172, doi: 10.1109/SAIN.2018.8673342.
L. Canales, W. Daelemans, E. Boldrini and P. Martínez-Barco, "EmoLabel: Semi-Automatic Methodology for Emotion Annotation of Social Media Text," in IEEE Transactions on Affective Computing, vol. 13, no. 2, pp. 579-591, 1 April-June 2022, doi: 10.1109/TAFFC.2019.2927564.
A. Yousaf et al., "Emotion Recognition by Textual Tweets Classification Using Voting Classifier (LR-SGD)," in IEEE Access, vol. 9, pp. 6286-6295, 2021, doi: 10.1109/ACCESS.2020.3047831.
Jonathan Herzig, Michal Shmueli-Scheuer, and David Konopnicki. 2017. Emotion Detection from Text via Ensemble Classification Using Word Embeddings. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR '17). Association for Computing Machinery, New York, NY, USA, 269–272. https://doi.org/10.1145/3121050.3121093
Adil Majeed, Hasan Mujtaba, and Mirza Omer Beg. 2021. Emotion detection in Roman Urdu text using machine learning. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (ASE '20). Association for Computing Machinery, New York, NY, USA, 125–130. https://doi.org/10.1145/3417113.3423375
Marco Polignano, Pierpaolo Basile, Marco de Gemmis, and Giovanni Semeraro. 2019. A Comparison of Word-Embeddings in Emotion Detection from Text using BiLSTM, CNN and Self-Attention. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization (UMAP'19 Adjunct). Association for Computing Machinery, New York, NY, USA, 63–68. https://doi.org/10.1145/3314183.3324983
Taiao Liu, Yajun Du, and Qiaoyu Zhou. 2020. Text Emotion Recognition Using GRU Neural Network with Attention Mechanism and Emoticon Emotions. Proceedings of the 2020 2nd International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI '20). Association for Computing Machinery, New York, NY, USA, 278–282. https://doi.org/10.1145/3438872.3439094
Panpan Li, Jun Li, Feiqiang Sun, and Peng Wang. 2017. Short Text Emotion Analysis Based on Recurrent Neural Network. In Proceedings of the 6th International Conference on Information Engineering (ICIE '17). Association for Computing Machinery, New York, NY, USA, Article 6, 1–5. https://doi.org/10.1145/3078564.3078569
T. Parvin and M. M. Hoque, “An Ensemble Technique to Classify Multi-Class Textual Emotion,” Procedia Computer Science, vol. 193, pp. 72–81, 2021, doi: 10.1016/j.procs.2021.10.008.
L. Kang, J. Liu, L. Liu, Z. Zhou, and D. Ye, “Semi-supervised emotion recognition in textual conversation via a context-augmented auxiliary training task,” Information Processing & Management, vol. 58, no. 6, p. 102717, Nov. 2021, doi: 10.1016/j.ipm.2021.102717.
R. Kumari, N. Ashok, T. Ghosal, and A. Ekbal, “Misinformation detection using multitask learning with mutual learning for novelty detection and emotion recognition,” Information Processing & Management, vol. 58, no. 5, p. 102631, Sep. 2021, doi: 10.1016/j.ipm.2021.102631.
D. Li, Y. Li, and S. Wang, “Interactive double states emotion cell model for textual dialogue emotion prediction,” Knowledge-Based Systems, vol. 189, p. 105084, Feb. 2020, doi: 10.1016/j.knosys.2019.105084.
T. T. Sasidhar, P. B, and S. K. P, “Emotion Detection in Hinglish(Hindi+English) Code-Mixed Social Media Text,” Procedia Computer Science, vol. 171, pp. 1346–1352, 2020, doi: 10.1016/j.procs.2020.04.144.
N. Shelke, S. Chaudhury, S. Chakrabarti, S. L. Bangare, G. Yogapriya, and P. Pandey, “An efficient way of text-based emotion analysis from social media using LRA-DNN,” Neuroscience Informatics, vol. 2, no. 3, p. 100048, Sep. 2022, doi: 10.1016/j.neuri.2022.100048.
Chowanda, A. et al. (2021) ‘Exploring text-based emotions recognition machine learning techniques on social media conversation’, Procedia Computer Science, 179, pp. 821–828. doi:10.1016/j.procs.2021.01.099.
Z. Li, H. Xie, G. Cheng, and Q. Li, “Word-level emotion distribution with two schemas for short text emotion classification,” Knowledge-Based Systems, vol. 227, p. 107163, 2021. doi:10.1016/j.knosys.2021.107163
K. Dheeraj and T. Ramakrishnudu, “Negative emotions detection on online mental-health related patients texts using the deep learning with MHA-BCNN model,” Expert Systems with Applications, vol. 182, p. 115265, 2021. doi:10.1016/j.eswa.2021.115265
D. Yasmina, M. Hajar, and A. M. Hassan, “Using YouTube comments for text-based emotion recognition,” Procedia Computer Science, vol. 83, pp. 292–299, 2016. doi:10.1016/j.procs.2016.04.128
A. Gupta and S. M. Srinivasan, “Constructing a heterogeneous training dataset for emotion classification,” Procedia Computer Science, vol. 168, pp. 73–79, 2020. doi:10.1016/j.procs.2020.02.259
F. M. Plaza-del-Arco, M. T. Martín-Valdivia, L. A. Ureña-López, and R. Mitkov, “Improved emotion recognition in Spanish social media through incorporation of lexical knowledge,” Future Generation Computer Systems, vol. 110, pp. 1000–1008, Sep. 2020, doi: 10.1016/j.future.2019.09.034.
Z. Halim, M. Waqar, and M. Tahir, “A machine learning-based investigation utilizing the in-text features for the identification of dominant emotion in an email,” Knowledge-Based Systems, vol. 208, p. 106443, Nov. 2020, doi: 10.1016/j.knosys.2020.106443.
Shrivastava, K., Kumar, S. & Jain, D.K. An effective approach for emotion detection in multimedia text data using sequence based convolutional neural network. Multimed Tools Appl 78, 29607–29639 (2019). https://doi.org/10.1007/s11042-019-07813-9
Hasan, M., Rundensteiner, E. & Agu, E. Automatic emotion detection in text streams by analyzing Twitter data. Int J Data Sci Anal 7, 35–51 (2019). https://doi.org/10.1007/s41060-018-0096-z
Dhar, S., Gour, V. & Paul, A. Emotion recognition from lyrical text of Hindi songs. Innovations Syst Softw Eng (2022). https://doi.org/10.1007/s11334-022-00520-z
Parvin, T., Sharif, O. & Hoque, M.M. Multi-class Textual Emotion Categorization using Ensemble of Convolutional and Recurrent Neural Network. SN COMPUT. SCI. 3, 62 (2022). https://doi.org/10.1007/s42979-021-00913-0
K., A., P., D., Sam Abraham, S. et al. Readers’ affect: predicting and understanding readers’ emotions with deep learning. J Big Data 9, 82 (2022). https://doi.org/10.1186/s40537-022-00614-2
Ghanbari-Adivi, F., Mosleh, M. Text emotion detection in social networks using a novel ensemble classifier based on Parzen Tree Estimator (TPE). Neural Comput & Applic 31, 8971–8983 (2019). https://doi.org/10.1007/s00521-019-04230-9
Ma, H., Wang, J., Qian, L. et al. HAN-ReGRU: hierarchical attention network with residual gated recurrent unit for emotion recognition in conversation. Neural Comput & Applic 33, 2685–2703 (2021). https://doi.org/10.1007/s00521-020-05063-7
Khalil, E.A.H., Houby, E.M.F.E. & Mohamed, H.K. Deep learning for emotion analysis in Arabic tweets. J Big Data 8, 136 (2021). https://doi.org/10.1186/s40537-021-00523-w
Angel Deborah, S., Mirnalinee, T.T. & Rajendram, S.M. Emotion Analysis on Text Using Multiple Kernel Gaussian.... Neural Process Lett 53, 1187–1203 (2021). https://doi.org/10.1007/s11063-021-10436-7
Yang, H., Alsadoon, A., Prasad, P.W.C. et al. Deep learning neural networks for emotion classification from text: enhanced leaky rectified linear unit activation and weighted loss. Multimed Tools Appl 81, 15439–15468 (2022). https://doi.org/10.1007/s11042-022-12629-1
Sailunaz, K., Dhaliwal, M., Rokne, J. et al. Emotion detection from text and speech: a survey. Soc. Netw. Anal. Min. 8, 28 (2018). https://doi.org/10.1007/s13278-018-0505-2
Alswaidan, N., Menai, M.E.B. A survey of state-of-the-art approaches for emotion recognition in text. Knowl Inf Syst 62, 2937–2987 (2020). https://doi.org/10.1007/s10115-020-01449-0
Elkobaisi, M.R., Al Machot, F. & Mayr, H.C. Human Emotion: A Survey focusing on Languages, Ontologies, Datasets, and Systems. SN COMPUT. SCI. 3, 282 (2022). https://doi.org/10.1007/s42979-022-01116-x
Downloads
Additional Files
Published
Issue
Section
License
The copyright of the article that accepted for publication shall be assigned to Jurnal Sisfokom (Sistem Informasi dan Komputer) and LPPM ISB Atma Luhur as the publisher of the journal. Copyright includes the right to reproduce and deliver the article in all form and media, including reprints, photographs, microfilms, and any other similar reproductions, as well as translations.
Jurnal Sisfokom (Sistem Informasi dan Komputer), LPPM ISB Atma Luhur, and the Editors make every effort to ensure that no wrong or misleading data, opinions or statements be published in the journal. In any way, the contents of the articles and advertisements published in Jurnal Sisfokom (Sistem Informasi dan Komputer) are the sole and exclusive responsibility of their respective authors.
Jurnal Sisfokom (Sistem Informasi dan Komputer) has full publishing rights to the published articles. Authors are allowed to distribute articles that have been published by sharing the link or DOI of the article. Authors are allowed to use their articles for legal purposes deemed necessary without the written permission of the journal with the initial publication notification from the Jurnal Sisfokom (Sistem Informasi dan Komputer).
The Copyright Transfer Form can be downloaded [Copyright Transfer Form Jurnal Sisfokom (Sistem Informasi dan Komputer).
This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s). After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted. The copyright form should be signed originally, and send it to the Editorial in the form of scanned document to sisfokom@atmaluhur.ac.id.
