Positional-attention based bidirectional deep stacked AutoEncoder for aspect based sentimental analysis

Zheng W., Zhang S., Yang C., Hu P.

Murugaiyan S., Uyyala S.R.

The process of detecting sentiments of particular context from human speech emotions is naturally in-built for humans unlike computers, where it is not possible to process human emotions by a machine for predicting sentiments of a particular context. Though machines can easily understand the content-based information, accessing the real emotion behind it is difficult. Aspect-based sentiment analysis based on speech emotion recognition framework can bridge the gap between these problems. The proposed model helps people with autism spectrum disorder (ASD) to understand other’s sentiments expressed through speech data about the recently purchased product based on various aspects of the product. It is a framework through which different sound discourse documents are characterized into various feelings like happy, sad, anger, and neutral and label the sound with aspect-wise sentiment polarity. This study proposed a hybrid model using deep convolutional neural networks (DCNN) for speech emotion recognition, bidirectional long short term memory (BiLSTM) for speech aspect recognition, and rule-based classifier for aspect-wise sentiment classification. In the existing work, sentiment analysis was carried out on speech data, but aspect-based sentiment analysis on speech data was not carried out successfully. The proposed model extracted standard Mel frequency cepstral coefficient (MFCC) features from customer speech data about product review and generated aspect-wise sentiment label. Enhanced cat swarm optimization (ECSO) algorithm was used for selection features from the extracted feature in the proposed model that improved the overall sentiment classification accuracy. The proposed hybrid framework obtained promising results on sentiment classification accuracy of 93.28%, 91.45%, 92.12%, and 90.45% on four benchmark datasets. The proposed hybrid framework sentiment classification accuracy on these benchmark datasets were compared with other CNN variants and shown better performance. Sentiment classification accuracy of the proposed model with state-of-art methods on the four benchmark datasets was compared and shown better performance. Aspect classification accuracy of the proposed with state-of-art methods on the benchmark datasets was compared and shown better performance. The developed hybrid model using DCNN, BiLSTM, and rule-based classifier outperformed the state-of-art models for aspect-based sentiment analysis by incorporating ECSO algorithm in feature selection process. The proposed model will help to perform aspect-based sentiment analysis on all domains with specified aspect corpus.

Gu T., Zhao H., He Z., Li M., Ying D.

Aspect-based sentiment analysis aims to analyze the sentiment polarity of a given aspect. The graph convolutional neural network model is widely used. However, most existing research focuses on mining the context-word-to-aspect-word dependencies of dependency trees based on the sentence itself without using much text-related external knowledge. In addition, the problem of reasonably capturing words outside the multihop grammatical distance and edge label hinders the effect of GCN. This paper proposes a graph convolutional network that fuses external knowledge (sentiment lexicon and part-of-speech information) (EK-GCN). Specifically, we conduct a statistical study on part-of-speech and construct a part-of-speech matrix to fully consider the influence of denying words, degree words, and other words that affect sentiment expression in sentences on sentiment classification. Then, an external sentiment lexicon is used to assign sentiment scores to each word in the sentence to construct a sentiment score matrix to highlight the weight of sentiment words, which to a certain extent, compensates for the fact that the syntactic dependency tree cannot capture edge labels. In addition, we design a Word-Sentence Interaction Network (WSIN), which can fully consider the information of the current aspect word and interact with the context information of the reviews to filter useful sentence information. We conduct experiments on four benchmark datasets, and the excellent experimental results demonstrate the effectiveness of our model. The results also verify that fully integrating external knowledge can assist in completing aspect-based sentiment analysis tasks.

Wang X., Pan X., Yang T., Xie J., Tang M.

Abstract Aspect-based sentiment analysis aims to identify the sentiment polarity of aspects in a given sentence. Although existing neural network models show promising results, they cannot meet the expectations in the case of a single network structure and limited dataset. When an aspect term composes more than one word, many models use the coarse-grained attention mechanism but lead to the unsatisfactory results. Besides, the relative distance between words in a sentence is always out of consideration. In this paper, we propose a model based on the interaction matrix and global attention mechanism to improve the ability of aspect-based sentiment analysis. First of all, the relative distance features of words in a sentence are initialized to enrich word embedding. Second, classic neural networks are applied to extract the essential features of word embedding in a sentence, such as long short-term memory and convolutional neural network. Third, an interaction matrix and global attention mechanism are combined to calculate weighted scores and measure relationships between aspect terms and context words. Finally, sentiment polarity is represented through a softmax layer. Experimental results on restaurant, laptop and twitter datasets show that the performance of the proposed model is superior to other methods.

Liang B., Su H., Gui L., Cambria E., Xu R.

Aspect-based sentiment analysis is a fine-grained sentiment analysis task, which needs to detection the sentiment polarity towards a given aspect. Recently, graph neural models over the dependency tree are widely applied for aspect-based sentiment analysis. Most existing works, however, they generally focus on learning the dependency information from contextual words to aspect words based on the dependency tree of the sentence, which lacks the exploitation of contextual affective knowledge with regard to the specific aspect. In this paper, we propose a graph convolutional network based on SenticNet to leverage the affective dependencies of the sentence according to the specific aspect, called Sentic GCN . To be specific, we explore a novel solution to construct the graph neural networks via integrating the affective knowledge from SenticNet to enhance the dependency graphs of sentences. Based on it, both the dependencies of contextual words and aspect words and the affective information between opinion words and the aspect are considered by the novel affective enhanced graph model. Experimental results on multiple public benchmark datasets illustrate that our proposed model can beat state-of-the-art methods.

Birjali M., Kasri M., Beni-Hssane A.

Sentiment analysis (SA), also called Opinion Mining (OM) is the task of extracting and analyzing people’s opinions, sentiments, attitudes, perceptions, etc., toward different entities such as topics, products, and services. The fast evolution of Internet-based applications like websites, social networks, and blogs, leads people to generate enormous heaps of opinions and reviews about products, services, and day-to-day activities. Sentiment analysis poses as a powerful tool for businesses, governments, and researchers to extract and analyze public mood and views, gain business insight, and make better decisions. This paper presents a complete study of sentiment analysis approaches, challenges, and trends, to give researchers a global survey on sentiment analysis and its related fields. The paper presents the applications of sentiment analysis and describes the generic process of this task. Then, it reviews, compares, and investigates the used approaches to have an exhaustive view of their advantages and drawbacks. The challenges of sentiment analysis are discussed next to clarify future directions. • Sentiment analysis is constantly evolving through approaches, data and models. • The paper provides an unprecedented and comprehensive survey on sentiment analysis. • Traditional and recent models are discussed, compared and classified. • Pointing out the reasons to select the proper model for sentiment analysis. • The paper summarizes the sentiment analysis models to monitor future trends.

Chandrasekaran G., Nguyen T.N., Hemanth D. J.

Kastrati Z., Dalipi F., Imran A.S., Pireva Nuci K., Wani M.A.

In the last decade, sentiment analysis has been widely applied in many domains, including business, social networks and education. Particularly in the education domain, where dealing with and processing students’ opinions is a complicated task due to the nature of the language used by students and the large volume of information, the application of sentiment analysis is growing yet remains challenging. Several literature reviews reveal the state of the application of sentiment analysis in this domain from different perspectives and contexts. However, the body of literature is lacking a review that systematically classifies the research and results of the application of natural language processing (NLP), deep learning (DL), and machine learning (ML) solutions for sentiment analysis in the education domain. In this article, we present the results of a systematic mapping study to structure the published information available. We used a stepwise PRISMA framework to guide the search process and searched for studies conducted between 2015 and 2020 in the electronic research databases of the scientific literature. We identified 92 relevant studies out of 612 that were initially found on the sentiment analysis of students’ feedback in learning platform environments. The mapping results showed that, despite the identified challenges, the field is rapidly growing, especially regarding the application of DL, which is the most recent trend. We identified various aspects that need to be considered in order to contribute to the maturity of research and development in the field. Among these aspects, we highlighted the need of having structured datasets, standardized solutions and increased focus on emotional expression and detection.

Ray P., Chakrabarti A.

Social networks have changed the communication patterns significantly. Information available from different social networking sites can be well utilized for the analysis of users opinion. Hence, the organizations would benefit through the development of a platform, which can analyze public sentiments in the social media about their products and services to provide a value addition in their business process. Over the last few years, deep learning is very popular in the areas of image classification, speech recognition, etc. However, research on the use of deep learning method in sentiment analysis is limited. It has been observed that in some cases the existing machine learning methods for sentiment analysis fail to extract some implicit aspects and might not be very useful. Therefore, we propose a deep learning approach for aspect extraction from text and analysis of users sentiment corresponding to the aspect. A seven layer deep convolutional neural network (CNN) is used to tag each aspect in the opinionated sentences. We have combined deep learning approach with a set of rule-based approach to improve the performance of aspect extraction method as well as sentiment scoring method. We have also tried to improve the existing rule-based approach of aspect extraction by aspect categorization with a predefined set of aspect categories using clustering method and compared our proposed method with some of the state-of-the-art methods. It has been observed that the overall accuracy of our proposed method is 0.87 while that of the other state-of-the-art methods like modified rule-based method and CNN are 0.75 and 0.80 respectively. The overall accuracy of our proposed method shows an increment of 7–12% from that of the state-of-the-art methods.

Ren F., Feng L., Xiao D., Cai M., Cheng S.

Aspect based sentiment analysis (ABSA) is the task of identifying fine-grained opinion polarity towards a specific target in a sentence, which is empowering experts and intelligent systems with enriched interaction capabilities. Most of approaches to date usually capture semantic relations between target and context words based on RNNs (Recurrent Neural Networks) or pre-trained models (e.g. BERT). However, due to computational complexity and size constraints, these models are often hosted in the cloud. Enabling ABSA models to run on resource-constrained end-devices with quick response time is still challenging and not yet well studied. This paper presents distillation network (DNet), a lightweight and efficient sentiment analysis model based on gated convolutional neural networks for on-device inference. Through combining stacked gated convolution with attention mechanism, DNet can distill aspect-aware context information from unstructured text progressively, achieving high performance with less inference latency and reduced model size. Experiments on SemEval 2014 Task 4 and ACL14 Twitter datasets demonstrate that our approach achieves the state-of-the-art performance. Furthermore, compared with the BERT-based model, DNet reduces the model size by more than 50 times and improves the responsiveness by 24 times.

Kastrati Z., Imran A.S., Kurti A.

Students' feedback is an effective mechanism that provides valuable insights about teaching-learning process. Handling opinions of students expressed in reviews is a quite labour-intensive and tedious task as it is typically performed manually by the human intervention. While this task may be viable for small-scale courses that involve just a few students' feedback, it is unpractical for large-scale cases as it applies to online courses in general, and MOOCs, in particular. Therefore, to address this issue, we propose in this paper a framework to automatically analyzing opinions of students expressed in reviews. Specifically, the framework relies on aspect-level sentiment analysis and aims to automatically identify sentiment or opinion polarity expressed towards a given aspect related to the MOOC. The proposed framework takes advantage of weakly supervised annotation of MOOC-related aspects and propagates the weak supervision signal to effectively identify the aspect categories discussed in the unlabeled students' reviews. Consequently, it significantly reduces the need for manually annotated data which is the main bottleneck for all deep learning techniques. A large-scale real-world education dataset containing around 105k students' reviews collected from Coursera and a dataset comprising of 5989 students' feedback in traditional classroom settings are used to perform experiments. The experimental results indicate that our proposed framework attains inspiring performance with respect to both the aspect category identification and the aspect sentiment classification. Moreover, the results suggest that the framework leads to more accurate results than the expensive and labour-intensive sentiment analysis techniques relying heavily on manually labelled data.

Aiello A.E., Renson A., Zivich P.N.

Disease surveillance systems are a cornerstone of public health tracking and prevention. This review addresses the use, promise, perils, and ethics of social media– and Internet-based data collection for public health surveillance. Our review highlights untapped opportunities for integrating digital surveillance in public health and current applications that could be improved through better integration, validation, and clarity on rules surrounding ethical considerations. Promising developments include hybrid systems that couple traditional surveillance data with data from search queries, social media posts, and crowdsourcing. In the future, it will be important to identify opportunities for public and private partnerships, train public health experts in data science, reduce biases related to digital data (gathered from Internet use, wearable devices, etc.), and address privacy. We are on the precipice of an unprecedented opportunity to track, predict, and prevent global disease burdens in the population using digital data.

Rajput A.

Recent advances in Big Data have prompted healthcare practitioners to utilize the data available on social media to discern sentiment and emotions’ expression. Health Informatics and Clinical Analytics depend heavily on information gathered from diverse sources. Traditionally, a healthcare practitioner will ask a patient to fill out a questionnaire that will form the basis of diagnosing the medical condition. However, medical practitioners have access to many sources of data including the patients’ writings on various media. Natural Language Processing (NLP) allows researchers to gather such data and analyze it to glean the underlying meaning of such writings. The field of sentiment analysis—applied to many other domains—depends heavily on techniques utilized by NLP. This work will look into various prevalent theories underlying the NLP field and how they can be leveraged to gather users’ sentiments on social media. Such sentiments can be culled over a period of time thus minimizing the errors introduced by data input and other stressors. Furthermore, we look at some applications of sentiment analysis and application of NLP to mental health. The reader will also learn about the NLTK toolkit that implements various NLP theories and how they can make the data scavenging process a lot easier.

Yang C., Zhang H., Jiang B., Li K.

Aspect-based sentiment analysis aims to predict the sentiment polarities of specific targets in a given text. Recent researches show great interest in modeling the target and context with attention network to obtain more effective feature representation for sentiment classification task. However, the use of an average vector of target for computing the attention score for context is unfair. Besides, the interaction mechanism is simple thus need to be further improved. To solve the above problems, this paper first proposes a coattention mechanism which models both target-level and context-level attention alternatively so as to focus on those key words of targets to learn more effective context representation. On this basis, we implement a Coattention-LSTM network which learns nonlinear representations of context and target simultaneously and can extracts more effective sentiment feature from coattention mechanism. Further, a Coattention-MemNet network which adopts a multiple-hops coattention mechanism is proposed to improve the sentiment classification result. Finally, we propose a new location weighted function which considers the location information to enhance the performance of coattention mechanism. Extensive experiments on two public datasets demonstrate the effectiveness of all proposed methods, and our findings in the experiments provide new insight for future developments of using attention mechanism and deep neural network for aspect-based sentiment analysis.

Drus Z., Khalid H.

This paper is a report of a review on sentiment analysis in social media that explored the methods, social media platform used and its application. Social media contain a large amount of raw data that has been uploaded by users in the form of text, videos, photos and audio. The data can be converted into valuable information by using sentiment analysis. A systematic review of studies published between 2014 to 2019 was undertaken using the following trusted and credible database including ACM, Emerald Insight, IEEE Xplore, Science Direct and Scopus. After the initial and in-depth screening of paper, 24 out of 77 articles have been chosen from the review process. The articles have been reviewed based on the aim of the study. The result shows most of the articles applied opinion-lexicon method to analyses text sentiment in social media, extracted data on microblogging site mainly Twitter and sentiment analysis application can be seen in world events, healthcare, politics and business.

Cai N., Li S., Xu J., Tian Y., Zhou Y., Liao J.

Automatic intention recognition in financial service scenarios faces challenges such as limited corpus size, high colloquialism, and ambiguous intentions. This paper proposes a hybrid intention recognition framework for financial customer service, which involves semi-supervised learning data augmentation, label semantic inference, and text classification. A semi-supervised learning method is designed to augment the limited corpus data obtained from the Chinese financial service scenario, which combines back-translation with BERT models. Then, a K-means-based semantic inference method is introduced to extract label semantic information from categorized corpus data, serving as constraints for subsequent text classification. Finally, a BERT-based text classification network is designed to recognize the intentions in financial customer service, involving a multi-level feature fusion for corpus information and label semantic information. During the multi-level feature fusion, a shallow-to-deep (StD) mechanism is designed to alleviate feature collapse. To validate our hybrid framework, 2977 corpus texts about loan service are provided by a financial company in China. Experimental results demonstrate that our hybrid framework outperforms existing deep learning methods in financial customer service intention recognition, achieving an accuracy of 89.06%, precision of 90.27%, recall of 90.40%, and an F1 score of 90.07%. This study demonstrates the potential of the hybrid framework to automatic intention recognition in financial customer service, which is beneficial for the improvement of the financial service quality.