Service chatbots: A systematic review

• This review conducts a quantitative analysis of state-of-the-art service chatbot. • Deep and reinforcement learnings dominate the most used chatbot design techniques. • Twitter dataset emerges to be the most popular dataset used for chatbot evaluation. • Accuracy becomes the most frequently used performance evaluation metric for chatbot. Chatbots or Conversational agents are the next significant technological leap in the field of conversational services, that is, enabling a device to communicate with a user upon receiving user requests in natural language. The device uses artificial intelligence and machine learning to respond to the user with automated responses. While this is a relatively new area of study, the application of this concept has increased substantially over the last few years. The technology is no longer limited to merely emulating human conversation but is also being increasingly used to answer questions, either in academic environments or in commercial uses, such as situations requiring assistants to seek reasons for customer dissatisfaction or recommending products and services. The primary purpose of this literature review is to identify and study the existing literature on cutting-edge technology in developing chatbots in terms of research trends, their components and techniques, datasets and domains used, as well as evaluation metrics most used between 2011 and 2020. Using the standard SLR guidelines designed by Kitchenham, this work adopts a systematic literature review approach and utilizes five prestigious scientific databases for identifying, extracting, and analyzing all relevant publications during the search. The related publications were filtered based on inclusion/exclusion criteria and quality assessment to obtain the final review paper. The results of the review indicate that the exploitation of deep learning and reinforcement learning architecture is the most used technique to understand users’ requests and to generate appropriate responses. Besides, we also found that the Twitter dataset (open domain) is the most popular dataset used for evaluation, followed by Airline Travel Information Systems (ATIS) (close domain) and Ubuntu Dialog Corpora (technical support) datasets. The SLR review also indicates that the open domain provided by the Twitter dataset, airline and technical support are the most common domains for chatbots. Moreover, the metrics utilized most often for evaluating chatbot performance (in descending order of popularity) were found to be accuracy, F1-Score, BLEU (Bilingual Evaluation Understudy), recall, human-evaluation, and precision.