A generative chatbot with natural language processing

Abstract

The goal in this thesis is to create a chatbot, a computer program that can respond verbally to a human in the course of simple day-to-day conversations. A deep learning neural network model called the Transformer is used to develop the chatbot. A full description of a Transformer is provided. The use of a few different Transformer-based Natural Language Processing models to develop the chatbot, including Generative Pre-Training 2 (GPT2), are shown. For comparison a Gated Recurrent Unit (GRU) based model is included. Each of these are explained below. The chatbot code is installed on a small device such as the Raspberry Pi with speech recognition and speech-to-text software. In this way a device that can carry out a verbal conversation with a human might be created. For the GRU-based model a Raspberry Pi 3B with 1GB RAM can be used. A Raspberry Pi 4B with 4GB of RAM is needed to run a chatbot with the GPT2.