Summary

This project was conducted during my first year of Master Degree. I provided a solution for Pepper robot to handle Speech Detection, Speech Recognition and Natural Language Processing (NLP) in order to answer Human requests. This pipeline detects the user's input voice by analysing the evolution of ambient sound level. The Speech Recognition part is powered by the Google API Google Cloud Speech-To-Text and NLP by using deep learning approaches such as Mbot combined with rule-based APIs (e.g. Spacy). In addition to those solutions, I used a Dialog Act classifier. This classifier is used to adapt the system response to the type of utterance. For example if the sentence is considered as a Action-Command the system will run Mbot as an intents analysis, otherwise the classic rule-based model will be used. Finally I used the COMeT for Commonsense Inference approach to retrieve user intents and needs.