Practical Natural Language Processing by Sowmya Vajjala
Author:Sowmya Vajjala
Language: eng
Format: epub
Publisher: O'Reilly Media
Published: 2020-06-17T00:00:00+00:00
A Pipeline for Building Dialog Systems
We discussed various NLP tasks, such as classification and entity detection, throughout Chapters 4 and 5. Now, we’ll utilize some of them to describe an example pipeline to build a dialog system. Figure 6-3 depicts a complete pipeline of a dialog system with various components. We’ll discuss the utility of each component and data flow through the pipeline.
Figure 6-3. Pipeline for a dialog system
Speech recognition
Usually, the dialog system works as an interface between human and machine, so the input into the dialog system is human speech. Speech recognition algorithms transcribe speech to natural text. In industrial dialog systems, state-of-the-art [5] speech-to-text models are used, which is beyond the scope of this book. If you’re interested in speech models, refer to [5] for an overall view.
Natural language understanding (NLU)
After transcribing, the system tries to analyze and “understand” the transcribed text. This module encompasses various natural language understanding tasks. Examples of such tasks are sentiment detection, named entity extraction, coreference resolution, etc. This module is primarily responsible for gathering all possible information that is implicitly (sentiment) or explicitly (named entities) present in the input text.
Dialog and task manager
Once we obtain information from the input, a dialog manager, as shown in the figure, gathers and systematically decides which pieces of information are important or not. A dialog manager is a module that controls and guides the flow of the conversation. Imagine this as a table containing information extracted in NLU steps and stored concurrently for all utterances in the ongoing conversation. The dialog manager develops a strategy via rules or other complex mechanisms, such as reinforcement learning, to effectively utilize the information obtained from the input. Dialog managers are mostly prevalent in goal-oriented dialogs since there’s a definite objective to reach via the conversation.
Natural language generation
Finally, as the dialog manager decides a strategy for responding, the natural language generation module generates a response in a human-readable form according to the strategy devised by the dialog manager. The response generator could be template based or a generative model learned from data. After this, a speech synthesis module converts the text back to speech to the end user. For more information on speech synthesis tasks, take a look at [6] and [7].
Download
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.
Computer Vision & Pattern Recognition | Expert Systems |
Intelligence & Semantics | Machine Theory |
Natural Language Processing | Neural Networks |
Algorithms of the Intelligent Web by Haralambos Marmanis;Dmitry Babenko(8300)
Test-Driven Development with Java by Alan Mellor(6721)
Data Augmentation with Python by Duc Haba(6636)
Principles of Data Fabric by Sonia Mezzetta(6387)
Learn Blender Simulations the Right Way by Stephen Pearson(6284)
Microservices with Spring Boot 3 and Spring Cloud by Magnus Larsson(6158)
Hadoop in Practice by Alex Holmes(5958)
Jquery UI in Action : Master the concepts Of Jquery UI: A Step By Step Approach by ANMOL GOYAL(5806)
RPA Solution Architect's Handbook by Sachin Sahgal(5553)
Big Data Analysis with Python by Ivan Marin(5363)
The Infinite Retina by Robert Scoble Irena Cronin(5247)
Life 3.0: Being Human in the Age of Artificial Intelligence by Tegmark Max(5147)
Pretrain Vision and Large Language Models in Python by Emily Webber(4327)
Infrastructure as Code for Beginners by Russ McKendrick(4086)
Functional Programming in JavaScript by Mantyla Dan(4038)
The Age of Surveillance Capitalism by Shoshana Zuboff(3950)
WordPress Plugin Development Cookbook by Yannick Lefebvre(3801)
Embracing Microservices Design by Ovais Mehboob Ahmed Khan Nabil Siddiqui and Timothy Oleson(3606)
Applied Machine Learning for Healthcare and Life Sciences Using AWS by Ujjwal Ratan(3577)
